What is diverse beam search?

Diverse Beam Search is a modification of the classic beam search algorithm used for sequence generation (for example, in language models).

The main idea is to increase the diversity of generated options to avoid overly similar or repetitive results.

How it works:

• Instead of simply selecting the top-N most probable continuations, the beam is divided into several groups (diverse groups).
• In each group, the best candidates are selected considering a penalty for similarity to those already chosen in other groups.
• This encourages the search for diverse paths, not just the most probable ones.

Applying diverse beam search is useful when you need to obtain several high-quality but different options, for example, in dialogue tasks, translation, or creative text generation.