What is BLEU and in what tasks is it used? What are the problems with this metric?

BLEU (Bilingual Evaluation Understudy) is a metric for assessing the quality of machine translation and other text generation tasks. It measures how much the generated text matches one or more reference translations based on n-gram overlap.

Main features of BLEU:

• Calculates n-gram precision (usually from 1 to 4).
• Introduces a penalty for translations that are too short (brevity penalty).

Problems with BLEU:

• Does not consider semantic similarity, only exact word matches.
• Sensitive to style and phrasing, which can underestimate good but paraphrased translations.
• Less informative for short sentences.

It is used in machine translation, text summarization, and other NLP tasks where the quality of generated text is important.