What is BERTScore and why is it better than BLEU for evaluation of generation?

BERTScore is a metric for evaluating the quality of generated text, based on semantic similarity between the generated and reference texts. Unlike BLEU, which compares exact n-gram matches, BERTScore uses word embeddings from the BERT model to compute cosine similarity.

Advantages of BERTScore over BLEU:

• Considers semantics, not just surface-level matches.
• Can evaluate synonymous and paraphrased variants that BLEU might underestimate.
• More robust to language variability and word reordering.

Example: if the generated text conveys the same meaning but with different words, BLEU might give a low score, while BERTScore would give a high score, reflecting the actual quality of generation.

Thus, BERTScore is better suited for evaluating generative models where semantic closeness is important, rather than literal matching.