What are reference-free and reference-based metrics?

Reference-based metrics in machine learning and natural language processing are methods of evaluating the quality of a model that require a reference answer or set of answers for comparison. For example, in machine translation or text generation tasks, BLEU, ROUGE, and METEOR are often used — they compare the generated text with the reference.

Reference-free metrics (also called unreferenced metrics) evaluate the quality of the model's output without direct comparison to a reference answer. They may use internal features of the text, language models, or other heuristics to assess quality. Such metrics are useful when reference data is absent or limited.

Example: for evaluating dialogue systems, reference-free metrics can assess coherence, relevance, and grammaticality of the response without a reference, whereas reference-based metrics require pre-prepared correct answers.