What is mean reciprocal rank (MRR) and where is it used in search evaluation?

Mean Reciprocal Rank (MRR) is a metric for evaluating the quality of search and recommendation systems. It measures the average of the reciprocal ranks of the first relevant result in the list of results for a set of queries.

Formula:

[ MRR = \frac{1}{Q} \sum_{i=1}^Q \frac{1}{rank_i} ]

where (Q) is the number of queries, (rank_i) is the position of the first relevant result for the i-th query.

MRR is used in information retrieval tasks, including evaluating models based on NLP and embeddings, to understand how quickly a user receives a relevant answer. The higher the MRR (closer to 1), the better the search quality.