How are MAP and MRR metrics calculated in ranking tasks?

MAP (Mean Average Precision) and MRR (Mean Reciprocal Rank) are quality metrics in ranking tasks that evaluate how well a model orders relevant elements.

• MAP is the average of Average Precision over all queries. Average Precision for a single query is the average of precision values at positions where relevant documents are found.

Formula for AP for a single query:

[ AP = \frac{1}{R} \sum_{k=1}^N P(k) \times rel(k) ]

where:

• (N) — number of retrieved documents,
• (R) — number of relevant documents,
• (P(k)) — precision at position (k),
• (rel(k)) — indicator of relevance at position (k) (1 if relevant, 0 otherwise).

MAP is the mean AP over all queries.

• MRR is the average of the reciprocal rank of the first relevant document over all queries.

Formula for one query:

[ RR = \frac{1}{rank} ]

where (rank) — position of the first relevant document in the list.

MRR is the mean RR over all queries.

Example: if for three queries, the first relevant documents are at positions 1, 3, and 2, then

[ MRR = \frac{1}{3} \left(1 + \frac{1}{3} + \frac{1}{2}\right) = 0.611 ]