What is DCG and why is normalization needed?

DCG (Discounted Cumulative Gain) is a metric for evaluating the quality of ranking in information retrieval and recommendation systems. It measures the usefulness of search results considering their position: relevant documents higher in the list contribute more.

The DCG formula takes into account the relevance of each result and reduces its impact as the position increases, usually using logarithmic discounting:

DCG_p = rel_1 + rel_2 / log_2(2) + rel_3 / log_2(3) + ... + rel_p / log_2(p)

Normalization is necessary to compare results from different queries with varying numbers of relevant documents. For this, the IDCG (Ideal DCG) is calculated — the maximum possible DCG for a set of relevant documents sorted by decreasing relevance. Then, DCG is normalized by IDCG:

NDCG_p = DCG_p / IDCG_p

This allows obtaining a value in the range [0,1], where 1 indicates an ideal ranking. Normalization is important for an objective comparison of ranking quality across different queries and systems.