What is Bayesian target encoding?

Bayesian target encoding is a method of encoding categorical features in machine learning tasks, based on using the statistics of the target variable with Bayesian smoothing.

The idea is to replace a category with a number reflecting the average value of the target variable for that category, but taking into account the overall mean and the number of observations to avoid overfitting on rare categories.

The smoothing formula usually looks like this:

[ \text{EncodedValue} = \frac{\text{mean_target_for_category} \times n + \text{global_mean} \times k}{n + k} ]

where:

• (n) — the number of observations in the category,
• (k) — the smoothing parameter (prior mean weight),
• (\text{global_mean}) — the mean across the entire dataset.

This approach helps reduce noise and overfitting, especially when working with rare categories, and is often used in boosting algorithms for tabular data.