What is target encoding for categorical features over time and how to avoid leakage?

Target encoding for categorical features in time series is a method of replacing categorical values with numerical ones based on the statistics of the target variable (for example, the mean target value for each category). In the context of time data, it is important to avoid leakage, which occurs when encoding uses future data relative to the current point in time.

To prevent leakage, target encoding for each time point uses only information available up to that point. For example, the mean target value is calculated for all previous observations of each category, excluding the current and future ones. This can be implemented using cumulative averages or rolling windows with delay.

Example in Python (pandas):

import pandas as pd

df = pd.DataFrame({
    'category': ['A', 'B', 'A', 'B', 'A'],
    'target': [1, 0, 1, 1, 0],
    'time': pd.date_range('2023-01-01', periods=5)
})

df = df.sort_values('time')

# Cumulative mean of target by category, shifted by 1 to avoid using the current target

df['target_enc'] = df.groupby('category')['target'].apply(lambda x: x.shift().expanding().mean())

print(df)

Thus, target encoding reflects historical statistics without leaking future information.