What is back leakage?

Back leakage in the context of machine learning and data quality testing is a situation where information related to the future gets into the model or test data intended for training or validation on past data. This leads to unrealistically inflated model performance on test data, as the model essentially "peeks" into the future.

Examples of back leakage:

• Time series: Including future values of a time series (e.g., tomorrow's stock price) in features or labels for training a model that should predict the past or present.
• Leakage from test set into training: Random inclusion of data from the test set into the training set, or using information from the test set when processing the training set (e.g., feature scaling based on statistics of the entire dataset, including the test set).
• Feature generation: Creating features based on data collected after the time point for which the prediction is made.

Consequences of back leakage:

• Unrealistic performance metrics on test data.
• Poor model performance in real-world conditions (when applied to new, future data).
• Incorrect conclusions about the quality of the model.

Preventing back leakage is critically important for building reliable models and conducting proper testing.