What methods are used to detect data drift in a pipeline?

Data drift detection in a pipeline is an important step for maintaining model quality and prediction accuracy. The main methods include:

• Statistical tests: comparing the distributions of current data with training data using tests such as Kolmogorov-Smirnov, Cramér-von Mises, and chi-square for categorical features.

• Distribution distance metrics: calculating distances such as Kullback-Leibler divergence, Jensen-Shannon divergence, and Wasserstein distance.

• Model metric monitoring: tracking changes in quality metrics (accuracy, precision, recall, AUC) over time, which indirectly indicates drift.

• Training a drift detector: building a model that classifies whether a sample belongs to the old or new distribution, and determining the presence of drift based on its accuracy.

• Visualization: creating histograms, boxplots, t-SNE for visual detection of changes.

Example of using the Kolmogorov-Smirnov test in Python:

from scipy.stats import ks_2samp

# old_data and new_data are arrays of feature values
stat, p_value = ks_2samp(old_data, new_data)
if p_value < 0.05:
    print("Data drift detected in feature distribution")
else:
    print("No drift detected")

Regular monitoring and automation of these methods help to timely identify drift and take measures to update models.