4 · Drift & Retraining

4. Detecting Data & Concept Drift in Production

Kolmogorov-Smirnov tests, Population Stability Index (PSI), and automated retraining triggers.

10 min read · 2 MCQs

Data Drift vs Concept Drift

Data Drift (Covariate Shift): The distribution of input features $P(X)$ changes (e.g. inflation shifts user transaction amounts). Concept Drift: The underlying relationship between input features and target outputs $P(Y|X)$ changes (e.g. consumer purchasing habits change during a global event).

Statistical Drift Tests

Statistical metrics like the Kolmogorov-Smirnov (KS) test for numerical data and Population Stability Index (PSI > 0.2 indicates significant shift) monitor live inference telemetry. When drift is confirmed, automated pipelines trigger data extraction, validation, and shadow model evaluation.

Chapter quiz

2 questions · pass mark 75%

1. What is concept drift?
2. A Population Stability Index (PSI) score above 0.2 typically indicates: