A predictive model does not freeze the world at the moment it is deployed. Customers change behavior, factories replace components and fraud patterns adapt. Model drift is the mismatch that develops between the conditions a model learned from and the conditions it now sees.
A customer-risk model starts receiving many new accounts from a channel it barely saw in training. Even if the input feed is technically healthy, performance for those customers may differ. Review results by cohort and compare them with the original deployment population. Retraining is one option, but correcting the acquisition data or changing the decision rule may be more appropriate.
Monitor more than one signal
Track input changes, data quality and output distributions. Where true outcomes arrive later, compare predictions with those outcomes too. A sudden drop in a field's completeness may be a broken integration rather than genuine business change. A shift in customer mix may be real and require a different response.
Monitoring needs an owner and a calendar. A dashboard of drift scores that nobody reads creates false comfort. Decide which changes trigger a source-system check, which require segment evaluation, and which justify a controlled model update. Keep the old model and evaluation dataset available for comparison. After an update, watch actual decisions as well as statistical scores; a new version may alter who receives attention even when overall accuracy improves.
Define an investigation path
Choose thresholds that trigger review, not automatic retraining by default. Inspect source changes, labeling delays and segment performance. Keep a known baseline and the ability to roll back a model. Document who can approve a new version and how it will be tested.
Watch the decision, not just the score
Suppose a churn model still predicts reasonably well, but the retention team no longer has a suitable offer for its highest-risk group. The business value has drifted even if the statistical measure has not. Pair model monitoring with workflow and outcome monitoring, and revisit the original decision the system was meant to improve.
