A model that was accurate at launch quietly gets worse. This guide explains model drift, why it happens, and how to catch it.
Model drift is the degradation of a machine learning model's accuracy over time as the real world changes and diverges from the data the model was trained on. A model that performed well at deployment can silently become unreliable as inputs and relationships shift.
A model learns patterns from historical data, but the world keeps moving: customer behaviour changes, systems evolve, new categories appear. As live data drifts away from the training distribution, the model's predictions become less accurate, often without any error or crash to signal the problem. Left unmonitored, a model can quietly make worse and worse decisions.
There are two main forms. Data drift is when the distribution of the input data changes, the model sees inputs unlike its training set. Concept drift is when the relationship between inputs and the correct output changes, so even familiar inputs now map to different answers. Both erode accuracy and need to be detected.
Drift is caught by continuously monitoring input distributions, prediction quality and outcome feedback against a baseline. When drift is detected, the response is typically to retrain the model on fresh data or, if quality has dropped too far, to roll back. In production, this monitoring and response is what keeps models trustworthy.
Ops Singularity's AI/ML and Agent Ops pillar monitors models in production for drift and quality regression and takes governed action, flagging, retraining triggers or rollback, so models stay trustworthy. Explore the AI/ML and Agent Ops pillar.
Model drift is caused by the real world changing so that live data diverges from the data a model was trained on, either the input distribution shifts (data drift) or the relationship between inputs and outputs changes (concept drift).
By continuously monitoring input data distributions, prediction quality and outcome feedback against a baseline, and alerting or acting when they deviate. The usual response is to retrain the model or roll it back.
Bring a real problem. We will show you Sentinel investigate, act and verify end to end, with every action reversible and audited.