Four questions. One accountability chain.

In Part 2, we asked whether the institution can prove what actually happened. Now we move to the next question: what if the system changes without anyone noticing?

The chain continues: from evidence to monitoring.

A bank builds a credit risk model on three years of data and puts it into production. At launch it correctly identifies the large majority of default risk, and everyone involved can see that it works, because the numbers are there in front of them. Some months later it is still running. It still produces a score for every applicant, still integrates cleanly with everything downstream, still throws no error and triggers no alert. And it is quietly worse than it was, because the applicants coming through the door are no longer the applicants it was trained on, and the model is scoring today's population against yesterday's world.

This is the failure that has no symptom. A server that goes down announces itself. An integration that breaks throws an exception. A model that drifts does none of that. It keeps running, keeps producing plausible outputs, and degrades along a curve that is invisible from inside the system, because nothing the system reports has changed. The predictions still arrive. Only their quality has moved, and quality is the one thing an ordinary monitoring dashboard does not measure.

The first two articles in this series were about a single decision: who was authorised to act on it, and whether the bank could prove what the system did. This one is about time. A deployment that was answerable on the day it launched does not stay answerable on its own, because the system it was built to govern does not hold still. And the mechanism that is supposed to catch the drift is, in most institutions, the one thing least equipped to see it.

Why silence is the danger

Drift is dangerous because it is silent, and silence buys it time to compound. A model that degrades a little each week, across thousands of decisions, can process an enormous number of subtly wrong outcomes before anyone notices the trend line moving, and by the time degraded performance shows up in a business metric, the damage is already distributed across every decision made in the interim.

The industry evidence for how common this is has become hard to wave away. A widely cited McKinsey figure puts the share of companies experiencing noticeable AI performance degradation within the first year at around 40%. A larger study across multiple industries found that the substantial majority of machine learning models degrade over time, and that most businesses observed performance declines without having monitoring adequate to catch them early. Across enterprise deployments the trend line is consistent: degradation in production is the expected baseline, not the exception. A model is optimised against historical data at a fixed moment. It is, by construction, a snapshot of the world as it was, and the operating environment does not stand still to preserve the snapshot's accuracy.

There are two ways this happens, and the distinction matters because it determines what a bank would even look for. In the first, the inputs change: the mix of applicants, transactions, or customers shifts, while the underlying relationship the model learned still holds. In the second, the relationship itself changes: the same inputs no longer imply the same outcomes, because the world has moved. A credit model trained on an applicant pool averaging a strong credit profile can fail quietly when the incoming mix shifts toward a weaker one, even though nothing about the model has changed. The model did not break. The ground under it moved, and the model has no way of knowing that, because it only sees the inputs it is given and has no view of the population it is no longer seeing.

Why the AI you deployed is not the AI you validated

There is a version of this problem specific to generative and agentic systems that the classical framing misses, and it is worse in an instructive way.

The traditional account of drift assumes the model is fixed and the data moves. For a conventional statistical model that is right. For a modern AI system it is incomplete, because the system can drift even when its weights are frozen. The model can be exactly the model you validated, unchanged, and the deployment can still degrade, because the things around the model have moved: the prompts have been edited, the retrieval context has shifted, the usage patterns have changed, the definition of a good answer has quietly risen.

The most dangerous version of this is the one the bank does not control at all. When a system runs on an external foundation model served through an API, the vendor can update what sits behind that API at any time, pushing a new checkpoint or a backend optimisation with no change the bank can see. The integration code is identical. The behaviour is not. A model a bank validated one week can reason differently the next, on the same inputs, because the thing it is calling was quietly replaced, and the bank finds out only if it was watching the outputs closely enough to notice. You validated a system at a moment in time. What is running is an unvalidated variant wearing the original model's name.

This is the operational reality that connects back to the beginning of this series. The first pieces argued that AI deployments fail because organisations bolt a capable model onto a workflow without deciding who governs it. Drift is the same problem extended along the time axis. A boundary was drawn at launch: these cases the system handles, those go to a human; this threshold triggers review, that one does not. That boundary was correct for the system as it behaved on launch day. But the system does not behave the same way six months later, and the boundary has not moved, because nobody owns moving it. The deployment is being governed by a decision that describes a system that no longer exists.

The consequence is that "we validated this model before deployment" is not the reassurance it sounds like. Validation is a photograph. It certifies that the system was sound at a moment, under conditions that were true at that moment. It says nothing about whether the system is sound now, and for a system that is non-deterministic to begin with and mutable through its surrounding context, the gap between the validated state and the current state opens the day the system goes live and widens continuously afterward.

What the regulator now expects

The supervisory framework has moved to meet exactly this, and the move is recent enough that many banks have not adjusted to it.

The revised US model risk guidance issued in April 2026, SR 26-2, elevates ongoing monitoring from a periodic exercise to a continuous one. Where a bank could once treat validation as an annual event, with a review committee and a sign-off, the expectation now is real-time performance tracking for material models, with monitoring explicitly tied to drift, portfolio shifts, data relevance, and changing market conditions between validation cycles. The framework's own language on this is not about box-ticking. An examiner, in the accounts of those who deal with them, is less interested in whether the annual review happened than in whether the bank's monitoring is actually capable of catching a problem before it does damage.

That last point contains the part most banks will find uncomfortable, and it is the heart of this article. SR 26-2 retains a principle from its predecessor called effective challenge, and the principle has two halves that are easy to collapse into one. The first is that whatever a bank uses to test a model must be genuinely independent of the model it is testing: you cannot grade a system with its own outputs. The second is organizational. Effective challenge requires an independent party with the technical competence and the standing to contest the team that built the model, a second line that can tell the first line it is wrong and make it hold. A bank can have the metric right and the organization wrong, and the challenge still fails, because the people who could raise the alarm have neither the independence nor the authority to be heard.

This requirement usually fails through circularity rather than outright absence. A bank has a monitoring process, it produces green dashboards, the review is documented, and none of it ever catches anything, because the process was never actually capable of catching a problem. The challenge was procedural rather than substantive. It measured the things the model was already good at and never tested the thing that was quietly going wrong, and no one with the standing to question it was positioned to notice.

Drift is where circular monitoring gets exposed, because drift is precisely the failure that green dashboards do not show. If a bank monitors latency, throughput, and error rates, all of which can be perfect while decision quality collapses, its monitoring is measuring the health of the plumbing and calling it the health of the decisions. The fraud model that lets through twice the usual rate of fraud while passing every system health check is not a hypothetical; it is the standard illustration, because it is what happens when the signal a bank watches is independent of the outcome it actually cares about. Effective challenge, applied to drift, means the monitoring has to track the quality of the decisions, against something that does not depend on the system whose decisions are in question.

What catching it actually requires

Detecting drift is a solved problem in principle, but it requires instrumenting the thing that is hard to measure rather than the things that are easy, and doing it as the system runs.

It starts with a baseline. A bank cannot detect that decision quality has shifted if it never recorded what decision quality looked like at launch. This means capturing the distribution of the system's outputs when it is known to be working, so there is something for later behaviour to be compared against. Without that reference, drift stays invisible: it is present, but there is nothing to measure it against, and a great many deployments have no such baseline because nobody captured one before going live.

From the baseline, the discipline is watching the right distributions and setting thresholds that trigger before the system has spent a long time operating outside its validated range. The relevant signals are the ones about decisions rather than infrastructure: the distribution of the model's outputs, shifts in the inputs it is receiving, changes in the mix of cases, movement in confidence or in the rate at which humans override the system. Standardising which of these a bank tracks matters more than it appears, because when different teams measure drift differently, enterprise-wide governance becomes impossible and each system's health becomes a local dialect that no one above it can read.

And detection has to connect to a response, or it is just a more sophisticated way of watching the decline. A signal that decision quality has moved needs to route somewhere: to a recalibration of the boundary, a retraining, a tightening of what the system is allowed to decide alone, or at minimum a human who is now accountable for deciding what to do about it. Detection without a defined response is the monitoring equivalent of a smoke alarm with no one in the building. It records the fire accurately and changes nothing.

The gap tends to reveal itself the moment an institution actually examines a mature deployment, because the same three absences turn up again and again. There is no record of what the system's decision quality looked like at launch, so there is no baseline for current behaviour to be measured against. There is no tripwire that would fire on a slow degradation, so nothing short of a customer complaint or a regulator's question would surface it. And there is no one who receives a drift signal and is empowered to act on it, so even a detected problem has nowhere to go. A bank that finds all three in one of its systems is not monitoring that system in any sense a regulator would accept. It is watching dashboards that confirm the system is running, which it is, right up until the damage is large enough to see without them.

The first article asked who is authorised to stop the system. The second asked whether the bank could prove what it did. This one asks a question that sits underneath both, because it decides whether the answers stay true: how would the bank know the system had changed. A deployment that was governed, evidenced, and sound on launch day is none of those things a year later if nobody is watching the thing that actually degrades, in a way that could actually catch it. And the mechanism that watches has to be built to see decision quality, independent of the system it is judging, because the one failure a bank cannot afford is the monitoring that reports everything is fine while the decisions quietly stop being.

Authority, evidence, and monitoring are all standing operational capabilities. Each must be maintained long after the deployment team that built it has disbanded. That leaves the final structural question: who, inside the institution, actually holds them?

Sources

• On the prevalence of silent degradation: a McKinsey figure commonly cited at roughly 40% of companies experiencing noticeable AI performance degradation within the first year; a multi-industry study reporting that a large majority of ML models degrade over time and that most organisations lack monitoring to catch it early (2026 industry write-ups). Figures vary by source; treat as directional and verify against the primary studies before citing specific numbers.

• On drift in banking specifically, including the applicant-pool-shift example: industry panel reporting from banking technology conferences (2026).

• On drift with frozen weights (prompt drift, context shift, moving definition of "good enough," silent vendor model updates): GenAI observability and continuous-evaluation writing (2026).

• Interagency guidance SR 26-2 / OCC Bulletin 2026-13 (Federal Reserve, OCC, FDIC, 17 April 2026), superseding SR 11-7: elevation of ongoing monitoring from periodic to continuous, monitoring tied to drift and changing conditions, and the retained "effective challenge" pillar requiring the challenge to be independent of the model. Verify scope and current status against primary sources and counsel; SR 26-2 is supervisory guidance, not a prescriptive rule, and places generative/agentic AI outside its formal scope.

• On drift-detection metrics (population stability, distribution shifts, confidence and override-rate movement) and the need for standardised signals and a baseline captured at launch: enterprise MLOps and model-monitoring literature (2026).