Four questions. One accountability chain.

In Part 1, we asked who has the authority to stop the machine. Now we move to the next question: once someone acts, can the institution prove what actually happened?

The chain continues: from authority to evidence.

Consider a case that stands in for many. An automated trading firm is investigated. Investigators ask for the audit trail: which model version was running, what inputs it received, what outputs it produced, and how those outputs became trading decisions, on every transaction across the previous eighteen months. The firm cannot produce it. The information exists in principle, since the model ran, the trades executed, and the data passed through, but no one built the traceability that would let the firm reconstruct any of it. The logs that do exist are inconsistent, partially stored, and not linked to specific model versions in a way that makes reconstruction possible. An investigation like this runs into seven figures in legal and consulting fees before it closes.

The AI need not have done anything wrong. The firm still cannot prove that, and an audit trail that cannot prove a system behaved is indistinguishable from one concealing an error. The exposure is not a bad model. It is the inability to answer for a good one.

The first article in this series argued that most banks cannot say who is authorised to stop a malfunctioning AI system. This is the next question, and it is the one that arrives the moment someone is authorised and does act: can the institution prove what the system actually did, to someone who was not in the room and is not inclined to take its word for it. Most cannot. And the reason is not negligence but design: the thing they built to answer the question was built to answer a different one.

A log is not evidence

Almost every AI system in production generates logs. The logs record that a decision happened, roughly when, and what the outcome was. A typical entry captures the prompt sent to the model, the response it returned, a timestamp, and perhaps a token count: telemetry that mimics an audit trail without functioning as one under examination.

The distinction that separates the two is the distinction between recording that something happened and recording why. A log that reads "risk factors were evaluated" or "criteria were assessed" describes that a process occurred. It contains no information about what the process actually produced: which factors, weighted how, against what threshold, with what confidence, on what data. When an examiner asks why a specific loan was declined or a specific transaction flagged, the log that documents that governance happened cannot answer, because it never captured the thing the examiner is asking about. It recorded that a decision was reached while discarding what the decision rested on.

This is why the common reassurance, that a system is logging everything, misses the point. The question is never whether a system logs. It is whether what it logs will survive cross-examination, and the honest answer for most deployments is that it was never designed to. The logging was built for engineering, to debug failures and monitor uptime. Defending a decision to a regulator is a different task requiring a different record: one that captures what the system concluded and on what basis, in enough detail to reconstruct the decision and challenge it. Knowing merely that the system ran is not that record.

Why you cannot fix this afterward

The instinct, once the gap is visible, is to close it retroactively: pull the logs together, sign them, aggregate them, run them through a compliance tool, and produce the evidence after the fact. This does not work, and the reason it does not work is the hardest thing in this article to accept, because it forecloses the cheap solution.

Evidence has to be captured at the moment of the decision. It cannot be reconstructed from adjacent systems afterward, because a reconstruction and a record are not the same kind of thing, even when they contain identical information. Consider the difference between two statements. The first: "a human approved this change, captured and signed at the moment of approval." The second: "we believe a human approved this change, because a ticket existed, a role existed, an approval workflow completed, and the logs suggest approval occurred." Every clause in the second statement can be true. None of it is evidence. It is an inference assembled from systems that were never designed to jointly attest to a single event, and an examiner who understands the difference will treat it as exactly that.

The problem sharpens with non-deterministic systems, which is what most modern AI is. A stochastic model can return different outputs for the same input across two runs. This means that even if the input is preserved perfectly, re-running the system later does not reliably reproduce the original decision. The reasoning that produced the actual output existed only at the moment it was produced, and if it was not captured then, it is not recoverable now. You cannot re-derive it, because the system that would re-derive it may derive something else. For a deterministic process, a complete record of the inputs is nearly as good as a record of the decision, because the inputs regenerate the output. For a non-deterministic one, the inputs do not regenerate anything, and the record of the decision is the only evidence that will ever exist.

This is why practitioners who have worked the problem converge on a single principle, stated more or less the same way across the field: evidence must be produced as a byproduct of how the system runs, not reconstructed after the fact. If the execution does not generate structured evidence at each decision step, no logging layer added later can manufacture it. The trail has to be a property of the architecture, designed in before the system goes live, because the one thing that cannot be retrofitted onto a decision is the record of how it was actually made.

The reproduction trap

There is a deeper version of this problem that the current regulatory framework has walked into without resolving, and it explains why "just validate the model" is not the escape it appears to be.

The revised US model risk guidance issued in April 2026, SR 26-2, carries forward the long-standing expectation that models be validated, which in practice means their outputs must be reproducible enough to test against a benchmark. Traditional model risk management was built on this: a model is a function, the same inputs produce the same outputs, and validation checks that the outputs are correct and stable. The EU AI Act's high-risk provisions carry a parallel expectation of explainability and auditability for systems making consequential decisions about people.

Generative and agentic systems do not satisfy this cleanly, which regulators have noticed. SR 26-2 explicitly places generative and agentic AI outside its formal scope, describing the technology as novel and rapidly evolving, while directing banks to apply appropriate governance to it anyway and signalling that specific guidance is coming. This produces an awkward middle position that a great many banks are currently occupying: they have deployed systems that the existing validation framework does not cleanly cover, under a supervisory expectation that they be governed as if it did, with the detailed rules still unwritten. The systems the framework handles well are the ones many banks are moving away from. The systems they are adopting most aggressively are the ones with the least settled guidance.

The formal research on this is blunter than the regulation. Recent theoretical work has begun proving that post-hoc reconstruction of an agentic system's reasoning is not merely hard but, under specified conditions, structurally impossible: once evaluation and execution are separated, a record written after the action cannot faithfully establish what was authorised before it, and no amount of downstream enrichment closes the gap. The details are technical and the field is young, but the direction should concentrate the mind. The reconstruction problem is not a tooling deficiency that better software will shortly solve. For some classes of system it appears to be a property of the architecture, which means the only place to solve it is at design time, in the decision to capture evidence as the system runs rather than to hope it can be assembled later.

What answering requires

Set against all of this, a defensible record contains nothing exotic. The practitioners building for examination have largely converged on the same elements.

The record has to pin the model identity and version to each decision, so that a decision can be tied to the exact system that produced it even after the model has been updated. The inputs the system acted on need preserving too, along with the data lineage behind them: where the data came from and whether it was filtered or altered before it reached the model. Most demanding of all is the reasoning itself, captured at the moment of decision, the factors weighted and thresholds applied, rather than a generic note that factors were considered. Where the system produces a confidence or uncertainty score, that belongs in the record as well, because a low-confidence decision warrants different scrutiny than a high-confidence one. So does every human action taken against the decision, any override, approval, or escalation, logged with the identity of the person and what they saw when they acted.

Two properties govern the whole record. It has to be immutable, because a log that can be altered after the fact is evidence of nothing, and a field in that log asserting its own immutability is not immutability. And it has to be reconstructible by an outsider: preserved in a form that lets an independent reviewer follow one decision end to end, within a bounded window, without the institution engineering the answer after the request arrives.

The test of whether a bank has this is simple, and a bank can run it on itself before a regulator runs it. Take one AI-influenced decision from the last quarter, chosen at random. Produce, from records made at the time, the model version that decided it, the inputs it used, the basis on which it decided, the confidence it held, and the person who reviewed it and what they saw. If that packet can be assembled from what already exists, the system is answerable. If assembling it requires inference from adjacent systems, or re-running the model, or a memo written now about what probably happened then, the system is not answerable, and the gap will not be visible until the day it is examined, at which point it is a finding rather than a fix.

The first article ended on the observation that a bank which never assigned authority over its AI had built a deployment it could not answer for. This is the same failure seen one layer down. Assigning the authority to act is necessary, but the person holding it is defenceless without the runtime evidence to show what was done and why. A named owner with no defensible record is in the same position as the trading firm: acting in good faith, over a system that did nothing wrong, unable to prove it. The authority to answer and the evidence to answer with are two different things, and a deployment needs both, designed in from the start, because neither can be added to a decision after it has already been made.

Yet even with clear authority and complete records, a third exposure remains. Both depend on knowing what the system is supposed to be doing, and an autonomous system does not hold still. It drifts from the behaviour it was validated on, quietly, while every individual decision still looks correct. The next question is how an institution catches that before an examiner does.

Sources

• Himadri Roy, "When Your AI System Gets Investigated, What Evidence Will You Have?", June 2026 (account of an investigated trading firm unable to reconstruct its AI decision trail). Details vary across secondary accounts; treat the case as illustrative and verify specifics before relying on them.

• Interagency guidance SR 26-2 / OCC Bulletin 2026-13, revised model risk management guidance issued April 2026 by the Federal Reserve, OCC and FDIC, superseding SR 11-7; generative and agentic AI placed outside formal scope with a request for information and further guidance signalled.

• EU AI Act, high-risk provisions on explainability and auditability; Annex III obligations and Article 50 transparency duties (2 August 2026).

• On evidence-as-byproduct and the logs-versus-why-trail distinction: MLflow, "Why Audit AI Decision Making: A 2026 Guide" (June 2026); industry writing on proof bundles and authorization events captured at write time versus reconstructed from telemetry (2026).

• On the formal limits of post-hoc reconstruction: "Atomic Decision Boundaries" (arXiv 2604.17511, 2026) and "The Accountability Horizon" (arXiv 2604.07778, 2026). These are recent theoretical results; cited as emerging research, not settled law, and read them before relying on the specific claims.

• On required audit-trail fields and retention: industry compliance checklists mapping SOX, HIPAA, ECOA, GDPR Article 22 and EU AI Act traceability expectations (2026).