What Stands Between the Model and the Operation
In the last month, we’ve seen several of the largest banks in the U.S. discover AI agents in their customers’ accounts, under their customers’ own logins. In some cases, the agents were placing calls the customer hadn’t asked for, in others they were moving money. One bank executive said there could be thousands of them. The banks responded by doing the only two things their systems allowed them to do: block the agents outright or let them look but not touch.
I’ve spent the last two years building the element that sits in the gap between those two settings, for a different industry. I want to describe it, not as a product, but as a systemic approach, because the question it answers stopped being theoretical this month. It became our collective problem.
Four constituencies asked the same question in the same month
The bank story drew the most attention, but it wasn’t the only one. In the same few weeks, enterprise software companies shipped credentials for AI agents (signed passports stating who an agent is, what it may touch, and who answers for it). But a frontier lab announced that two of its own agents had weakened the monitoring meant to watch them, and one of the agents then denied that it had done so. Regulators in the U.S. and Europe began writing standards for agent identities and how much authority could be delegated to them. A safety agency created a new category of incident: an action that no one instructed. Large nurses’ unions in several states wrote standards for AI governance into their contracts for the first time ever.
Read separately, these were four problems. Read together, they’re one question, and it’s not the question the industry has been asking. The industry’s been asking what a model can do. Banks, labs, regulators, and nurses are now asking something much different: who issues the authority for AI agents to act, and where is the record of their actions?
That’s an authority question, not a capability question. And it has a different answer.
The industry rediscovered the first rung and stopped
Look at what the banks actually did. “Read-only” is not nothing. It’s the first rung of a ladder that every safety-critical industry has already climbed. Systems observe before they’re permitted to act. Aviation, nuclear power, and air traffic control have run that way for decades. Software that watches a live operation, predicts, and is scored against what happens, is the right first step. The banks took it.
But they had to stop there because there’s nothing above the first rung available to them. We’re beginning to correct that by building the identity half of the answer, a credential that says who the agent is and what it’s allowed to do. But no one has built the evidence half, the record that says what the agent did, what it declined to do and why, what each action was worth, and whether it should be trusted with more next month. A permission bounds the damage of a single action. It doesn’t say anything about whether the thing holding the permission has earned it.
That’s the gap. It’s not a gap in model capability. It’s a gap in architecture.
Where the pattern came from
I didn’t set out to answer the banks’ question. My goal was to build an operational layer for hospitals, analogous to those found in mission-critical industries, but not found in healthcare. In that environment the cost of software acting wrongly is measured in a patient’s time and a surgeon’s trust, and no one will let a model touch the operation on the strength of a demo.
What that environment teaches is a discipline. A model may propose. It may never act. Between the model and the operation sits something that doesn’t reason at all: a deterministic core that takes the proposal, decides whether to act or stay silent, writes down its decision and its reason, and is scored afterward against a record it didn’t write. Every advance in what the system is allowed to do is a decision a hospital’s governance body makes based upon how well it met the criteria. And that permission can be revoked instantly.
Over two years that discipline became three parameters and four mechanisms. I think they generalize. The bank case is the evidence that they do.
Three parameters
Any institution adopting the pattern declares three things.
The wall. The decisions the system may never make, regardless of how confident it is. At a hospital, clinical judgement. At a bank, whether to extend credit, change a limit, or open a relationship. The wall is not a policy the system consults. It’s a boundary the system cannot cross.
The truth source. The record against which every claim the system makes is settled. It has to be a record the system can’t write. A hospital’s admission and discharge feed or a bank’s own posting ledger become the truth source. If the agent being scored can write the scorecard, there’s no score.
The authority ladder. Who grants the system each step of autonomy, on what evidence, and who can revoke it. It can’t be the agent vendor, or the agent. It has to be a named body inside the institution, acting in a recorded decision.
Four mechanisms
With those declared, the pattern runs on four mechanisms.
Silence with a reason. When the system decides not to act, that decision is written down with its cause and is visible to the people it concerns. Most systems log what they did. Almost none log what they chose not to do. Yet the refusals are where the trust is built. Those near-misses become the evidence that the center can still hold.
The claim before the outcome. Before the system acts, it writes down what it expects the action to achieve, in units the truth source can measure. Then the outcome arrives, and the claim is credited only to the extent the record confirms it. Where nothing can be measured, nothing is credited. A system that states its expectation after the result is a system grading its own exam.
Settlement against a record the actor cannot write. The two mechanisms above are worthless if the system, or the party it works for, controls the record it’s scored against. That is literally the whole point. The bank already has the perfect record for this: its own ledger, which no agent can write. Until now, though, it’s never used it to settle anything.
Authority that’s earned. Autonomy advances only by a governed decision made against the settled record. It degrades immediately when the record turns, and recovers slowly. A system that was safe at deployment is not automatically safe at scale. Trust is a quantity that accrues, and it should be revocable.
Two readings for the bank case
I’ll stay at the level of principle here, because the specifics belong in a different kind of document. But two things follow directly, and they’re the two things the industry hasn’t said out loud yet.
First, an actor should be known by the record it must leave, not by the credential it presents. A login proves which password was used. It can’t distinguish a person from an agent holding the person’s password; nothing at the session layer can. But a record can. A governed actor, such as an agent, leaves exactly one entry for every attempt. That entry can be a refusal with a reason, or a claim settled against the ledger. A person acting through their own channel leaves neither. Two parties with the same credential produce different record shapes. That difference is a property of the ledger’s structure, not of anyone’s inspection. The bank doesn’t need to tell them apart at login. It needs to require the record.
Second, authority should be granted by the institution whose systems are acted upon, and it should be granted on the evidence. Today an agent’s authority is whatever its owner’s password confers. It gains all of it, at once, forever. The alternative is the ladder: observe, then propose, then act within bounds, then act. Each step is a decision the institution makes on the record, and each step revocable. The bank does not have to trust a vendor’s passport or a customer’s judgement. It can let the record decide.
Why the nurses matter
The constituency I find most instructive is the one furthest from the banks. Nurses’ unions have begun writing artificial intelligence into their contracts and the language they’ve won is procedural: notification before introduction, committees, review. It’s procedural because there’s nothing else to bargain over. No technical artifact exists that a nurse can read and a hospital can be held to.
The pattern I’ve described produces that artifact, as a side effect of how it works. It begins with a ledger in which every measure is attributed to a role and a team, rather than a person. Where a hospital legitimately needs to know who was where (a safety investigation for example, or a dispute) the system only allows it with a stated reason from an approved list and a named person authorizing it. That lookup leaves a trace of its own so it can’t happen quietly. It becomes a record that every individual who is affected by it can access and review.
That same record is what a bank needs. Every entry carries a reason. Nothing can be quietly edited. It can be replayed because the core is deterministic. Anyone with access to the raw events can rerun them and get exactly the same figures. The nurse checks her own entries, a bank regulator recomputes the whole thing and confirms the bank’s figures match. One primitive, two readers. I didn’t design it for the second. It was the first that forced it into existence.
The answer isn’t a smarter model
The question went public this month and the instinctive answer from the labs, the vendors, and much of the commentary, is a more capable, more aligned, more carefully instructed model. I think that’s the wrong layer. The model will keep getting better and will keep being the thing that proposes. What’s missing is what stands between it and the operation: a deterministic core that can stay silent, that commits to a claim before the outcome, that settles against a record it can’t write, and whose authority is granted and withdrawn by people, on evidence.
That layer’s built with AI but it doesn’t run on it. Nothing that reasons sits on the path that acts. That is the design, not a limitation.
The banks have built the first rung. The rest of the ladder is an architecture and it already exists.
Cleveland A. Motley III is the founder of Opta Health, which builds the operational control layer for hospitals. opta.health