The most expensive agent build we ever inherited had burned four hundred thousand pounds by the time the operations director rang us, and the agent was technically running. Nobody was using it. The vendor had demoed a charming thing in week two: an agent that read shipment exceptions from a carrier portal, reconciled them against the order management system, and drafted the customer email. By month three the portal had quietly changed its login flow twice, the OMS team had refused to issue a service account on policy grounds, and the customer-service supervisor had stopped opening the drafts because two of them had quoted the wrong delivery window to a tier-one retailer. We wrote our four-question filter the week after that call. Every prospective build now goes through the same four questions before we put a number on a page, and the answers tell us, and the buyer, whether there is a real agent here or just a slide in someone's board pack.
1. Can you describe the workflow in two sentences?
We ask the operator, not the executive sponsor, the person who actually does the work today, to describe the workflow the agent will take over. Two sentences. Subject, verb, object. Trigger, action, outcome. If they can do it, we keep talking. If the description runs to a paragraph, we stop and rescope.
A national insurer asked us last year for a claims-triage agent. The head of operations gave us a clean two-line brief in the first email. Then we got on a call with the team leader who runs the triage desk in Leeds. Her description ran for four minutes. It depended on whether the claim was motor or property, whether the policy was direct or broker-introduced, whether the customer had a prior claim in the last eighteen months, whether the damage estimate sat above or below the underwriter referral threshold, and whether the loss adjuster had already been instructed. That is not one workflow. It is at least six, with branches she had internalised so thoroughly she no longer noticed them. Six workflows means six integration surfaces, six sets of edge cases, six places where the agent's behaviour has to be specified, audited, and reversed. Six workflows is a programme of work, not a build.
Most agent ideas that fail at this question were never agent ideas. They were unspecified workflows in an AI hat. The team had been carrying the branching logic in their heads for years and assumed any sufficiently clever model would carry it too. Models do not carry institutional context. They carry whatever you put in the prompt and the tools you give them. If the workflow is not specifiable in two sentences, it is not specifiable in a prompt either.
The two-sentence test is also a forcing function for the buyer. It turns a vague aspiration, we want AI to handle our exceptions, into a concrete decision about which exception, which trigger, which downstream action. The insurer eventually picked one of the six: motor claims under five thousand pounds, direct policies only, no prior-claim history. That agent shipped in eleven weeks and now handles about forty per cent of inbound motor volume. The other five workflows are still done by humans. They will get their turn. They were never going to get it as part of the first build.
2. What is the single most constraining boundary on this work?
Every workflow has one binding constraint. Not three, not a list. One. The constraint that, if violated, makes everything else the agent did irrelevant. Name it before the design starts.
A mid-sized bank in Manchester came to us with a customer-service agent that had been in production for nine months. It was answering account questions well enough that complaints had dropped. Then the internal audit team ran a reconstruction test on fifty interactions: pick any answer the agent gave, reproduce the exact reasoning and inputs that produced it, six months after the fact. Eleven of the fifty could not be reproduced. The retrieval layer had been swapped twice, the prompt had been edited by three different engineers, and nobody had versioned the tool definitions. The agent was switched off the following Monday. Nobody had named auditor reconstruction as the binding constraint at the start, so nobody had designed the logging or the version pinning that would have made it survivable.
The constraint differs by sector but the pattern repeats. In healthcare it is almost always HIPAA Minimum Necessary: the agent must touch the smallest possible slice of protected health information needed for the task, and you must be able to demonstrate that boundary to an auditor. In legal it is playbook fidelity: the agent's redlines must match the firm's negotiation playbook clause by clause, because deviating is what loses the matter. In banking it is the reconstruction test described above. In retail it is the refund authority limit: past a certain pound threshold, a human must approve, and the agent must not invent ways around the threshold. In manufacturing it is line-down risk: anything the agent does that could halt the line carries a cost measured in tens of thousands of pounds per hour, so its scope of unilateral action is narrow by definition.
Name the constraint on day one and it shapes the build from the first design session: which data the agent is allowed to read, what gets logged, where human approval sits in the loop, how the audit trail is structured. Name it later and you are refactoring all of that under deadline pressure, which is the most expensive way to do it. The Manchester bank rebuilt the agent six months later with full prompt and tool versioning. The rebuild cost more than the original.
Without integration access you are not building an agent. You are building screen-scraping infrastructure wearing an agent costume, and it will break the first Tuesday the vendor changes a button.
3. Do you have API or integration access, or do we have to scrape?
This is the question that kills around seventy per cent of attractive agent ideas, and it is the one buyers most want to skip. The team has imagined an agent reading the data and acting on it. We ask, concretely, where the data lives and how the agent will reach it. Is there a documented API. Is there a service account the agent can use. Is the system on the company network or behind a vendor SSO with mandatory MFA. Has the system owner agreed to grant access, in writing.
A logistics planner we spoke to last quarter wanted an agent that would pull shipment status from three carriers, two of whom did not expose a public API and one of whom had a sandbox API that was rate-limited to a hundred calls an hour. The team's plan was to scrape the two carrier portals with a headless browser. We have built scrapers. They work. They are also fragile, expensive, and politically messy. Fragile because the target system can change a CSS class or a redirect on a Tuesday and your agent silently breaks until somebody notices the queue backing up at four in the afternoon. Expensive because every scraper needs its own monitoring, its own retry logic, its own login-session management, and a human on rota for when the scrape breaks at the end of a quarter. Politically messy because the carrier you never spoke to will eventually find out you are driving their portal with a robot, and the conversation that follows is not the one you wanted to be having.
We answer this question candidly with the buyer, workflow by workflow. Sometimes the answer is clean: the OMS has a REST API, the team owns the credentials, we can move. Sometimes the answer is that two of the five systems involved are reachable and three are not, in which case the agent shrinks to the reachable part and we have an honest conversation about whether the smaller scope still pays for itself. Sometimes the answer is no, and the right recommendation is to spend three months pushing the system owners for access before any agent work begins. The logistics planner ended up doing exactly that: they negotiated a proper API contract with the largest of the three carriers, scoped the agent to that lane only, and parked the other two until the commercial team could open the door. The worst outcome is to start building on the assumption that integration access will appear later. It almost never does.
4. Who keeps override authority, and what does the rollback look like?
The last question is the one that separates a production agent from a demo. We ask two things: which named human keeps the authority to overrule any decision the agent makes, and what the rollback looks like the day the agent has to be switched off. If either answer is missing, no compliance team will sign off and no operations team will trust the cut-over, regardless of how good the model is.
A fashion retailer in Birmingham launched a refunds agent last winter without a kill switch. The product owner had argued that kill switches were a sign of low confidence in the build. On the third Saturday after launch, a regex in the order-matching tool started returning the wrong order line for a particular SKU family, and the agent quietly approved about twelve hundred refunds on the wrong items overnight. When the duty manager noticed at seven on Sunday morning, the only way to stop it was to email the vendor and wait. Sixteen thousand pounds of unrecoverable refunds left the building between three and seven. The post-mortem listed two missing things: a button anyone in operations could press, and a daily reconciliation that would have caught the divergence by midnight.
Override authority is not a philosophical commitment. It is a button, an account, a workflow. The named human must be able to reverse a single agent action, restore the prior state, and have that reversal logged in a way the auditor will accept. Rollback is the same thing at scale: the agent is misbehaving and you need to revert the whole operation to manual within a shift, without losing the in-flight cases the agent had picked up. Both have to be designed in from day one. Bolting them on after launch is the kind of work that gets quoted at three weeks and takes three months, because every action the agent took in the interim now needs a backwards-compatible reversal path.
Three patterns cover most of what we build, depending on risk profile and the binding constraint named in question two. Most builds combine at least two of them.
- Per-decision override. The operator sees every action the agent proposes, can accept or reverse it individually, and every override is logged with reason codes. Used when each decision carries material consequence on its own: clinical triage, large refunds, contract redlines.
- Threshold-rollback. The agent runs autonomously while its confidence score, error rate, or downstream rejection rate stays above an agreed threshold. The moment the metric crosses the line, the workflow reverts to manual queueing and an alert fires. Used for high-volume, low-individual-stakes work where you cannot have a human in every loop but you also cannot tolerate quiet drift.
- Full kill-switch. A single button, owned by named people in operations and compliance, that stops the agent entirely and routes its inflow back to the manual queue within minutes. Every build has one. Treat it as a non-negotiable, not a feature.
When we lay these patterns out for a buyer, the conversation shifts from can the agent do this to who owns it once it does. That is the conversation that produces something operations and compliance will accept on go-live day.
Why the filter is an accuracy filter, not a sales filter
The four questions look like a way to disqualify work. They are not. They are a way to be accurate about what we agree to build, so the work we agree to actually ships and stays shipped. Teams that answer all four cleanly, yes we can describe it, yes we know the constraint, yes the integrations exist, yes override sits with the supervisor, ship more agents, not fewer. They have done the thinking before the build, so the build is mostly building. Teams that say yes to all four casually are the teams we meet again in month three of someone else's project, asking for help with a portal that just changed its login flow.
What to do on Monday morning
Pick the agent idea at the top of your backlog and try this in a half-hour meeting. Get the operator in the room, not the sponsor. Ask them to describe the workflow in two sentences and write what they say on a whiteboard. Ask them to name the one constraint that, if broken, makes the work pointless. Ask them, system by system, whether there is an API and a service account already in hand or whether somebody still needs to ask. Ask them who, by name, will press the button when the agent goes wrong. If three of the four answers come back clean, you have a scope. If two or fewer do, you have homework, and that homework is cheaper to do now than to discover in month four. We would rather lose a quote in the first call than find out, six months later, that the agent we built has nowhere to live.