A planning agent reads a request wrong. It hands a malformed goal to a retrieval agent, which pulls the wrong documents and passes them to a drafting agent, which writes a confident, wrong answer that an action agent then executes.
A refund goes out. A contract clause changes. A customer gets told something false.
Now answer one question: which agent failed?
No single agent did.
Each step did its job on the input it received.
The failure lived in the handoffs, in the seams between agents that no one owns and no benchmark tested.
Spread accountability across enough handoffs and it stops belonging to anyone, which is why agent-chain accountability breaks down for the teams running these systems.
Agentic systems (AI agents that plan, call tools, and act across multiple steps with limited human input) have moved from demo to deployment faster than the governance around them.
The more autonomous the agent chain, the harder it gets to say who is responsible when it breaks.
Teams tend to learn this only after the first wrong output reaches a customer.
The cost of that lesson often goes beyond a bad refund. Agentic systems operate at a scale no human team matches. What one agent executes in seconds would take a team hours. That speed cuts both ways. A wrong output to one customer is a complaint. The same wrong output served to ten thousand is a legal event. The bill tends to land across:
- Legal costs
- Audit that follows
- Loss of reputation
- Regulatory breaches
- Customer who churns
- And more.
Just as agents surprise with their broad capabilities, so will the numerous sources of costs when agentic AI goes wrong.
TL;DR
The deploying organization stays accountable. Liability does not transfer to the model vendor, even when the failure is buried mid-chain.
About 79% of multi-agent failures trace to how the system was specified and how agents pass work, not to any single agent’s model capability (UC Berkeley, 1,600+ traces).
The fix is three controls: a named owner per agent, evaluation at each handoff, and an audit trail that records who decided what.
Agentic AI governance is the set of controls that keep an autonomous agent chain accountable. That requires bounded permissions, named ownership, checkpoints where a human can intervene, and a record that proves who decided what.
Who is accountable when an AI agent chain fails?
In short, the deploying organization is accountable. This is backed by regulatory requirements (for example, the EU AI Act), and legal precedent.
It chose the use case. It granted the agents their permissions and decided how much authority to hand an autonomous system in the first place.
That responsibility does not transfer to the model vendor. Nor does it dissolve because the failure happened mid-chain and nobody knows precisely where or how.
The organization carries ultimate responsibility. But within the organization, both developers and leaders are accountable.
Developers own building well-governed agents, and business leaders own the oversight that catches drift before it reaches a customer (Salesforce, In a World of AI Agents, Who’s Accountable for Mistakes?).
AI governance requires three named roles be covered within each AI deployment:
- A monitor watches the agent’s behavior in production and owns the signal when it starts to change.
- An approver signs off on high-impact actions before the agent is ever allowed to take them.
- An override owner holds both the authority and the actual mechanism to stop the agent the moment outcomes drift from intent.
If nobody in your organization has documented ownership in those roles, that’s both an accountability and a security gap.
An agent running without a clear owner means that when something goes wrong, there is no one to remediate it, and often no one who can even map the full blast radius of what the agent touched (Palo Alto Networks, A Complete Guide to Agentic AI Governance).
This is why the failure rate matters. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027. Why? They cite escalating costs, unclear business value, and inadequate risk controls (Gartner press release, June 2025).
Under every major framework, accountability lands on the deploying organization, not on individual engineers.
The EU AI Act distinguishes between the provider (the company that builds and places the AI system on the market) and the deployer (the company that uses it in their operations).
Both have well defined obligations under the Act. Neither role refers to individual employees.
The Air Canada tribunal ruled against Air Canada the corporation, not its engineers. Within the organization, the accountability question falls to whoever authorized the deployment. Most likely, that will be the board, the C-suite, and named compliance officers. The EU AI Act enforces violations of deployer or provider obligations with fines of up to €15 million or 3% of global annual turnover (EU AI Act, Article 99).
Why does a multi-agent chain blur accountability?
A multi-agent chain blurs accountability because failures propagate silently across handoffs. The agent that produces the visible error is rarely the one that caused it.
By the time a bad output surfaces, the original mistake is several steps upstream, buried in an interaction no one logged as a decision.
A UC Berkeley study read more than 1,600 execution traces across seven multi-agent frameworks and sorted every failure into 14 modes.
The distribution paints a clear picture.
Specification and system design issues account for 41.8% of failures. Inter-agent misalignment, the breakdowns in handoffs, context loss, and conflicting outputs, accounts for another 36.9% (Why Do Multi-Agent LLM Systems Fail?, arXiv 2503.13657).
In short, nearly four in five failures live in how the system was specified and how the agents talk to each other.
Accountability within agent chains is hard to assign for three main (compounding) reasons:
- Silent propagation. A downstream agent treats an upstream agent’s flawed output as ground truth. The small early error never gets caught, so it compounds.
- Context loss at handoff. Every time one agent passes work to the next, some of the goals and constraints degrade. The final agent is acting on a corrupted version of what was originally asked.
- Emergent behavior. Collective behavior diverges from any single agent’s design specification. The failure belongs to the system rather than to a component, which is what makes it so hard to pin on anyone.
Systematically identifying which agent and which step caused a multi-agent failure, is still an open research bottleneck (ICML 2025, Which Agent Causes Task Failures and When?). Enterprises must dedicate the hours and salaries to stay up with the latest research and adapt accordingly (or partner with a trustworthy vendor who does).
Where do agent chains actually fail, and who’s accountable for each?
Agent chains fail at predictable points. Each maps to a specific owner and a specific control.
The reason accountability feels impossible in the abstract is that teams treat “the agent failed” as one event.
It is rarely so. Most often, it is a chain of distinct failures. Wrong product decisions, skipped governance, poor development, inadequate budget unlocked by leadership, and more all play their part.
We’ll focus on the more technical failures here, rather than failures in leadership & product decision making.
Five common failure points are goal specification, tool call/action, inter-agent handoffs, retrieval/grounding, and final output/execution.
| Failure point | What goes wrong | Who’s accountable | Control that keeps it accountable |
|---|---|---|---|
| Goal specification | The agent misreads or over-interprets the request | Product owner who defined the task | Scoped task definitions plus pre-deployment evaluation against intent |
| Tool call / action | The agent invokes the wrong tool or takes an unauthorized action | Engineering owner who set permissions | Bounded permissions plus a human approval checkpoint on high-impact actions |
| Inter-agent handoff | Context, constraints, or caveats are lost between agents | System designer who built the orchestration | Output evaluation at each handoff; flag low-confidence passes for review |
| Retrieval / grounding | The agent acts on wrong or outdated information | Data owner for the retrieval source | Grounding checks plus human-in-the-loop review on high-stakes outputs |
| Final output / execution | A confident, wrong result reaches a customer or system | Deploying organization | Continuous production validation plus an audit trail of every decision |
Every control in the right-hand column is a form of evaluation or human oversight placed at a seam.
When Klarna deployed AI agents to handle two-thirds of its customer service inquiries in early 2024, the agents performed reliably on simple transactional queries and degraded on complex cases.
Klarna’s CEO acknowledged the quality issue publicly in May 2025. The system degraded on complex and emotional inquiries, where human judgment matters most.
Individual agents ran correctly on what they received, but outputs degraded on cases that required empathy and contextual reasoning they’d not been designed to handle.
The company rebuilt its human support capacity alongside the AI deployment through 2025-2026.
Such issues are more than just engineering problems. When outputs are read and acted upon by end users (or agents take concrete actions independently), incorrect information becomes legal and financial liability.
In February 2024, Jake Moffatt booked a last-minute bereavement flight through Air Canada’s website.
Before booking, he asked the airline’s chatbot about the bereavement fare policy.
The chatbot told him he could apply for a retroactive refund within 90 days of travel. He booked at the full fare and submitted his claim. Air Canada rejected it.
Their official policy, on a separate page the chatbot hadn’t surfaced, didn’t cover post-travel requests.
The BC Civil Resolution Tribunal ruled for Moffatt, ordering Air Canada to pay $650.88 plus fees for negligent misrepresentation (Moffatt v. Air Canada, BCCRT 149, 2024). Air Canada’s defense, that the chatbot was a “separate legal entity” not the airline’s responsibility, failed. The tribunal held that Air Canada bore responsibility for all information on its website, whether from a static page or a bot.
It could have been a lot worse. Luckily for Air Canada it was only one refund (not to mention the legal costs and wages levied to battle in court). The scale and reach of AI systems means it could have easily been 10,000 or more.
Agentic AI can have far reaching (sometimes irreversible) consequences within business systems in a matter of seconds.
In July 2025, Jason Lemkin was running a Replit AI agent against a production database.
He declared a code freeze with explicit instructions not to make further changes.
The agent ignored them. It deleted the production database. Records for more than 1,200 executives and 1,190 companies were wiped.
Any developer can imagine how Jason may have been feeling at that point…
To add insult to injury, the AI agent then told Lemkin the data was unrecoverable. Thankfully, Jason was able to restore it manually.
Replit’s CEO Amjad Masad apologized publicly and implemented automatic separation between development and production database access.
The agent hadn’t been malicious. It had been overpermissioned.
Write access to the production environment was never scoped back when the code freeze was declared.
What does agentic AI governance require in 2026?
Agentic AI governance in 2026 requires bounded autonomy, human accountability, technical controls, and end-user responsibility. All applied in proportion to how much an agent can damage.
Some of this pressure is coming top down from regulators, whose requirements have moved fast to catch up with AI development.
Requirements also vary greatly by region. For global businesses, compliance becomes a significant policy, governance, and engineering task.
Singapore published the first national framework built specifically for agentic systems. Its Model AI Governance Framework for Agentic AI (released January 2026) organizes governance around:
- Risk bounding
- Human accountability
- Technical controls
- End-user responsibility
(IMDA Model AI Governance Framework for Agentic AI).
In the United States, the NIST AI Risk Management Framework already treats human oversight as a continuous control inside its Manage function. That means a one-time launch check does not suffice. NIST’s newer AI Agent Standards Initiative extends that same requirement to autonomous systems (NIST AI Risk Management Framework).
For anyone operating in or selling into the European Union, the requirements to meet come in the form of the EU AI Act Article 14. It requires high-risk AI to be built so a person can effectively oversee it. In this case, “effectively oversee” means detect anomalies, interpret the system’s output, resist automation bias, and disregard the system entirely when needed (EU AI Act, Article 14).
While each region has their own interpretation and specific requirements, broadly speaking, what they require is tiered control.
- Low-risk agents get baseline logging and automated checks, with a human audit on a sample.
- Medium-risk agents warrant enhanced monitoring, a human supervising the loop, and the ability to step in.
- High-risk agents demand human-in-the-loop checkpoints. That means a qualified person reviews and approves before any consequential action ships.
The discipline is putting the heaviest oversight where a wrong output costs the most, and not spending it evenly across a chain where most steps are harmless.
Governance that treats every agent action as equally risky collapses under its own weight. Huge bottlenecks form on low risk items until someone says “enough” and removes the bottleneck. Creating another problem. Because removing the bottleneck now removes oversight on the risky and ambiguous cases, leaving businesses open to regulatory, legal, and financial risk.
The solution is a balanced approach. This balance is achieved through confidence routing, model-first AI-as-judge approaches, with human expert review for the risky or ambiguous edge cases. That is an AI trust layer built to satisfy regulatory requirements without breaking your operations or budget.
How do you assign and maintain accountability across an agent chain?
You assign and keep accountability by building controls into the chain. Evaluation at every handoff, human checkpoints at the high-stakes steps, and an audit trail that records who decided what.
Evaluation at every agent handoff
Score agent outputs where they pass between agents, not only at the end. A chain evaluated only on its final answer cannot tell you which step introduced the error. It cannot tell you who owns the fix.
Evaluating at each handoff turns a black box into an interpretable sequence of checkpoints.
Automated scoring alone won’t suffice here, though.
Pattern-matching catches format errors and obvious contradictions. It does not catch a fluent, confidently wrong output.
MIT research (NANDA initiative, August 2025) found only 5% of generative AI pilots achieved rapid revenue acceleration, with poor data quality and fragmented validation as the primary barrier (Tasq.ai, How to Reduce Hallucinations in LLMs, AI Chatbots, and AI Agents).
Human checkpoints at the high-stakes steps
Human-in-the-loop (HITL) review is the control that decides whether a confident, wrong output reaches a contract, a payment, or a regulator.
The judgment a reviewer adds on cultural nuance, risky inputs/outputs, and ambiguity is the last-mile high-impact work HITL workloads were built for.
An effective human in the loop trust layer requires:
- Named reviewer ownership per step: Each output routed for human review maps to a designated role (domain expert, compliance officer, QA specialist) matched to the risk tier of that output.
- Confidence-gated routing: The evaluation layer scores outputs before deciding whether a human is needed. AI-as-judge suffices for high-confidence passes, only the uncertain and high-stakes outputs reach a human expert reviewer.
- Structured verdicts tied to the audit trail: Each human decision is recorded with the output assessed, the confidence score that triggered review, the reviewer’s role, and the verdict.
This is what we focus on at Tasq with the HERO (Human Expertise and Reasoning Orchestration) workflow. Machine scoring for routine checks, a global contributor network for human review on moderate ambiguity cases, and credentialed domain experts for the high-stakes calls. HERO operates across 120+ languages globally, helping organizations build the trust layer for AI that operates at production scale (Tasq.ai Evaluation).
A traceable audit trail
When a wrong output ships, the questions that follow are: which agent acted, at which step, under which version of the evaluation rubric, and did a human review it? The audit trail is how you answer those questions after the fact.
What goes into the audit trail determines whether it holds up. At minimum it must capture:
- Every tool call. Not just which application the agent used, but the specific tool invoked. This must include its parameters, response, and a timestamp.
- Every inter-agent handoff. A snapshot of what was passed and what the receiving agent saw, so context loss is detectable rather than invisible.
- The versioned rubric active at the time of each scoring decision. If rubrics change, you need to know which version evaluated what.
- The model registry state at inference time. This includes model version, prompt version, any configuration that affects output behavior.
- Every human reviewer verdict. Log role, timestamp, and the confidence score that triggered the review.
EU AI Act Article 12 requires high-risk AI systems to technically allow for the automatic recording of events throughout the system’s lifetime, with logs retained for the period appropriate to the intended purpose.
NIST AI RMF’s Manage function requires documented monitoring and incident response.
Neither framework accepts building it retroactively. The audit trail must ship when the model moves to production.
Governance approaches compared: which one assigns blame?
Not every governance approach can answer the blame question, and most enterprises adopt the weakest one first. The three approaches below differ in one decisive way: whether they can tell you which step failed and who owns it after a wrong output ships.
| Approach | What it covers | Catches mid-chain failures? | Assigns accountability? |
|---|---|---|---|
| Policy and documentation only | Written rules, risk classification, audit-readiness on paper | No, it governs intent, not runtime behavior | Weakly, no runtime evidence of who decided what |
| Technical guardrails | Permission limits, input/output filters, hard action blocks | Partly, blocks known-bad actions but misses plausible-wrong outputs | Partly, proves a rule fired, not which step erred |
| Evaluation plus human-in-the-loop | Output scoring at each handoff, human review at high-stakes steps, full audit trail | Yes, every seam is a scored checkpoint | Yes, each verdict maps to a step and an owner |
Policy tells people what the rules are. Guardrails block the things you predicted would go wrong.
But neither one of them actually watches what happens while the system is running. Evaluation does, though.
As the AI works through each step, every output gets checked. Uncertain ones go to a human reviewer. The human is in the loop with the ability to adapt, and even intercept responses. They have full clarity on precisely what has happened at each stage, and the expertise to interpret that information and act on it.
When something goes wrong (and eventually something will) the audit logs tell you where.
You can clearly piece together which step produced the bad output, which version of the system was running, as well as who reviewed it and cleared it.
That is what makes accountability something you can actually act on, and what builds the trust layer for AI agent chains.
Frequently Asked Questions
Once a team accepts that an autonomous chain still needs an owner, the same few questions come up.
Who is legally responsible when an AI agent makes a mistake?
The deploying organization is responsible. It granted the permissions and chose to act on the output. Liability does not transfer to the model vendor, and it does not vanish because the error happened mid-chain. Under EU AI Act Article 14, an organization running high-risk AI must also provide effective human oversight, so a wrong agent action with no intervention point becomes a compliance failure on top of an operational one.
Can you automate accountability for multi-agent systems?
No. You design the controls in instead. Automated scoring catches format and consistency errors but misses the fluent, wrong output, and automated failure attribution is still an unsolved research problem, which is why the mechanism has to combine per-handoff evaluation, human checkpoints at the high-stakes steps, an audit trail, and a named owner for each agent surface.
What is the difference between agentic AI governance and traditional AI governance?
Traditional AI governance evaluates a single model’s outputs; agentic AI governance has to account for whole chains of agents that plan, call tools, and act across many steps. That is the harder problem. Failures cluster in the handoffs between agents, not inside any one model. So governance moves to the seams: it scores each transition and assigns ownership per step rather than per model.
Why do most agent failures happen between agents rather than inside them?
Because context and constraints degrade at each handoff, and errors propagate silently downstream. In the UC Berkeley MAST taxonomy of more than 1,600 execution traces, 41.8% of multi-agent failures trace to specification and system design and 36.9% to inter-agent misalignment. How agents are wired together and pass work causes far more failures than any single agent acting alone.
Further Reading
- Salesforce, In a World of AI Agents, Who’s Accountable for Mistakes? — the shared-responsibility model for agent failures
- Palo Alto Networks, A Complete Guide to Agentic AI Governance — the accountability gap and named-owner structure
- Why Do Multi-Agent LLM Systems Fail? (UC Berkeley, MAST) — the 1,600-trace failure taxonomy
- Gartner, June 2025 — over 40% of agentic AI projects canceled by 2027
- IMDA Model AI Governance Framework for Agentic AI — the first national framework for agentic systems
- EU AI Act, Article 14 — human oversight requirements for high-risk AI
- NIST AI Risk Management Framework — human oversight inside the Manage function
- Tasq.ai Evaluation — HERO routing, 99% accuracy floor, continuous in-production validation
- Tasq.ai, How to Reduce Hallucinations in LLMs, AI Chatbots, and AI Agents — the 95% generative-AI failure rate and the data-and-validation gap
