When the AI Gets It Wrong, Who Carries It?

The model was wrong. The person got blamed.

Here is the shape of it. A report reaches a client with a figure in it that does not exist, or a letter quotes an obligation that was repealed. The work was AI-assisted, and a human read it, or half read it, and pressed send.

Within the hour someone senior asks the question that decides everything: who approved this?

That question has an answer, and the answer is almost always the most junior person in the chain. Not because they were careless. Because they were closest. They were the last human hand on the work, which makes them the only party the organisation can name, sanction and be seen to have dealt with. The vendor is not in the room. The model cannot be performance managed. The decision to put the tool into that workflow was made a level or two above the failure and is not on trial. So accountability settles where it fits rather than where it belongs, and nobody feels they have done anything unfair.

Why blame lands on whoever was closest

There is a name for that position, and it is nearly a decade old.

In Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction, published in Engaging Science, Technology, and Society in 2019, Madeleine Clare Elish introduced the concept "to describe how responsibility for an action may be misattributed to a human actor who had limited control over the behavior of an automated or autonomous system." The comparison is deliberate: "While the crumple zone in a car is meant to protect the human driver, the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator." Or, more bluntly, "the technology is maintained as faultless, while the human operator becomes the faulty feature of the system."

Her cases were the partial meltdown at Three Mile Island and the crash of Air France Flight 447, and her description of the Three Mile Island operators reads like a bad Monday in a modern office. They "knew the system was malfunctioning, but they did not have sufficient information or authority to take corrective actions." Elish is explicit that she is writing about cultural perceptions of blame rather than legal liability, and primarily in an American context. Take the shape, not a claim about your jurisdiction.

The newer empirical work is more mixed than the concept alone suggests, and worth being precise about rather than recruiting. In Responsibility Attribution in Human Interactions with Everyday AI Systems, presented at the 2025 CHI Conference in April 2025, Joe Brailsford, Frank Vetere and Eduardo Velloso put 321 participants through shared human-AI decision scenarios. The useful finding is an asymmetry. Where the outcome was positive, participants were more likely to ascribe shared responsibility to both the human and the AI. Where it was negative, they were more likely to attribute responsibility to a single entity, though not consistently to the human or to the AI. Credit spreads. Blame narrows to one point.

Where it narrows to depends on what people know. Takahiro Tsumura and Seiji Yamada, in Effects of knowledge and importance on responsibility in human-AI decision making in Scientific Reports in December 2025, ran 588 participants and found prior knowledge of the agent shifted responsibility away from the user and toward the agent and its developer, rising further on the developer as the topic was seen as more important. So the evidence does not say the nearest human always carries it, and neither study watched an organisation handle a real failure.

Our reading of the gap is this. Outside the organisation, blame floats to the developer, a legitimate target. Inside it, the developer is a contract, and a contract cannot be spoken to on Monday morning. The only actor in reach is the employee. That is why the crumple zone survives even when everyone believes the tool was at fault.

Why this is a leadership failure, not an individual one

Look at what the person who pressed send was working with. They were given the tool, usually with a productivity expectation attached, and the same deadline as before or a shorter one, because the tool was meant to make it faster. And in most cases they were never given an explicit standard for what checking this output required, how long it should take, or what authority they had to hold the work when the check did not clear.

Accountability without authority and without time is not accountability. It is exposure. And it was allocated by whoever set the deadline and deployed the tool, which is to say by a leader, before the person ever opened the file.

Elish is careful that a moral crumple zone is more than a scapegoat, and the difference is structural: it arises where control over an action has become distributed but responsibility has not been redistributed with it. Deploying AI into a workflow distributes the control. Almost nobody redistributes the responsibility to match.

For an Australian leader there is a second reason to take this seriously, and it is not only about fairness. Safe Work Australia's model Code of Practice on managing psychosocial hazards at work, published in July 2022, lists lack of role clarity as a psychosocial hazard, "uncertainty, frequent changes, conflicting roles or ambiguous responsibilities and expectations." It separately lists poor support, covering tasks where workers have "inadequate training, tools and resources for a task." Making somebody answerable for catching a model's errors without telling them what the check is, or funding the time to do it, sits inside both. The Code is general guidance and duties vary by jurisdiction, but the direction is clear.

This sits downstream of deciding who gets to make which calls, which we covered in decision rights are the leadership job AI just made urgent. Decision rights are the authority to act. This is who answers when the action turns out to be wrong.

How do you build the accountability map this week?

The remedy is smaller than it sounds. Not a policy, a framework or a committee. Four lines per workflow, written down and said out loud, and it takes an afternoon.

Assume review tiers already exist, or that you are building them, which we set out in the review tax. This is the layer above: not how much checking a workflow needs, but who answers for it when it fails.

1. List the recurring AI-assisted workflows that produce something a third party sees. Client documents, board papers, regulator correspondence, published analysis. Internal and low stakes stays off. Five to eight items, not thirty.
2. Name one person accountable for release. By name, not a team and not a role in the abstract. If two people would both plausibly say "that is not really mine", you have found a crumple zone in advance, which is the point.
3. State what they are accountable for, and what they are not. This is the line that does the work. Not the model behaving. A listed set of checks: every figure traced to the source system, every citation opened, every legal reference confirmed as current. Write the actual checks. A vague standard is not a standard, it is a trapdoor.
4. State the resourcing, and name the escalation point. How long the check takes, what access it needs, and the standing authority to hold the release when it does not clear. If the check takes twenty minutes and the deadline allows five, you have written the crumple zone with better formatting. Name who the held work goes to, and say plainly that holding an output is never a fault.

Then say it to the team, because an unstated map is worth nothing:

"[MANAGERNAME] is accountable for [WORKFLOW] going out. That means the checks on this list and nothing beyond them, and the time to do them is in the plan. If something wrong gets out after those checks were properly done, that one is mine."

That last sentence is the whole article, and most leaders will not say it until they have written the first three.

A worked example. [TEAM] produces a monthly client-facing performance pack, now AI-drafted and finalised by [ANALYSTNAME], eighteen months into the role. The old arrangement was implicit: [ANALYSTNAME] pressed send, so [ANALYSTNAME] owned it. The new map makes [MANAGERNAME] accountable for release, lists four checks against the source system, funds forty minutes for them, and instructs [ANALYSTNAME] to escalate rather than ship anything failing a check. Nothing about the tool changes. What changes is that a failure now has a named owner senior enough to fix the workflow that caused it.

What does a leader do in the first hour after a failure?

Three things, in order.

Take the public accountability yourself, by name, before anybody asks who did it. Not a shared "we", which reads as evasion, and not a passive "an error occurred". If the first thing the organisation hears is a search for the individual, that is the culture you just built.

Separate the process question from the person question and run them on different clocks. The process question is urgent and starts immediately. The person question, if there is one at all, is slow, and does not belong in the same week.

Then ask the questions that keep the inquiry above the individual:

• What did the standard say the check was, and did it exist in writing before this happened?
• Was that check done, and if not, what stopped it?
• Was the time, access and authority to do it properly there on the day?
• What in the workflow let a wrong output survive all the way to a customer?

And the questions that guarantee a crumple zone, worth knowing precisely because they feel so natural: who approved this, why did you not catch it, how did this get past you. Each is answerable, each terminates at the nearest human, and each leaves the workflow that produced the failure exactly as it was.

If it does become a person question, it has to survive being looked at. On the Fair Work Ombudsman's account of unfair dismissal, the Fair Work Commission weighs whether there was a valid reason, whether the employee was given a reason and a chance to respond, and, on underperformance, whether they had been warned beforehand. A quiet mark against someone for missing a check that was never defined and never funded is not a valid reason. General information, not legal advice.

The judgement boundary

Some of this cannot be pushed down, and the list is short enough to memorise.

The decision to put AI into a given workflow is a leadership decision, and so is the risk that follows. So is the resourcing of the review, because only a leader can move a deadline or fund the time, and an unfunded check is a decision to accept the failure rate that comes with skipping it. So is what gets said publicly when something goes wrong, which most visibly shows which accountability model you actually run. None of it is a technology question, and all of it lands on managers who already have too much, a resourcing problem covered in your AI rollout landed on your managers.

What genuinely sits with the individual is narrower, and should be said just as plainly. Following the standard that was actually set and actually resourced. Saying so, at the time, when it was not. Not representing an unchecked output as a checked one. That is a real accountability, and naming it precisely is what lets you defend the person against everything outside it.

Do this Monday

1. Write down the last AI-assisted thing that went out wrong, and who carried it. If the name is the most junior person in the chain, you have your evidence and did not need a study to get it.
2. Pick your three highest-exposure AI-assisted workflows. The ones where a wrong output reaches a client, a regulator or a board.
3. Write the four lines for one of them. Owner, checks, resourcing, escalation. Twenty minutes.
4. Say the last sentence out loud. Tell the named owner that if something wrong gets out after the listed checks were properly done, you carry it. Said afterwards it sounds like damage control. Said in advance it is the whole control.

Every AI-assisted workflow you run has an accountability model already. You either wrote it, or proximity wrote it for you, and proximity always picks the same person. The only question is whether you find out which one you have before or after something goes wrong.

References

• Madeleine Clare Elish, "Moral Crumple Zones", Engaging Science, Technology, and Society, vol. 5, 2019, pp. 40-60.
• Brailsford, Vetere and Velloso, "Responsibility Attribution in Human Interactions with Everyday AI Systems", CHI 2025, 25 April 2025.
• Tsumura and Yamada, "Effects of knowledge and importance on responsibility in human-AI decision making", Scientific Reports, 23 December 2025.
• Safe Work Australia, "Managing psychosocial hazards at work", model Code of Practice, July 2022.
• Fair Work Ombudsman, "Unfair dismissal", accessed July 2026.