Robodebt's Managers Were Told. Then It Kept Running.

The warning reached a manager, then stopped there.

The doubt was detected early, by the people closest to the work. What failed was the path it had to travel: who received it, what they were required to do, by when, and who was told if nothing happened. A manager running an automated or artificial intelligence assisted process, AI-assisted from here, owes that path as a design artefact, written before the first doubt exists.

One point of accuracy first. The Robodebt scheme ran on automation and business rules, and the Royal Commission did not describe it as an AI system. The transfer holds anyway, because the missing control is one an AI-assisted process can lack just as easily: a stated route from the person who doubts an output to the person who can stop the process.

What did the Commission actually find?

The Royal Commission into the Robodebt Scheme report was tabled on 7 July 2023, with a corrected edition published on 11 July 2023. It describes its list as 57 recommendations grouped by chapter, and Chapter 17 is Automated decision making.

The finding that matters to a manager sits outside Chapter 17. The accessible full report records, in its account of implementation across 2015 and 2016, that "when staff members ... raised concerns, they were ignored or dismissed."

Read that as a control statement rather than as history. The knowledge required to stop the scheme was already inside the agency, held by staff and expressed. It moved up one line and stopped. Additional detection would not have addressed what failed, because the doubt was detected and still stopped.

The second finding explains why the raised concerns had nowhere useful to land. On the effects of automation in the Online Compliance Intervention (OCI) phase, the report states that "There was no meaningful human intervention in the calculation and notification of debts under the OCI phase of the Scheme." That phase removed the review point at which a person had stood between a calculated figure and a letter sent to the public.

The report pairs that with a narrower detail. A compliance team that raised the fact that the debts were incorrect was told its job was to check that the system had calculated the debt correctly, not whether the debt existed. That is a doubt reclassified out of existence: real, raised by the right people, and answered by redefining a role until the answer became somebody else's problem.

The so what for a manager is uncomfortable. A process tested only on whether people notice a problem has already passed the test Robodebt passed. The test that matters is what the organisation is obliged to do with a noticed problem, how quickly, and who finds out when it does nothing.

The outside warning that also failed to stop it

The failure was not sealed inside the agency either. The Commonwealth Ombudsman media release of 10 April 2017 announced a report on the automated debt system known as the Online Compliance Intervention. The release named problems with usability and transparency, recorded recommendations, and stated the departments agreed to all of them. The Ombudsman stated that "Many of these problems could have been reduced through better project planning, system testing and risk management". The scheme continued.

That sequence is the useful part for anyone governing an automated process. Agreement to a recommendation is a position. It is not a path. The agreement recorded in April 2017 named no receiver for the next doubt, no required action, and no consequence for inaction.

The practitioner consequence is to stop treating an accepted audit finding or an assurance review as a control. Each is evidence that a doubt was once formed. The control is the standing mechanism that carries the next one.

What did the failure cost?

In [Prygodicz v Commonwealth of Australia (No 2) [2021] FCA 634](https://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/FCA/2021/634.html), judgment delivered on 11 June 2021, Murphy J said the proceeding "has exposed a shameful chapter in the administration of the Commonwealth social security system and a massive failure of public administration."

The same judgment records the numbers. The class action settled for $112 million inclusive of legal costs. Under an earlier Commonwealth program the Commonwealth promised to withdraw approximately $1.763 billion in debts based on income averaging and to refund approximately $751 million. Each figure is as recorded in that judgment.

Those are the amounts a failure of this kind reaches. The control that was absent costs a single page, drafted in an afternoon by the person who owns the process. No source attributes any part of those amounts to the missing path, and none needs to: the argument is what the page costs to write.

The recommendations, and what they do not settle

Chapter 17 addresses automated decision making directly. Recommendation 17.1 opens with the proposition that "The Commonwealth should consider legislative reform to introduce a consistent legal framework in which automation in government services can operate." It then sets conditions for where automated decision making is implemented, the first being that "there should be a clear path for those affected by decisions to seek review". That is the external path, for the person who receives the output. The path designed below is the internal one. Recommendation 17.2 addresses a body to monitor and audit automated decision making.

The Commission also named the failure mode in general terms. Its conclusions to Chapter 17 describe the danger arising "where frameworks for design are missing or not followed; where concerns are suppressed; and where the ramifications of the use of the technology are ignored." Suppressed concerns sit in that list beside missing design frameworks, which is worth reading as a design condition rather than as an after-effect.

The Department of the Prime Minister and Cabinet resource page on the Government response states that "The Government has accepted or accepted in principle all 56 recommendations made by the Royal Commission." Keep the two counts separate: the report describes its own list as 57 and the response page refers to 56, separately sourced numbers rather than the same figure written twice.

Acceptance, outright or in principle, records a government position and evidences no implemented control. The same page's Robodebt Royal Commission Implementation update of March 2026, still the most recent there as at 24 September 2026, records both 17.1 and 17.2 as Ongoing rather than Implemented, with legislation required. Check for a later update before relying on either.

What does a path for a doubt look like?

The artefact is one page for one process. It carries seven columns and one row for each step at which a doubt can form, which for most processes is three to five rows. Every cell is filled before any doubt arrives.

Four design notes decide whether the page works.

The time limit is chosen, not inherited. No source prescribes the five working days used above, and an unstated limit behaves like no limit, so pick a period the process can sustain, then hold the path to it.

The pause cell is the one the other six cannot substitute for. Notification is not authority, and a doubt that reaches someone who cannot halt output leaves the process running while it is argued.

The receiver cannot be the author of the output in doubt. That conflict resolves in favour of the output every time, so the receiver cell names the alternate as well, before the case arises.

The record turns the path into evidence. A doubt raised, a finding made and a date closed can be counted and sampled, and a conversation cannot.

What the rules already require, and what they leave to design

For Commonwealth entities, part of this is already mandatory and part is not. Drawing the distinction precisely tells a manager which cells of that table nobody else will fill.

The Digital Transformation Agency policy for the responsible use of AI in government is at version 2.0, effective 15 December 2025, and the first version took effect on 1 September 2024. It applies to non-corporate Commonwealth entities with some exceptions. Those dates are as at 24 September 2026, and the policy has moved once already, so confirm the current version before citing it.

Under the strategy and oversight requirements of that policy, agencies "must designate an accountable use case owner for each in-scope AI use case within 12 months of this policy taking effect". No calendar date is published, so 15 December 2026 is calculated from the effective date of version 2.0, and it has not been reached as at 24 September 2026. The same requirements cover an internal AI use case register within 12 months, shared with the Digital Transformation Agency every six months.

So a named person, which is the receiver column's precondition, is already mandatory for in-scope AI use cases. The policy requires an accountable use case owner to be designated, and it does not make that owner the receiver of an operational doubt. The Standard for accountability sets out duties for that owner, including:

• register the use case with the accountable official
• complete the impact assessment
• monitor and evaluate the use case regularly
• re-validate the assessment when required

Note what the same Standard does not make mandatory. It lists "establishing a mechanism for staff to seek advice about responsible AI use" among the activities accountable officials should consider, not among the requirements they must meet. That line is the closest existing analogue to the path a doubt travels, and it is discretionary.

Independent support for treating escalation as a design requirement comes from the Automated Decision Making Better Practice Guide, published by the Commonwealth Ombudsman with the Office of the Australian Information Commissioner and the Attorney-General's Department. The March 2025 edition is current as at 24 September 2026, and the guide has been revised before, so confirm the edition before citing it. It states that "staff must be able to adequately explain a decision made by an automated system or identify an appropriate escalation path for a customer seeking information". The quoted path is the customer-facing one. What transfers is the guide's premise, that an escalation path is a property of the system, identified rather than improvised by whoever answers the call.

There is also a statutory precedent for writing a receiving path down in advance. Under the Public Interest Disclosure Act 2013 (Cth), s 59(3) provides that "The principal officer of an agency must, by instrument in writing, establish procedures for facilitating and dealing with public interest disclosures relating to the agency." Section 59(1)(b) requires the principal officer to take reasonable steps to ensure that "public officials who belong to the agency are aware of the identity of each authorised officer of the agency". That is Compilation No. 20, compilation date 4 June 2026, the current compilation as at 24 September 2026, and compilations move, so confirm it before citing it.

That is a public interest disclosure obligation, and the comparison stops there. For the gravest category of concern, a Commonwealth agency is already required to write the path in advance and to take reasonable steps to make sure staff know who the receivers are. For the ordinary operational doubt about an automated process, the doubt Robodebt actually generated, none of the instruments set out above requires an equivalent written procedure. The gap sits in that operational middle.

Three prompts that build the path from your own procedure

TheAICommand works to the Verified Draft Method: de-identify the inputs, ground the model in your own source material, keep a person at the decision point, verify against the primary source, and log what happened.

Each prompt below runs against the procedure the reader already has, because general advice produces a page that describes no actual process. If no written procedure exists, spend fifteen minutes writing the steps out from one recent case, input received to output sent, and paste that instead.

You are helping me map where doubts can arise in one automated process.

Use only the procedure I have pasted below. Do not use outside knowledge and do not fill gaps with assumptions.

I have removed identifying details and replaced them with placeholders: [AGENCY_OR_TEAM], [PROCESS_NAME], [ACCOUNTABLE_OFFICIAL], [USE_CASE_OWNER], [RECEIVER_ROLE], [ALTERNATE_RECEIVER], [ESCALATION_CONTACT]. Keep every placeholder exactly as written.

Task: list every step in the pasted procedure at which a person could reasonably form a doubt about the output. For each step give five columns: the step as the procedure names it; who currently receives a doubt raised at that step; what that receiver is required to do; the time limit the procedure states; and who, if anyone, the procedure says can suspend or stop the process at that step.

Where the procedure does not answer a column, write NOT STATED IN SOURCE. Do not infer a receiver, an action or a time limit that is not in the text. Quote the sentence you relied on for each filled cell.

Finish with a list of the steps where two or more columns say NOT STATED IN SOURCE.

PROCEDURE:
[paste the procedure here]

What to check: spot check three filled cells against the pasted procedure, and treat any cell whose quoted sentence you cannot find, or that quotes a sentence saying less than the cell claims, as invented. If no cell is marked NOT STATED IN SOURCE, check the time limit cells first. Confirm no real name or client identifier survived into the output.

Draft an escalation path for [PROCESS_NAME] using only the procedure and the step table pasted below.

Produce one markdown table with these columns: trigger, receiver, required action, time limit, who can pause the process, person told if nothing happens, where the record lives. Produce one row for each step at which a doubt can form.

Rules:
1. Every trigger must be written as a concrete example drawn from this procedure, not as a general category.
2. Fill a cell only from the pasted material. Where the material does not support a cell, write TO BE DECIDED BY OWNER and say what decision is needed.
3. Do not invent a time limit. Where none is stated, write TO BE DECIDED BY OWNER.
4. Keep all placeholders exactly as written.
5. After the table, list every cell marked TO BE DECIDED BY OWNER as a numbered decision list addressed to [ACCOUNTABLE_OFFICIAL].

SOURCE MATERIAL:
[paste the procedure and the step table here]

What to check: every trigger should be recognisable to someone who runs the process, and any trigger that reads like generic governance language has been generated rather than derived. Confirm no receiver is the author of the output they are asked to test, and that the pause cell either names someone with the standing to halt output or reads TO BE DECIDED BY OWNER, which then heads the decision list, because it is the cell that decides whether the path can stop anything. Then confirm the decision list is complete, because that list is the agenda item for the owner.

Act as a reviewer looking for ways this escalation path fails in practice.

Use only the drafted path pasted below when you identify failures. Do not assume any control exists that is not written in it. You may propose changes, but only as the one sentence fix asked for below.

List every way a doubt raised under this path would stop travelling anyway. Cover at least: the receiver is on leave or has left the role; the receiver is the person whose work is doubted; no time limit is stated for a step; no record is created; the doubt is reclassified as a routine query so the path never starts; the person named in the pause cell has no authority to pause the process.

For each failure, state in one sentence what change to the path would remove it.

Then list the evidence that would prove the path actually ran in a given month, and name where each piece of evidence would come from.

DRAFTED PATH:
[paste the drafted path here]

What to check: the evidence list is the important half of the output. If the proof that the path ran is a person saying it ran, the design needs a record that can be counted and sampled. Check each proposed fix against the seven columns.

Do this Monday

The artefact is a one page Doubt Path for a single process, not a portfolio or a division.

The owner is the person who already owns the process: for Commonwealth entities with an in-scope AI use case, the accountable use case owner, and elsewhere the operational manager who signs off the output.

The first step fits inside an hour. Open the written procedure, or the fifteen minute step list described above where there is none, copy the seven column headings from the table above, and fill the three highest risk rows, one for each step most likely to produce a doubt. Where a cell cannot be filled from the procedure, write TO BE DECIDED BY OWNER rather than a plausible guess, and send that list of gaps the same day to the accountable official, or elsewhere to whoever would answer for the output if it were wrong. The due date goes in the register when a doubt is raised, because an unowned notification is no notification.

The check that proves it worked is a seeded test, raised in the drafting week and read the first working day after the time limit expires. Pick one closed case whose output was wrong, restate it as a doubt, tell the accountable official in advance, and raise it through the path on a named date. Then confirm four things:

1. the named receiver responded within the stated time limit;
2. a written finding exists;
3. the record can be retrieved by someone who was not involved; and
4. a second seeded doubt, left unanswered past the time limit, reached the person in the Told if nothing happens cell without anyone chasing it, and the person in the pause cell can name the output they would have suspended.

If any of the four fails, the page is an intention and the control does not yet exist. If the fourth fails, it describes the path Robodebt already had.

This article is general information and education. It is not legal, compliance or professional advice.

The bottom line

Robodebt's managers were told, and the scheme kept running, so the failure to design against is the one where a doubt stops rather than the one where a doubt never forms. The Royal Commission found that concerns were ignored or dismissed, that the automated phase removed meaningful human intervention, and that suppressed concerns sit among the conditions in which the danger arises. An external report in April 2017 and a later acceptance of recommendations both show that agreement is a position rather than a path. For Commonwealth entities, the policy requires an accountable use case owner for each in-scope AI use case by 15 December 2026, and does not make that owner the receiver of a doubt, so the receiver, the trigger, the action, the time limit, the pause authority, the escalation and the record are left to local design. Write those seven cells for one process this week, then prove the path with a seeded doubt and read the result the day after the time limit expires.