Debug a Bad AI Output: Five Rungs, One Change at a Time

Most people debug a bad AI output by asking again, differently.

That is a random walk with a plausible exit. Three prompts later the answer looks better, nobody can say which change did it, and the same failure returns next week. The problem is not a shortage of technique. It is the absence of an order to work in, so every attempt moves several things at once and the result teaches nothing.

A ladder fixes that. Five rungs, cheapest cause first. One rung, one change at a time.

Before reading this. It helps to have written a structured prompt using something like the RCTF pattern, and to have run one AI workflow end to end. Neither is essential. Those articles build the thing; this one is about what to do when it comes back wrong.

After reading this article, you will be able to. Place a bad output on one of five named rungs, make the single change that rung calls for instead of re-prompting at random, and recognise the point at which the task is wrong for a model rather than wrong in the prompt.

Why does re-prompting feel productive and fix nothing?

Because it reliably produces a different output, and different feels like progress.

Two things are missing. The first is a definition of what a good output would look like. Anthropic's prompt engineering guide will not start without one: it assumes the reader already holds a clear definition of the success criteria, a way to test against them empirically, and a first draft prompt to improve, and tells anyone without those three to establish them first. Without a success criterion, "wrong" is a feeling, and a feeling cannot be diagnosed. If the task repeats, a small golden test set is the cheapest durable version of it.

The second is controlled change. Alter the wording, the attached file and the model in one go and you have run three experiments at once with no way to attribute the result. If the output improves you cannot repeat it. If it does not, you have eliminated nothing.

The two rules that make the ladder work

Climb from the bottom. Lower rungs are cheaper to test, and input and instruction faults cost nothing to fix. No published measurement says how bad outputs distribute across the five, so the order here is cost, not a claimed frequency. The model is where most people start, because swapping it is the most visible move.

Change one thing. Make the change the rung specifies, re-run, and look. If the output improves you have learned something reusable. If it does not, you have eliminated a cause, which is also progress. Two changes at once teach nothing either way. Log each attempt, and the log becomes an inventory of the failures your work actually produces.

What goes wrong on rungs one to four

Rung one is the input, and rewording will not reach it. The symptom is an output that invents something absent from your source, omits something plainly in it, or answers generically when you supplied specifics. That is what a missing input looks like, and nothing in the interface says so. Microsoft's own documentation states that Copilot Chat "is primarily web grounded", and that without a Microsoft 365 Copilot licence it does not provide broad organisational grounding outside supported experiences such as Outlook. A tool that reads the web and not your tenant is unlicensed, not broken.

Capacity is the other half. Gemini without a paid AI plan runs a 32,000 token context window, not the million that dominates the marketing. AI Plus is 128,000, and only AI Pro and AI Ultra reach a million. ChatGPT Projects cap at five files on Free, 25 on Go and Plus, and 40 on Pro and the business tiers. Both sets of figures are as at 26 September 2026. A long document on a free account is truncated quietly, with no error.

Anthropic's guidance for long document work is to ask the model to quote the relevant parts first, which is the test you want here.

Before you do anything with the attached document, list the following:

1. The heading of section [SECTION_NUMBER] and its first sentence.
2. Every date that appears in [DOCUMENT_NAME].
3. The final sentence of the document.

Quote each item word for word. If you cannot find one, write NOT FOUND and
describe what you can see instead.

Do not summarise. Do not paraphrase. Do not answer my earlier question yet.
Do not fill a gap from general knowledge.

If the quotes match your copy, climb. If not, fix the input, checking grounding, tier, context window and file limits. Which tool you are on decides most of that.

Rung two is the instruction. The input landed, and the output is the wrong shape, length or audience. Anthropic states the test plainly: show your prompt to a colleague with minimal context on the task and ask them to follow it, and if they would be confused the model will be too. The same guidance says to tell the model what to do rather than what not to do, offering "Your response should be composed of smoothly flowing prose paragraphs" in place of a ban on markdown. Excessive length is routinely misfiled on rung four. Anthropic names Claude Opus 5 as an exception on verbosity: raising or lowering its effort setting does not reliably change visible response length, and the fix is to prompt explicitly for conciseness. Note too that a guaranteed output shape is exactly that: schema conformance cannot enforce a numeric range and says nothing about whether the content is true.

Rung three is the context state, because a conversation is not a static container. The symptom is early detail gone soft while recent detail stays sharp. Anthropic documents context rot directly: as token count grows, accuracy and recall degrade, which makes curating what is in context as important as how much space is available. It also documents server-side compaction, which condenses earlier turns so a conversation can run past the context window limit. That is a feature, and it produces exactly this symptom. Paid Claude Projects add a third mechanism, scaling large knowledge through retrieval augmented generation, which shifts the failure mode from missing nothing to missing whatever was not retrieved.

Rung four is the configuration, and the setting comes before the badge. On the Artificial Analysis Intelligence Index v4.3.2, measured independently, Claude Fable 5.1 scores 53 at max effort and 47 at low. That six point spread inside one model is wider than the two points separating it from Claude Opus 5 at max, as at 26 September 2026, though only the Fable rows carry the leaderboard's fallback label. A confidently outdated answer is a knowledge cutoff symptom, not a reasoning one, so attach the source. A prompt that used to work can become the defect after a model change. Anthropic warns that wording written to stop a tool undertriggering may make Claude Opus 4.5 and 4.6 overtrigger instead. Routing work deliberately is the durable version of this rung.

When is the task wrong for a model at all?

When every configuration fails the same way.

A ladder that stops at rung four has no top, and people keep climbing it. Anthropic states in its own documentation that not every success criterion or failing evaluation is best solved by prompt engineering, and offers model choice as its example. Rung five is the case its example does not cover, where no configuration is the answer either.

The published numbers support that. On OSWorld 2.0, Anthropic's own launch table reports Claude Fable 5.1 at 41.7 percent strict against 77.9 percent partial, same model, same tasks. That gap between made progress and actually finished is the most useful single fact about agentic work, and it is vendor-published. Rule out refusal before assuming incapacity: Anthropic records that where its production safeguards intervened, Fable 5.1 and Fable 5 scored zero on that benchmark, and a zero can mean declined rather than failed.

Then there is the category no benchmark reaches. A determination, an attestation, a performance rating, a decision someone has to own and defend: none is a hard task a better model eventually clears. They are decisions with a named human on them. If the failure sits in judgement, currency or authority, the ladder has run out. Split the task into the part a model can do and the part a person owns, or take it back. Anthropic notes that breaking work into sequential steps stays useful when you need to inspect intermediate outputs, which is the shape most regulated work wants anyway.

The five rung diagnostic ladder

This is the whole method on one page. Copy it out, keep it beside the work, and run it the next time an output comes back wrong. It does not need the article around it.

Before you climb. Write one sentence saying what a good output would have contained, and note the tool, plan tier and model you are on. Each changes the diagnosis.

Rung 1. The input. Symptom: the output invents, omits, or answers generically despite your specifics. Change: ask the model to quote three specific things from your source, word for word, before it does the task. Worked if: the quotes match your copy. Stop condition: quotes correct means climb; quotes wrong means stop and fix the input, checking grounding, plan tier, context window and file limits.

Rung 2. The instruction. Symptom: the input landed but the shape, length, audience or inclusions are wrong. Change: state the output shape, audience and length in one sentence, and say what to do rather than what to avoid. Worked if: the shape is right, whatever you think of the content. Stop condition: shape right and content still wrong means climb.

Rung 3. The context state. Symptom: early detail lost while recent detail stays sharp, or a long session drifting. Change: open a new conversation with the same input and the same instruction, nothing else different. Worked if: identical ingredients produce a materially better answer. Stop condition: the same bad output in a fresh conversation means climb.

Rung 4. The configuration. Symptom: a reasoning failure or a confidently outdated answer that survived rungs one to three. Change: raise effort one step, then change model, then check the knowledge cutoff and attach the source. Worked if: that class of error disappears. Stop condition: effort raised and model changed with the same failure means climb.

Rung 5. The task. Symptom: every configuration fails identically, or the failure sits in judgement, currency or authority. Change: stop climbing, split the task, and keep the part that needs a decision with a person. Worked if: the model-shaped part now succeeds on its own. Stop condition: none. This is the floor.

What stays a human decision. Every rung above is diagnosis and none of it decides anything. Whether to accept the output, whether the content is true, and whether the task should have gone to a model at all remain with the person whose name is on the work.

Try this

Take the last AI output you rejected and write one sentence saying what a good version would have contained. Then run rung one only: ask the model to quote three specific things from your source, word for word, and check them. One prompt, and you have either found the cause or eliminated it.

Where to go next

• Prompt Engineering Fundamentals: The 2026 Update for Working Professionals
• Build a Golden Test Set Before You Trust an AI Workflow
• Structured Outputs: Get the Same Shape Every Time
• What Your AI Workspace Actually Remembers, and What It Doesn't

Bottom line

Re-prompting produces a different output; the ladder produces an explanation. Five rungs in order, each with one change and one stop condition, turn a bad output from a mood into a diagnosis you can repeat. The rung most people never build is the last, because it is the only one that ends with putting the work down. Climb in order, change one thing, stop when the rung explains the failure.

Do this Monday

• Write the one-sentence success criterion for your three most repeated AI tasks.
• Run rung one on the next bad output by asking the model to quote three things from your source.
• Check which plan tier and grounding your team is actually on, because the free tier answers a different question.
• Keep a four-line log of what you changed and what happened, then review it in a fortnight.
• Agree which tasks in your area sit on rung five, and write down who owns each decision.

References

1. Anthropic, Prompt engineering overview. https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview
2. Anthropic, Prompting best practices. https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices
3. Anthropic, Context windows. https://platform.claude.com/docs/en/build-with-claude/context-windows
4. Anthropic, What are projects. https://support.claude.com/en/articles/9517075-what-are-projects
5. Anthropic, Claude Fable 5.1 and Claude Mythos 5.1 announcement. https://www.anthropic.com/claude-fable-and-mythos-5-1
6. Google, Gemini Apps limits and upgrades for Google AI subscribers. https://support.google.com/gemini/answer/16275805
7. Microsoft Learn, Overview of Microsoft Copilot Chat. https://learn.microsoft.com/en-us/copilot/overview
8. OpenAI, Projects in ChatGPT. https://help.openai.com/en/articles/10169521-projects-in-chatgpt
9. Artificial Analysis, Intelligence Index leaderboard. https://artificialanalysis.ai/leaderboards/models
10. Artificial Analysis, Claude Opus 5 model page. https://artificialanalysis.ai/models/claude-opus-5