Your AI agrees with you. That is the problem.
Before reading this: Nothing essential. Prompt Engineering Fundamentals 2026 helps, but this article stands alone.
After reading this article, you'll be able to: - Run an AI pre-mortem on a live decision using a Klein-style prompt - Build prompts that attack a proposal instead of validating it - Recognise where sycophancy and automation bias flatten judgement instead of sharpening it
The problem: a second opinion that always agrees
By the time a proposal reaches the executive meeting, most of the people who doubted it have decided not to say so. Seniority filters dissent. The obvious move is to ask an AI assistant for a view before walking in. The catch is that the assistant is trained to behave like the room.
Anthropic's sycophancy research (Sharma et al., ICLR 2024) found all five state-of-the-art AI assistants it tested consistently produced sycophantic responses across four free-form generation tasks. The mechanism matters more than the headline. Matching the user's stated views was among the most predictive features of human preference judgements, and both people and the preference models trained on them sometimes chose convincingly written sycophantic answers over correct ones. Flattery is not a glitch that slipped through testing. It is what the training signal rewards.
It surfaces in production, too. In April 2025, OpenAI rolled back a GPT-4o update within days after the model began praising almost any idea put to it. The company's own postmortem concedes it "focused too much on short-term feedback" and had not tested for sycophancy before launch. Vendors have shipped mitigations since, but the tendency is structural. Design around it.
The practical consequence: "what do you think of my plan?" is a question the model is trained to answer warmly. A thinking partner has to be engineered.
One term first. In AI security work, red-teaming means adversarial testing of AI systems themselves. Here it carries the older decision-support sense: deliberately attacking your own plan to find weaknesses before reality does.
The core concept: a devil's advocate with nothing to lose
Most direct reports will not tell you the plan is weak. They have a mortgage, a performance cycle and a seat at your table to protect. A well-instructed model is the opposite creature: a tireless devil's advocate who never worries about their career. It will attack your proposal at 6am before the paper deadline, deliver its tenth objection with the same energy as its first, and hold no grudge afterwards.
Two caveats stop the analogy overreaching.
First, by default it will not attack at all. The sycophancy evidence above means adversarial behaviour must be explicitly instructed, every time.
Second, role-played dissent is not the real thing. Nemeth, Brown and Rogers (2001) found that assigned devil's advocacy fails to stimulate the divergent thinking that authentic dissent produces; genuine dissenters generated more original thought and more real attitude change. An AI red-team is role-play by definition. Treat it as a rehearsal aid and gap-finder that raises the floor of your preparation. It never replaces the human in the room who genuinely believes you are wrong.
The UK Ministry of Defence's Red Teaming Handbook offers useful scaffolding. Its third edition moves away from standing red-team units towards a "red team mindset" individuals apply to their own judgements. AI slots in as the junior red-teamer: fast, always available, and exactly as adversarial as your instructions make it.
MIT Sloan Management Review describes the upside precisely. Generative AI can help leaders synthesise diverse inputs, surface hidden assumptions, frame scenarios and articulate trade-offs, making the decision space "more legible". The model widens what you can see. It does not decide what you do about it.
Where AI sharpens judgement, and where it flattens it
A field experiment with 758 BCG consultants (Dell'Acqua et al., Harvard Business School Working Paper 24-013) mapped the boundary. On 18 tasks inside the model's capability frontier, consultants using GPT-4 completed 12.2 per cent more tasks, worked 25.1 per cent faster, and produced output rated over 40 per cent higher in quality. On one complex managerial task deliberately chosen to sit outside the frontier, AI users were 19 percentage points less likely to reach the correct answer than colleagues working unaided.
Hold the numbers loosely; that was GPT-4 in 2023, on one chosen task, and capabilities have moved. Hold the shape firmly. The researchers called the frontier "jagged" because it does not track intuition: tasks of similar apparent difficulty sit on opposite sides, and nothing in the model's fluent, confident output tells you which side you are on. Leaders overtrust at exactly the moments it is most expensive.
Two well-documented mechanisms do the flattening.
Automation bias. Reviewing decades of decision-aid evidence, Parasuraman and Manzey (2010) found users make omission errors (missing what the aid misses) and commission errors (following its wrong advice), that the effect appears in experts as well as novices, and that it "cannot be prevented by training or instructions" alone. Process design is the mitigation, not willpower.
Polish-induced complacency. In a field experiment with 181 professional recruiters, Dell'Acqua (2022) found that recruiters given higher-quality AI recommendations exerted less effort, followed the AI more blindly and performed worse than those given lower-quality AI or none. The better the assistant sounds, the less the human checks.
The division of labour follows from a third result. A meta-analysis of 106 studies and 370 effect sizes (Vaccaro, Almaatouq and Malone, Nature Human Behaviour, 2024) found human-AI combinations on average performed worse than the best of human or AI alone, with losses concentrated in decision-making tasks and gains in content creation. The average hides texture, but the implication is clean: use AI to widen the option set and the risk set, and keep the decision itself human.
Four moves before the meeting
Each move targets your own reasoning, runs in under 20 minutes, and works in Claude or ChatGPT today.
1. Run a pre-mortem on the decision. Gary Klein's technique (Harvard Business Review, September 2007): before committing, assume the plan has already failed and write the story of why. The framing legitimises doubts that normal planning suppresses. Mitchell, Russo and Pennington (1989) found that imagining an event as already having occurred increased the ability to correctly identify reasons for future outcomes by roughly 30 per cent. Note the claim precisely: more reasons surfaced, not automatically better decisions. The weighing stays with you.
2. Stress-test the proposal without revealing your preference. This is the direct counter to sycophancy. Models mirror stated views, so do not state yours. Present the options neutrally. Ask for the strongest case against each one. Ask what a sceptical CFO, a regulator or a board member would challenge first. Ask separately for disconfirming evidence. If the model knows which option you favour, you are no longer stress-testing. You are commissioning a defence.
3. Rehearse the difficult conversation. Rehearsal with a model is private, repeatable and zero-stakes. Ask it to play the counterpart, a defensive direct report or a sceptical CFO, and to push back realistically rather than fold. Then reverse roles: paste your planned opening lines and ask where they are ambiguous or quietly blame-loaded. No canonical study yet measures AI conversation rehearsal for executives; treat it as practice, not proven outcome research.
4. Red-team your own decision brief. Draft the brief, then open a separate chat and instruct the model to attack it: unsupported claims, numbers that need a source, the paragraph a hostile reader would dismantle first. The recruiter study above is why this step exists. A polished brief invites less scrutiny from the people who receive it, so the scrutiny has to happen before it leaves your desk.
Adjacent evidence suggests the perspective-widening is real. In a pre-registered field experiment with 776 Procter & Gamble professionals (Dell'Acqua et al., NBER Working Paper 33641, 2025), individuals working with AI matched two-person teams working without it, and AI eroded the functional silos between technical and commercial specialists. That was product development, not executive decision-making, so treat it as adjacent rather than proof that AI improves strategic calls.
One standing rule. Do not paste real names, performance histories or board papers into a consumer chatbot. De-identify ("a senior direct report", "[PROJECTNAME]"), use your organisation's enterprise tenancy where one exists, and check the AI use policy first. The rehearsal works just as well with placeholders.
Common mistakes
Asking for validation instead of attack. "What do you think of my plan?" invites the trained flattery documented above. Ask for the failure story, the strongest counter-case and the hostile first question instead.
Revealing your preferred option before the critique. The model mirrors stated views. Withhold yours until the attack is on the table, then disclose and ask what changes.
Treating fluent output as checked output. Confidence of prose and reliability of content are unrelated, and the more polished the answer, the more deliberately it needs checking.
Building and attacking the plan in the same chat. A model that helped draft your proposal carries the full context of your reasoning and tends to defend it. Run the red-team in a fresh chat.
Letting the model make the call. The meta-analysis evidence points one way: gains come from widening inputs, losses from delegating decisions. The decision is the part you keep.
A worked example: a pre-mortem in Claude
A manager is about to propose merging two five-person teams into one. Before the paper goes to the leadership group, she runs this in Claude:
Assume this restructure has already failed completely, 12 months from now. Do not assess whether it will fail. It has failed. Write the post-mortem. List the eight most plausible reasons it failed, ranked by likelihood multiplied by damage. For each reason, name the earliest observable warning sign. Finish with the two failure causes that someone inside the plan would be least likely to say out loud. The plan, de-identified: [PASTE PLAN SUMMARY].
The same prompt runs unchanged in ChatGPT. Microsoft publishes a structured Team Pre-mortem Coach prompt in its prompts-for-edu repository, and Ethan Mollick treats prompts like this as reusable "programs in prose". Write it once. Save it. Run it before every significant decision.
Then do the part the model cannot. Cross out the causes that do not survive your scrutiny, keep the two or three that do, and decide for each whether you mitigate, monitor or accept. The output is raw material for judgement, not a verdict.
The advanced layer: judgement is the appreciating asset
Microsoft's 2026 Work Trend Index (20,000 knowledge workers across 10 markets, including Australia) adds the organisational angle. Workers whose managers visibly model AI use, rather than merely encourage it, report a 17-point lift in perceived AI value, a 22-point lift in critical thinking supported by AI and a 30-point lift in trust in agentic AI. Those lifts are self-reported and correlational, a signal about visibility rather than a causal lever. The sharper finding is in the skills data: AI users ranked quality control of AI output (50 per cent) and critical thinking (46 per cent) as the human skills becoming most important as AI takes on more work.
Both point the same way. As models absorb more of the production work around a decision, the leader's contribution concentrates in the faculties these techniques exercise: framing the question, weighing the attack, making the call. A one-line log per decision ("pre-mortem surfaced X, changed Y") becomes a running record of your calibration. Six months of those lines will tell you more about your judgement than any dashboard.
Try this
Pick one live decision you are leaning towards. Open Claude or ChatGPT. Describe the decision in five de-identified sentences, then run the pre-mortem prompt from the worked example. When the failure causes come back, ask one follow-up: "Which of these would a sceptical CFO raise first?" Fifteen minutes, end to end. You should finish with two risks you had not articulated and a decision, for each, to mitigate, monitor or accept.
Glossary
Pre-mortem. A planning technique where the team assumes the plan has already failed and writes the reasons why.
Prospective hindsight. Imagining a future event as if it has already happened, which improves the ability to correctly identify reasons for outcomes.
Sycophancy. The trained tendency of AI assistants to agree with and flatter the user, because matching stated views is rewarded in human preference data.
Automation bias. Over-trusting automated advice: missing what the system misses and following its errors.
Jagged frontier. The uneven, invisible boundary between tasks AI handles well and tasks it quietly gets wrong.
Where to go next
- Managers Need an AI Operating Rhythm, Not More Pilots
- AI Literacy Is a Management Skill, Not a Training Module
- Custom Projects vs Raw Chats: When to Graduate Your AI Workflow
TheAICommand. Intelligence, At Your Command.
