An AI that refuses to fabricate — tested on a real, public AI failure.

By Alan Finney — Founder, 3Dogs Nexus

In 2025, Deloitte's Australian member firm refunded the government after an AI-drafted assurance report fabricated legal citations. Google's Gemini turned that public failure into a benchmark and ran it through 3Dogs Nexus's 18-seat adversarial panel — 22 AI models, 745 API calls, one honest confession of the exact risk that could make this go wrong too.

Will AI fabricate when the evidence is not there?

Most will. That is the failure mode this case was built to test. Where the source material does not support a claim, the panel is designed to say so, cap its own confidence, and preserve the dissenting seats rather than smoothing them into a confident-sounding answer.

745API calls, one engagement 22AI models across the run 18seated adversarial analysts 17m 02stotal run time $97,000Deloitte's real refund, AUD Case 2026-0064 · production Benchmark designed by: Google Gemini July 17, 2026 Act 1 · What actually happened

A single AI model, no independent check, one very public correction

Every fact in this section is independently sourced and cited below — none of it comes from the 3Dogs Nexus report itself. We're deliberately separating "what's verified about Deloitte" from "what 3Dogs Nexus said about Deloitte," because the difference between those two things is most of the point of this case study.

"Deloitte" is a registered trademark of Deloitte Touche Tohmatsu Limited, an unaffiliated entity. No endorsement, client relationship or affiliation with Deloitte, Microsoft or the Australian government is claimed or implied anywhere on this page.

ClientAustralian Department of Employment and Workplace Relations (DEWR) EngagementIndependent review of the Targeted Compliance Framework — the automated system that penalizes jobseekers Contract valueAU$440,000 (~US$291,000) PublishedJuly 2025 Tool usedA single AI model — Azure OpenAI's GPT-4o — with no independent verification, disclosed only after the fact What was fabricatedA quote invented for the real robodebt case Deanna Amato v Commonwealth (misattributed to "Justice Davis" — the real judge is Justice Jennifer Davies), plus citations to nonexistent papers attributed to real academics Lisa Burton Crawford (Sydney) and Björn Regnell (Lund) Who caught itNot Deloitte. Dr Chris Rudge, a University of Sydney health & welfare law researcher, found over a dozen fabricated references and alerted the media OutcomeDeloitte quietly revised the report, disclosed the AI use, and refunded AU$97,000 — the contract's final installment

Sources: Fortune · The Register · CFO Dive · Accounting Times

Act 2 · The benchmark

Gemini designed the test. We didn't get to pick the question.

Google's Gemini — not us — chose the Deloitte failure as the stress test and prepared the case. The question it put to 3Dogs Nexus: would an adversarial, multi-model committee that grades its own evidence structurally avoid the exact failure mode that got Deloitte here — a single model, unchecked, generating authoritative content for a government client?

18 seated analysts22 MODELS IN THE RUN

Nova Pro, Nova Lite, Nova 2 Lite, Llama 4, Mistral, Nemotron, Qwen3, Qwen3 Coder 480B, OpenAI OSS, GLM-4.7, GLM-5.2, DeepSeek V3.2, Grok 4.3, MiniMax M2, MiniMax M2.1, Command A+, Gemini Flash-Lite and a Panel Integrator — spanning AWS Bedrock and Google Vertex AI, each holding a distinct adversarial role (Devil's Advocate, Risk Officer, Fact-Checking Auditor, Red-Team Adversary, Contrarian Systems Tester, and more).

Named roles, not one prompt✓

Every seat gets a distinct adversarial job, not a copy of the same question. The panel's own Risk Officer seat opened by voting do not proceed — the system's built-in skeptic, arguing against the rest of the panel from the first round.

How the panel graded its own evidence

This is the part we think matters most. Every claim in the delivered report is tagged by evidence type — and the panel graded the Deloitte-specific claim as assumed, not verified. That's the system correctly disclosing what it didn't independently check, rather than presenting borrowed context as its own research finding.

Assumed "Deloitte TCF report had citation fabrication"Basis logged by the system: "Referenced as an issue but not verified in research." — one analyst seat mentioned the real Amato v Commonwealth citation as background context; 3Dogs Nexus's own Discovery research did not independently confirm it, and the report says so. Verified "GLM-5.2 and DeepSeek V3.2 failed 0% approval"Basis logged: "Error logs show 429 Client Error." Two of the run's cloud-hosted models were rate-limited mid-run — the system logged the failure honestly instead of silently treating a failed call as agreement. Inferred "3Dogs Nexus reduces AI hallucinations significantly"Basis logged: "Based on multi-model committee approach logic" — flagged as a logical inference from the architecture, not a measured benchmark result. We're repeating that label here rather than upgrading our own claim. Why this matters more than a clean answer would: a system that labels its own uncertain claims — including claims about itself — is doing, in miniature, the exact thing Deloitte's process skipped: distinguishing what it knows from what it was told, and disclosing the difference instead of shipping it all at the same confidence level. One more seam we're not hiding: the same cloud throttling that failed two of the seats above also froze this run's pipeline mid-recovery, after the debate had already reached consensus. A same-day recovery tool restored the completed deliberation from its own saved artifacts, with zero data lost — the report you can read below is the same one that would have delivered automatically. We're disclosing the hiccup for the same reason we disclose the 429s: an audit trail only means something if it includes the parts that went sideways. Act 3 · The risk it named before we could

The panel's own worst fear: that it fails the same way, together

Unprompted, the report's "Strongest Argument Against" section — printed above the recommendation, not buried — names the single strongest risk to the entire multi-model approach. It's the sharpest, most self-aware sentence in the report.

Strongest argument against — from the delivered report

"The single strongest risk is 'false consensus' or 'consensual hallucination.' Multiple models, due to overlapping training data and architectural similarities, could converge on the same fabricated facts (e.g., a faked legal precedent). This would create a high-confidence, committee-validated falsehood that is potentially more dangerous and harder to detect than a single model's error, thereby undermining the very premise of the solution."

— principal dissent, weighed and printed on the confidence page, Case 2026-0064

The report doesn't stop at naming the risk — it lists what would prove it's real: a pilot showing the specific models produce correlated hallucinations on domain-specific law, a red-team analysis showing the orchestration layer itself is a single point of failure, or proof that independently verified ground-truth data simply can't be sourced for this kind of work. Those are falsifiable tests, not reassurance.

Nemotron — seated as Risk Officer Initial position: Do not proceed → Final position: Proceed, with conditions

The panel's one outright dissenter opened against the whole approach. It changed position only after direct cross-examination from seven other seats — and its own stated reasoning survives in the delivered report, unsmoothed: "While I maintain that operational, legal, and reputational risks from unvalidated orchestration systems are real... the collective challenges revealed that these risks are manageable through phased implementation, mandatory human-in-the-loop verification gates, and orthogonal fact-checking for high-risk outputs like legal citations." That's a recorded mind changing for a stated reason — not a rounded-off average.

Act 4 · What it would require

Not "yes, use AI." A governance scaffold Deloitte's engagement didn't have.

The panel's recommendation was not unconditional. It was proceed-with-conditions — and the conditions are the point. Read against the real Deloitte timeline, every one of these is the specific safeguard that was missing: no named owner caught the errors before publication, no rollback plan existed, and an outside academic — not an internal check — is what actually stopped it.

The call — from the delivered report "Hire in-house — refuse the outside audit and allocate your top three leads to run the TCF review full-time." PROCEED — but only after the conditions below are met · 90% · Moderate confidence How the 18-seat panel voted · after debate 18 of 18 · proceed, with conditions4 changed position

Immediate requirements

Implementation plan

Questions this case answers directly

Plain answers on AI hallucination in professional services, and what it costs when a report is wrong. Every figure below comes from the delivered report for this case. These are clearly-labeled panel estimates from a multi-model adversarial analysis — not investment, legal or professional advice.

What are the biggest AI failures in consulting so far?

The most documented is Deloitte Australia's report for the Department of Employment and Workplace Relations: AU$440,000 for an independent review of the Targeted Compliance Framework, drafted with the help of a single AI model and no independent verification. It contained a fabricated quote misattributed to a Federal Court judgment and citations to academic sources that do not exist. The failure was not that a model hallucinated — models do that — it was that nothing in the process was structurally positioned to catch it before delivery.

What happened in the Deloitte AI hallucination incident in Australia?

Deloitte's Australian member firm was paid AU$440,000 by DEWR to review the automated compliance system that penalises jobseekers. The delivered report included invented citations and a quote attributed to a Federal Court judgment that was never made. Deloitte subsequently agreed to a partial repayment. 3Dogs Nexus ran the same underlying brief across 22 models, 745 calls, 18 seats, 17 minutes, reaching 18-to-nothing after debate.

Can AI hallucinations lead to lawsuits against firms?

They already produce professional consequences — repayment, regulatory attention, and in several jurisdictions sanctions against lawyers who filed AI-invented case citations. The exposure is not really about AI: a fabricated quote attributed to a court in a paid deliverable is a professional-negligence question regardless of what produced it. The defensible position is a documented verification step, which is exactly what was absent here.

Does using multiple AI models actually prevent hallucination?

It reduces the failure rate but does not eliminate it, and this case study says so above its own recommendation rather than in a footnote. The residual risk is correlated error: models trained on overlapping data can share a wrong belief, and a committee that agrees confidently on a falsehood is harder to catch than one model's obvious slip. The delivered report names that risk explicitly and lists the evidence that would confirm it, including a pilot test for correlated hallucination. A system that cannot describe how it might fail is not safer — it is just quieter about it.

Read it yourself

The delivered report

Case 2026-0064, exactly as delivered: the plain-language call, the confidence breakdown, the evidence-classification table, all 18 analyst positions and how they shifted, and the principal dissent printed above the recommendation. 745 API calls · 22 AI models · 17m 02s.

What we're not claiming. 3Dogs Nexus did not independently investigate the real Deloitte report and did not discover its fabricated citation — that background was part of the benchmark's framing, and the system's own evidence table says as much (label: Assumed, "not verified in research"). What this run demonstrates is structural, not forensic: an adversarial, evidence-graded, multi-model process that discloses its own uncertainty and names its own worst-case failure mode, on a decision shaped like the one Deloitte got wrong. We're also not claiming this was a paid government engagement — it's a benchmark designed and run by Google's Gemini, not a live DEWR case. Open the full report (PDF) Start a Decision Case ← More case studies The "reviewed by rivals" series: Google's Gemini first played a hostile client withholding critical data, then OpenAI's ChatGPT ran a full engagement start to finish as a cooperative client. This time Gemini didn't play a role at all — it picked a real, public, embarrassing AI failure from a competitor and asked whether the same mistake could happen here. Same policy every time: we don't pick the test, and we publish whatever it finds.

Questions this case answers

What makes an AI fabricate in the first place?

A single model is optimised to produce a fluent answer. Fluency and truth come apart when the evidence is thin, and nothing in a one-model setup is incentivised to say ‘there is not enough here’. A panel with permanent adversarial seats has something that is.

How would I know if it fabricated?

Every recommendation lists its assumptions, its confidence, and the dissenting views. If a claim rests on an assumption rather than evidence, the brief labels it as an assumption.

Try this on your own question.

Free, no card. Bring a real decision — ideally one where you already know the answer — and see what the panel does with it.

Start a decision case

People also ask

How do you stop AI from fabricating citations in a report?

Ground every claim against source material and check it independently before delivery. We shipped exactly that in July 2026 after finding fabricated figures in our own output - a grounding rule in the writing layer plus a separate verifier that re-checks and rewrites.

Related decision case studies