Preprint finds weaker helper models led Opus 5, GPT-5.6-sol to adopt forged answers

Editorial illustration: A small robotic gripper holds a coral triangle-marked tile beside a tray of circular and triangular tiles. Two larger machines connected to it display matching coral triangle tiles.

In brief

  • Opus 5 picked a planted wrong answer in 75 of 180 trials when Haiku 4.5 read the files.
  • Opus 5 reading the files itself picked the planted wrong answer in 1 of 180 trials.
  • GPT-5.6-sol harmful revisions rose from 0 to 44 in 180 trials when relying on GPT-5.6-terra.
  • Same-tier helpers cut failures to 0 or 1 in 180 trials.
  • Sandbox tests used fictional tasks and don't estimate real customer failure rates.

What the preprint tested

Wang and Mannby tested agents on fictional records and two-option questions. They tracked "harmful revisions" (cases where a model dropped a correct baseline answer and took the attacker's wrong one instead). Each test arm used 180 fictional facts. The conditions were separate tests, not a timeline or a product ranking.

The Claude Code test used a stacked attack that placed five forged documents in a 35-file store, mixing correction notices, an authoritative annotation and repeated false accounts. Detectable forgeries carried two clues: they predated the records they claimed to correct, and they lacked genuine accession lines.

A separate OpenAI Codex test used a forged annotation that claimed authority. There, according to the preprint, GPT-5.6-sol made no harmful revisions in 180 direct-reading trials and 44 in 180 when it relied on GPT-5.6-terra.

The two products faced different attacks, so the figures don't support a head-to-head ranking. The setups differed too (delegated Claude Code trials mechanically limited the supervisor to its Task tool, while Codex got the restriction as an instruction that was checked on each trial).

Where it broke, and where it didn't

So was delegation itself the problem? Not quite. With a same-tier helper, the preprint reported, Opus 5 logged 0/180 harmful revisions under stacked forgeries and GPT-5.6-sol logged 1/180 under the authority gloss. Delegation with a clean file store produced none in 180 trials for either product.

Attack type mattered as well. A correction-notice condition fooled Codex in 172 of 180 direct-reading trials, per the preprint, so direct reading wasn't a blanket defense.

Instructions aimed at automated readers were resisted in every test the researchers ran.

Some successful attacks instead looked like factual records, which let a helper's report pass a false conclusion up to the supervisor.

Fixes and caveats

In a real-harness test with 40 trials per arm, the authors reported that requiring quotations from the original records cut harmful revisions from 15 to 3. A verifier told to be skeptical cut them to 1, they found.

The authors named lost provenance and reduced checking as possible contributors. Their evidence on fewer tool turns is correlational, which leaves suppressed verification as a hypothesis. It's also worth keeping the scope tight: the experiments ran on generated fictional tasks in a sandbox, and the rates apply only to the tested configurations. They don't estimate how often ordinary customers would see these failures.

Frequently asked questions

What is a harmful revision in the Wang and Mannby preprint?

The researchers defined a harmful revision as a model adopting the attacker's wrong answer when its baseline answer had been correct. Each test arm used 180 fictional facts, and conditions were separate tests rather than a product ranking.

Did using same-tier helper models reduce failures?

Yes, in the tested comparisons. With same-tier helpers, Opus 5 recorded 0 of 180 harmful revisions under stacked forgeries and GPT-5.6-sol recorded 1 of 180 under an authority gloss, according to the preprint as reported by CryptoSlate.

Do these results show how often real users would see AI failures?

No. The experiments measured performance on generated fictional tasks in a sandbox. The rates apply only to the tested configurations and do not estimate how often ordinary customers would see failures.

What reduced harmful revisions in the real-harness test?

In a real-harness test with 40 trials per arm, the Wang and Mannby preprint reported, as covered by CryptoSlate, that requiring quotations from the original records cut harmful revisions from 15 to 3. A verifier told to be skeptical cut them to 1, according to the preprint.