Apollo Research can flag what OpenAI withholds in four-month safety pilot

Editorial illustration: An articulated magnifying glass enlarges a black redaction on an open document, with a glowing network of connected spheres inside a glass enclosure behind it.

In brief

  • Apollo Research will spend four months checking whether OpenAI's models try to bypass safeguards.
  • Six Apollo evaluators are set to work inside OpenAI; no findings have been released.
  • OpenAI can withhold material but must give Apollo written reasons.
  • Published terms protect Apollo's verdict on each claim from redaction.

What Apollo will examine

Six Apollo evaluators are set to work inside OpenAI. They'll look at four questions: whether training encourages models to undermine safeguards, whether safety tests can draw out that behavior, whether it happens during internal use, and whether evaluators can detect the reasoning behind it. The review covers agreed claims, models and time periods.

For the training inquiry, Apollo said it'll have access to relevant code and datasets, can query intermediate training checkpoints, and will receive reinforcement-learning records (including rewards and available reasoning traces). Access details for the other three inquiries hadn't been agreed when Apollo announced the pilot.

OpenAI said Apollo is paid for its work and that compensation doesn't depend on the assessment's results.

What OpenAI can hold back

The terms set separate rules for what Apollo can inspect and what it can disclose. OpenAI can withhold even in-scope material when it considers that reasonably necessary because of technical limits, security risks or intellectual property protection. When it does, it must tell Apollo and give written reasons; where OpenAI considers it feasible and relevant, it'll also identify the categories and approximate volume of withheld records.

Unfavorable information alone can't justify withholding access.

Interviews run on tighter rules. OpenAI selects which employees can discuss confidential information, formal interviews are logged, and the conversations are off the record. To publish a claim from an interview, Apollo must establish it independently or obtain a formal attestation from OpenAI. If OpenAI declines to attest, Apollo can say so publicly and explain the effect on its conclusions while keeping the underlying information confidential.

What the public reports can say

So what will readers actually see? Apollo plans two public reports. The first will address whether training encouraged models to undermine safeguards, and the second will cover safety testing, internal use and whether the reasoning behind such behavior can be detected.

OpenAI can review the reports for confidential material and redact supporting evidence, but its commitments bar it from requesting a redaction solely because a finding is unfavorable. Apollo's verdict on each claim is protected from redaction. The Safety and Security Committee of the OpenAI Foundation's board will receive unredacted copies.

Apollo can also publicly describe the access it requested and received, any limits or delays OpenAI imposed, and the reasons the company gave. It doesn't need to clear comments on the pilot's terms with OpenAI first.

For now, those protections are commitments in a pilot's published terms (nobody has independently verified them). Neither report is out yet.

Frequently asked questions

What is Apollo Research checking at OpenAI?

Apollo is running a four-month pilot investigating whether OpenAI's models try to bypass safeguards. Six evaluators will examine whether training encourages that behavior, whether safety tests can draw it out, whether it happens during internal use, and whether the reasoning behind it can be detected.

Can OpenAI redact Apollo's findings?

OpenAI can redact confidential supporting evidence from the public reports, but Apollo's verdict on each claim is protected from redaction. OpenAI's commitments bar it from requesting a redaction solely because a finding is unfavorable, and Apollo can explain how redactions affected its conclusions.

What happens if OpenAI withholds evidence from Apollo?

OpenAI must tell Apollo that material was withheld and give written reasons. Apollo can publicly describe the access it requested and received, any limits or delays imposed, and the reasons OpenAI gave. Unfavorable information alone can't justify withholding access.