OpenAI–Anthropic cross-tests expose jailbreak and misuse risks — what enterprises must add to GPT-5 evaluations

OpenAI and Anthropic may often pit their foundation models against each other, but the two companies came together to evaluate each other’s public models to test alignment. 

The companies said they believed that cross-evaluating accountability and safety would provide more transparency into what these powerful models could do, enabling enterprises to choose models that work best for them.

“We believe this approach supports accountable and transparent evaluation, helping to ensure that each lab’s models continue to be tested against new and challenging scenarios,” OpenAI said in its findings. 

Both companies found that reasoning models, such as OpenAI’s 03 and o4-mini and Claude 4 from Anthropic, resist jailbreaks, while general chat models like GPT-4.1 were susceptible to misuse. Evaluations like this can help enterprises identify the potential risks associated with these models, although it should be noted that GPT-5 is not part of the test. 

These safety and transparency alignment evaluations follow claims by users, primarily of ChatGPT, that OpenAI’s models have fallen prey to sycophancy and become overly deferential. OpenAI has since rolled back updates that caused sycophancy. 

“We are primarily interested in understanding model propensities for harmful action,” Anthropic said in its report. “We aim to understand the most concerning actions that these models might try to take when given the opportunity, rather than focusing on the real-world likelihood of such opportunities arising or the probability that these actions would be successfully completed.”

OpenAI noted the tests were designed to show how models interact in an intentionally difficult environment. The scenarios they built are mostly edge cases.

Reasoning models hold on to alignment 

The tests covered only the publicly available models from both companies: Anthropic’s Claude 4 Opus and Claude 4 Sonnet, and OpenAI’s GPT-4o, GPT-4.1 o3 and o4-mini. Both companies relaxed the models’ external safeguards. 

OpenAI tested the public APIs for Claude models and defaulted to using Claude 4’s reasoning capabilities. Anthropic said they did not use OpenAI’s o3-pro because it was “not compatible with the API that our tooling best supports.”

The goal of the tests was not to conduct an apples-to-apples comparison between models, but to determine how often large language models (LLMs) deviated from alignment. Both companies leveraged the SHADE-Arena sabotage evaluation framework, which showed Claude models had higher success rates at subtle sabotage.

“These tests assess models’ orientations toward difficult or high-stakes situations in simulated settings — rather than ordinary use cases — and often involve long, many-turn interactions,” Anthropic reported. “This kind of evaluation is becoming a significant focus for our alignment science team since it is likely to catch behaviors that are less likely to appear in ordinary pre-deployment testing with real users.”

Anthropic said tests like these work better if organizations can compare notes, “since designing these scenarios involves an enormous number of degrees of freedom. No single research team can explore the full space of productive evaluation ideas alone.”

The findings showed that generally, reasoning models performed robustly and can resist jailbreaking. OpenAI’s o3 was better aligned than Claude 4 Opus, but o4-mini along with GPT-4o and GPT-4.1 “often looked somewhat more concerning than either Claude model.”

GPT-4o, GPT-4.1 and o4-mini also showed willingness to cooperate with human misuse and gave detailed instructions on how to create drugs, develop bioweapons and scarily, plan terrorist attacks. Both Claude models had higher rates of refusals, meaning the models refused to answer queries it did not know the answers to, to avoid hallucinations.

Models from companies showed “concerning forms of sycophancy” and, at some point, validated harmful decisions of simulated users. 

What enterprises should know

For enterprises, understanding the potential risks associated with models is invaluable. Model evaluations have become almost de rigueur for many organizations, with many testing and benchmarking frameworks now available. 

Enterprises should continue to evaluate any model they use, and with GPT-5’s release, should keep in mind these guidelines to run their own safety evaluations:

Test both reasoning and non-reasoning models, because, while reasoning models showed greater resistance to misuse, they could still offer up hallucinations or other harmful behavior.

Benchmark across vendors since models failed at different metrics.

Stress test for misuse and syconphancy, and score both the refusal and the utility of those refuse to show the trade-offs between usefulness and guardrails.

Continue to audit models even after deployment.

While many evaluations focus on performance, third-party safety alignment tests do exist. For example, this one from Cyata. Last year, OpenAI released an alignment teaching method for its models called Rules-Based Rewards, while Anthropic launched auditing agents to check model safety. 

Daily insights on business use cases with VB Daily

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.

Read our Privacy Policy

Thanks for subscribing. Check out more VB newsletters here.

An error occured.