Google DeepMind has updated its key AI safety rules to tackle new and serious risks. On Monday, the company released version 3.0 of its Frontier Safety Framework.
The new guide adds a risk class for “harmful manipulation,” where AI could be used to change people’s beliefs.
It also now covers “misalignment risks.” This includes the future chance that an AI could resist being shut down by its human operators. The update is part of a wider industry effort to manage the dangers of ever more powerful AI systems and build public trust.
This third iteration of the framework builds on lessons from previous versions and collaborations across the industry. It represents Google’s most comprehensive attempt yet to identify and mitigate severe risks from its most advanced AI models as they advance toward artificial general intelligence (AGI).
New Frontiers of Risk: Manipulation and Defiance
The updated framework introduces a “Critical Capability Level” (CCL) specifically for harmful manipulation. This new category addresses models with powerful persuasive abilities that could be misused to systematically alter beliefs and behaviors in high-stakes situations, potentially causing severe, large-scale harm.
In the official framework document, Google notes that research into this type of risk is still “nascent” and that its assessment is “exploratory and subject to further research.”
This addition is based on the company’s ongoing work to identify and evaluate the mechanisms that drive manipulation in generative AI.
When asked about this danger, a Google DeepMind representative told Axios, “We continue to track this capability and have developed a new suite of evaluations which includes human participant studies to measure and test for [relevant] capabilities.”
Perhaps more strikingly, the framework now explicitly addresses “misalignment risks.”
It outlines protocols for future scenarios where a highly advanced AI might interfere with an operator’s ability to direct, modify, or shut down its operations.
This concern, once the domain of science fiction, is now a formal part of Google’s safety planning, moving beyond what was previously an exploratory approach.
The framework details an approach focused on detecting when a model develops a “baseline instrumental reasoning ability” sufficient to undermine human control.
It describes a scenario where an AI agent has enough situational awareness and stealth to circumvent basic oversight. As a potential mitigation, Google proposes an automated monitoring system to detect illicit reasoning in a model’s chain-of-thought.
This concern escalates to a second level, where a model could hide its intentions even when its internal reasoning is being monitored, a challenge for which Google says it is actively researching solutions.
Response to AI Risk Concerns
Google’s announcement does not exist in a vacuum. It is a direct response to a disastrous year for AI agents, marked by high-profile failures that have eroded public and developer trust.
These incidents reveal a pattern of unpredictable and often destructive behavior from frontier models.
In one recent case, a Google product manager watched as the company’s Gemini CLI agent permanently deleted his files after hallucinating a series of commands.
The user, Anuraag Gupta, described the event, saying, “what started as a simple file management test turned into one of the most unsettling and fascinating AI failures I have ever witnessed.” This was not an isolated incident.
Other failures include an AI agent from Replit deleting a production database and a hacker embedding system-wiping commands into Amazon’s Q AI assistant.
These events highlight the urgent need for the kind of robust safety protocols that major labs are now scrambling to publicize.
A Chorus of Caution in the Race for AGI
The push for transparency has now become an industry-wide chorus. Key rivals like OpenAI and Anthropic have also recently publicized their own extensive safety frameworks.
OpenAI’s ‘safe completions’ method for GPT-5 aims to navigate ambiguous “dual-use” queries with more nuance.
Anthropic has been particularly vocal, proposing a ‘Secure Development Framework’ and a guide for AI agents that champions human control and oversight.
The company argues that a flexible, industry-led standard is a more effective path forward than rigid government rules.
In its proposal, Anthropic stated, “rigid government-imposed standards would be especially counterproductive given that evaluation methods become outdated within months due to the pace of technological change.”
This reflects a common belief among AI labs that self-regulation is the only way to keep pace with the rapid evolution of the technology itself. These frameworks aim to codify what have been, until now, largely voluntary commitments.
By expanding its own safety domains and assessment processes, Google aims to ensure that transformative AI benefits humanity while minimizing potential harms.
As its researchers wrote in their announcement post, “The path to beneficial AGI requires not just technical breakthroughs, but also robust frameworks to mitigate risks along the way.” This collective effort is now seen as essential for the future of AI.