Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
AI agents have safety and reliability problems. Although agents would allow enterprises to automate more steps in their workflows, they can take unintended actions while executing a task, are not very flexible and are difficult to control.
Organizations have already raised the alarm about unreliable agents, worried that once deployed, agents might forget to follow instructions.
OpenAI even admitted that ensuring agent reliability would involve working with outside developers, so it opened up its Agents SDK to help solve this issue.
However, Singapore Management University (SMU) researchers have developed a new approach to solving agent reliability.
AgentSpec is a domain-specific framework that lets users “define structured rules that incorporate triggers, predicates and enforcement mechanisms.” The researchers said AgentSpec will make agents work only within the parameters that users want.
Guiding LLM-based agents with a new approach
AgentSpec is not a new large language model (LLM) but rather an approach to guide LLM-based AI agents. The researchers believe AgentSpec can be used for agents in enterprise settings and self-driving applications.
The first AgentSpec tests integrated on LangChain frameworks, but the researchers said they designed it to be framework-agnostic, meaning it can also run on AutoGen and Apollo ecosystems.
Experiments using AgentSpec showed it prevented “over 90% of unsafe code executions, ensures full compliance in autonomous driving law-violation scenarios, eliminates hazardous actions in embodied agent tasks and operates with millisecond-level overhead.” LLM-generated AgentSpec rules, which used OpenAI’s o1, also had a strong performance and enforced 87% of risky code and prevented “law-breaking in 5 out of 8 scenarios.”
Current methods are a little lacking
AgentSpec is not the only method for helping developers give agents more control and reliability. Other approaches include ToolEmu and GuardAgent. The startup Galileo launched Agentic Evaluations, a way to ensure agents work as intended.
The open-source platform H2O.ai uses predictive models to improve the accuracy of agents used by companies in finance, healthcare, telecommunications and government.
The AgentSpec said researchers said current approaches to mitigate risks, like ToolEmu, effectively identify risks. They noted that “these methods lack interpretability and offer no mechanism for safety enforcement, making them susceptible to adversarial manipulation.”
Using AgentSpec
AgentSpec works as a runtime enforcement layer for agents. It intercepts the agent’s behavior while executing tasks and adds safety rules set by humans or generated by prompts.
Since AgentSpec is a custom domain-specific language, users must define the safety rules. There are three components to this: the first is the trigger, which lays out when to activate the rule; the second is to check to add conditions; and the third is enforce, which enforces actions to take if the rule is violated.
AgentSpec is built on LangChain, though, as previously stated, the researchers said AgentSpec can also be integrated into other frameworks like AutoGen or the autonomous vehicle software stack Apollo.
These frameworks orchestrate the steps agents need to take by taking in the user input, creating an execution plan, observing the result, and then deciding if the action was completed and, if not, planning the next step. AgentSpec adds rule enforcement into this flow.
“Before an action is executed, AgentSpec evaluates predefined constraints to ensure compliance, modifying the agent’s behavior when necessary. Specifically, AgentSpec hooks into three key decision points: before an action is executed (AgentAction), after an action produces an observation (AgentStep), and when the agent completes its task (AgentFinish). These points provide a structured way to intervene without altering the core logic of the agent,” the paper states.
More reliable agents
Approaches like AgentSpec underscore the need for reliable agents for enterprise use. As organizations begin to plan their agentic strategy, tech decision leaders also look at ways to ensure reliability.
For many, agents will eventually autonomously and proactively do tasks for users. The idea of ambient agents, where AI agents and apps continuously run in the background and trigger themselves to execute actions, would require agents that do not stray from their path and accidentally introduce non-safe actions.
If ambient agents are where agentic AI will go in the future, expect more methods like AgentSpec to proliferate as companies seek to make AI agents continuously reliable.
