Global AI governance is currently at a critical juncture. Rapid advancements in technology are presenting exciting opportunities but also significant challenges. The rise of AI agents — AI systems that can reason, plan, and take direct action — makes strong international cooperation more crucial than ever. To create safer and more responsible AI that benefits people and society, we must work collectively on a global scale.
Partnership on AI (PAI) has been deeply engaged in these conversations, bridging the gap between AI development and responsible policy.
Our team has crossed the globe, connecting with partners and collaborators at key events this year, from the AI Action Summit in Paris, to the World AI Conference in Shanghai, and the Global AI Summit on Africa in Kigali. This builds on the discussion at PAI’s 2024 Policy Forum and the Policy Alignment on AI Transparency report published last October, both of which explored how AI governance efforts align with one another and highlighted the need for international cooperation and coordination on AI policy.
Our journey now takes us next to the 80th session of the United Nations General Assembly (UNGA), taking place in New York this week.
In addition to marking the 80th anniversary of the UN, this year’s UNGA is a call for renewed commitment to multilateralism. It also serves as the official launch of the new UN Global Dialogue on AI Governance. The UN is a crucial piece of the global AI governance puzzle, as a universal and inclusive forum where every nation, regardless of size or influence, has a voice in shaping the future of this technology.
To celebrate this milestone anniversary, PAI is bringing together its community of Partners, policymakers, and other stakeholders for a series of events alongside the UNGA. This is a pivotal moment that demands increased global cooperation amid a challenging geopolitical environment. Our community has identified two particularly important and challenging areas for global AI governance this year:
The opportunities and challenges of AI agents (with 2025 dubbed the “year of agents”) across different fields, including AI safety, human connection, and public policy
The need to build a more robust global AI assurance ecosystem; AI assurance being defined as the process to assess whether an AI system or model is trustworthy
To inform these important discussions and build on our support for the UN Global Digital Compact, PAI is bringing both topics to the attention of the community of UN stakeholders through a series of UNGA side events and publications on both issues. The issues align with the mandates of two new UN AI mechanisms: the UN Independent International Scientific Panel on AI and the Global Dialogue.
The Scientific Panel is tasked with issuing “evidence-based scientific assessments” that synthesize and analyze existing research on the opportunities, risks, and impacts of AI.
Meanwhile, the role of the Global Dialogue is to discuss international cooperation, share best practices and lessons learned, and to facilitate discussions on AI governance to advance the sustainable development goals (SDGs), including on the development of trustworthy AI systems; the protection of human rights in the field of AI; and transparency, accountability, and human oversight consistent with international law.
AI agents are a new research topic that the international community needs to better understand, considering opportunities and potential risks in areas such as human oversight, transparency, and human rights. We expect this topic to be taken up by the Scientific Panel and brought to the attention of the Global Dialogue.
PAI’s work on AI agents includes three key publications:
A Real-time Failure Detection Framework that provides guidance on how to monitor and thereby prevent critical failures in the deployment of autonomous AI agents, which could lead to hazards or real-world incidents that can harm people, disrupt infrastructure, or violate human rights.
An International Policy Brief that offers anticipatory guidance on how to manage the potential cross-border harms and human rights impacts of AI agents, leveraging foundational global governance tools, i.e., international law, non-binding global norms, and global accountability mechanisms.
A Policy Research Agenda that outlines priority questions that policymakers and the scientific community should explore to ensure that we govern AI agents in an informed manner domestically, regionally, and globally.
At the same time, we believe a robust AI assurance ecosystem is crucial to enabling trust and unlocking opportunities for adoption in line with the SDGs and international law. Both the Scientific Panel and the Global Dialogue can help fill significant research and implementation gaps in this area.
Looking ahead, we will expand our focus on AI assurance, with plans to publish a white paper, progress report, and international policy brief at the end of 2025 and early 2026. These publications will touch on issues ranging from the challenges to effective AI assurance, such as insufficient incentives and access to documentation, to AI assurance needs in the Global South.
We hope these contributions will not only inform discussions at the UN but also in other important international AI governance forums, including the OECD’s Global Partnership on AI Expert Group Meeting in November, the G20 Summit in November, and the AI Impact Summit in India next year.
The global conversation on AI governance is still in the early stages, and PAI is committed to ensuring that it is an inclusive, informed, and effective one. To stay up to date on our work in this area, sign up for our newsletter.