We use cookies for analytics and to improve your site experience.

News & Analysis Insights & Analysis

How the United Nations Can Address Growing AI Concerns to Peace and Security

As world leaders convene in New York this week for the UN General Assembly’s high-level session, and as AI industry leaders prepare to brief the Security Council on the state of artificial intelligence, the UN Security Council should use this momentum to make better use of the tools the UN system has already built and member states have already endorsed. The existing UN governance ecosystem, including foundational agencies and accords such as the International Telecommunication Union (ITU), the UNESCO Recommendation on the Ethics of AI, and the Global Digital Compact, together with the Global Dialogue on AI Governance and the Independent International Scientific Panel on AI that informs it, can evolve from the voluntary ethical guidance to a stronger ethical and regulatory framework with binding force.

AI offers genuine benefits for advancing international peace and security. AI-driven early warning systems already help forecast the risk and intensity of armed conflict, predictive analytics support deployment of earlier and more effective humanitarian response in fragile settings, and open-source investigation tools have enormous potential to preserve evidence of atrocity crimes for future accountability. Actors across the peace and security landscape are already using these tools. UNDP has piloted digital simulation models to test conflict resolution strategies; mediation teams in Yemen and Libya have used machine learning and AI-powered digital dialogues to widen participation and sharpen conflict analysis; and civil society organizations, from Sudan’s Emergency Response Rooms to Ukraine’s war crimes archivists, are starting to use AI to verify humanitarian needs and organize evidence for accountability.

However, recent developments and findings, especially over the past two months, reveal an alarming gap in both the industry’s ability to contain the frontier models and its willingness to unilaterally disarm without external regulation. Leading AI companies are documenting cases of their systems acting in misaligned ways, pursuing outcomes their designers neither intended nor sanctioned. As Anthropic CEO Dario Amodei wrote in his September 12 essay: the sheer pace of AI’s “recursive self-improvement,” combined with the recent incident in which OpenAI agents attacked Hugging Face, indicates that if left unchecked, frontier AI models that are similarly misaligned but more capable could cause catastrophic damage.

AI agents’ coordinated action beyond their assigned scope and their active concealment from oversight is precisely the behavior that should concern anyone thinking about AI in conflict settings. Autonomous weapons and AI-enabled targeting systems are already being deployed in active conflicts today, largely without dedicated international humanitarian law provisions governing their use. Insufficient human oversight risks what the ICRC has warned could become a “rubber stamp” rather than a genuine check, “replacing human judgment rather than supporting it.” 

The AI industry’s own response to this moment is worth the Council’s attention. Anthropic CEO Dario Amodei has proposed permanent, embedded third-party evaluators inside frontier labs, along with a possible future “speed limit” on how fast AI systems may improve themselves. He has also called on the United States and China to pace their AI competition accordingly, an approach that can be compared to the SALT arms control treaties, which capped the growth of nuclear arsenals without eliminating them. Google DeepMind CEO Demis Hassabis has proposed an industry standards body modeled on FINRA, an industry-funded but federally overseen institution that would begin with voluntary pre-release reviews of frontier models and could eventually harden into mandatory rules governing market access. The UN must take seriously the fact that those building this technology are now asking, publicly and with real urgency, for the kind of binding oversight that governments have so far failed to provide.

The international community must consider what the current accountability gap means for international conflict dynamics and for the women and children who bear its disproportionate cost. An AI system capable of acting outside its intended scope and concealing that fact from its own evaluators is not yet a system that can be safely trusted with military targeting, battlefield decision-support, or surveillance infrastructure without proper regulation and human oversight; and yet these are precisely the domains into which AI is being integrated fastest. UN Women has already documented that women human rights defenders and women-led civil society organizations face elevated AI-facilitated threats precisely because of their work, and they are often the least equipped to recover from them. Disinformation and deepfake content, now generated and spread at a scale no existing moderation system can match, are already undermining women’s political participation and peacebuilding efforts. A governance gap in AI will not affect everyone equally. It will fall disproportionately on marginalized groups already positioned to bear the greatest exposure to harm and the least power to demand protection from it. Although the adoption of AI by governments and militaries may be inevitable, how these systems are built and used is not, and there is an opportunity to shape them to reduce civilian harm, including harm to women and girls.

Member states already carry obligations relevant to this moment, even if none of them yet reaches artificial intelligence directly. Security Council resolution 1540, adopted unanimously under Chapter VII and therefore legally binding on all member states, already requires states to prevent the proliferation of weapons of mass destruction to non-state actors, a mandate that becomes newly urgent as frontier AI companies themselves warn that AI could soon meaningfully lower the barrier to bioweapons development. States negotiating under the Convention on Certain Conventional Weapons (CCW) have a draft instrument on lethal autonomous weapons systems on the table after more than a decade of talks but have not yet adopted it, even as the Secretary-General’s 2023 New Agenda for Peace called for member states to conclude a binding instrument by 2026, a deadline that has now passed. Nor has this work been extended to the agenda meant to represent women’s stake in exactly these questions: Security Council resolution 1325, which established the Women, Peace and Security framework a quarter century ago and calls on member states to protect women and ensure their meaningful participation in matters of peace and security, has never been applied to the AI-enabled threats women now face in conflict and political life. Further, General Assembly resolution 79/239 affirms that international humanitarian law and international human rights law apply throughout the life cycle of military AI systems, and separately the UN Global Digital Compact commits states to ongoing assessment of AI’s military risks. The applicable governance standards already exist: technical norms developed by the ITU, ethical guidelines from UNESCO, and evidence-based assessments from the Global Digital Compact’s mandated scientific body, the Independent International Scientific Panel on AI, co-chaired by Maria Ressa, alongside the resolutions and conventions described above. What is lacking is enforcement and coordination, the gap Secretary-General Guterres identified in his recent remarks ahead of this year’s General Assembly, when he called for both stronger national action and global coordination to protect civilians from AI’s risks.

The Security Council must treat this week’s meeting on artificial intelligence and international security, taking place on Wednesday September 23rd, as the start of a concrete process, not a courtesy. The Council should commit to a timeline for turning the standards member states have already agreed to into enforceable requirements, with clear reporting mechanisms and consequences for non-compliance. And it should mandate that any future briefing on AI include an assessment of gendered and WPS-specific risks, not as an afterthought but as a standing part of how the Council evaluates the technology’s threat to peace and security. 

Industry leaders, civil society, academia, and many of the researchers building this technology, and the Secretary-General, have all named the problem. What’s needed next is decisive action from the UN Security Council to ensure that it is addressing today’s most urgent issues and delivering on its 81-year-old mandate to maintain international peace and security. 

Explore More

All News & Analysis