Opinion: Humanity’s Fate Can’t Be Entrusted to Algorithms

Artificial intelligence (AI) is fundamentally transforming both the global economy and the fabric of daily human life, but this progress is bringing new governance challenges that must not be overlooked. As AI technologies advance, both opportunities and unprecedented risks are arising, prompting urgent calls for comprehensive global responses [para. 1].

At the 80th session of the United Nations General Assembly in New York, the UN Security Council hosted its first open debate dedicated to “AI and International Peace and Security.” In this landmark session, UN Secretary-General António Guterres announced the creation of an independent panel of 40 scientific experts, the launch of a “Global AI Governance Dialogue,” and the establishment of a global fund to build AI capacity. These initiatives reflect the world’s heightened focus on the profound—and sometimes alarming—impact of AI on the future of humanity. Optimism about technological advances is tempered with serious concerns about the potential risks and the need for robust safeguards [para. 2].

During the debate, Guterres stressed that critical decisions, especially those involving nuclear weapons or acts of war, must never be left to AI algorithms. He urgent all nations to pursue a legally binding international treaty on AI governance by the end of 2026, marking a crucial step toward a coordinated global response. This call builds on earlier high-level meetings, like the UN Security Council’s discussion in July 2023 on the “Opportunities and Risks for International Peace and Security” presented by AI. The pace and unpredictability of AI’s advancement have prompted growing recognition that if misused or allowed to operate beyond human oversight, AI could pose catastrophic risks that transcend individual nation’s capacities to manage. Global collaboration, swift action, and comprehensive regulation are deemed essential [para. 3].

The future of AI remains uncertain, and expert opinions are varied. The Artificial Intelligence Security Governance Framework 2.0, released in September, identifies a wide range of security challenges: model algorithm flaws, data quality issues, vulnerabilities in network systems and information content, and the potential for catastrophic derivative risks arising from the technology’s misuse or abuse. These risks originate from all parts of the AI ecosystem—training data, algorithms, computing infrastructure, and application scenarios—necessitating a complex, multilayered approach to security and governance involving government, businesses, technology providers, and civil society [para. 4].

However, growing geopolitical tensions could worsen governance challenges. The United States opposes “centralized global AI governance,” arguing it could stifle innovation and undermine national sovereignty, whereas European countries generally favor stronger international rules. These disagreements risk turning AI governance into a tool of great-power competition, potentially marginalizing smaller nations and deepening global inequalities through technological exclusion and high investment barriers [para. 5].

Effective AI governance does not entail hindering technological development but instead aims to minimize risks and foster equitable growth and benefit for all. Initiatives like China’s “Global AI Governance Initiative,” issued in 2023, call for “intelligence for good,” equal development rights, and the establishment of an international AI governance institution under the UN [para. 5].

The importance of harnessing expert knowledge is reflected in the recent “Global Call for AI Red Lines,” launched at the UN General Assembly and supported by over 200 leading figures—including 10 Nobel laureates. This initiative demands robust international agreements and enforcement mechanisms on AI “red lines” by 2026, warning that some advanced AI systems are already displaying harmful autonomous behavior [para. 6].

Ultimately, as AI increasingly permeates all aspects of human existence, the world faces a critical question: can humanity control AI’s trajectory, or will it cede too much power to algorithms? The answer will depend on the will, coordination, and responsibility exercised collectively at the global level [para. 7].