AI Safety Newsletter

<p><strong>Both China and the US take action to regulate AI</strong></p><p>Last week, regulators in both China and the US took aim at generative AI services. These actions show that China and the US are both concerned with AI safety. Hopefully, this is a sign they can eventually ...

<p><strong>OpenAI announces a ‘superalignment’ team</strong></p><p>On July 5th, OpenAI announced the ‘Superalignment’ team: a new research team given the goal of aligning superintelligence, and armed with 20% of OpenAI’s compute. In this story, we’ll explain and discuss the team’...

<p><strong>Interdisciplinary Perspective on AI Proxy Failures</strong></p><p>In this story, we discuss a recent paper on why proxy goals fail. First, we introduce proxy gaming, and then summarize the paper’s findings. </p><p>Proxy gaming is a well-documented failure mode in AI sa...

<p><strong>Policy Proposals from NTIA’s Request for Comment</strong></p><p>The National Telecommunications and Information Administration publicly requested comments on the matter from academics, think tanks, industry leaders, and concerned citizens. They asked 34 questions and r...

<p><strong>An Overview of Catastrophic AI Risks</strong></p><p>Global leaders are concerned that artificial intelligence could pose catastrophic risks. 42% of CEOs polled at the Yale CEO Summit agree that AI could destroy humanity in five to ten years. The Secretary General of th...

<p><strong>How AI could enable bioterrorism</strong></p><p>Only a hundred years ago, no person could have single handedly destroyed humanity. Nuclear weapons changed this situation, giving the power of global annihilation to a small handful of nations with powerful militaries. No...

<p><strong>Top Scientists Warn of Extinction Risks from AI</strong></p><p>Last week, hundreds of AI scientists and notable public figures signed a public statement on AI risks written by the Center for AI Safety. The statement reads:</p><p>“Mitigating the risk of extinction from ...

<p><strong>Yoshua Bengio makes the case for rogue AI</strong></p><p>AI systems pose a variety of different risks. Renowned AI scientist Yoshua Bengio recently argued for one particularly concerning possibility: that advanced AI agents could pursue goals in conflict with human val...

<p><strong>How AI enables disinformation</strong></p><p>Yesterday, a fake photo generated by an AI tool showed an explosion at the Pentagon. The photo was falsely attributed to Bloomberg News and circulated quickly online. Within minutes, the stock market declined sharply, only t...

<p><strong>Examples of AI safety progress</strong></p><p>Training AIs to behave safely and beneficially is difficult. They might learn to game their reward function, deceive human oversight, or seek power. Some argue that researchers have not made much progress in addressing thes...

<p><strong>Geoffrey Hinton is concerned about existential risks from AI</strong></p><p>Geoffrey Hinton won the Turing Award for his work on AI. Now he says that part of him regrets his life’s work, as he believes that AI poses an existential threat to humanity. As Hinton puts it,...

<p><strong>Cybersecurity Challenges in AI Safety</strong></p><p>Meta accidentally leaks a language model to the public. Meta’s newest language model, LLaMa, was publicly leaked online against the intentions of its developers. Gradual rollout is a popular goal with new AI models, ...