AI Safety Newsletter

<p><strong>Challenges of Reinforcement Learning from Human Feedback</strong></p><p>If you’ve used ChatGPT, you might’ve noticed the “thumbs up” and “thumbs down” buttons next to each of its answers. Pressing these buttons provides data that OpenAI uses to improve their models thr...

<p><strong>Automatically Circumventing LLM Guardrails</strong></p><p>Large language models (LLMs) can generate hazardous information, such as step-by-step instructions on how to create a pandemic pathogen. To combat the risk of malicious use, companies typically build safety guar...

<p><strong>White House Unveils Voluntary Commitments to AI Safety from Leading AI Labs</strong></p><p>Last Friday, the White House announced a series of voluntary commitments from seven of the world's premier AI labs. Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and Op...

<p><strong>Both China and the US take action to regulate AI</strong></p><p>Last week, regulators in both China and the US took aim at generative AI services. These actions show that China and the US are both concerned with AI safety. Hopefully, this is a sign they can eventually ...

<p><strong>OpenAI announces a ‘superalignment’ team</strong></p><p>On July 5th, OpenAI announced the ‘Superalignment’ team: a new research team given the goal of aligning superintelligence, and armed with 20% of OpenAI’s compute. In this story, we’ll explain and discuss the team’...

<p><strong>Interdisciplinary Perspective on AI Proxy Failures</strong></p><p>In this story, we discuss a recent paper on why proxy goals fail. First, we introduce proxy gaming, and then summarize the paper’s findings. </p><p>Proxy gaming is a well-documented failure mode in AI sa...

<p><strong>Policy Proposals from NTIA’s Request for Comment</strong></p><p>The National Telecommunications and Information Administration publicly requested comments on the matter from academics, think tanks, industry leaders, and concerned citizens. They asked 34 questions and r...

<p><strong>An Overview of Catastrophic AI Risks</strong></p><p>Global leaders are concerned that artificial intelligence could pose catastrophic risks. 42% of CEOs polled at the Yale CEO Summit agree that AI could destroy humanity in five to ten years. The Secretary General of th...

<p><strong>How AI could enable bioterrorism</strong></p><p>Only a hundred years ago, no person could have single handedly destroyed humanity. Nuclear weapons changed this situation, giving the power of global annihilation to a small handful of nations with powerful militaries. No...

<p><strong>Top Scientists Warn of Extinction Risks from AI</strong></p><p>Last week, hundreds of AI scientists and notable public figures signed a public statement on AI risks written by the Center for AI Safety. The statement reads:</p><p>“Mitigating the risk of extinction from ...

<p><strong>Yoshua Bengio makes the case for rogue AI</strong></p><p>AI systems pose a variety of different risks. Renowned AI scientist Yoshua Bengio recently argued for one particularly concerning possibility: that advanced AI agents could pursue goals in conflict with human val...

<p><strong>How AI enables disinformation</strong></p><p>Yesterday, a fake photo generated by an AI tool showed an explosion at the Pentagon. The photo was falsely attributed to Bloomberg News and circulated quickly online. Within minutes, the stock market declined sharply, only t...