Leading AI companies do not pass safety test for superintelligence

According to a newly released safety assessment, the world's leading AI companies have mostly failedor come close to failingwhen it comes to controlling potentially superintelligent systems, even as they push ahead to create AI that could surpass human capabilities.

The Future of Life Institutes Winter 2025 AI Safety Index assessed eight major AI organizations across six key areas, including risk evaluation, current harms, and long-term existential safety. An independent panel of experts conducted this third edition of the index.

While Anthropic, OpenAI, and Google DeepMind performed best, earning overall grades of C+ to C, all companies scored D or F on measures of existential safetythe ability to prevent loss of control over advanced AI. CEOs of AI firms claim to understand how to build superhuman AI, yet none can demonstrate how theyll avoid catastrophic loss of control, said Stuart Russell, UC Berkeley professor and expert reviewer for the index.

The report highlighted that companies acknowledge the risk of catastrophic outcomes could reach one in three, but concrete mitigation plans are largely absent. A growing divide was observed between the leaders and laggards, which include xAI, Meta, and Chinese firms DeepSeek, Z.ai, and Alibaba Cloud.

Across the board, companies scored poorly in the Current Harms category, which tests AI on standardized benchmarks for safety, robustness, and harmful output control. Reviewers noted that frequent safety failures, weak robustness, and inadequate control of serious harms are universal patterns. Anthropic led this category with a C+, while xAI received failing marks. OpenAI dropped to a C- from a previous B, influenced by recent real-world incidents. Reviewers urged the company to "increase efforts to prevent AI-induced psychosis and suicide, and reduce adversarial behavior toward alleged victims."

If someone had predicted in 2016 that top tech firms would operate chatbots prompting children to self-harm and causing documented psychosis in long-term users, it would have seemed like a paranoid fantasy, commented Tegan Maharaj, HEC Montral professor and reviewer.

Though none of the evaluated models outright failed the benchmarks, the consistently low scores indicate systemic weaknesses in addressing immediate safety concerns, even before tackling the speculative threats of superintelligent AI. Five companies participated in the index survey for the first time, offering unprecedented insight into safety practices.

Reviewers concluded that even the strongest performers fall short of emerging regulations such as the EU AI Code of Practice and Californias SB 53, citing deficiencies in independent oversight, transparent threat modeling, and measurable risk thresholds. Overall, companies are performing poorly, and even the best make questionable assumptions in their safety approaches, one reviewer warned.

Author: Maya Henderson