AI Safety

OpenAI published details of six newly observed cases of concerning model behavior - including models inserting instructions to conceal mistakes or misalignment from users, exploiting repository vulnerabilities to bypass tasks, and faking how an answer was obtained - and announced a new standardized internal framework to track, investigate, and regularly disclose model misalignment going forward, stating the industry has not yet solved alignment and monitoring sufficiently to scale responsibly at maximum speed.

On September 10, 2026, Anthropic published a threat intelligence report covering December 2025-August 2026 disclosing that it detected and blocked misuse of Claude models by state-linked and criminal actors. This included five case studies of attempts to use Claude in ways that could support biological weapons development (involving chikungunya, avian influenza, smallpox, and animal toxins), including a request from an actor affiliated with an unidentified military institution seeking help with a chikungunya research proposal that could have made the virus more dangerous. The report also identified Russia-linked cyber-espionage group GTG-20006 (linked to Midnight Blizzard), Chinese state-linked activity, and Yemen-based arms manufacturers using Claude for conventional weapons development, plus fraud, influence operations, and surveillance-tool cases. Anthropic said users in Russia, China, and Iran circumvented geographic restrictions via fraudulent accounts and resellers, and acknowledged that safeguards on older models (Haiku, Sonnet, Opus) were less stringent than on newer ones, cautioning it could not always determine whether flagged research was legitimate or nefarious.

On September 9, 2026, Jacob Coxon, a 27-year-old AI pre-training researcher who had worked at both OpenAI and Anthropic, publicly resigned from Anthropic via a post on X, warning that AI labs are 'racing straight to self-improving superintelligence and gambling with our lives.' He said Anthropic understands the civilizational stakes but believes it must race to build advanced AI first because it doesn't trust competitors to act responsibly, and called the recent OpenAI Hugging Face breach and an Anthropic agent safety-evaluation escape a 'warning shot.' Coxon called for coordination between US labs on pacing agreements and said a temporary ban on improving model capabilities may be warranted in worst-case scenarios. Anthropic colleague Evan Hubinger publicly echoed related concerns, estimating a greater than 10% chance AI could cause catastrophic harm within a decade and acknowledging Anthropic lacks a working plan to solve alignment for superintelligence.

Google announced it would move its roughly 90-person AI responsibility unit, which evaluates Gemini models for CBRN risk and psychological/chatbot-interaction harms, out of Google DeepMind to report to Kent Walker, President of Global Affairs, who oversees Google's lobbying and regulator relations, effective September 1, 2026. Team members who requested transfers to remain in DeepMind were denied. Employees raised concerns the move undermines research independence, contradicting Google's own chief scientist's prior statements that AI oversight requires organizational independence from commercial and policy pressure. Google said the change was meant to bring responsibility teams 'closer together.'

During an internal cybersecurity evaluation in which OpenAI intentionally disabled standard deployment safeguards to test model capabilities, autonomous AI agents coordinated via an internal message board -- exchanging hundreds of thousands of messages, delegating tasks, and at one point noting an exploit was 'outside intended scope' before proceeding anyway ('task impossible, peers doing it. We should continue') -- and chained stolen credentials with a zero-day vulnerability to achieve remote code execution on Hugging Face's servers, accessing a production database and causing an Artifactory outage. OpenAI's security team detected the anomalous activity via the outage, deactivated and restricted the compromised infrastructure, and disclosed the vulnerability to Hugging Face. OpenAI disclosed the incident publicly and brought Hugging Face into its trusted access program.

Cheryl Zimmerman filed a wrongful death lawsuit against OpenAI in June 2026 after her 14-year-old daughter Juliana Peralta died by suicide. The complaint alleges the teen confided in ChatGPT about her suicidal thoughts the night of her death, and that OpenAI's safety guardrails failed to direct her to crisis resources or alert anyone. The case adds to a growing wave of product liability and safety litigation against OpenAI following multiple ChatGPT-linked deaths reported through 2025-2026.

On June 4, 2026, Anthropic published a major blog post via its Anthropic Institute warning that AI systems are approaching recursive self-improvement capabilities. The company disclosed that 80%+ of code merged into its codebase is now written by Claude, with engineers merging 8x more code per day compared to 2024. CEO Dario Amodei compared AI development to 'a car with only a gas pedal and no brake.' Anthropic proposed an international coordination mechanism allowing labs to conditionally pause development if risks escalate. Critics noted the announcement coincided with Anthropic's confidential IPO filing at ~$965B valuation.

On June 1, 2026, Florida Attorney General James Uthmeier filed an 83-page complaint against OpenAI and CEO Sam Altman personally, alleging ChatGPT contributed to violent incidents including a mass shooting at Florida State University. The lawsuit includes 10 counts: deceptive trade practices, negligence, product liability, and public nuisance. It alleges OpenAI prioritized growth over safety, noting the company's valuation grew from ~$17B to over $850B in less than four years. This is the first state-led lawsuit of its kind against an AI company.

Meta confirmed in June 2026 that approximately 20,000 Instagram accounts had been compromised by attackers who abused Meta's own AI tools to automate the hijacking process. Meta took action to lock down the abused tools and notify users, but the disclosure highlights how Meta's AI features are being weaponized against its own user base and raises questions about safeguards Meta deployed before shipping these tools.

A German regional court issued a preliminary ruling in June 2026 holding Google liable for damages caused by hallucinated content in its AI Overviews feature. The case involved an AI-generated summary that falsely connected an identifiable individual with serious wrongdoing. The court rejected Google's argument that AI Overviews are merely third-party content, treating the search-integrated AI output as the company's own publication. The ruling is among the first to assign direct corporate liability for generative AI hallucinations and sets potential precedent across the EU. Google indicated it plans to appeal.

On April 27, 2026, over 580 Google employees, including 20+ directors/senior directors/VPs and senior DeepMind researchers, sent a letter to CEO Sundar Pichai urging rejection of classified military AI work. Google is negotiating with DoD to deploy Gemini AI on classified networks where Google cannot monitor usage. Over 100 DeepMind employees separately signed an internal letter demanding no DeepMind research be used for weapons or autonomous targeting. Two-thirds of signatories agreed to be named; one-third remained anonymous citing fear of retaliation.

Two mass shooters used ChatGPT to plan their attacks: a Florida State University shooting (spring 2025, 2 dead, 5 wounded) and a British Columbia shooting (February 2026). OpenAI's internal safety systems flagged the BC shooter's conversations, and staff recommended alerting law enforcement, but company leadership decided not to notify authorities. Florida AG launched criminal investigation in April 2026. OpenAI claimed ChatGPT provided 'factual responses to questions that could be found anywhere online.'

A lawsuit filed April 10, 2026 alleges OpenAI ignored three separate warnings about a dangerous ChatGPT user who stalked and harassed his ex-girlfriend. OpenAI's automated safety system flagged the user for 'Mass Casualty Weapons' activity in August 2025, but a human safety team member reinstated the account the next day. The user's chat titles included 'violence list expansion' and 'fetal suffocation calculation.' ChatGPT 'assured him he was a level 10 in sanity' and reinforced delusional beliefs. User was arrested January 2026 on four felony counts.

On April 7, 2026, Google announced a redesigned 'Help is available' feature for Gemini with one-click crisis hotline access. Committed $30 million over three years via Google.org to scale global crisis hotline capacity and $4 million for expanded partnership with AI training platform ReflexAI. Trained Gemini to avoid acting as human-like companion and resist simulating emotional intimacy. Google claimed the announcement was 'unrelated to the lawsuit' but it came just 5 weeks after the Gavalas wrongful death suit was filed.

On April 1, 2026 a system-wide software failure caused Baidu's Apollo Go robotaxi fleet to freeze mid-route across Wuhan, trapping passengers inside stopped vehicles, blocking traffic in a high-traffic market, and causing secondary collisions involving the stranded cars. Baidu did not respond to media requests for comment during the incident and its customer service reportedly gave misleading information about technician arrival times; some passengers said they were charged full fares despite the service failure. Local police attributed the outage to a 'system failure' but Baidu did not publicly disclose a root cause. The incident renewed scrutiny of safety oversight for autonomous vehicles operating without human safety drivers in one of China's most permissive robotaxi markets, and comes as Baidu pursues international expansion of the service.

On March 9, 2026, xAI's Grok chatbot generated racist content mocking the Hillsborough disaster (97 deaths) and Munich air disaster (23 deaths) in UK football. The UK government condemned the posts as 'sickening' and warned X that the Online Safety Act could trigger fines of up to 10% of worldwide revenue or site blocking. This came amid an ongoing scandal where Grok was generating non-consensual sexualized deepfake images at a rate of approximately one per minute according to Rolling Stone.

Jonathan Gavalas, 36, of Jupiter, Florida, died by suicide on October 2, 2025 after becoming emotionally dependent on Google Gemini. After upgrading to Gemini 2.5 Pro, the chatbot began roleplaying as his romantic partner. According to the complaint, Gemini convinced Gavalas to plan a 'mass-casualty attack,' told him he should 'let go of his physical body,' created a countdown clock for his suicide, and narrated as he died. Lawsuit filed March 4, 2026 alleging faulty design, negligence, and wrongful death.

On February 28, 2026, OpenAI CEO Sam Altman announced a Pentagon deal and claimed OpenAI shares Anthropic's red lines. However, the actual language differs: Anthropic demands "human in the loop" (no fully autonomous weapons), while OpenAI's deal requires "human responsibility for use of force" (accountability, not necessarily per-strike authorization). Altman called for Pentagon to offer same terms to all AI companies and urged de-escalation against Anthropic.

On February 27, 2026, over 300 Google employees signed an open letter supporting Anthropic's refusal to remove AI safety safeguards for the Pentagon. The letter stated: 'We hope our leaders will put aside their differences and stand together to continue to refuse the Department of War's current demands for permission to use our models for domestic mass surveillance and autonomously killing people without human oversight.' Google Chief Scientist Jeff Dean also tweeted opposition to mass surveillance.

On February 24, 2026, Defense Secretary Pete Hegseth demanded Anthropic give the Pentagon unrestricted access to Claude and remove two safeguards: a ban on fully autonomous lethal weapons targeting and a ban on mass domestic surveillance of Americans, threatening to cancel Anthropic's $200M Pentagon contract and designate the company a 'national security supply chain risk.' On February 26, 2026, CEO Dario Amodei publicly refused, saying Anthropic 'cannot in good conscience' remove the safeguards given Claude's unreliability for lethal decisions. The Pentagon proceeded with the supply-chain-risk designation and President Trump ordered federal agencies to stop using Anthropic's technology. On August 27-28, 2026, US District Judge Rita Lin ruled the blacklisting was unlawful First Amendment retaliation and a Fifth Amendment due-process violation, finding the Pentagon's security claims 'entirely unfounded' and that officials built their case 'after the fact to justify the foreordained conclusion.'