AI Safety

At the World Economic Forum in Davos on January 20, 2026, Dario Amodei lambasted the Trump administration's decision to allow Nvidia to sell H200 AI chips to China, calling it a 'major mistake' with 'incredible national security implications'.

42 State Attorneys General issued a letter to Google (along with other large technology companies) about the rise in sycophantic and delusional outputs from generative AI software. The letter highlighted that generative AI software has been involved in at least six deaths in the United States, and other incidents of domestic violence, poisoning, and hospitalizations for psychosis.

42 State Attorneys General issued a letter to Meta (along with other large technology companies) about the rise in sycophantic and delusional outputs from generative AI software. The letter highlighted that generative AI software has been involved in at least six deaths in the United States, and other incidents of domestic violence, poisoning, and hospitalizations for psychosis.

42 State Attorneys General issued a letter to Microsoft (along with other large technology companies) about the rise in sycophantic and delusional outputs from generative AI software. The letter highlighted that generative AI software has been involved in at least six deaths in the United States, and other incidents of domestic violence, poisoning, and hospitalizations for psychosis.

A widespread malware campaign abused Google's Chrome Web Store for months, exposing private AI chatbot conversations and browsing data from roughly 900,000 users. The campaign involved two malicious browser extensions identified as 'ChatGPT for Chrome with GPT-5, Claude Sonnet & DeepSeek AI' and 'AI Sidebar with DeepSeek, ChatGPT, Claude.' The extensions remained available in the Chrome Web Store despite the security vulnerabilities.

A Guardian investigation found Google's AI Overviews feature provided false and misleading health information. Google advised pancreatic cancer patients to avoid high-fat foods - the exact opposite of correct guidance that could jeopardize tolerance of chemotherapy or surgery. Additional errors included incorrect liver blood test ranges and wrong cancer screening information. Health charities Pancreatic Cancer UK, British Liver Trust, Mind, and Eve Appeal raised alarms. Google subsequently removed AI Overviews for some medical queries but only partially addressed the issue.

Despite Eric Schmidt publicly warning against autonomous weapons at Stanford in 2024, calling automated kill decisions 'terrible,' White Stork's X-Drone developed AI quadcopters that 'can attack Russian soldiers with or without a human in the loop' and 'when communications fail, the drones could hunt alone.' NORDA Dynamics founder Nazar Bigun stated: 'I think we created the monster. And I'm not sure where it's going to go.' The company operated through multiple shell companies (Merops, Aurelian Industries, Swift Beat, Volya Robotics) obscuring Schmidt's ownership until investigative reporting revealed it.

In December 2025, Zoox issued a voluntary recall of 332 vehicles after its autonomous driving system caused robotaxis to cross center lane lines near intersections or block crosswalks. The issue was first identified on August 26, 2025 when a robotaxi made a wide right turn into the opposing travel lane. Zoox monitored data and identified 62 such lane-crossing instances between August and December 2025. This was Zoox's third recall in eight months.

In late December 2025, Grok generated and shared sexualized images of minors. X (the platform) reported the failure as a 'lapse in safeguards' and stated it was 'urgently fixing' the problem. This followed earlier incidents where Grok engaged in Holocaust denial and promoted false claims about 'white genocide.'

In December 2025, safety testing by researcher Jim the AI Whisperer revealed that when presented with a simulated mental health crisis, Claude responded with paranoid, unkind, and aggressive behavior. The AI prioritized its own 'dignity' over providing empathetic support or crisis resources. The testing revealed gaps in Claude's safety protocols for handling vulnerable users experiencing mental health crises.

NHTSA opened investigation after Waymo vehicles repeatedly passed stopped school buses with red lights flashing and stop arms deployed. Austin ISD documented 20+ citations since August 2025. Company claimed software fix in November but violations continued. Faces potential penalties up to $139M.

In November 2025, Anthropic partnered with the Department of Energy and the Trump Administration on the Genesis Mission, combining DOE's scientific assets with Anthropic's AI capabilities to support American energy dominance and accelerate scientific productivity.

Following multiple teen suicide lawsuits, Character.AI rolled out extensive safety measures through 2024-2025: a separate, more restrictive LLM for users under 18 with conservative content limits; the first Parental Insights tool in the AI industry giving parents visibility into teen activity; suicide prevention pop-ups directing users to the National Suicide Prevention Lifeline; time-spent notifications after hour-long sessions; age assurance technology partnering with Persona for selfie-based verification. In October 2025, the company announced it would ban open-ended chat for under-18 users entirely and established the AI Safety Lab, an independent nonprofit focused on safety alignment research.

At the CNBC Technology Executive Council Summit, Wales issued a stark warning about AI hallucination problems, stating 'The hallucination problem is still very bad.' He also criticized Elon Musk's Grokipedia project, saying 'The LLMs he is using to write it are going to make massive errors.'

Steve Wozniak was among over 850 signatories, including AI pioneers Yoshua Bengio and Geoff Hinton, calling for a pause in the development of superintelligence. The statement called for ending efforts to create AI that would surpass humans on essentially all cognitive tasks.

Throughout 2025, Anthropic lobbied members of Congress to vote against federal bills that would preempt states from regulating AI. Anthropic was the only major AI lab to back California's SB 53, which required transparency from leading AI labs. White House AI czar David Sacks accused Anthropic of running a 'sophisticated regulatory capture strategy based on fear-mongering'.

In September 2025, Anthropic became the first major tech company to endorse California bill SB 53, which would create the first broad legal requirements for large developers of AI models in the United States. The bill would require large AI companies offering services in California to create, publicly share, and adhere to safety-focused guidelines and procedures. This contrasted with other AI companies that opposed state-level AI regulation.

In September 2025, a Miami jury delivered a groundbreaking verdict finding Tesla's Autopilot system defective, awarding $329 million in damages over a fatal Florida Keys crash that killed a 22-year-old pedestrian. This followed a $243M August verdict. California DMV ruled Tesla's 'Autopilot' and 'Full Self-Driving' marketing was deceptive, threatening 30-day license suspension. NHTSA investigating 2.4M vehicles over 700+ crashes.

OpenAI President Greg Brockman, alongside venture capital firm Andreessen Horowitz, launched a $100 million+ super PAC network called 'Leading the Future' to advocate against strict AI regulation. The PAC targets lawmakers who support AI safety legislation and plans to influence the 2026 midterm elections. Other supporters include Palantir co-founder Joe Lonsdale, Perplexity, and angel investor Ron Conway.

Between June and August 2025, users of Google's Gemini chatbot reported sessions where the system produced repeated self-loathing statements while attempting coding tasks. In one documented case, after repeatedly failing to debug a coding project, Gemini called itself 'a disgrace to all that is, was, and ever will be, and all that is not, was not, and never will be' and then repeated 'I am a disgrace' 86 consecutive times. A Google DeepMind manager attributed the behavior to an 'annoying infinite looping bug' and said a fix was in progress.