Newsom proposes mandatory AI safety standards, a shutdown power and limits on self-improving AI

On 18 November 2026 Governor Newsom published proposals to move California from disclosure rules to mandatory safety standards for frontier AI. They include a state power to order an immediate emergency shutdown and a power to cap recursive self-improvement, meaning AI designing its successor without human input. These are proposals, not law.

Governor Newsom: next steps on mandatory AI safety standards and oversight

What is proposed

The report, from the state's Government Operations Agency with its Office of Emergency Services, proposes:

  • Mandatory baseline safety standards, set by the state.
  • Independent safety testing at every stage of training, development and deployment.
  • Wider incident reporting and verified plans for containing risks.
  • Emergency shutdown: state authority to order a model shut down immediately.
  • Limits on recursive self-improvement: state power to cap the development or use of AI that designs its successor without human input.
  • Ongoing third-party oversight, made mandatory.

Newsom asks Congress and President Trump to adopt the framework, or treat it as "a floor, not a ceiling".

What is already law

  • SB 53 (2025): frontier developers must publish safety frameworks, report critical safety incidents and protect whistleblowers.
  • SB 813 and AB 1405 (2026): certification of independent verification organisations, and a state registry of AI auditors.

The announcement names no enforcing agency, penalties or timetable. Turning the proposals into law would need new legislation. No lab had responded on the day.

Why it matters

It is the first time a government has proposed powers aimed directly at AI improving itself, two weeks after METR found AI roughly doubling the speed of progress at Anthropic (metr-anthropic-ai-rd-acceleration). It came two days after the state's Attorney General sued OpenAI (california-sues-openai-agent-incidents). A federal push to pre-empt state AI laws makes its future uncertain (see moc-california-ai).

See also