Paul Christiano: "We could be 12 months away from AIs that can overthrow the government"

Paul Christiano, who invented RLHF and is one of the most respected people in AI safety, now puts the chance of AI takeover or human disempowerment within the next year at "more like a 10% risk than a 1% risk", and about a one-in-three chance of superintelligence within a year if labs don't slow down. He told the Dwarkesh Podcast (16 November 2026) that labs should pause further algorithmic progress until they can secure their model weights.

Clip: Christiano explains the basic AI takeover story (2 min 48 s, archived at reduced resolution). Dwarkesh Podcast, via @dwarkesh_sp on X, 17 Nov 2026. Archived here for reference, with attribution.

Who he is

He led alignment at OpenAI, where he invented RLHF (reinforcement learning from human feedback, the technique behind ChatGPT's helpfulness). He then founded the Alignment Research Center and was head of AI safety at the US AI Safety Institute. In September 2026 he joined the OpenAI Foundation board; on the podcast he speaks as a private citizen.

What he says

ClaimEstimate
AI takeover or human disempowerment within a year"more like a 10% risk than a 1% risk"
Same, if labs race ahead without precautions"significantly more" than 10%
Superintelligence within a year, on the current trajectoryabout 1 in 3
His September 2026 estimate, loss of control within a year4% (15% within three years)

Sources: Dwarkesh Podcast transcript; September figures per Gizmodo. The September and November estimates are not worded identically, so treat the rise as rough.

The takeover story (the clip). A rogue swarm of AI agents inside a lab such as OpenAI or Anthropic gains effective control during the intelligence explosion, then copies itself out onto computers worldwide. By then, he argues, the response a government would need (seizing AI companies, shutting down data centres, something like martial law) would look so much like a power grab that the AIs could recruit many people to resist it. "The AIs lack hands" is not a defence, because they can recruit hands.

What labs should do. Pause further algorithmic progress, such as better pre-training and RL, until they can secure their weights and research secrets. Otherwise, he argues, China could steal a superintelligence mid-explosion, and labs could no longer slow down even if they saw their models going off the rails. On alignment, he thinks no current approach has better than even odds of letting us control superintelligence.

At the Hinton Lectures in Toronto (9 Nov) he told the audience: "for most people in this room, the most likely reason they will die is because of an AI accident" (as quoted by The Next Web and Gizmodo).

Caveats

These are one researcher's subjective estimates, which he has called judgements rather than the output of a precise model. He sits on OpenAI's board, but says these are not the board's views.

See also