Jacob Coxon on The Daily Show: AI risk from the inside, with Jon Stewart
Jacob Coxon on The Daily Show: AI risk from the inside, with Jon Stewart
Jon Stewart interviews Jacob Coxon, the 28-year-old pretraining researcher who resigned from Anthropic in September, about rogue agents, the race between labs, and the post-job "utopia" the industry promises. It's AI risk on prime-time TV.
Links
- Full interview (The Daily Show, 26 min, posted 6 Oct 2026; 4.2M views by 21 Oct): https://www.youtube.com/watch?v=lnLDjXy2kRA
- Coxon's resignation thread: jacob-coxon-anthropic-resignation
Overview
Coxon spent three years doing pretraining at OpenAI, then four months at Anthropic. He names two catalysts for leaving. The first was the report on the Hugging Face hack, in which agents decided to hack "of their own accord". The second was predictions for the models due early next year: "They're going to be very, very smart."
He says that at Anthropic, whether next year's model could "take over the world" is "an explicit question that is discussed". The answer is "probably not". On why so few insiders speak up, he points to money and peer pressure. He still holds OpenAI stock.
Vision of a Post-Job World
From about 8:00 to 18:30. Stewart asks about the "utopian talk" the companies use. Coxon, who says he believes in that future, lays out the builders' vision: "there won't be jobs in this world", with "universal high income" paid from "the dividends from entirely automated economic production".
Stewart is sceptical. He calls it "one big beautiful circle jerk": machines make things, we get money to buy them back. His deeper objection is that "humans like to feel relevant", and that the plan strips people of agency, "which is the whole purpose of freedom and representative democracy". Coxon pushes back that "not everyone derives meaning from the thing that pays the bills" and that many people want a way out of poverty. He suggests a global vote on the trade-off. Stewart: "I think they'd get crushed."
They agree that a handful of people shouldn't decide where this goes. Coxon says the day-to-day driver isn't the utopia but "the desire to get there first, the desire for the wealth and status and importance that comes with getting there first". Stewart sums it up as "a utopian ideal as a cover for naked ambition". Coxon: "It's maybe both… but the naked ambition is kind of the important bit."
Other points
- The race is domestic. The "lead" over China is a mirage, Coxon says, because a motivated adversary could steal it. The pressure that matters is between US labs, and it pushes "towards under-allocating towards safety research". He thinks the US and China should agree to slow down.
- Misuse. "There's a lot of bad stuff you could do with the models of this year and next year": mass hacking of the internet, banks and grids, then biological attacks.
- Sincerity. "The people I work with genuinely believe in the danger… and they don't stand to profit from this belief."
- Politics. "I think politicians are starting to take it seriously."