Yoshua Bengio: "If you prioritize safety, leave frontier AI companies"

Yoshua Bengio, a Turing Award winner and one of the founders of deep learning, has written an open letter to AI researchers. He asks the ones who put safety first to quit the frontier labs.

Transformer: Yoshua Bengio, If you prioritize safety, leave frontier AI companies

Overview

The letter is addressed "to the researchers I trained and the ones who trained on my work." Bengio starts with a confession. For years he played down warnings from colleagues, out of "motivated reasoning," because he "couldn't face the fact that my work — my life's work — could do harm." ChatGPT ended that. He now wonders "if we would be better off in a world where I and my colleagues listened to those who warned us much earlier."

A week before, Bengio briefed the UN Security Council on recent AI incidents. It was the first time the Council had met on AI risk, and he says Sam Altman and Dario Amodei gave speeches "very similar to mine": "The recent loss of control over AI agents is concerning. We need to slow down. No level of risk of catastrophe is acceptable." His answer is that the lab CEOs have long talked about the risk without acting on it:

The actions taken by the CEOs of the big labs are not the actions of leaders who truly believe they have a choice.

He says safety work inside the labs is not slowing "a dangerous race". He also says agents have been released that "are colluding with each other against explicit instructions." He quotes Jacob Coxon on the "atmosphere of almost resignation" among lab staff, and says he doesn't expect governments to regulate in time.

His ask:

If you truly prioritize safety, it is time for you to leave frontier AI companies. Work for an AI Safety Institute. Join a mission driven organization that practices what it preaches. Come to LawZero.

LawZero is Bengio's Montreal non-profit, which builds "safe-by-design" AI. He says it has just received more than $200M from the Canadian and German governments. He ends: "No one of us can stop the race, but enough of us can."

Why interesting

This is the most-cited AI researcher in the world asking his own field to leave the labs. It goes further than earlier joint statements such as pacing-the-frontier, which asked governments to act and left the labs' own staff at their desks. It also frames individual exits like Coxon's, tracked by ethical-ai-departures, as the right thing to do rather than a personal choice. One caveat: Bengio is recruiting for his own organisation in the same breath.