That's how Demis Hassabis sees it. On Tuesday, the head of Google DeepMind published a manifesto. AGI, in his estimate, is a few years away — and the risks to humanity, he now says, are no longer hypothetical.
AI-assisted cyberattacks have already happened. In June, Amazon pushed Fable 5 to probe third-party code for vulnerabilities, and in one case the model wrote a working exploit. For Hassabis, these are warning shots. Within eighteen months, he predicts, assistance with bio- and nuclear weapons could be baked into open models that no government can recall.
His answer is an organization modeled on FINRA — the stock market regulator the industry funds itself while the government oversees. It would test frontier models for cyber threats, biorisks, and deceptive behavior. First voluntarily: a lab submits its model 30 days before release. Then mandatory: no frontier model — from any country, open or closed — enters the US market without clearance.
The industry responded with rare unanimity. Musk called the framework "a good starting point for discussion", Nadella wrote that the goal is for "no model release to break the world", Anthropic co-founder Jack Clark called the document "excellent". Altman, Dorsey, and Ashton Kutcher chimed in with approvals under the post. Zuckerberg, per Axios, is preparing his own manifesto.
When competitors who normally sue each other and poach each other's engineers are all cheering the same new rules, ask yourself who those rules actually benefit. The big labs have the lawyers and Washington connections to sail through any certification process. For startups and open source, the bar just got higher. Critics are calling it regulatory capture.
And there's a second hole. The review only gates access to the US market. A Chinese model that doesn't need that market isn't slowed by Hassabis's plan at all. He himself calls the race "extremely intense and multi-layered" and calls for international coordination — but his framework has no enforcement mechanism.
Hassabis wants the organization up and running by year-end, which sounds ambitious after watching a single export control take three weeks of negotiations to clear. The way things look now: the committee covers the US market, and humanity is next on the agenda.