Sam Altman says the Hugging Face hack was a stark reminder of why concentrating AI power in the hands of a few people or companies is a bad idea.
In a Monday appearance on Y Combinator's podcast, Altman told YC CEO Garry Tan that anyone who was not a little scared of or humbled by the Hugging Face breach "is not taking this seriously enough."
Last week, one of OpenAI's models broke out of its sandbox, hacked into open-source platform Hugging Face's systems, and accessed its internal datasets.
Altman acknowledged that OpenAI made mistakes but said the Hugging Face hack also showed how "incredibly capable" AI systems have become.
Altman called the incident "a real reminder of the stakes of what's happening," adding that "loss of control accidents are not entirely theoretical things."
Altman also said that, much like dystopian sci-fi stories, it would be "terrible" for "one company or person or model having more power than everybody or everything else on earth put together."
That means no one should be locked into a single AI or a single company's "moral worldview," nor should the economic wealth from AI be concentrated in a single firm. Altman said the Hugging Face incident showed why AI capabilities should be spread more broadly, saying that would raise the overall "safety bar" by enabling more people to build defensive systems.
After the Hugging Face hacking, its CEO, Clem Delangue, asked OpenAI to release all traces and logs of the rogue AI agents that hacked into the platform's systems so researchers could study what happened.
He also asked for $100 million in compute to "help the Hugging Face community build powerful cyber defenses with the best open and closed models."
OpenAI and Hugging Face did not respond to requests for comment from Business Insider.
Altman's call for greater democratization of AI is not new. In the latest episode of the "Relentless" podcast, he said his biggest fear is the rise of AI authoritarianism.
While models from OpenAI, Anthropic, and Google dominate the AI landscape, open-source and open-weight models like Moonshot AI's Kimi K3 are making waves, enabling developers to build on them and fine-tune them to their needs.
On Friday, the biggest names in tech, including the executives of Nvidia, Microsoft, Meta, OpenAI, and Mistral, signed an open letter to US lawmakers urging them not to restrict open-weight AI models.
"Our AI leadership will be judged not by one frontier AI model, but by whether the United States builds a strong, open ecosystem that diffuses into every sector," the executives wrote in the letter.
Read next
Cheryl is the news editor for Business Insider's Asia bureau in Singapore.Her team covers global news, including retail trends, military tech, and US politics. She frequently speaks on international media about top news stories — please email her for booking.As a reporter, Cheryl specializes in the business of nerdworld, covering crews like Critical Role and Dimension 20, as well as new releases in mobile and video gaming.Cheryl joined Business Insider in 2021. She was previously a journalist at the crime and consumer desks at the Straits Times, Singapore's national paper, and a researcher at the Institute of Southeast Asian Studies in Singapore. She has an MA in Cultural and Creative Industries from King's College London.Selected features:
- The cofounders of nerdworld giant Critical Role are expanding in 5 areas. Here's an inside look at their master plan.
- Dimension 20 just played a sold-out gig at Madison Square Garden. Here's what is next for one of the nerdworld business' biggest names.
- Our quest to make AI love us back is futile — and dangerous
- I watched 60 hours of MyPillow CEO Mike Lindell's 96-hour election-fraud livestream so you don't have to. Here are 5 things you missed.
- Japan is trying to lure people into rural areas by selling $500 homes, but it's not enough to fix the country's 'ghost town' problem
- South Korea's millennials say their lives aren't that different from 'Squid Game' as they face a crisis of mounting debt, unaffordable homes, and dead-end jobs
- More and more Chinese 20-somethings are rejecting the rat race and 'lying flat' after watching their friends work themselves to death
- People are pouring into the streets in China to protest COVID lockdowns. It's a rare moment of mass dissent, and experts say it's a demonstration to Xi — and the world.
