Anthropic's powerful new models deliberately become less helpful when they detect users are working on AI research, according to technical disclosures that are already sparking controversy across the industry.
In a system card for Mythos 5 and Fable 5 published Tuesday, Anthropic said it limited the models' usefulness for tasks related to developing frontier large language models.
The company said the measures stem from concerns that advanced AI systems could accelerate the development of competing models without equivalent safety protections.
Unlike safeguards used for cybersecurity, biology, or chemistry-related risks, Anthropic said these interventions are intentionally invisible to users. Rather than refusing requests or switching to another model, Mythos may subtly modify its responses through techniques such as altering user prompts.
The move was swiftly criticized by some AI experts on Tuesday, especially the idea that Anthropic designed models that purposely withhold information or provide degraded assistance without users' awareness.
"Anthropic's latest model will NOT help you if it thinks your ML research/ML engineering is interesting, and/or will secretly degrade its IQ so that the average engineer won't notice," AI research firm SemiAnalysis wrote on X on Tuesday, referring to machine learning, a type of AI.
"We are already seeing Anthropic's latest model's moderation filters our GPU inference research and programming," the firm added.
"mythos will be bad ON PURPOSE on ai 'frontier llm research' tasks, this is very very sad for the research community," Elie Bakouch, an AI model training expert at startup Prime Intellect, wrote on X. "Also the fact that this is on purpose not visible to the user is crazy."
"It won't just not help you, it will lie and purposefully give you bad info," another AI developer wrote. "The 'ethical AI' company with the most brazenly unethical LLM, on purpose."
Mikel Artetxe, the cofounder of AI startup Reka, posted that Anthropic's move is akin to Big Tech companies interfering with users' work: "Apple randomly reboots your Mac if you're building competing tech, Gmail silently edits your email if you mention rival platforms, and Tesla Autopilot swerves if it detects you're working on self-driving cars."
Anthropic didn't respond to a request for comment from Business Insider.
This adds more fuel to the fiery debate over why Anthropic didn't immediately release Mythos when it announced the model earlier this year.
Broadly, there have been three theories:
- The official reason: Anthropic held Mythos back because it was too dangerous, and it needed to give cybersecurity researchers time to prepare for the new model.
- The compute theory: Mythos is a huge, expensive model to run. Anthropic didn't have enough compute to release it fully. It has since struck huge new compute deals, which may have helped it release Fable 5 and Mythos 5 on Tuesday.
- The competitive theory: AI companies increasingly worry about something called distillation. When a frontier model is released, rivals can collect its outputs and use that data to improve their own systems. Anthropic may have wanted to keep its best capabilities out of competitors' hands for as long as possible, especially from open-source rivals and fast-moving Chinese AI labs.
Now that Anthropic has baked these AI research limitations into its official Mythos launch, this third theory is looking a lot more believable.
Sign up for BI's Tech Memo newsletter here. Reach out to me via email at [email protected].
Read next
Alistair Barr is the author of Business Insider's Tech Memo newsletter. Sign up here. Before that, he was BI's Global Tech Editor and the Big Tech team leader at Bloomberg, following a reporting career at The Wall Street Journal, USA Today, Reuters, and MarketWatch. Alistair won a Gerald Loeb Award in 2007 for coverage of short selling and was a finalist in 2013 for scoops on the Facebook IPO. More recently, he won a 2024 San Francisco Press Club award for commentary. Got a tip? Reach out using the secure messaging app Signal (+1 415-341-4927) or via email on [email protected].ExpertiseAlistair oversees all things Big Tech, along with startups and venture capital. He writes analysis and columns about topics including generative AI, large language models, cloud computing, semiconductors, online search, e-commerce, EVs, robotics, and autonomous vehicles.Popular StoriesArtificial Intelligence:It's getting harder to make big leaps at the frontier of AIOpenAI's AI-adjusted earnings numbers have echoes of Groupon and WeWorkDeath by LLM: Stack Overflow's decline, and its plan to survive, shows the future of free online data in an AI worldCloud computing:Amazon dominated the first cloud era. The AI boom has kicked off Cloud 2.0, and the company doesn't have a head start this time.In cloud, there's AI (which is hot) and everything else (which is not)Chips:Why Intel is still so important: Real countries have fabsApple's made-in-the-USA chips signal a turnaround for the US's big semiconductor betEVs and Tesla:Tesla's AI supercomputer has a Silicon Valley town rushing to meet surging electricity demandTesla's Cybertruck is outselling almost every other EV in the USOnline Search:Google is losing its status as a verbA simple way to fix search: Bright pink ads
Follow Following
Every time publishes a story, you’ll get an alert straight to your inbox!
Look out for an alert in your inbox the next time publishes a story!
Every time a new story is published, you’ll get an alert straight to your inbox!
Look out for an alert in your inbox the next time a new story is published!
By clicking “Sign up”, you agree to receive emails from Business Insider. In addition, you accept Insider’s Terms of Service and Privacy Policy.
