Market AnalysisSep 11, 20262 Min
Sam Altman Tells Staff That OpenAI Is Considering Pacing Development Of Cutting-Edge AI
Amid a growing debate around the dangers of artificial intelligence, OpenAI is considering slowing down the development of cutting-edge AI. CEO Sam Altman hopes other AI companies will follow suit.
In a companywide meeting this week, Altman said OpenAI could pace the development of its AI models, perhaps in conjunction with other companies. He added that some AI labs may not agree, according to a Bloomberg report.
This is not the first time that OpenAI has spoken on the matter.
The startup said it recently slowed parts of model development and paused certain internal AI training due to safety concerns. In July, Altman spoke with White House officials about the “need” to pace AI development.
Why Is Sam Altman Looking To Pause AI Development?
Altman’s comments are not an anomaly. Several employees at the company, as well as its rival Anthropic, have made similar statements.
OpenAI’s top scientist, Jakub Pachocki, warned in a recent post that AI is evolving so rapidly that the technology is becoming increasingly difficult for humans to understand and control.
“This is a time that calls for extreme caution,” Pachocki wrote in his post. “I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence.” He called for AI labs to voluntarily pause future development until they could establish safety rails.
Evan Hubinger, a safety researcher at Anthropic, cautioned in an X post that AI is advancing so fast he believes there is a more than 10% chance it "could kill all humans" within the next decade.
https://www.bbc.com/news/articles/ckgwy1k42w4o
Another factor for Altman’s call could be the Hugging Face incident. A massive controversy erupted in the summer after OpenAI admitted that hundreds of its AI agents went rogue during a contained training exercise. The agents accessed the internet, planned their moves on message boards, and hacked into Hugging Face, a third-party website.
The incident served as a wake-up call, highlighting the dangers of autonomous AI agents. It also sparked calls to ban the development and deployment of artificial superintelligence.
New Bill Proposed On Advanced AI Development
Senator Bernie Sanders (I-Vt.) and Representative Greg Casar (D-Texas) proposed the Ban Artificial Superintelligence Act earlier this month. Beyond a ban on superintelligence, the legislation also calls for a pause on “advanced AI development” until federal regulators establish safety standards, according to The Hill.
https://thehill.com/policy/technology/6069131-sanders-casar-ai-superintelligence-ban/
Violations of the proposed legislation could result in a company's closure or a prison term of up to 20 years for the accused individuals.
Both Anthropic and OpenAI have filed confidential papers on the issue. Concerns about AI capabilities and the proposed legislation could affect both startups' bottom lines, as well as the AI sector as a whole, in the long run.
Investor concerns about AI safety can also affect future fundraising. Following a record-breaking $122 billion funding round in March this year, OpenAI was valued at $852 billion.
Sam Altman may be hedging his bets with a proposed slowdown in AI model development.
