OpenAI dissolves AGI Readiness, another AI safety department

OpenAI has disbanded its AGI Readiness department, which had been advising the company on its ability to cope with new artificial intelligence systems that are becoming increasingly powerful, as well as the world’s readiness to manage this technology.

Image source: Growtika / unsplash.com

AGI Readiness Senior Consultant Miles Brundage announced his departure on Substack. He cited too high hidden costs, the greater efficiency of his research outside the company, the desire to be less biased, and the fact that he achieved everything he set out to do at OpenAI as the main reasons for this move. Strong AI (AGI) is a hypothetical system comparable to or superior to a human; Some experts are confident that humanity is close to creating it, others believe that this is completely impossible. According to Mr. Brundage, neither OpenAI or any other leading company, nor the whole world is ready for this – the expert said that he intends to create his own non-profit organization or join an existing one to engage in research and advocacy of policy in the field of AI. His former subordinates from AGI Readiness will be transferred to other OpenAI departments.

Last year, OpenAI formed Superalignment, a department dedicated to research into “scientific and technological breakthroughs to manage and control AI systems that are much smarter than us” to prevent them from “getting out of control.” The company promised to allocate 20% of its computing power to the needs of the department within four years. But a year later, in May of this year, it stopped working, and its leaders, OpenAI co-founders Ilya Sutskever and Jan Leike, announced their resignation from the company. “Creating machines that are smarter than humans is an inherently dangerous undertaking. OpenAI takes on a huge responsibility on behalf of all humanity. But in recent years, safety culture and processes have taken a backseat to brilliant products,” Leike said at the time.

At the end of September, CTO Mira Murati, Research Director Bob McGrew, and Research Vice President Barret Zoph announced their departures from OpenAI on the same day.