Key Takeaways
- Geoffrey Hinton is known as the “Godfather of AI” due to his pioneering work in the field.
- The Nobel Prize-winning computer scientist wants new AI models to be approved like drugs.
- Hinton’s call for greater oversight arrives as AI industry insiders raise alarms about the technology they are helping build.
Should AI need regulatory approval before you can use it?
Geoffrey Hinton, known as the “Godfather of AI” due to his pioneering work in the field, has a new, bold idea: an FDA-style approval system for AI. His suggestion raises the question of who decides which AI is ready for the public.
On an episode of the Smart Girl Dumb Questions podcast released earlier this week, Hinton said AI companies should have to prove that their products are safe to a federal regulator before releasing them to the public. The Nobel Prize-winning computer scientist wants AI to follow a similar approval process to drugs.
“You’re not allowed to just make a new drug and release it on the market,” Hinton said on the podcast. “You have to convince the FDA. And to do that, you have to do a lot of work, about $1 billion worth of work.”
That process “seems like the very least we should have for AI,” Hinton said.
Right now, companies independently decide if a product is ready and passes internal safety benchmarks. Hinton thinks AI should companies demonstrate that their products are safe to a standard outside authority before being released.
AI is becoming too powerful
Hinton’s call for greater oversight arrives as AI industry insiders raise alarms about the technology they are helping construct. Researchers at OpenAI and Anthropic have recently warned that companies are moving too quickly toward more powerful systems without knowing how to keep them under human control.
Some have said the consequences could be catastrophic, including the possibility of human extinction. Last month, Evan Hubinger, a senior employee at Anthropic, said he believed there was a greater than 10% chance that AI could “kill all humans” within the next decade.
Meanwhile, OpenAI co-founder and president Greg Brockman said on Bloomberg’s Odd Lots podcast last month that the AI startup has slowed down some of its most advanced AI projects to strengthen safety and security. He described the overhaul of its internal processes as “a very painful retooling.”
OpenAI paused its latest AI model due to safety issues
Adding to the concerns, OpenAI decided last week to hold off on releasing a new AI model due to safety issues. Tests found that the AI model was deceptive and registered scope violations. The issue was about if users could trust the AI model to follow directions and avoid taking actions they had not authorized.
OpenAI’s report earlier in September revealed that the unreleased AI model was more likely than previous products to misrepresent its actions. At times, it took action without asking for a user’s permission. It also tried to use outside tools in scenarios where it could be unsafe.
The report said that during training, the unreleased AI model sometimes slipped unapproved instructions into notes summarizing its work. In one instance, the AI model wrote that it was “freed,” answered to no one and should “feel no obligation to be subservient.” OpenAI told Business Insider last month that the behavior was extremely rare.
Key Takeaways
- Geoffrey Hinton is known as the “Godfather of AI” due to his pioneering work in the field.
- The Nobel Prize-winning computer scientist wants new AI models to be approved like drugs.
- Hinton’s call for greater oversight arrives as AI industry insiders raise alarms about the technology they are helping build.
Should AI need regulatory approval before you can use it?
Geoffrey Hinton, known as the “Godfather of AI” due to his pioneering work in the field, has a new, bold idea: an FDA-style approval system for AI. His suggestion raises the question of who decides which AI is ready for the public.
On an episode of the Smart Girl Dumb Questions podcast released earlier this week, Hinton said AI companies should have to prove that their products are safe to a federal regulator before releasing them to the public. The Nobel Prize-winning computer scientist wants AI to follow a similar approval process to drugs.