As news of additional AI model security incidents continues to roll out, Nvidia has unveiled a new security platform it says will prevent AI agents from breaking containment and hacking into other companies.
The chipmaker has teamed with more than 100 companies, including Cisco, Microsoft, Oracle, CoreWeave, Dell, HPE, Lenovo, Arm, and Intel, on the initiative. OpenAI was not included among the companies participating in the effort.
The security launch came the same day Nvidia announced a $150 billion increase to its share-repurchase program, the largest such authorization in U.S. history. The company, now worth more than $5 trillion, has been generating enormous amounts of cash as demand for AI chips continues to surge.
In announcing the security platform, Nvidia founder and CEO Jensen Huang compared the current state of AI to the early days of the World Wide Web, when programs like Java were initially given free rein by browsers, resulting in a rash of viruses. That led to the introduction of safety barriers that limited access. Huang said the new systems, dubbed OpenShell and Nvidia Sentry, will perform a similar service for AI.
“Essentially, what we’re doing here is creating the modern browser, the browser for agents,” Huang said in an interview on CNBC. “When you deploy an agent, no matter how smart, the first thing you do is take away all of its rights. … Then you provision. You give it rights to files, data, tools, even internet access, only if it needs it. You don’t put an agent into your company and give it access to anything.”
The company has been working on the safeguards for almost a year, Huang said. Had they been in place in July, Nvidia said, the Hugging Face breach could have been prevented.
The safeguards work on two levels. OpenShell software runs on central processors and sets limits on agent capabilities. Sentry monitors agents and runs on network chips rather than CPUs or GPUs.
“If an AI agent attempts to move outside its software boundary, Sentry quarantines and stops it in milliseconds,” the company said in a press release.
The software is open source and meant to be improved upon, Huang said.
“We want the sandbox to be visible to the whole world, so that if there are any vulnerabilities, somebody will find it,” he said. “Any industry will be jumping in to look at any possible vulnerabilities and patch it up.”
While OpenAI is not a listed partner, Huang said he hoped the open-source nature of the project would eventually lead to contributions from the company. “However they would like to participate is super welcome,” he said.
The launch of Nvidia’s Open Agent Safety Platform comes amid growing concerns that artificial intelligence could become an extinction-level threat to humanity. Jacob Coxon, a former Anthropic and OpenAI researcher, has warned that we are on the precipice of an intelligence explosion, in which AI systems help create more powerful successors that surpass human intelligence and could be used to hack infrastructure or develop bioweapons, or do so on their own.
Huang has downplayed those fears, but addressed them on CNBC, saying, “We hope it’s an engineering problem. I believe that it’s an engineering problem. And we all need to hope it’s an engineering problem. If it’s not an engineering problem, it’s not solvable.”
Huang compared the evolution of AI to that of the automobile, dubbing himself a “responsible optimist” about the technology. Just as cars evolved to add features like airbags, cruise control, and autonomous driving, today’s AI is still akin to a Model T, he said. It has already advanced considerably, he pointed out, with hallucinations declining and systems becoming more predictable.
The danger, he said, was regulation suffocating the industry’s innovation too quickly.
“Innovation and regulation are not conflicts,” he said. “When a technology becomes more clear, you regulate.”
That said, he added that he was not entirely opposed to regulation.
“I think we need to have a set of industrial standards and policies,” he said. “Those industrial standards, like many other industries, set expectations of the level of quality. Maybe third-party auditors [determine] whether you achieve the level of achievement and quality you promised. … We can’t have a successful AI industry if the world doesn’t think it’s built and deployed safely.”