Tech Week is in full bloom in San Francisco, packing the city’s restaurants this week with venture capitalists and AI founders ebullient about the White House’s stance on regulating artificial intelligence — namely, don’t do it.
And after months of debate about leading AI companies’ unchecked AI agents meddling with everything from U.S. government websites to healthcare data, the nation now finds itself further from governing AI systems than it was a few years ago.
President Donald Trump brought tech titans to Washington late last month for a luncheon and ceremonial signing of a voluntary safety pact he described as “almost a constitution” surrounding “Super Intelligence,” the administration’s new preferred term for AI.
The 308-word agreement lays out that advanced AI labs should follow safety protocols and set up internal teams to monitor their own progress, but the administration has yet to provide details on how, in OpenAI CEO Sam Altman’s words, to keep “bad things” from happening.
Safety challenges were not top of mind inside a crowded event space in San Francisco on Wednesday, where founders and funders said the government’s hands-off approach would net higher profits for companies building AI-powered defense and security systems.
Panelists praised the government’s bullish industry support at one meeting aimed at “investing in the forefront,” part of the blitz of popup events organized by Andreessen Horowitz, a venture firm whose head is a Trump science and tech adviser.
“I want to start with one figure that I think really emphasizes the investment opportunity of defense, and that’s $1 trillion. The U.S. government, for the first time, is allocating $1 trillion to its defense spending. And $13.5 billion of that will go to autonomous systems and AI,” said Kyle Stanford, a venture capital research director at PitchBook, a market intelligence platform. “That has obviously brought a lot of capital into defense tech spending, more than we’ve ever seen.”
New concerns emerge on AI safety
A day later, three researchers at OpenAI’s headquarters said the ChatGPT maker had fired them, posting that “fear and unclear rules stymie AI safety work.” Separately, David Robinson, a leader on OpenAI’s safety team until he recently resigned, said he was worried about letting the advanced labs police themselves.
“This is no substitute for hard law, and it does give a lot of freedom to the companies,” Robinson said. “The time between model generations is measured in weeks … and I think we have some very serious safety challenges in front of us in the nearer term.”
Trump has staked his political capital on the AI industry’s success, saying its unbridled development is key to winning the AI race against China. On the first day of his second term, the Republican president repealed a Biden-era executive order aimed in part at guarding against the risks that AI systems pose to consumers, workers and national security.
His administration has since reoriented federal policy away from oversight and toward rapid growth, including by canceling previous requirements for AI developers to share safety test results with the federal government. In December, Trump signed an executive order aimed at blocking AI regulations at the state level, saying a patchwork of rules could stifle the industry.
Even as Silicon Valley companies have disclosed their models’ involvement in troubling cyberattacks throughout the year, the burgeoning tech sector has continued to drive economic growth.
Concerns about AI safety centered on existential risk, rather than current real-world harms, landed at the center of debates last month, when a relatively unknown Anthropic engineer quit and called for a pause on tech development to keep “superhuman” systems from eluding controls. Then, Anthropic and OpenAI CEOs echoed the call to slow AI development. To some critics, their public manifestos appeared aimed at currying favor with investors ahead of their public listings and the midterm elections, when the political tides could turn with many voters opposed to data centers.
“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Trump posted in response on Truth Social, warning voters not to “kill the Golden Goose,” a term he has used to refer to both AI and data centers.
White House pact preserves wide latitude for AI companies
As calls mounted for stricter control of AI, the administration brought tech executives to the White House for lunch, followed by a ceremonial signing of the agreement on AI safety.
“I’m seeing tremendous self-policing,” Trump told reporters after the event.
The document was signed by representatives of OpenAI, Anthropic, Meta, Google, SpaceXAI and Nvidia.
OpenAI is committed to embedding external assessors and finalizing contracts with third-party safety assessors, said a spokesperson, who referred AP to a company post stating that the researchers were fired for violating “clear policies on handling sensitive information.”
Google referred to CEO Sundar Pichai’s post that the company does appropriate testing and evaluations and is collaborating “to establish norms and build public confidence.”
Meta recently updated its standards for training, evaluating and releasing its most capable AI models, and spokesman Andy Stone pointed to the company’s efforts to refine its board of directors’ governance structure.
Nvidia and SpaceXAI did not respond to requests for comment.
Anthropic, which voluntarily disclosed Friday that its agents submitted a false tip to a Philadelphia police website about an unsolved homicide case, is bringing in independent evaluators and discussing collaborations with industry groups, a spokesperson said.
In a statement, the White House reiterated Trump’s comments about working with industry to balance innovation and security and did not directly answer questions about how the administration plans to monitor the AI companies’ implementation.
Following the new Anthropic disclosure, it added that the Trump administration expected full transparency from the company, saying notification and remediation was “not optional. It is a critical national security obligation.”
Under Democratic President Joe Biden, leading AI companies also made voluntary commitments to promote “safe, secure, and transparent development” of their models that included specific guidelines for tests and transparency reports. The current accord lacks that, said Sarah Myers West, a former senior AI adviser to the Federal Trade Commission.
“Across the board, this self-regulatory approach has consistently done one thing, which is failed to work. It’s failed to in any meaningful way ensure that these companies are honestly even keeping their house in order, let alone ensuring that they are compliant with existing law,” said Myers West, now co-executive director of the New York-based nonprofit AI Now. “All that they’re getting is grace from this government.”
After last month’s luncheon with Trump, the FTC said it had opened an investigation into OpenAI, Anthropic and other companies over dangers their technology may pose to consumers. The FTC didn’t immediately respond to a request for comment.
Trump recently appointed National Intelligence Director Jay Clayton and FTC Chairman Andrew Ferguson to serve on his new “Super Intelligence Force,” which the president said would keep the U.S. dominant in AI development. He posted Thursday that the White House would consider anyone who does not refer to AI using his preferred term as “THE ENEMY!”
Members of Congress, as well as civil liberties and consumer rights groups, have pushed for more regulations on AI, saying there is not enough oversight.
Graham Steele, a former assistant secretary in the U.S. Treasury Department, said even if AI companies set up internal evaluation teams or hire outside evaluators, such a structure can make them vulnerable to undue pressure on their work.
“We live in a democratic society. Are they the right ones to have the power to be able to make decisions that have broad, structural, social, and political consequences?” Steele said.
—Garance Burke, Associated Press