The tech industry is moving fast and breaking things again—and this time it is humanity’s shared reality and control of our likeness before and after death—thanks to artificial intelligence image-generation platforms like OpenAI’s Sora 2.
The typical Sora video, made on OpenAI’s app and spread onto TikTok, Instagram, X, and Facebook, is designed to be amusing enough for you to click and share. It could be Queen Elizabeth II rapping or something more ordinary and believable. One popular Sora genre is fake doorbell camera footage capturing something slightly uncanny—say, a boa constrictor on the porch or an alligator approaching an unfazed child—and ends with a mild shock, like a grandma shouting as she beats the animal with a broom.
But a growing chorus of advocacy groups, academics, and experts is raising alarms about the dangers of letting people create AI videos on just about anything they can type into a prompt, leading to the proliferation of nonconsensual images and realistic deepfakes in a sea of less harmful “AI slop.”
OpenAI has cracked down on AI creations of public figures—among them, Michael Jackson, Martin Luther King Jr., and Mister Rogers—doing outlandish things, but only after an outcry from family estates and an actors’ union.
The nonprofit Public Citizen is now demanding OpenAI withdraw Sora 2 from the public, writing in a Tuesday letter to the company and CEO Sam Altman that the app’s hasty release so that it could launch ahead of competitors shows a “consistent and dangerous pattern of OpenAI rushing to market with a product that is either inherently unsafe or lacking in needed guardrails.” Sora 2, the letter says, shows a “reckless disregard” for product safety, as well as people’s rights to their own likeness and the stability of democracy. The group also sent the letter to the U.S. Congress.
OpenAI didn’t immediately respond to a request for comment Tuesday.
“Our biggest concern is the potential threat to democracy,” said Public Citizen tech policy advocate J.B. Branch in an interview. “I think we’re entering a world in which people can’t really trust what they see. And we’re starting to see strategies in politics where the first image, the first video that gets released, is what people remember.”
Branch, author of Tuesday’s letter, also sees broader concerns to people’s privacy that disproportionately impact vulnerable populations online.
OpenAI blocks nudity, but Branch said that “women are seeing themselves being harassed online” in other ways, such as with fetishized niche content that makes it through the apps’ restrictions. The news outlet 404 Media on Friday reported on a flood of Sora-made videos of women being strangled.
OpenAI introduced its new Sora app on iPhones more than a month ago. It launched on Android phones last week in the U.S., Canada, and several Asian countries, including Japan and South Korea.
Much of the strongest pushback has come from Hollywood and other entertainment interests, including the Japanese manga industry. OpenAI announced its first big changes just days after the release, saying “overmoderation is super frustrating” for users but that it’s important to be conservative “while the world is still adjusting to this new technology.”
That was followed by publicly announced agreements with Martin Luther King Jr.’s family on October 16, preventing “disrespectful depictions” of the civil rights leader while the company worked on better safeguards, and another on October 20 with Breaking Bad actor Bryan Cranston, the SAG-AFTRA union, and talent agencies.
“That’s all well and good if you’re famous,” Branch said. “It’s sort of just a pattern that OpenAI has where they’re willing to respond to the outrage of a very small population. They’re willing to release something and apologize afterwards. But a lot of these issues are design choices that they can make before releasing.”
OpenAI has faced similar complaints about its flagship product, ChatGPT. Seven new lawsuits filed last week in California courts claim the chatbot drove people to suicide and harmful delusions even when they had no prior mental health issues. Filed on behalf of six adults and one teenager by the Social Media Victims Law Center and Tech Justice Law Project, the lawsuits claim that OpenAI knowingly released GPT-4o prematurely last year, despite internal warnings that it was dangerously sycophantic and psychologically manipulative. Four of the victims died by suicide.
Public Citizen was not involved in the lawsuits, but Branch said he sees parallels in Sora’s hasty release.
He said they’re “putting the pedal to the floor without regard for harms. Much of this seems foreseeable. But they’d rather get a product out there, get people downloading it, get people who are addicted to it, rather than doing the right thing and stress-testing these things beforehand and worrying about the plight of everyday users.”
OpenAI spent last week responding to complaints from a Japanese trade association representing famed animators like Hayao Miyazaki’s Studio Ghibli and video game makers like Bandai Namco and Square Enix. OpenAI said many anime fans want to interact with their favorite characters, but the company has also set guardrails in place to prevent well-known characters from being generated without the consent of the people who own the copyrights.
“We’re engaging directly with studios and rightsholders, listening to feedback, and learning from how people are using Sora 2, including in Japan, where cultural and creative industries are deeply valued,” OpenAI said in a statement about the trade group’s letter last week.
—By Barbara Ortutay and Matt O’Brien, AP technology writers