A talking cat recently sat next to me on the couch, thanked me for interviewing his creator, and proceeded to tell me a series of dad jokes.
It was the work of a company called Pixi Platforms, which recently launched an iPhone and iPad app that can send the augmented reality equivalent of talking greeting cards to other users of iOS devices. Other featured characters include a talking robot who can play a mean game of tic-tac-toe and Pixi’s newest creation: an envelope that can follow a user across the room, moving its flap as it delivers a message.
Cofounder and CEO Mark Drummond, who previously worked for Apple and Siri namesake SRI International, has worked on augmented and virtual reality projects for headsets like the Apple Vision Pro. But he says he’s realized handheld devices can be just as good, if not better, for short and simple experiences. For one thing, they’re already in the hands of pretty much everyone, and for another, they can be viewed by more than one person at once.
“It’s like a magic looking glass,” Drummond says.
The iPhone still has enough input devices and smarts to let the character see and hear what’s going on around it. The cat, for instance, will react to dog barks or music played in its vicinity. And Drummond says having them actually appear to pay attention is critical to making them feel actually present in the room.
But, he emphasizes, audio and video data isn’t uploaded to the cloud. Instead, while AI frontier models are used by Pixi’s team of four to develop its app, lighter-weight AI running on the phone itself controls the actual characters, helping to safeguard user privacy.
“These devices, if you’re really engaging them in an AR sense, they’re sucking up a lot of very personal data,” he says. “And we just decided that should never leave the device.”
That’s likely to be a comfort to recipients of Pixi messages, which can be customized and sent through iMessage, which integrates closely with the company’s Pixi Garden app. The app must be installed to view an incoming Pixi character, with recipients prompted to install it when they get an iMessage text with one attached.
The privacy element will likely also be a comfort to the kind of brands Drummond and his colleagues hope to woo to the platform. He envisions the app as being a go-to platform for big IP owners who want to give users the ability to send each other Pixi messages featuring their characters. Those would be generally developed to assuage content owner concerns, Drummond says, so a cartoon character aimed at children wouldn’t be able to send vulgar messages or react to cursing in its presence, but an edgier comic book character might have looser limitations baked into its code and configuration.
“We’re positioning a character as software,” he says. “Our ultimate goal is to help bring those things to life in a brand-safe way.”
For early brand engagements, Drummond says the company would likely work with IP owners to develop the character’s in-app behavior. As the underlying technology continues to rapidly advance, the platform could be increasingly self-serve. In the not-too-distant future, end users might also be able to build custom characters, perhaps via plain language prompting similar to existing AI image and video creation tools. Already, content largely can be specified using a standard called Universal Screen Description (USD), and different characters can have the ability to sense and respond to particular stimuli, like the virtual cat’s acute awareness of barking dogs.
An upcoming character will likely be a version of Lewis Carroll’s Alice—based on public domain IP, rather than versions owned by Disney or other brands, Drummond says—that will be designed to have the family-friendly sensibilities of a Victorian schoolgirl. She’ll be a familiar character to users while also showcasing how Pixi creations can avoid engagement with off-color material, Drummond says.
“You put something obscene in front of Alice, she is blind to it,” he says.
The exact revenue model is yet to be determined, though it’s likely to depend on those brand deals, perhaps driven by around upcoming movies and TV shows. Drummond says investors include Andrew Ng’s AI Fund, Newmay Ventures, and the visual effects company Framestore, though he declined to specify how much the company has raised.
An Android app, plus the ability to send Pixi creations via other messaging apps, will likely follow in the future, though Drummond says the iPhone and iPad are well equipped for AR purposes, making them a natural first start.
“We know we live or die on iOS first,” he says.