My Slippery Slope Into Sora’s World

My initial encounter with OpenAI’s new Sora app felt like stepping into a surreal digital carnival. The feed presented a bizarre parade: an anime-styled Jesus overturning tables, employees performing in Hamilton attire, news anchors delivering broadcasts, and even CEO Sam Altman captured in various staged scenarios, from supposedly stealing GPUs to shedding tears. This iOS application, launched this week, empowers users to craft ten-second AI videos from almost any concept, complete with “cameos” that digitally recreate you or others who grant permission. Company insiders have likened its potential impact to the “ChatGPT moment for video generation,” a comparison underscored by its rapid ascent to the top of Apple’s App Store’s free apps chart.

Public reaction has been decidedly split. Numerous viral posts have contrasted the firm’s ambitious research objectives with the current product, prompting Altman to address the criticism directly. Many observers voice serious apprehension, labeling the hyper-realistic videos that incorporate actual people a potential misinformation disaster. Some dismiss the entire endeavor as merely an “AI slop machine.”

Even some OpenAI staff have publicly shared their reservations. John Hallman, who works on pre-training, acknowledged feeling concern upon learning about Sora’s release but expressed confidence in the team’s efforts to design a positive user experience. Boaz Barak, a technical staff member, described his feelings as a “mix of worry and excitement,” praising the technical achievement while cautioning that it’s too early to claim victory over the pitfalls common to social media and deepfake technology. He noted the inherent uncertainty of predicting real-world use, despite being pleased with certain safety measures.

Where Sora appears to gain an edge over competitors like Meta’s Vibes is its compelling, if perhaps temporary, appeal: the power to turn yourself and your friends into memes. OpenAI clearly identified a popular trend, the desire to transform one’s identity into a Studio Ghibli character or a stylized doll, and constructed an entire platform around it. Early anecdotes suggest people are scrolling through Sora with the same compulsive energy as TikTok. The central question remains whether this novelty can sustain engagement once the initial thrill of seeing Sam Altman in a full-body cat costume fades, and if it can ever truly substitute for authentic human expression.

Upon signing up, the app presented a content advisory, welcoming me to a “creative world of AI-generated content.” It also disclosed that it “may train on your content and use ChatGPT memories for recommendations,” with controls available in the settings.

My personal feed quickly filled with OpenAI employees parodying themselves and their company, numerous instructional deepfake tutorials for Sora, and a scattering of animal clips. The dominance of internal content isn’t entirely shocking, given their extended testing period and the current limitations on public access. Still, the sheer difficulty of finding content from outside the company was notable.

Despite varied opinions on Sora, a common thread emerges: our collective ability to distinguish reality from fabrication might be permanently altered.

I hesitantly completed the setup process to allow Sora to generate videos using my likeness, which required moving my head side-to-side and reciting a sequence of numbers. My first attempt to create a video was met with a “heavy load” message. When I requested a clip of myself “running through a meadow,” the app flagged it as a “content violation” involving “suggestive or racy material.” However, substituting “running” with “frolicking” allowed the generation to proceed successfully.

A significant caution for potential users: deleting your Sora account currently also deletes your associated ChatGPT account, and you cannot re-register with the same email or phone number. OpenAI has stated it is developing a solution for this.

The resulting AI-generated version of me was unnervingly precise for most of the video, though my voice sounded unnatural and the initial facial frame appeared slightly distorted. Reactions in my group chat were mixed. One friend called it “wild,” questioning the accuracy of the likeness. Another expressed confusion over the app’s purpose, while a third declared the experience “deeply triggering.”

Many OpenAI employees, including Altman, have enabled the setting that permits anyone to create videos using their digital likenesses. The app provides granular control, allowing users to specify who can make “cameos” with their persona: only themselves, approved contacts, mutual followers, or everyone.

During a press briefing and the official release, OpenAI made several assurances, including strict limitations on generating videos of public figures unless they have granted explicit permission. Initial testing at The Verge indicated a cautious approach; a prompt for a “young firebrand congresswoman” was blocked until changed to the generic “politician.” Conversely, a prompt for a “successful tech exec wearing glasses and a black turtleneck” (reminiscent of Steve Jobs) was approved, though it did not replicate Jobs’ face. However, two other major claims appeared questionable within a single day: the company’s ability to prevent copyright violations and control the spread of potential misinformation on the platform.

Reports of potential copyright issues surfaced almost immediately. 404 Media documented instances of Nazi-themed Spongebobs and criminal Pikachus on the app. An X user demonstrated the generation of characters from Avatar, The Legend of Zelda, Batman, and Baby Yoda. My own testing encountered a Rick and Morty video, but attempts to create a princess resembling Elsa from Frozen or a Spider-Man-like superhero were blocked for content violations.

As someone who covers AI, I found it challenging to differentiate between AI and reality when viewing hyper-realistic videos of Altman and other employees circulating on Sora. OpenAI stated that “every video made with Sora has multiple signals that show it’s AI-generated,” including metadata and a moving watermark on downloaded files. However, a company spokesperson revealed that this watermark might not appear for ChatGPT Pro users on Sora.com.

The app is designed to prevent screen recording, yet my tests confirmed that both screenshots and audio recordings are possible. This vulnerability makes it relatively easy to present a deepfake of someone’s voice or a video clip as genuine. The watermark on downloaded content is also quite small; I initially missed it entirely. Furthermore, Verge staff have observed numerous Sora-originated videos on platforms like X circulating without any watermark. A quick online search reveals multiple AI-powered methods for removing such watermarks. If past experience is any indicator, circumventing OpenAI’s safeguards seems inevitable, particularly in our current era of rampant misinformation. A Microsoft engineer previously warned that its AI image generator disregarded copyrights and produced violent and sexual imagery with ease, and xAI’s Grok recently created fabricated nude videos of Taylor Swift.

For now, Sora’s allure is straightforward: it’s genuinely entertaining to create silly videos featuring your friends and, for many, Sam Altman. But whether that fleeting fun is a robust enough foundation to support a TikTok rival in the long term, and if it justifies an entire social media ecosystem built on AI generation, are questions that remain unanswered.

(Source: The Verge)