Sora might have a 'pervert' problem on its hands

We need to talk about something uncomfortable when it comes to Sora 2 — OpenAI's new video-making app that I've truly enjoyed using.

Looking at some of the videos people have made — starring me — a little red light went off in my mind: This looks like fetish content.

Here's what happened: A few days into playing around with Sora, I noticed an unsettling video someone had made of me. I've allowed anyone to make "cameos" using my face. (You don't have to do this: You can choose settings that make your likeness private, or open to just your friends — but I figured, why not? And left my likeness open to everyone, just like Sam Altman.)

I found a stranger had made a video where I appeared pregnant. A quick look at the user's profile, and I saw that this person's entire Sora profile was made up of this genre — video after video of women with big, pregnant bellies. I recognized immediately what this was: fetish content.

Like I said, I'd been having a lot of fun on Sora, letting friends and even strangers make silly videos with my face. But the idea that someone was making a video that had some potentnial sexual gratification element made me feel fairly icked out.

Over the next week or so, I noticed a few more videos that I immediately recognized from years of poking around the underbelly of the internet as niche fetishes: belly inflation videos where my stomach inflated massively, Violet Beaugregard-style, or giantess videos where I and two other women towered over a cityscape (and another where we were miniature). Some of these accounts make these kinds of videos with purely AI-generated women, not cameos of real people — I found a bunch of accounts like this dedicated to making vore, foot worship, crushing, and other niche, non-nude fetishes.

A significant portion of the videos people made with my face are fetish content

At first, it was just one or two videos. But it picked up steam. In the last week or so, 10 out of the 25 most popular cameos using my face are various fetishes, including one where I'm a centaur-woman pregant with octoplets. It's not just me, either. I've seen this kind of content made with cameos of other women: female creators, another woman tech reporter, and a female employee of a prominent venture-capital firm.

Of the recent "cameos" people made of me in Sora, a large portion are fetish videos of belly inflation, pregnancy, cloning, or centaurs. Sora

As a journalist who has covered internet culture for a long time, I've seen enough hand-drawn images of Sonic the Hedgehog showing feet and pregnant Barack Obama over the years not to be shocked or to yuck someone's yum. I've also experienced enough online harassment that I have a thick skin about this stuff. But there is something different and unsettling here: It's people being able to use my face (easily) to create content for potential sexual gratification without my consent. And if I, an internet-poisoned person, feel weird about this, I imagine a normal person really would hate it.

Copyright and celebrity likeness issues on Sora

OpenAI is clearly still working out the kinks (no pun intended) with its AI video app Sora 2. Launched only this month, it's still invite-only but has topped the App Store charts. The obvious initial concerns were around misinformation and potentially copyright-infringing content. The first few days of Sora were rife with non-sanctioned versions of Pokémon, "Family Guy" or Nintendo characters.

OpenAI didn't respond to my multiple requests for comment for this story.

Sora doesn't allow you to make videos with the likenesses of living celebrities, although it does allow for dead celebrities. This led to a massive wave of videos starring Bob Ross, Stephen Hawking, and Martin Luther King, Jr. After complaints from the King family, OpenAI announced it was "pausing" use of the likeness of Dr. King.

This is part of an overall pattern of how OpenAI has approached he concept of copyright and intellectual property: asking forgiveness, not permission.

Whether that pattern extends to how non-nude, quasi-sexual content is being made on Sora will be another question.

Sora allowed kids to make videos of themselves with a porn star

Apart from the pregnancy and inflation fetishes, there was some other troubling content I saw. A popular female porn star is on Sora, and I saw a handful of videos of young men who looked to be under 18 making videos of themselves or their friends with her. One video was of her and another boy lying in bed together (fully clothed). He appeared to be maybe 11 years old.

I came across other gray-area videos, like an adult man who had made about 20 videos with variations of a prompt asking for a preteen girl in a grass skirt, dancing.

OpenAI has announced new parental controls for teen accounts, which would potentially limit how adults and teens could interact on something like Sora. It's unclear what its rules around something like this are — is it OK for teenage accounts to interact with porn stars on Sora? Can adults create cameos of kids? I'm not sure!

There's really two separate issues at hand: Should users be allowed to make fetish content of any woman who is stupid enough (like me) to allow anyone to make cameos of her? And how do you stop people from making fetish content of purely AI-generated characters that aren't cameos of real people? Does OpenAI want to stop that? Maybe OpenAI thinks it's fine for people to make belly-flation or foot-fetish videos as long as they're not of a real person.

For now, I keep going back to a thought I had early on while scrolling Sora: There's hardly any women on here, and it's no wonder why. Women innately undertsand the risk of letting anyone make vidoes with their faces — the likihood of something being creepy is extremely high. These fetish videos are kind of goofy — I have to admit I even cracked up at a little at the centaur one — but overall, it's an icky and somewhat menacing feeling seeing a lot of them.

Although this genre of content has always existed online, it's never been so easy to make it using a real person's face. OpenAI has anticipated the most obvious content violations with nudity or overtly sexual content, but it might not have anticipated the wave of niche fetish videos now flooding the app.