As AI porn generators get better, the stakes get higher

https://techcrunch.com/2023/07/21/as-ai-porn-generators-get-better-the-stakes-raise/

by rntn • 3 years ago

91 116 3 years ago

rcoveson2• 3 years ago

Can we please just skip to 2-5 years from now when image/video generation is so good that it's as easy to fake it as it is to lie in textual form?

Most important information is textual, and there's really nothing at all that binds text to reality. So we have laws about fraud and libel for important matters. Trust is a hard problem! But we're working on it, and images were never much of a solution to it anyway.

Most of the time, we just don't care. Many stories are untrue; explicitly fictional. Some of them are untrue in ways that could emotionally damage others (gossip, insults). That's just a part of social life, to be mitigated socially, not judicially.

Images are going to be just like text. Let's all just embrace that already. You can't lie any more or less with them than you can text, once we all (inevitably) stop trusting them as though they were difficult to fabricate. There's a little turmoil coming, but there's less if we all just decide to get over it now rather than cling desperately to the idea that images are somehow more deeply intertwined with the real world than text is.

enumjorge• 3 years ago

I get what you're saying, and I do think we should start preparing for a future where images are not something we can trust because I think we'll inevitably get there, but images are different than text. They are information dense and can elicit an emotional reaction much faster than text can. Think of the fake images that showed Trump being violently arrested. Something like that can become viral much more easily than text.

As for a world where we all learn to distrust images, I don't think we will fully get there. You still have people falling for email scams that by now are very clearly documented. Even a small percentage of people who are living in a different reality can be problematic for the rest of us.

mitthrowaway23• 3 years ago

A photo of Trump being violently arrested provokes a reaction because we assume it captures a real scene. A painting or a political cartoon showing the same scene would not, even if it were quite realistically painted. Before the camera was invented, paintings and engravings like that were the best anyone could do to transmit a scene, and now that AI can generate fake photorealistic images, photos will gradually have to be seen the same as a skilled painting. It might capture reality, or it might not.

majormajor6• 3 years ago

Trends that will mitigate harassment done with stuff like this are already underway, but they're mostly ones that run contrary to the original spirit of the internet:

* don't accept unsolicited messages or images from people you don't already know

* don't leave your corporate-moderated walled garden of choice to search for stuff on the open internet

* don't put too trust much into any information distributed primarily online

I think these things will all be necessary but they are far more dystopian than the internet that I once hoped for.

dtech• 3 years ago

I wonder how much of that is innate and how much of that is because we're not used to fake pictures on a scale AI will allow yet

mgkimsal• 3 years ago

and I do think we should start preparing for a future where images are not something we can trust

I'm not sure we can entirely 'trust' images/photos anyway right now. Even a 'genuine/undoctored' image, paired with partial/misleading info or outright lies has an impact on what we take from an image anyway.

tennisflyi• 3 years ago

Pretty sure those 2-5 years will never be realized due to Zeno’s paradox.

IanCal2• 3 years ago

Images are going to be just like text. Let's all just embrace that already. You can't lie any more or less with them than you can text, once we all (inevitably) stop trusting them as though they were difficult to fabricate

People trust text and that's clearly written by someone not a snapshot of reality. There's a reason libel is a thing.

zb3• 3 years ago

I don't see a problem with them. Can't we focus on more important (I mean actual) problems?

majormajor6• 3 years ago

Let's say your boss, or spouse, received fake photorealistic pornographic images of you with your boss's spouse...

Why are you expressing annoyance about people focusing on the "unimportant" problem of than dealing with the fallout of "AI fake porn generators" than about other people causing these problems in the first place by focusing on the problem of generating faked porn. There's no real importance to generating this stuff.

Enginerrrd• 3 years ago

Honestly, I think such a scenario will only be relevant for a few more years. After which, photorealistic deepfakes will be so easy to produce reliably that they simply won't have the credibility.

I don't know what happens to journalism and misinformation at that point, but it's not good. State level actors have been doing this already. India just recently got caught putting cheeky smiles on the faces of protestors after being brutalized by police to suggest they were faking their distress for their cause.

hateful• 3 years ago

I think the idea here is that he should treat that image or video the same as an email or text message describing it. It gets real interesting when you're doing a real-time deep fake. What even is reality anymore?

jcadam2• 3 years ago

That's it, I'm getting a UUID tattooed onto my backside.

mitthrowaway23• 3 years ago

Let's say your boss, or spouse, received fake photorealistic pornographic images of you with your boss's spouse...

If they're naive enough to actually believe that image, you can just reply with one showing them in bed with your spouse. That should settle the matter.

spywaregorilla3• 3 years ago

Let's say your boss, or spouse, received fake photorealistic pornographic images of you with your boss's spouse...

... in a world where it is understood that fake realistic images are easy to be made then this is not any more credible than a text without pictures.

pessimizer9• 3 years ago

Let's say your boss, or spouse, received fake photorealistic pornographic images of you with your boss's spouse...

Let's say your boss received an anonymous email accusing you of being a pedophile. What's the difference here in the world where a deepfake is no harder to make than an anonymous email?

An anonymous email will probably be a hell of a lot harder to accomplish than a deepfake, actually.

hennell• 3 years ago

There are actual problems, they're mentioned in the article. Just because it doesn't affect you, doesn't mean it's not a problem or important to others.

mc322• 3 years ago

If this obviates the need for the exploitative sex video industry so be it. The sooner it’s relegated to history the better.

If people never the less want to be exhibitionists and do it, that’s fine.

Now, what’s true for real life should be true for synthetic productions. Depictions in real life that would be illegal should remain illegal in the synthetic world. Though I wonder if Japan will insist on pixelation of the groin.

throwaway77897• 3 years ago

Depictions in real life that would be illegal should remain illegal in the synthetic world.

Why? How is it any of your/our business what porn people generate and masturbate to in private?

lelanthran• 3 years ago

Depictions in real life that would be illegal should remain illegal in the synthetic world.

What's your reasoning for this? Murder is illegal, yet depicted all the time in movies.

So is robbery, rape, any number of illegal activities. How do you propose to convince a population to make all those depictions illegal?

Or is it only sex that you have a hangup with?

tetrep• 3 years ago

Depictions in real life that would be illegal should remain illegal in the synthetic world.

I think creative works would be severely limited if nothing illegal could be depicted within them. If you're arguing only pornographic content should carry this restriction, I'd be interested as to why the same crime depicted in a non-pornographic film is better (in the sense that you think it should be allowed / not made illegal) than the same crime depicted in a pornographic film.

Even restricted to only pornography, I don't see why I (as a hypothetical creative) should not be allowed to include a description of something illegal within the context of a pornographic film. i.e. I think it should be (legally speaking) fine for me to make a hot a steamy porno that also includes, as a core part of the porno, a scathing critique of my local town's property tax implementation while a bank robbery is ongoing. If it matters, let's say the main characters / people having sex are also doing the bank robbing.

If you're trying to restrict what crimes are illegal to depict (rather than all crimes), then I'm not sure what metric you're using for that. Is it only explicitly sexual crimes that should not be in sexual films? Because there's plenty of crimes that are not intrinsicly sexual but that can be sexual because of the observer, e.g. Masochists can be aroused by illegal non-sexual physical abuse, would it be okay to have that illegal non-sexual physical abuse in an otherwise explicitly sexual pornographic film?

If you're concerned about people "getting off" to crimes, I don't think your restrictions are going to mitigate that at all. People are aroused by the abuse in Uncle Tom's Cabin[0]. If you're trying to prevent people from being aroused by "bad" things, I don't think you'll see much success. Hell is other people, after all.

[0]: I'm reading "The Masochistic Pleasures of Sentimental Literature" by Marianne Noble and it references a letter that's relevant here: "Even in my early childhood I loved to revel in ideas about the absolute master of one man over others. The thought of slavery had something exciting in it for me, alike whether from the standpoint of master or servant. That one man could possess, sell or whip another caused me intense excitement; and in reading 'Uncle Tom's Cabin' (which I read at about the beginning of puberty) I had erections."

Same letter, different reference, is quoted on page 65 here of "The Erotics of History" by Donald L. Donham: https://www.jstor.org/stable/10.1525/j.ctt2204r8x.12

Tao33003• 3 years ago

a scathing critique of my local town's property tax implementation

Woof. Is it hot in here??

mc322• 3 years ago

If people want to get aroused by looking at plain old rocks and then act that out on the rocks, well, I’ll take that deal.

Animats• 3 years ago

Pornhub has an AI R&D operation. So far, they've mostly been restoring really old porno films from 1900-1920. Presumably they have work underway on generating content directly. As they point out, they have plenty of training data.

hef19898• 3 years ago

It is always amazing how much technological impact the porn industry has. Media standards, streaming, filming, you name it. But then one of the first large internet use cases was porn.

130e13a• 3 years ago

it still is one of the largest internet use cases today, iirc ~30% of all web traffic is porn.

Jeff_Brown• 3 years ago

One of the first uses of film, too. In fact one of the earliest uses of writing -- mosr Egyptian writing fragments are either receipts or (text) porn.

pessimizer9• 3 years ago

As far as I can tell, the only thing in danger is employment in the porn industry. The only porn stars left will be the ones that are really just like any other eCeleb living from panhandling from the parasocial.

The "deepfake" armageddon that's been blathered on and on about is obviously horseshit. If people can make a deepfake of anybody in five minutes, nobody is going to get fired (without getting the shit sued out of them) for deepfakes of them showing up. Also no one is going to believe in the pictures, even if they are real. This is a blessing to the future victims of revenge porn, one that people whose photographs were spread around before the dawn of generative AI won't be treated to.

This has always been a media panic to lay the groundwork for more censorship and more surveillance. We'll have to be licensed to run an algorithm if they have anything to do with it, and our models will require a Microsoft signing key.

resolutebat• 3 years ago

The friendly article describes a case where a small-town school teacher was, in fact, fired because of deepfaked images.

CamperBob23• 3 years ago

And now imagine a world where deepfake porn is ubiquitous. Does the teacher still get fired?

CamperBob23• 3 years ago

If people can make a deepfake of anybody in five minutes, nobody is going to get fired (without getting the shit sued out of them) for deepfakes of them showing up.

Exactly. At some point people will just stop caring, which is the best of all possible outcomes.

jrflowers3• 3 years ago

This is a good point. People are quick to worry about the real people that have been victims of this stuff and the people that will certainly be victimized by this stuff between now and an indeterminate point in the future, but we should really be thinking about the theoretical people that won’t be harmed in that possible future.

anigbrowl7• 3 years ago

This is beyond naïve. People get worked up about rumors of litter boxes in schools for students who are also furries: https://www.edweek.org/leadership/litter-boxes-in-schools-ho...

It doesn't matter whether it's true or not. Many people will act as if it's true, either because they are fools or frauds. IF they don't get the result they want they'll claim the truth is being suppressed and demand inquiries and investigations. Regardless of the facts, such false claims will waste resources and cause personal reputational damage.

We live in a country where weather forecasters get death threats for mentioning climate change, hospitals get bomb threats for treating kids with gender dysphoria, and librarians are condemned as agents of Satan. Relying on collective rationality and technical awareness is foolish.

https://www.theguardian.com/us-news/2023/jul/16/chris-glonin...

https://www.cnn.com/2022/09/15/us/boston-hospital-bomb-threa...

https://www.latimes.com/politics/story/2023-01-27/school-lib...

abeppu2• 3 years ago

nobody is going to get fired (without getting the shit sued out of them) for deepfakes of them showing up

The WaPo article cited that mentions the school teacher who was fired does _not_ mention any litigation. It seems like a failure to engage with the material if you're saying "no one is going to be impacted by humiliating deepfakes" in reporting that specifically mentions someone who was. Even if there was litigation, that puts a cost on the former employee which they may not be able to easily afford, and my understanding is that very specific criteria must be shown for most dismissed employees to get anything.

pessimizer9• 3 years ago

The schoolteacher who was fired was fired in the past. I'm talking about the future. If I can make deepfakes of every teacher in the school, good luck firing one of them.

-----

edit:

I'll put it up as a service; the protection of women from misogynist school administrations.

We've advanced beyond when women would be fired from teaching for getting married or becoming pregnant, but not much.

nickthegreek• 3 years ago

The pornstar of the future will just be a body with celeb face swap. Waiting to see the how the legal shoe drops on this growing industry.

npteljes5• 3 years ago

I feel the same. Tattoos, I think, have gone though a cultural perception change similar to what to deepfakes will happen. I don't expect deepfakes to be as accepted as tattoos are, but they won't pose a problem as many people will understand they are fake.

seydor2• 3 years ago

This is great cause soon nobody will believe any image is real.

krapp• 3 years ago

Why do people keep saying this?

People believe far less convincing things now. People see After Effects flares and badly animated CGI on Youtube and photoshopped images and think aliens are real and cities appear in the skies over China. People will believe whatever confirms their biases, regardless of its quality.

I don't think this is going to change much, one way or the other.

anigbrowl7• 3 years ago

To cope with their discomfort at not having a good answer for any of the potential problems. Also, reflexive denial is a very common tactic for avoiding responsibility. In an increasingly networked world, people can rely on others with the same personality type to back up their denials.

pessimizer9• 3 years ago

Why do people keep saying this?

Because there's no reasonable argument against it. What I want to understand is why people have such a low opinion of the intelligence of the people around them. I can't imagine a person so stupid that they would be aware that they can make a 1-click deepfake, but also be totally convinced by every photo they see. This would be somebody who runs away screaming when they see a train coming towards them on television (which people actually did at the dawn of film, but they got used to the new situation.)

People believe far less convincing things now. People see After Effects flares and badly animated CGI on Youtube and photoshopped images and think aliens are real and cities appear in the skies over China. People will believe whatever confirms their biases, regardless of its quality.

People believe it because it is harder to do now. They use a very sensible heuristic (often badly) that nobody would go through the trouble to fake X images for Y reason. The question is explicitly about what happens when you don't have to go through any trouble to create convincing fake images.

seydor2• 3 years ago

People believe whatever other people that they trust tell them. There will still be "reputable" sources to tell people what is real.

HDMI_Cable• 3 years ago

Yeah, its much more likely that people will use deepfakes and AI-generated photos/video to reconfirm their pre-existing biases, just as how we use manipulative news articles to confirm our opinions on topics. All that this will do if cement the death of an objective fact—at least for fields without strong empirical verification mechanisms.

singleshot_• 3 years ago

I think you might gain better insight into why people keep saying that if you think about your telephone. Twenty years ago when your phone rang, it was probably one of your friends or family. Today, when your phone rings, you just assume it’s a telemarketer, right?

I think I’m the future when you see a digital image or video you’ll think the same thing: here comes some jerk to scam me.

thebruce87m• 3 years ago

We’re not talking about outlandish things being faked. We’re talking about mundane things being faked with no way to prove otherwise.

From the article:

The parents at the school didn’t understand how that could be possible

These people will vote on what they see.

joshstrange• 3 years ago

A Washington Post piece recounts how a small-town school teacher lost her job after students’ parents learned about AI porn made in the teacher’s likeness without her consent. Just a few months ago, a 22-year-old was sentenced to six months in jail for taking underage womens’ photos from social media and using them to create sexually explicit deepfakes.

Following the link [0]:

People viewing explicit images of you without your consent — whether those images are real or fake — is a form of sexual violence, said Kristen Zaleski, director of forensic mental health at Keck Human Rights Clinic at the University of Southern California. Victims are often met with judgment and confusion from their employers and communities, she said. For example, Zaleski said she’s already worked with a small-town schoolteacher who lost her job after parents learned about AI porn made in the teacher’s likeness without her consent.
“The parents at the school didn’t understand how that could be possible,” Zaleski said. “They insisted they didn’t want their kids taught by her anymore.”

Holy shit that's fucked up. It wouldn't be right but I'd be hard pressed not to create deep-fakes of the people who fired me and/or the parents to show them how bullshit it is if I was fired for something I had no connection to and was actually the victim of.

[0] https://www.washingtonpost.com/technology/2023/02/13/ai-porn...

wmf5• 3 years ago

Seems like a clear case of wrongful termination.

masklinn• 3 years ago

At will employment being the law of the land, the only way to actually get a case of wrongful termination is to document that you’re firing the person for one of the few reasons which are unlawful (race, religion, sex, origin, disability, medical condition, or age, as well as additional restrictions the state may have in place).

“The parents are complaining and we’re chickenshits” is almost certainly not an unlawful reason, if they even bother justifying the dismissal (which most employers don’t).

anigbrowl7• 3 years ago

Indeed, but it's not just an issue with some obtuse administrator. School boards are heavily politicized these days, and if complaining parents were told that there was no good legal basis to terminate someone, it's entirely possible that they'd crowd the school board meeting and make a big stink about it.

Factuality does not matter to a lot of people.

canadianwriter2• 3 years ago

If they told the parents they were photoshopped instead of AI made, would that have changed anything?

Curious if we change the language used if that changes how people think of it...

golol4• 3 years ago

People viewing explicit images of you without your consent — whether those images are real or fake — is a form of sexual violence.

What if you imagine it? What if you describe it to your friend? What if you sketch it on your tablet and then delete it?

wmf5• 3 years ago

That's thotcrime.

krastanov2• 3 years ago

I am sure there is a line that is difficult to draw between what is ok and what is not, but can we not at least agree that there are two very different extremes easily separated by any such line:

- most folks probably will agree that imagining sexual content with your teacher and keeping it to yourself is fine

- most folks probably will also agree that making photorealistic fake porn of your teacher and spreading it around is akin to defamation or sexual harassment

Let's take care of the extremes, then we can worry about the difficult middle.

npteljes5• 3 years ago

The crux is in the distribution. Does distribution enter your scenarios?

Imagining it? That can't be distributed, so no issue here. Describing it to friend? I think it's no issue, as the setting is private, and words are ephemeral. Sketch it on tablet? As long as no one sees it, it's fine.

Now, let's spice these up with distribution. What if you describe it not to a friend, but to an audience at a stand-up club? Or in a tweet? Or in a written publication? What if your sketch gets photographed, and put online? What if it's a semi-known person, and a tabloid gets ahold of the photo?

I think the first and second scenarios differ a lot. And that while both can be likened to sexual violence, the second group is what does real damage.

bottlepalm2• 3 years ago

I could say AI is going to wreck porn in that you can't compete with on demand pics/vids/VR of any fantasy you want.. but it's really going to be the least of our problems.

igetspam2• 3 years ago

The internet wrecked pornography. It's become a game of extremes. There are studies that indicate that kids today think the violence depicted in what we used to call "hardcore" is what's normal and expected. Both boys and girls think this and young women are suffering through humiliation and abuse instead of thinking that sex should be a mutually pleasurable act. AI is just going to make it worse and more extreme but it's already wrecked.

bottlepalm2• 3 years ago

I meant the business, but yea that too. Brave new world.

bloopernova• 3 years ago

Pornpen.ai unfortunately generates images based on user prompts. So everything is ridiculous anime proportions.

GaggiX• 3 years ago

I don't know why the article talks mostly about unstable diffusion, when looking at the most popular models on CivitAI it is clear that there are even better models for porn, making it clear that you don't need to raise $26,000 to make a good model, especially if the effort is distributed, users will train models, the best ones will be further merged and trained, literally the evolutionary process.

metadat• 3 years ago

https://civitai.com/

Interesting, this is the first I've heard of it. It does appear to be compelling, however.

Tao33003• 3 years ago

I'm guessing they don't want to publicize the real answer.

majormajor6• 3 years ago

There's a lot of head-in-the-sand here about "soon people will know better than believe images" which fails to realistically engage with the world in a couple of ways:

1) ignoring a problem and harms because "soon" we think such tricks may not work as well is a disservice to anyone unlucky enough to be harmed now

2) Photoshop and similar have been around for a long time now yet fake images and videos still often fool people on first glance, so it's a bit blindly optimistic to assume that in the future volume alone will change people's initial credulity

EDIT: elsewhere on HN today there was a link to an annonymous post on Blind claiming to be a Google employee on a PIP because of some sort of quota imposed on their manager. This may be true, it may not be. I don't have a way of judging personally, and obviously text on the internet has been full of bullshit for a long time. But is anyone here confident enough to claim that nobody who encountered that on HN or Blind would believe it as a result?

canadianwriter2• 3 years ago

I always bring this back to photoshopping. It's been around forever.

That teacher in the article - what if she was photoshopped instead of "deep faked" would anything have been different?

Photoshopping has been around forever.

Just a bit weird to me that people are predicting the AI end of the world with this stuff when its really just a slightly faster way to do something we've had for ages.

flangola73• 3 years ago

Photoshop doesn't create photorealistic live video of you. Even voices can be matched.

"Slightly" faster is vastly underselling it, deepfakes can be rendered in real time now. The same tech powers TikTok filters.

jrflowers3• 3 years ago

I love the people pointing out a theoretical future wherein deepfakes are so prevalent that only a rube would fall for them.

I can’t imagine reading “22 year old teacher gets fired for no fault of her own” and immediately writing up an explanation as to why nobody should be able to get in trouble for making and distributing deepfake porn of strangers in the future, but I am not so enlightened.

majkinetor• 3 years ago

Unless picture of me is signed by my private key, its fake. Simple as that.

Tao33003• 3 years ago

A deepfaked, gender-swapped image of Donald Trump, created with Unstable Diffusion

Here's the real scandal: techcrunch didn't blur out the nipples!

igetspam2• 3 years ago

I'm not good with generative AI. I always make acid trip soup. Is there a version of this where I can drop photos of my face and have it generate variations of me? Preferably a SFW version because I'm more curious about the level of difficulty in generation than becoming internet trash.