Cops bogged down by flood of fake AI child sex images, report says

https://arstechnica.com/tech-policy/2024/01/surge-of-fake-ai-child-sex-images-thwarts-investigations-into-real-child-abuse/

by isaacfrond • 2 years ago

45 103 2 years ago

arstechnica.com

4gotunameagain• 2 years ago

It will be interesting to see if/how this will modify their argument "for the children". It is a nuanced discussion.

Is it a crime ? Why / why not ?

Is there a victim, is there harm being done ?

Needless to say I personally find it appalling.

aketchum• 2 years ago

there would have to be illegal images in the training set right? which I would imagine would make the whole model illegal and therefore it outputs?

sanitycheck2• 2 years ago

I have luckily never encountered any of this stuff, real or fake, so I may be hopelessly naive about what's depicted, but... I can ask for a picture of naked Donald Trump wallowing in mud and such a picture will be constructed even though there are no photos of Donald Trump wallowing in mud (naked or otherwise). So I don't think the training set necessarily contains illegal images.

spacebanana72• 2 years ago

Not necessarily.

If the training data contained sexual images of consenting adults and legal images of children, many intelligent models could interpolate between the two.

dariosalvi78• 2 years ago

Not necessarily: the algorithms are perfectly capable to extrapolate, which makes the argument that the synthetic images "harm children" (as the article repeatedly tells) hard to defend.

To be clear, I find child sexual abuse appalling, but maybe synthetic images would keep some people "satisfied" and leave the real children alone?

wharvle2• 2 years ago

Pictures of naked kids aren’t necessarily illegal, or we’d be sending nearly every parent to prison (to take one example).

Besides, if it worked like that, training on anything under copyright would have a similar effect. These models have a bigger problem if they get “tainted” by a tainted training source. (Fingers crossed they do! But I doubt it)

harshreality2• 2 years ago

For truly realistic CSAM, there would probably need to be some quasi-illegal images in the training set.

But "CSAM" also includes "character looks under 18" even if they look fully developed, which an AI model could do without training on actual CSAM.

Whether that makes the model illegal is a separate question. Is a LLM illegal if a clever prompt can get it to output a copyrighted poem? Is a human illegal if they can draw "CSAM"?

harshreality2• 2 years ago

They (other than the cops investigating actual crimes against actual children) don't care about the children. They care about control. AI "CSAM" in particular (combined with deepfakes of adults made without the model's permission) is an emotional justification for more control in the form of draconian internet censorship and other preemptive measures.

Decades of megacorporation propaganda have paid off, and a fair portion of gen-z/y/x is hypervigilant about use without permission, even if the use is transformative and doesn't displace the original work. Their misguided concerns will play into the hands of the politicians seeking more control.

And it will do very little to prevent the spread of AI "CSAM"[1]. What are they going to do, regulate GPUs or any AI training software?

[1] Or any of the other concerns of today, like voice cloning or mimicry of personal likeness, like what just happened with Taylor Swift. The cat's out of the bag for images; OSS voice cloning is roughly at elevenlabs' quality, and it'll be surprising if there aren't, very soon, easy-to-use interfaces for voice cloning and tts.

malux85• 2 years ago

Is there a victim, is there harm being done?

While one might argue there’s no immediate victim, the problem is a kind of sexual hedonistic adaption,

At Gay saunas and sex shops it’s well known that playing porn where people don’t wear condoms greatly increases the incidence of condomless sex on the site, and playing porn with condom users greatly increases condom use. There are some saunas that ONLY play porn with condom users for this reason.

I imagine these synthetic images would do the same thing. They are fueling a dangerous fire. Soon images won’t be enough and given they’re now desensitised though frequent exposure, it’s easier to make the leap to reality

If you’re a chocolate addict, having a hobby of smelling chocolate is a really dumb idea.

shaky-carrousel3• 2 years ago

The images would desensitise them, and it would normalize them. But there is a huge step between having a fantasy and taking it to reality. There are people who fantasize about being raped, but would hate for it to become reality.

gitanovic• 2 years ago

It's not really clear to me what's your position. In the sense you state that you find it appalling, but what? The fact that there is a problem with such generative AI, the discussion that might follow, or its consequences.

Obviously the powers that be will try to use this at their advantage, trying to tighten a bit control over population, such as rendering AI models illegal, except a selected blessed few (read megacorp. ones)

IMO the general response shouldn't be just trying to defend the freedom position for freedom sake, but to find viable alternative solutions that don't mine the freedom of the population.

For example... we could try to use AI to detect if the images are AI generated or not :) And police should have the means to use AI, maybe given for free by the megacorp that benefit the most from AI

ineedasername• 2 years ago

Precedent for this exists. [1] US Supreme Court ruled that completely virtual images are okay, but those derived using innocent real images of children would still be illegal.

So the AI images likely are as well because the training set would contain innocent child images that helped generate them.

Training data may also unknowingly contain actual child porn. If so then any naked/sex images derive partially from child porn. And many jurisdictions have "strict liability" laws...

Find one single illegal image in the training set and the entire model would be tainted. And how many may have slipped through human review? [2]

[1] https://web.archive.org/web/20201109024622/https://www.nytim...

[2] https://www.theguardian.com/technology/2023/aug/02/ai-chatbo...

hiatus• 2 years ago

How do they determine the images are generated and not real? It's not mentioned in the article.

danuker• 2 years ago

If there is no way to tell, then reasonable doubt will become much more prevalent.

wharvle2• 2 years ago

If the volume of novel images shot up 10x right after these tools started becoming accessible, that’d let you make a good guess that most of it’s not real.

mdhb3• 2 years ago

It’s very obviously not going to be a meaningful legal distinction in terms of possession nor should it be when you think about it for more than 5 seconds.

It’s only particularly relevant in the context of terms of trying to identify the source of abuse and trying to stop or interrupt that.

nordsieck3• 2 years ago

It’s very obviously not going to be a meaningful legal distinction in terms of possession nor should it be when you think about it for more than 5 seconds.

I've thought about it for 5 seconds, and it's not obvious to me.

The whole possession of images is a crime is already a stretch, justified by the very real observation that the market for these images necessarily fuels abuse of real children.

But AI images cut that link.

I'm in complete agreement that the content is odious. But there's plenty of stuff that's odious and legal.

gizmo6862• 2 years ago

Under current US law, it is a very meaningful distinction. Possesing pictures of real child abuse. Ashcroft v. Free Speech Coalition (2002) makes this distinction very clear.

The argument that eliminating the market for pornography produced using real children necessitates a prohibition on virtual images as well is somewhat implausible because few pornographers would risk prosecution for abusing real children if fictional, computerized images would suffice. Moreover, even if the market deterrence theory were persuasive, the argument cannot justify the CPPA because, here, there is no underlying crime at all. Finally, the First Amendment is turned upside down by the argument that, because it is difficult to distinguish between images made using real children and those produced by computer imaging, both kinds of images must be prohibited.

With the current composition of the court, I'm not convinced this precedent will hold up if a case reaches them. But until that happens, the law seems quite clear.

gizmo6862• 2 years ago

It is still not particularly difficult to tell AI images from real ones. The problem is that "not particularly difficult" means minutues of a human looking closely at them. If you have a bunch of images, those minutes at up very quickly.

ToucanLoucan• 2 years ago

Not to mention all the people who already looked at the images that hopefully were filtered from the data set before training.

And none of these people likely received mental health care as compensation for that work.

dgrin91• 2 years ago

Is it? I know in the early AI image days there were issues with number of fingers or other weird things, but today there are plenty of images that I struggle to tell apart

kumarvvr• 2 years ago

I am worried about the changes in laws this will bring.

As on date, the state cannot force you to testify against your own interests. It is the states responsibility to ensure it finds the evidence to present to courts.

With situations like this, the chipping away of rights will begin.

How do the cops know which is real and which is fake? Make the convict tell.

The problem is a tech problem, which has to be solved by techies. Otherwise, we will land up into a quagmire of privacy and state protections eroding, or the AI technology itself will be heavily regulated / banned.

ghaff• 2 years ago

The problem is a tech problem, which has to be solved by techies.

When has that ever been the way it worked? Or arguably should have worked?

loceng• 2 years ago

TL;DR - Could fake AI child abuse content be being pumped out as part of manufacturing consent from the general population to bring in more controlling policy to then implement a totalitarian state?

shzhdbi09gv8ioi2• 2 years ago

They are already using child protection as an argument to implement totalitarian measures.

The most recent example that come to mind is the European Union Chat Control, which were meant to break all e2e encryption, enforcing on-device data scanning and fingerprinting to function.

thaumasiotes• 2 years ago

How do the cops know which is real and which is fake? Make the convict tell.

What would that accomplish? All you'll learn is that they're all fake.

londons_explore• 2 years ago

When other cultures try to impose their morals on the west, we say it is oppression (eg. compulsary headscarves). When we try to impose our morals on others, that's somehow totally different, and they need to follow our rules even if they're in a totally different country.

LtWorf7• 2 years ago

Hehehehehe, my code of conduct starts with "No USA cultural imperialism allowed" (https://codeberg.org/ltworf/international_code_of_conduct/sr...)

Some people loved it, some others told me I'll never find a job because I'm a bad person who can't work with others (I have a job).

edit: in case that wasn't clear, it still doesn't allow child porn…

namaria9• 2 years ago

Ironically in trying to transcend a conflict you never needed to transcend because you're not even a part of, you just sound like you picked a side.

justaman• 2 years ago

Modern progressive discourse argues that when the west exports their culture it is native-erasure and neo-colonialism(ie racist). So I think your argument is very much up for debate. Sometimes a part of someone's culture is trash. Sure the Mayans built cool temples, but they were also ultra violent.

It sure sounds a lot like you are defending pedos here...

blueflow• 2 years ago

"Modern progressive discourse" is an US culture export. Also is using "racism" as a substitute for "bad".

And what would be a pedo by US standards might be legal and fine in other countries (looking at Japan lolicon situation).

namaria9• 2 years ago

Why would you think Mayans are any more violent than other contemporary or later civilizations?

djoldman2• 2 years ago

It will be interesting to see how this plays out between two different scenarios:

1. Real-world photographs of people, minors or otherwise, that are altered via photoshop/AI/hand/etc. and then disseminated

2. Images, that appear or seem to be of minors, created entirely through synthetic means (photoshop/AI/hand/etc.) and then disseminated

Can drawings be illegal?

Spivak2• 2 years ago

Can drawings be illegal?

Yes, very much so.

https://en.m.wikipedia.org/wiki/Legal_status_of_fictional_po...

Wikipedia calls the US a legal grey area but it really isn't. It's just a long road to, "yep still illegal."

shzhdbi09gv8ioi2• 2 years ago

Can drawings be illegal?

For sure.

https://en.wikipedia.org/wiki/Legal_status_of_fictional_porn...

djoldman2• 2 years ago

Thanks for this! If wikipedia is correct, it seems that this is clearly answered.

The PROTECT Act also enacted 18 U.S.C. § 1466A into U.S. obscenity law:[122]
Section 1466A of Title 18, United States Code, makes it illegal for any person to knowingly produce, distribute, receive, or possess with intent to transfer or distribute visual representations, such as drawings, cartoons, or paintings that appear to depict minors engaged in sexually explicit conduct and are deemed obscene.
Thus, virtual and drawn pornographic depictions of minors may still be found illegal under U.S. federal obscenity law. The obscenity law further states in section C "It is not a required element of any offense under this section that the minor depicted actually exist."

The difficult part of this is the phrase, "...that appear to depict minors..."

mikhailfranco2• 2 years ago

Synthetic AI images (2) would have to be trained on a corpus of real images (1), except in the very unlikely event that many child-porn images are generated by photo-realistic rendering of 3D animated models (CGI), where the 3D models are entirely created and animated by real content creators (i.e. entirely synthetic).

So I think the default assumption for most realistic AI-generated images, is that they were generated from at least some real photographs.

I am not an expert in AI images, but perhaps there is a way for AI to be trained on real pictures of children (innocuous, clothed, etc.), combined with real images of adult porn to create novel realistic child porn.

shaky-carrousel3• 2 years ago

Hm, I'm thinking, if there's a way to convert a real image of a person into a manga version (there's ton of them on the internet) the reverse should be true. So you would just have to train it with manga (which is legal in Japan, for example), and then convert it to real. No real people dataset needed.

mikhailfranco2• 2 years ago

Japan seems to be an interesting case, where real porn was censored, so it encouraged thriving comic-book manga/anime/hentai cartoon porn, that seems to be legal there (you see sararīman reading them on the commuter trains!).

Censorship is the mother of metaphor - J.L.Borges

P.S. Japan does appear in the long list of legal exceptions in the Wikipedia article linked by sibling comments

A4ET8a8uTh0• 2 years ago

It is a steady drumbeat of 'think of the children'. I have a daughter I would personally love it if politicians and their ilk stopped thinking about children so hard ( and either found a better scapegoat or stop eroding right or neuter new tech ).

It is annoying and, frankly, it is keeping us from evolving as a species.

forinti2• 2 years ago

I wonder if fake images will mean less attacks on real children.

This is a really hard issue to tackle!

jdiff4• 2 years ago

We've had fake images for decades, with people justifying it the same way. "Drawings of children mean real children aren't being abused." I've yet to see the slightest shreds of evidence supporting that claim.

forinti2• 2 years ago

Please, don't think I'm trying to justify. This thing is just plain nasty.

But these images do bring us some new (terrible) questions to ponder. Especially if the police can't differentiate between what's real and what's not.

sp332• 2 years ago

Physically maybe, but knowing that someone is getting off to images of you as a young kid is still traumatising, especially if you still are a young kid when you find out.

glimshe• 2 years ago

I find child pornography abhorrent. But AI child pornography could actually help reduce child abuse - some abusers might get their fix from these images instead of needing actual children. This would be similar to the (yet unproven) claim that widespread access to porn has reduced physical sexual activity in general.

jdiff4• 2 years ago

People have been saying that for decades before generative AI was a glimmer in anyone's eyes. Even the story we're commenting is of someone who was doing both. It's time to retire that tired theory.

ben_w4• 2 years ago

As with all interventions, it is a claim that can only be tested by comparing abuse rates in two otherwise very similar jurisdictions, where the only difference is the availability of such material. To think otherwise is to presume that all members of the target group are identical, which is implausible.

Even attempting to run such a pure observational study of such a situation would be denounced, even if the researchers had no power to alter the laws.

If you did somehow manage to run such a study, I expect you'd find three things: (1) plenty of examples of people who were inspired to act due to the images they saw, (2) plenty of examples of people who used the images as a way out, (3) that there are way more abusers and victims than most people realise.

oatmeal13• 2 years ago

What you're saying is the market-share of the AI version that does not involve additional exploitation of minors in its creation is going up over time. The evidence is increasing that the theory is right.

If the AI version gets good enough, the supply of a substitute will real CP will go up, the price of real CP will go down, meaning there is less incentive to exploit real children.

xorcist• 2 years ago

If there's anything to learn from history, it's that desensitizing people isn't a good thing. Much like heroin wasn't a great way to get people off morphine.

spacebanana72• 2 years ago

In the 80s & 90s, many people feared that violence in video games, movies & music would desensitise young people to violence and lead to an increase in crime.

But the exact opposite happened. The generation who murdered and robbed their way through the streets of San Andreas committed much less IRL crime than their parents generation.

oatmeal13• 2 years ago

Who are you seeing as being desensitized in this scenario? People searching the net for CP are already desensitized.

willcipriano• 2 years ago

It turns out that for some people porn is a gateway drug to child abuse.

They get hooked and start looking for more and more extreme content, eventually that escalation has them watching child abuse material, then then go from that to the real thing.

It might be why pedophilia seems to be more and more common post internet. If all you have is Hustler magazine you can't escalate to children.

mcphage• 2 years ago

It might be why pedophilia seems to be more and more common post internet.

I suspect (admittedly without evidence) that it's not more common—it's just more socially unacceptable, and also easier to get caught.

ben_w4• 2 years ago

The age of consent in the UK was 12 between 1275 and 1875, so I suspect "seems" doesn't map to "is".

And I vaguely remember an 80s or 90s survey which said something like 1 in 6 American women were victimised as minors.

ceejayoz• 2 years ago

Serial killers and serial rapists tend to show a pattern of escalation, in both significance and frequency of the crime. I would not be shocked if crimes against children of this nature do as well.

guappa• 2 years ago

I think most killers aren't serial killers though.

runiq• 2 years ago

While I agree in principle, it turns out that real CSAM may be used during the training phase and that obviously has to be prosecuted.

vabyta77• 2 years ago

In Russia and Ukraine, simple possession of child pornography is not even illegal. Yet they still are the 2nd and 3rd top CSAM producing countries respectively...

ajsnigrutin3• 2 years ago

This will be a shitstorm... How do you determine the age of an AI generated personna?

lm284692• 2 years ago

If it looks like a 8 years old you won't convince the FBI that it actually is a DnD character that's 106 years old under a spell of rejuvenation, and even if you do it won't matter for your sentencing.

ajsnigrutin3• 2 years ago

I mean sure... but that's because there's an actual person photographed there. What happens when you get into "does (s)he look 17 or 18 in this photo?" and no way to prove anything, since the person does not exist?

chefandy2• 2 years ago

I imagine the same way they do for actual photographs in which they haven’t identified the victim, which I’d guess is a pretty big percentage. Stuff like this is never cut-and-dried, but the law deals with ambiguous situations all the time. The test for what even constitutes pornography compared to, say, medical images involving genitals is a good example.

ajsnigrutin3• 2 years ago

harimau777• 2 years ago

How would they differentiate generated images of people who look young from generated images of people who are underage?

mmvasq2• 2 years ago

Who is “law enforcement”. Not “cops” as per YCombinator headline. You can’t get a cop to do a thing about online content. I would really like to know who is feeling bogged down. I have a serious case for them that a person was actually drugged, trafficked and extorted.

thelastparadise• 2 years ago

You need to contact FBI child enforcement immediately.

FYI you could have some liability of you delay.

mmvasq2• 2 years ago

She is an adult. Yes adults get trafficked. Especially good looking ones falling into health and economic challenge. She has a daughter. Children are hurt when adults are trafficked- no one gets this. The trafficked party then used to extort me by someone actually pretending to be “cops” with cross platform analytics and data broker access.

Craziest thing you ever could have imagined and only an extremely rich tech narcissist can come up with what they did to a woman already being trafficked. They did so to secondarily traumatize me.

ethanbond• 2 years ago

If a child, contact NCMEC (National Center for Missing and Exploited Children)

https://report.cybertip.org/

24/7 phone line: +1-800-843-5678

Otherwise/in addition, FBI is a good bet.

genmud• 2 years ago

If this is real, please hit up ICE C3 [1]. They specialize in investigations for this type of thing, specifically sexual exploitation and trafficking.

[1] https://www.ice.gov/partnerships-centers/cyber-crimes-center

explorigin• 2 years ago

So many commenters didn't read the article.

The troubling case seems to illustrate how AI-generated child sex images can be linked to real criminal activity while also showing how police investigations could be bogged down by attempts to distinguish photos of real victims from AI images that could depict real or fake children.

cyanydeez• 2 years ago

sounds like doomers are still winning

Cops bogged down by flood of fake AI child sex images, report says

Related Stories