Effective CSAM filters are impossible because what CSAM is depends on context

by speckx • 2 years ago

29 15 2 years ago

The article is correct, however most people would rather CSAM filters with errors than just throwing up our hands and saying no CSAM detection is possible.

The solution to have human-in-the-loop appeals processes. And because companies don't want to pay for this, it should be enforced as regulation.

kelseyfrog2• 2 years ago

That's because opposition to CSAM filtering is expressed as a defense mechanism, in this case splitting[1] also known as black or white thinking. Either the filter is 100% accurate or we can't have it at all. Splitting ranks low(level 2)[2] on the hierarchy of defense mechanisms.

1. https://en.wikipedia.org/wiki/Splitting_(psychology)

2. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8555762/

snapcaster• 2 years ago

That's not the opposition to it though? I think you're attacking a strawman (also just pointing to some fields classification of something isn't actually an argument)

It's the invasion of privacy not accuracy rates of the model

dmitrygr• 2 years ago

most people would rather

Citation please. Once it is explained to them how it could work, what the failure cases are, what false positives cause (with real life examples, since it has already happened), etc, I doubt this would be even remotely true.

chatmasta• 2 years ago

Presumably that depends on what actions are taken based on the filter. If it’s a filter of “which images to not sync to cloud,” that’s a different discussion than “which images to trigger an alert to FBI for investigation and prosecution.”

kwere• 2 years ago

the only way to catch pedos, if anybody really cared, would be with honeypots and "trojanlike" content, which could infect csam collectors devices and transmit data to police. It also work as a scaring tactic, so to limit diffusion of such material

Fire-Dragon-DoL• 2 years ago

No? My toddler did not want to wear clothes at home for essentially 1 year (2 to 3). We took photo of him playing during the year to preserve memories of his 2 years old, but of course he is naked in all the photos. My wife joked the other day "wow 2023 photos are full of his butt!"

Is it csam? I doubt it, those photos are for me and my wife who had to put up with it.

He finally wears underwear now.

havkom• 2 years ago

The doctors example is good, however may not only be health communication between a parent and a doctor but among parents themselves to figure out what action (if any) to take in relation to a condition.

The hesitation to effectively communicate after CSAM may actually cost lives.

neutered_knot2• 2 years ago

I’m not advocating widespread filtering in this answer.

This article is essentially referring only to edge cases which require context to determine if images are CSAM. There are other cases where context is not needed and determining if something is CSAM is sadly easy.

For example, there is no appropriate context for images or videos of adults raping toddlers or infants.

viraptor• 2 years ago

Do you know the actual volume of those cases though? As far as I know lots of groups exchanging the photos are mainly dealing with photos from semi-public spaces. The actual rape images are extremely rare. (There was a post about it on HN years ago, but I can't find it)

hju22_-3• 2 years ago

Not to mention, at least where I live, it'd be illegal to train a model on actual, real media representing such cases that would let the model have a greater confidence rating in contextualizing which are or aren't actual child pornography. And I somehow doubt we'd get a good filter either way.

neutered_knot2• 2 years ago

I am tertiarily involved in some efforts to prosecute offenders in this area. I am not of aware what volume of traffic online would be these images, but am aware of some material in cases where people are caught and prosecuted. My understanding is that in actual prosecutions such images and videos are quite common.

S0y• 2 years ago

A good way to treat CSAM is to see it as radioactive material. You always want it to be accounted for. Some people have legitimate use for it (x-ray machines, nuclear power plants, etc...) these would be the doctor example in this article. and then you have the actual illegal uses of it, like someone trying to build a bomb in their backyard. In that sense, a CSAM filter shouldn't assume anything about the content, just be able to identify that its content that needs to be accounted for.

sterlind• 2 years ago

I assume by CSAM you mean any indecent image of a child, since there's only one legitimate use of material generated by, or for, child sexual abuse: as evidence, to catch and prosecute child molesters. that's the problem: the label "CSAM" implies intent as well as content.

unlike "CSAM," "radioactive" implies only content - content that has to be deliberately mined and enriched in a handful of factories, not something that can be accidentally generated by billions of families.

also, material can become CSAM without starting off as CSAM. imagine a baby monitor captures video of a toddler changing clothes. that's not CSAM yet - it's merely indecent. but now imagine a pedophile hacks the feed and shares it on a CP website. now it absolutely is, because intent (sexual gratification) and abuse (stealing and sharing the photos) have occurred.

we need nuanced labels to implement a nuanced policy.

Effective CSAM filters are impossible because what CSAM is depends on context

Related Stories