In Spain, girls report AI-generated nude photos of them being shared at school

https://english.elpais.com/international/2023-09-18/in-spain-dozens-of-girls-are-reporting-ai-generated-nude-photos-of-them-being-circulated-at-school-my-heart-skipped-a-beat.html

by agubelu • 3 years ago

21 74 3 years ago

english.elpais.com

palata22• 3 years ago

I would not be super proud to have contributed to that kind of technology. Raises the point: "is technology always neutral"? At least those working in technology should regularly think about it.

nik282000• 3 years ago

Yes technology is neutral (exceptions being things like the hollow point bullet) but not all applications are neutral. Remember when cameras got small enough to be included in pocket sized devices? They were abused and spent some time at the front of the 24hr news cycle as the subject of a moral panic. Eventually laws were adjusted and some technological adjustment's were made and the same will happen with "AI".

In this particular case the issue isn't even the technology it's parenting. The students generating these images are in their teens and are more than capable of understanding that using images of their classmates is not only immoral but also likely illegal. But when left to raise themselves using the iPad-Babysitter you end up with the moral free for all that we are seeing today. Until 18, parents should be held equally responsible for the actions of their kids.

Miner49er2• 3 years ago

Yes technology is neutral (exceptions being things like the hollow point bullet)

This is a bad example, hollow points have a legitimate use in the prevention of collateral damage, among other things. Unless you mean it's an exception because it's clearly a good technology, but I'd also say that's not true. It falls under neutral.

palata22• 3 years ago

Yes technology is neutral (exceptions being ...)

I find it interesting how you contradict yourself in your very first sentence. "Technology is neutral, except when it isn't". To me it means that it is not neutral, doesn't it?

giantg27• 3 years ago

I fail to see the points you are trying to make.

Ok, so some laws were adjusted around the use of small cameras. Yet there are still issues today. I mean, you have to actually catch someone using it. How many people are redlight scanning their airbnbs? Many people breaking the law just aren't getting caught. I fail to see how the laws really fixed anything. All that happened is people learned to live in this new reality and the news stopped reporting it because it's not new.

I'm not sure what you're getting at with the hollowpoint comment. In theory it could be applied for evil (murder) or could be applied for good (self defense, public defense, food, etc depending on one's beliefs about what is good vs evil).

isk517• 3 years ago

But when left to raise themselves using the iPad-Babysitter you end up with the moral free for all that we are seeing today.

20 years ago long before iPads when I was in high school there were definitely kids that would have done this exact same thing if the technology was available. Lets not pretend as though lose morals are a recent invention.

sim7c002• 3 years ago

I think technology isnt neutral, and people creating tech that can cause harm and subsequently making it available to the general public and easy to use are as immoral as the people using it. i love writing malware, but once it works, i delete it. If you want to see if you can create an AI that can make nudes of people thats fine, you can use it for yourself. or when it works decide atleast not to publish it. theres arguments for publishing it, and people ofcourse in favor of those arguments. thats ok, i am not though. i think its bad. you can write a paper about it and show some benign examples in it if you want the world to know these things are possible. there is no need imho to hand everyone a gun simply to prove guns are dangerous, and that is akin to what these tech creators are doing..

palata22• 3 years ago

I tend to agree. I feel like in today's world (where society may well collapse very soon because of what we did with technology - I'm thinking about climate and biodiversity), we engineers should think much more about what we do and the consequences it may have on the world.

Miner49er2• 3 years ago

I agree, but I think deep fake type technology is neutral. Some technology isn't.

Deep fakes aren't the issue here. This same thing could have been done before AI with Photoshop or even scissors and glue.

If any technology is at fault here, it's social media, because that's likely where whoever is making these is finding the pictures of the girls to feed the AI.

sim7c002• 3 years ago

you can also kill someone with your hands or a knife, that doesnt make guns less deadly. perhaps theres good use cases for deep fake tech, but realising people will use technologies for their own use cases instead the one you envisioned i think is important. That being said, this is a difficult topic and i suppose both sides xn have valid arguments. guns also save lives for instance. (sorry for keepong to use the gun example but i think ite easy relatable maybe a not a perfect example)

palata22• 3 years ago

I agree, but I think deep fake type technology is neutral. Some technology isn't.

That's probably a valid argument, but you should elaborate. What does this particular technology bring to society that balances for deep fakes that can destroy someone's life?

Cryptography can protect people, which sounds like a good thing, but it can protect bad actors, which is a bad thing. One could debate it, but overall it feels like cryptography is useful in many cases. On the other hand, I feel like we would be better off without e.g. blockchain or without e.g. SpaceX making space flights affordable for businesses in a world where the climate crisis may soon destroy us.

What about deep fakes then? What good does it bring that would balance the bad use-cases?

pixl973• 3 years ago

"that kind of technology"

I have a feeling you're going to have a problem with any kind of "general" technology in the future. Or, lets change the initial premise this this Heinlein quote

"Specialization is for insects," the science fiction author Robert A. Heinlein famously wrote. "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly."

The thing is with a generalized image creation software is it will do things you won't like. Be that representation of minors, or offending your king/glorious leader, religious sacrilege, or whatever it is you don't like.

Generalized software, much like humans, is capable of heresy.

palata22• 3 years ago

I have a feeling you're going to have a problem with any kind of "general" technology in the future.

I can agree to the argument that "everything can be used in a bad way". But that is not my point.

My point is: is any technology always worth existing? As a thought experiment, I can easily imagine a technology that would only be useful for torture, and I would consider it fundamentally evil. To the point where I would not want this technology to exist at all.

If you agree with that, then it means that it is possible to imagine a technology that should not exist. And consequently, it raises the question for every new technology: should this one exist, or not?

giantg27• 3 years ago

'Raises the point: "is technology always neutral"?'

Is technology ever neutral by those standards? Seems like anything that has legitimate uses could be used for evil too.

palata22• 3 years ago

Seems like anything that has legitimate uses could be used for evil too.

Ok, let's assume that anything that has legitimate uses can be used for evil. But what if the technology is mostly useful for evil? Would you call it neutral?

lrvick14• 3 years ago

It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.

There are potentially a couple of positive social outcomes that may emerge from this trend though.

1. everyone who sees a deepfake of themselves will be forever viscerally skeptical of any claim relying only on photos and videos socially shared online.

2. everyone who sees actual nude photos or videos of people they know will be skeptical they are real, thus taking away the power of revenge porn or ransom porn.

ToucanLoucan5• 3 years ago

It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination.

This is just casually dropped as though a photo-realistic AI image == an amateur painting, in production quality, in required effort and time, as though they're even remotely comparable. Which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy.

It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.

I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels.

Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world.

And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.

lrvick14• 3 years ago

Stopping someone from physically harming someone else can come with severe -punishment- as a deterrent and is much more serious than sharing -any- type of data online. Stopping someone from making a digital parody of someone else gets closer to free speech territory and is a lot more complicated.

You can make it illegal or try to detect and restrict sharing it in popular apps because it hurts feelings, just like you can outlaw pictures of religious imagery because they hurt feelings, but neither can ever be effectively enforced in an internet that allows open source applications and anonymity. See: piracy.

Giving up the right to online anonymity is a non starter as it would make it impossible for journalists and dissidents to communicate freely which is a requirement for democracy to function.

RetroTechie• 3 years ago

(..) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying.

You talk of punishment in the context of physical harm (fist vs. face), while deepfakes concern mental distress at worst. 2nd order effects like reputation damage, lost income etc can be adressed by lawyers.

Yes mental distress can be severe to some people. Or (for example) cause a group of people to harass someone.

But a) such harassment is a separate act from posting a deepfake. And

b) If deepfake postings happen often enough, it kinda takes the sting out. Few people today would be deeply disturbed if someone posts their head shopped onto a pornstar's body. That's easy enough that anyone can do it, it's been done many times, so the newsworthiness / shock value is gone.

If you know the victim in person, you'd probably be able to tell whether some video is fake or not. Or ask that person. If victim is a random stranger & deepfake disappears into the 'internet background noise', who cares.

Either way, when creating a deepfake is easy enough, it simultaneously grants a victim plausible deniability.

So ehm... "horrifying" is too strong a word imho. Try "annoying", "bad taste" or "disrespectful". Punishment (if any) should fit that.

jazzyjackson• 3 years ago

It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society.

terminally uncreative take. I'm actually curious what you mean by free society tho because clearly we are not totally without rules and taboos, what are we free to do and where do we draw the line?

imo deepfakes are not that different from revenge porn, it is an image of you being shared without your consent. doesn't matter that it's 90% synthesized, your image was used in a way designed to hurt you or at least your reputation. we have laws about libel and slander, where's the incompatibility?

giantg27• 3 years ago

There's a difference between between actually stopping it and laws. Sure, the law can be there but it's going to be hard to enforce and we need to see how the case law pans out.

What can you actually do to truly stop it? Ban deep fake software? How's that ban going to work out? To what degree is libel and slander present in current schools and how often is it successfully addressed?

lrvick14• 3 years ago

The incompatibility lies with it being unenforceable at scale even if illegal. Religious blasphemy is not legal in many countries. Sharing pirated content is not legal in many countries, but we all are on the same internet. Trying to lock up millions of people anonymously sharing data globally that is not even universally illegal is an absurd enforcement problem.

At some point enforcement becomes impossible without killing access to a free anonymous internet entirely.

helen___keller• 3 years ago

I think stopping sharing is possible it’s stopping generation that’s probably incompatible

AI is taking off like a rocket. Soon it’s going to be as easy as asking chatGPT to generate the photos/videos/whatever you want - and while chatGPT may be censored, there’s almost certainly going to be open source models that are not censored

ChainOfFools3• 3 years ago

It would not surprise me to find that in 30 years or so a new category of harm is codified into law which amounts to official and social condemnation of behavior that abuses the freedom of a free society. Or to put another way, behavior that pathologically elevates personal or individual freedom ahead of the net freedom of the society that forms an indiv9dual's supporting environment.

Particularly such behavior that in order to reduce the immediate harms it causes, would require reducing the freedom of society as a whole. A simple example, rather less sensational or revolting than pseudo-nude photos of schoolchildren but far more pervasive and destructive, and similarly immune to attack under today's laws and circumstances, would be noise pollution ordinances, which are seldom enforced but frequently abused. Such abuses degrade the environment in innumerable, but sub-actionable ways for hundreds of people so that one person can have something they want, ie straight pipes on a harley/whatever being gunned through residential streets at 2:00 a.m. in the morning.

I feel like this class of abusive behavior is on the rise (at least, in Spain as well as my country) and will cause a much more catastrophic breakdown of a free society if it remains unchecked, but that we have no means of curbing under our current frameworks without compromising those frameworks themselves. At the same time I'm really tired of this posture of conceding to assholes on the grounds that tjey effectively hold "free society" hostage as a shield for their individual need for validation and/or financial gain.

__turbobrew__• 3 years ago

For me personally I think it is less that this behaviour is on the rise, but that I’m getting older and it bothers me more. After hearing some stories about what it was like during my grandparents lifetime there were no bylaw officers or noise ordinances or pollution control, so I’m not convinced that we have it worse than a few generations ago.

seanmcdirmid• 3 years ago

I don't think it will take 30 years. The problem is that it will be very hard to prosecute, it will have to concern possession rather than distribution (since you don't need to distribute the artifacts, you just need to be able to generate them).

palata22• 3 years ago

but the types of measures it would take to stop it are incompatible with a free society.

Before going into enforcement, I would have talked about ethics :-). It feels like it is not too much to ask of people who can create new technology that they think about the consequences of their work.

ChainOfFools3• 3 years ago

yep. I'm getting really weary of these shallow protests about protecting anonymity and "technology" above all else, that are grounded in some sort of vague idea that technology is beyond moral reproach or has a mind of its own, and isn't in fact just an extension of ordinary, and in the case of this article, mean-spirited and cruelty-motivated human intent.

These deflecting arguments amount to no more than a modern irreligious version of "the s/devil/technology made us do it"

lrvick14• 3 years ago

Millions of people can program, and it only takes one to anonymously release code that anyone can use. Code is free speech. We -should- teach ethics, but we must also live in the reality that any code, text, or media that can be imagined will exist now. This genie is not going back in the bottle.

throw__away7391• 3 years ago

In a way this isn’t actually a new thing. Throughout most of human history there was nothing like a photograph and it was always possible to fake paintings and other depictions of anything.

It’s only this very narrow window of recent history when photos have existed and been broadly considered “proof” of reality while being difficult to forge. This time now appears to be coming to an end.

darkerside• 3 years ago

I'm reminded of the aliens on Galaxy Quest who had no conception of lying, and were all but unable to identify or understand it. Rationally, we may know that pictures and videos are not to be trusted, but we will never stop getting taken in, like an optical illusion.