The guy who made a tool to track women in porn videos is sorry

https://www.technologyreview.com/s/613607/facial-recognition-porn-database-privacy-gdpr-data-collection-policy

by theBashShell • 7 years ago

55 80 7 years ago

technologyreview.com

bem94• 7 years ago

It's things like this that make me think all engineers need more training on the social consequences of the tools they build.

This is an extreme case and I hope most engineers would look at this and understand how awful & damaging it could be; but given our influence on the world nowadays (as a class, if not always as individuals) we really need to be much more aware of what we're doing and how it affects people.

xwdv• 7 years ago

The problem is that engineers are not always the ones in charge of the entire end result. If you want to do something unethical you can just split it up into harmless individual tasks and give it to a dozen engineers and then put it all together. It’s obfuscation.

Some of the greatest atrocities we will see in tech (and we will see them) in the coming century will be the result of combining things created with good intentions.

Not much we can do I’m afraid except endure.

whenchamenia• 7 years ago

An engineer can quit if the project is unethical. Just 'enduring' is harmful to yourself, and others. Don't be a coward, at least stand up for your values.

todd8• 7 years ago

I agree, but I don’t think education will end up solving the problem. Somewhere out there, there is a sociopathic engineer that will make antisocial choices anyway. Software is so easy to copy that it proliferates too easily.

If a mechanical engineer creates some evil weapon, say a gun that shoots frozen mercury slugs (just to make up a ridiculous example), we don’t really have to worry about the proliferation of cryogenic weapons. When a software engineer pushes some questionable program’s source to github it’s game over.

internet_user3• 7 years ago

You do realize that morality and ethics can be drastically different from yours, and what you are proposing really amounts to "let's civilize those savages" ?

listenallyall• 7 years ago

What if a convenience store had security video of a clerk being shot and killed. Nobody recognizes the suspect's face in the video, so they want to use this tool to bring the killer to justice. Is that legal? Is it GPDR-complant? Clearly, the killer is not consenting.

Another scenario: a neo-Nazi protest rally occurs, and counter-protesters use this tool to determine who the Nazis are, in order to publicly shame them. Is this OK? Again, the Nazis have not consented to the use of their images to shame or "out" them.

You could go on and on... a vigilant neighbor filming a "porch pirate" stealing packages... someone filming people leaving a gay bar, an abortion clinic, a "massage parlor". Someone constantly running red lights. Who decides what is shaming vs what is "protecting people"?

jonnycomputer6• 7 years ago

I'm not sure I understand the GDPR argument here. Is collecting the facial information from actors and actresses in porn movies illegal? Or was it the access of their social media accounts? Why is the tallying of what is publicly consumable a violation of GDPR?

tobr• 7 years ago

GDPR deals with processing of personal data, not just collection or canonical records of it. If personal data passes through a system, it’s probably covered by GDPR.

barrkel• 7 years ago

The doctrine of public view meaning there are no rights to be enforced is not internationally recognised. Google even gave up on Street View in Germany, despite public streets being public.

There are good logical and moral reasons, IMO, why the doctrine is unsound. For example: individual photos are qualitatively different to movies, even though movies are made up of individual photos; the fixed interval conveys speed, which individual photos are quite poor at. Similarly, pervasive photography is different to individual photography: instead of an incident at a single location, someone's whole day may be mapped out, and who they're talking to, etc.

In other words, reasoning about privacy is not transitive or scale invariant. You can't say that the end result is ok or moral based on its component parts. Composition changes the thing qualitatively.

dahart• 7 years ago

I think this is a good question, and I’d like to know too, in specifics, not just “well, there’s personal data involved.” I’ve seen a lot of misunderstanding about GDPR and I’ve read parts of the legal code myself, and while it is clear why this software was a bad idea and how awful it might be for the actors, it’s not immediately clear exactly how this violates that law.

One thought perhaps is that the porn movies, even if it’s public information with consent, often do not identify the actors. The problem may be in connecting “anonymous” public information from one source with personally identifying public information from another source. But, this seems doubly problematic since all the information may be public with consent and since someone’s face might reasonably be considered identifying information. Does GDPR make it illegal to connect any personal data even when from public sources that all have consent?

jonnycomputer6• 7 years ago

apparently others don't like the question either. strange, because it was just a question. i don't really have an agenda, I'm just confused.

GuB-422• 7 years ago

In France, we have something called "droit à l'image". It means that if you can't appear on a picture without your authorization, with a few exceptions.

If you give an authorization, it has to be for a specific purpose and not generalized. So just because you gave an authorization to appear in a movie doesn’t mean you gave an authorization for other purposes, such as face matching. Same thing for social media.

There are also copyright issues. If France, you always own copyright (droit d'auteur), you may transfer the exploitation rights but the work is still yours. It means you still retain some level of control, in particular regarding decency.

France is just one European country but that's to show that there are limits to what you can do, even with publicly available data.

internet_user3• 7 years ago

What if your image has been cached by a CDN outside of France, and then downloaded and matched by someone who never interfaced with a French IP? Who is at fault here?

karl_schlagenfu• 7 years ago

How does this work when tourists are taking photographs? For example at the Eiffel Tower where it's almost impossible to take a photo without people being in the shot.

zamadatix2• 7 years ago

What are the GDPR repercussions for an open source project that does not make money, has no ties to an organization, holds only code, and handles no data?

kyberias• 7 years ago

If there is no data, obviously GDPR doesn't apply.

zamadatix2• 7 years ago

I'm more focusing on this section of the article where the legal source says 2 interesting things:

"just collecting the data is illegal if the women didn’t consent, according to Börge Seeger, a data protection expert and partner at German law firm Neuwerk. These laws apply to any information from EU residents, so they would have held even if the programmer weren’t living in the EU."

#1 the programmer is implicated (it's possible this is in the context of "the programmer" also being "user in Europe") #2 even if they weren't living in the EU

#2 is the particularly interesting one and also opens up the general question of "what happens if <external group> collects data but does not handle monetary transactions in Europe or deal with money abroad". GPDR is very much written around how to wrangle companies in financially, what if it's not a company?

majkinetor6• 7 years ago

Not sure how this relates to GDPR - the arguments are nonsensical. People tend to bump everything under GDPR now like its some form of gods commandment. If I simply sit in the cafe watching someone continuously near me, I am collecting data, and nobody can do anything about it.

When you leave your home, your privacy is gone.

When you make anything public your privacy is gone. Not sure anything is ethically wrong with the software existence, the problem seems to be data usage, just as if somebody found out his neighbors wife is in porn and went to twitter and said it publicly with few pictures as a proof, and people do that.

It's naive to think that this thing can/will go away, if anything, because of this news alone who knows how many more random programmers are retrying this (or even make business plans) at this very moment.

geezerjay2• 7 years ago

Not sure how this relates to GDPR

GDPR states that the lawful basis for processing of data depends, among other things, on:

* the individual's explicit consent to the processing of his or her personal data.

* protecting the vital interests of a data subject or another individual

This violates both criteria, thus is indirect violation of the GDPR.

majkinetor6• 7 years ago

* Face is not considered personal data AFAIK

* What the heck is vital interest ? For person playing in a movie it may as well be that everybody watch it.

Aengeuad4• 7 years ago

This has everything to do with the GDPR because the GDPR outlines the conditions you need to meet to process data, simply put actually gathering the data and using this tool is not legal within the EU.

If I simply sit in the cafe watching someone continuously near me, I am collecting data, and nobody can do anything about it.

But the tool is doing more than that as it's also using data gathered from crawling social media profiles and then matching people, you cannot do this without specific consent under the GDPR.

When you leave your home, your privacy is gone.
When you make anything public your privacy is gone.

GDPR provides controls on privacy, it does not say you are guaranteed privacy in every situation but that if somebody misuses your data in a way that violates your privacy you have rights to correct this misuse, e.g., the right to opt out and the right to erasure of data among others.

jobigoud• 7 years ago

Does GDPR applies to third party users of a website? Does it cover use of public data by other visitors or just use of private data by the site owner?

majkinetor6• 7 years ago

the right to opt out and the right to erasure of data among others.

Algorythm doesn't have to collect any data on you besides your image which is not considered private data given that you need to carry one. It can have single image and compare it to the porn collection without storing any results anywhere, but just returning 'XXX was a porn star in YYY'.

Tool can also create opt out mechanism which would have to be pretty complex as providing ZZZ in the name blacklist isn't appropriate solution given that many people share this name.

But the tool is doing more than that as it's also using data gathered from crawling social media profiles and then matching people, you cannot do this without specific consent under

Implementation detail. This can be totally avoided. Tool can accept picture and return the result.

buckminster• 7 years ago

If you sit in a cafe continuously staring at someone you will probably be asked to leave and never come back. Try it some time.

DanBC• 7 years ago

When you leave your home, your privacy is gone

This is not true in Europe.

orthoxerox2• 7 years ago

Russian imageboards found VK profiles of quite a few porn actresses and prostitutes using findface.ru. Of course, they immediately started harrassing them and sending messages to everyone in their friends list.

However, ethics aside, the cat is out of the bag. People can shame the guy into deleting the data, but computing power will only get cheaper and data matchers faster. Sooner or later some will do this again and launch a Tor site with an up-to-date database. We have to adjust to the idea that if you have a social media profile any public images of you will lead people to it sooner or later. And it will be sooner, not later, with every passing year.

sametmax2• 7 years ago

Yes, but that shoudln't prevent society to also make those particular use of tech very costly. It should be consider like holding a concealed weapon: you need a permit, or you get jail time.

Permit should not be hard to get, as long as you can justify professional or research need.

But letting this go is creating a society I don't want to live it. I already hate, living my anonymous life, when some stranger take my photo without asking. It happens more and more.

On the other end, people filming the police get in more and more trouble.

We got things reversed.

CydeWeys3• 7 years ago

How is this possibly enforceable though? The software is already out there and available. In countries that restrict guns, it's often quite hard to get one outside the legal channels (if there are any), so enforcement is pretty effective. But for something you can just download??

blablabla123• 7 years ago

I think that probably these things could even get completely prevented if people acted more consciously. It starts with DNS providers and hosters who should be aware of what people are doing with their services and shut them down if they aren't ethically good.

Probably Telcos and especially also CDNs should do the same. I don't talk about any packet inspection. But sometimes it's obvious what people are up to.

Basically everybody in society can do something against this by not looking away. Also more diversity in tech would have prevented a lot of malicious uses I think.

orthoxerox2• 7 years ago

If you are already a criminal, you aren't worried about the lack of a permit that much. Especially when you are a cybercriminal separated from your server by multiple cryptocurrency tumblers.

Forward intelligence teams already record the faces of people taking part in peaceful and legal protests. You can bet your ass they cross-ref the feed with their Linkedin and Facebook data dumps. The same tech is now becoming available to everyone.

The USA can prevent its people from putting an M2 on their truck. Mexico can't (in some states). Syria can't (almost anywhere). The internet is closer to Mexico than the US.

127• 7 years ago

Yes, absolutely. First criminal would be governments, who spy and track their own citizens with impunity.

internet_user3• 7 years ago

You mean, you want to prevent people from comparing public images against public videos on the public internet, by requiring them to obtain licensing?

This approach will just entrench FAANG even more and squeeze out SMBs even more.

keiru3• 7 years ago

I'm sure the same could be done with anything entropic enough to identify me, like my writing style (lexicon, commas, typos).

For a long time now I've assumed that everything I post online will eventually be cross-referenced back to me, if somebody cared enough.

samat• 7 years ago

The same story about Findface project was big news in Russia several years ago. They scraped the whole vk.com (Russian Facebook copycat with almost entire Russian population in it) and applied some very fine ML to find a person by his/her face.

2ch (4chan ancestor) users identified girls from porn and harassed them, outing them to their families and friends using this service.

Later company discontinued this service and now sells technology to the Russian government to monitor CCTV footage from Moscow subway.

VK kind of fixed data scraping issue (not sure if they actually changed much).

Usually Russia copies everything from the west, interesting to see western news follow Russian lead after a few years. Not saying I am proud of mother Russia in this case, just an observation.

gmuslera• 7 years ago

The bigger problem are not the right matches, but the wrong ones, specially if they are very similar. This should affect far more people than just ex-porn actresses.

keiru3• 7 years ago

I'm gladly surprised that "lookalike revenge porn" isn't already a thing (that I know of)

nkurz• 7 years ago

While false-positives are definitely a problem, why do you think they are a bigger problem? Do you feel the participants in pornography are "fair game" but the others are innocent? In some ways, the current level of inaccuracy could be considered an advantage, because it makes it more likely that there will be resistance. A 100% accurate system strikes me as more dangerous, as it's less likely for politicians or lawmakers to stand up for the victims.

keiru3• 7 years ago

Despite agreeing with your latter points, there's a reason why shrapnel damage is comparably serious.

Although most people who did some porn are most likely not proffesionals, most porn (by volume) is done by people who might even consider your finding part of their CV. So going by sheer numbers alone, there might be so much more cases of false positives. Now, even if it becomes terribly obvious how inaccurate it is, I don't think that matters much when it comes to weaponising it, because people are easily fooled or don't care. As long as you kind find a vid of a girl that looks just like Clara from accounting, you can spread it as a false rumor. Or not even claim it's her; it's just as embarrassing. And that's even assuming you could tell them appart, because the seed of doubt extends the problem to people who didn't even do porn.

For comparison think of deepfakes. Everybody knows it's not actually Emma Watson, but I bet she isn't particularly happy about it.

kabwj• 7 years ago

Why should pornography actors not have responsibilities for what they do? I don’t agree with this part where people who 1) put their faces in pornography and 2) put their faces in social media should not expect the two things to be correlated. Especially since the technology has already existed for some time and it cannot be stopped. Are we looking for regulation here? Is that exactly what we want, going back to cryptography export laws?

mannykannot• 7 years ago

The same principle would, of course, apply equally to this anonymous programmer.

Youcandothis4• 7 years ago

Why should pornography actors not have responsibilities for what they do?

They should have a right for privacy.

hurryskurry2• 7 years ago

How is the right to privacy being violated though?

Imagine you were a stripper or porn actress, your right to privacy wouldn't preclude someone recognizing you at the strip club or video.

Or suppose you went to the strip club or watched a video and then also happened to be on Facebook and saw the profile of someone you saw stripping at a strip club or in the video, matching the face and name in your head.

How is the slow case different fundamentally from the case where programming is used and throughput and speed is much higher and faster?

I mean would it be illegal for someone to just start going through Facebook profiles by hand trying to find an actress they just saw?

jadell2• 7 years ago

You're in a relationship with someone you trust. It's enough trust that you one day let them (or maybe even ask them to) photograph or film you having sex with them (or even just nude photographs, the level of explicitness really doesn't matter.) Days, months, even years later, things end badly.

Your ex posts the photos/videos of you on Pornhub, or 4chan, or anywhere.

Someone uses facial recognition to match the photos to your social media profiles. And then you get doxxed, outed, and the photos are shared with all your friends and family.

A) Are you a "pornography actor" in this scenario, or someone who trusted the wrong person? Is everyone who has ever had nude photos or videos taken of them a "pornography actor"? B) Just because something is technically possible, does that mean it should be legal? Do the ethics or the thing ever come into play? C) Do you believe that that laws/regulations are only effective if they prevent someone from doing something, and not effective if they only punish after-the-fact? Are deterrent laws useless?

bitL3• 7 years ago

Bypassing the stupid issue of harassing actresses by likely teenagers, isn't a husband-to-be right to know about the past of his fiancee? Is the moral right of not being outed outweighing the moral right of not being deceived in a very important matter? Where is the line?

jonnycomputer6• 7 years ago

What right of the husband-to-be are you talking about?

Is failing to tell someone something the same thing as deception?

Is this supposed right symmetric? Does bride-to-be have the right to know if the husband participated in porn video?

bitL3• 7 years ago

Is failing to tell someone something the same thing as deception?

Yes, absolutely. Imagine you are the only person that doesn't know your fiance was in porn and you committed to her without knowing that.

Is this supposed right symmetric? Does bride-to-be have the right to know if the husband participated in porn video?

Of course. Even though social stigma for males is lower, it has a devastating impact on their business relationships.

hurryskurry2• 7 years ago

If the videos are published, and the social media profiles public, how can it be illegal to match the two? Supposedly if they show their face they already accept the danger of being recognized, unless the video was published illegally.

That said, why would someone, once someone was identified, just go about harassing them and telling all their friends they do porn? Like who does that?

jddj• 7 years ago

The internet has enabled some people to deploy complex systems into New York datacenters from a beach in Thailand in a matter of seconds.

Unfortunately, it's also enabled a selection of less socially aware and/or socially responsible people to bypass the natural limits and filters that society had in place, to effectively cause suffering on a fairly broad scale.

Harrassing somebody on the other side of the world from the safe anonymity of your bedroom was once logistically very difficult, as was reaching significant numbers of people with a wildly malicious idea if you couldn't first find people in your immediate (physical) social circle and community to vouch for you/it.

Now, we have entire online communities which have embraced this new normal and provided these folk with the tools and an audience.

DougN7• 7 years ago

My guess is there are many that did porn when they were younger (and foolish?) but have since moved on with life and might want to forget that episode or their life. So having it resurface 10 years later would be very uncomfortable.