Facebook criticised over handling of reports of child exploitation content

http://www.bbc.com/news/technology-39187929

by ComputerGuru • 9 years ago

82 88 9 years ago

bbc.com

akerro2• 9 years ago

Just post a few pictures of breastfeeding and it will be closed imminently.

cr0sh4• 9 years ago

But a murder scene absolutely drenched in blood and body parts would be a-ok.

akerro2• 9 years ago

Yes, video from decapitating a woman in Mexico was alright to keep public for months.

camtarn• 9 years ago

"The BBC first asked Facebook for an interview about its moderation system in late-2015, and repeated the request following this follow-up investigation. The social network's director of policy Simon Milner agreed to be interviewed last week, on condition the BBC provided examples of the material that it had reported, but had not been removed by moderators. The BBC did so, but was reported to the UK's National Crime Agency as a consequence."

That's either Kafka-esque levels of bureaucratic crazy, or a very well-concocted trap...

gggy• 9 years ago

I always thought that the very notion that information can be illegal is like a contradictory premise that logically leads to crazy conclusions.

matthewmacleod2• 9 years ago

I don't think information is generally ever illegal, but rather the distribution of it is.

throwaway2016a• 9 years ago

My thought has always been that this, in additional to drugs is better treated on the supply side and the demand side is better treated as a health issue.

Although in the US we have stupid rules like if two teens who are dating share photos of themselves with each other they can wind up on a registry for life.

tankenmate• 9 years ago

In most western jurisdictions having child porn is a strict liability offence, there is no issue of mens rea; if you have it you are guilty regardless of the reason.

shp0ngle• 9 years ago

It's funny how "information wants to be free" proponents change tunes when it's their private e-mails and browsing habits.

Anyway. Yeah, there is hell lot of information that should be illegal. It's not just the information itself, it doesn't exist in a context-free vacuum

andrepd2• 9 years ago

Can you elaborate on that? I can think of several situations where information can and should be illegal.

djsumdog4• 9 years ago

It is legal to post photos of kids shot up by soldiers. Sebastião Salgado has entire collections of photography featured in museums around the world of genocide; even children in Africa laid waste by gunfire.

..and I'm sure somewhere, someone is jerking off to one.

When someone rapes a child, they put something into them; into their body, cause permanent mental and sometimes physical damage or death and leaves them with a high chance of developing PTSD.

When someone shoots a child, they put something into them; into their body, cause permanent mental and usually physical damage or death and leaves them with a high chance of developing PTSD.

I don't know what the correct solution is either. Both are terrible terrible things. But this disconnect shows where the priorities are in our current social structures.

makomk• 9 years ago

Reading between the lines, I think they may have been expecting the BBC to send them links to where the images could be found on their site and were sent actual copies of the images instead.

Latty4• 9 years ago

And are then probably bound by law to report it. This seems like a cheap swing from the BBC when all Facebook are doing is following the letter of the law, which I can't blame anyone for doing when it's this kind of law - people are not kind to anyone guilty of an offense involving child pornography.

roywiggins2• 9 years ago

And Facebook reported the BBC reporters to the police for sending Facebook examples of the content that the reporters found- on Facebook- which Facebook wouldn't remove.

Latty4• 9 years ago

As mentioned earlier, this seems like Facebook were asking for links to the offending content, got sent the actual content itself, and were then bound by law to report it. It also seems unlikely there is an overlap between the things they reported to the police and the things they wouldn't remove.

megous4• 9 years ago

It shouldn't matter. If BBC had the links, they obviously had the content too. BBC's computers downloaded the CP and stored it. Therefore Facebook should report them either way.

Distinction between sending links sending images + links makes no sense in this case. Both proves BBC downloaded CP.

awjr• 9 years ago

Well, if they downloaded the pictures then emailed them to Facebook, the BBC is breaking the law. If they sent links to the groups in question, then no.

megous4• 9 years ago

And that's the reason not to report abuse to the fucking Facebook. Just report it to the police directly.

ccrush• 9 years ago

Theyre often under order not to remove terror or child abuse content so the government can track the perpetrators.

TallGuyShort2• 9 years ago

What kind of orders? Do you have a citation?

Neliquat• 9 years ago

While I seriously doubt this is the case here, there is precedent for such things. See the FBI led TOR sting for example. However 'darknet'=/=Facebook.

Starwatcher2001• 9 years ago

Really? Do you have a reference for this?

lozf• 9 years ago

I'm pretty sure that images could be removed from public view yet still provided to the authorities along with any logs pertaining to their upload, to facilitate tracking.

MaxfordAndSons2• 9 years ago

I was thinking something similar. Though I doubt it would be an "order" in the case of a child abuse sting, it seems possible that the initial response was so tepid because there were competing motivations within FB to maintain cooperation on a sting vs. be responsive to the BBC.

6stringmerc• 9 years ago

Wow, this is quite an ugly story for Facebook, especially hot on the heels of Zuckerberg championing the platform as the key to building global community. I'll stand by my assertion that not all communities are good ones worth supporting. Thus, it puts a lot more responsibility on Facebook to arbitrate than I generally feel is logical in a capitalistic system. There are flaws. Like this for instance:

When provided with examples of the images, Facebook reported the BBC journalists involved to the police and cancelled plans for an interview.

It subsequently issued a statement: "It is against the law for anyone to distribute images of child exploitation."

I much more expected Facebook to blame their AI - which, as Zuckerberg claimed, is going to clean up Facebook just you wait and see - rather than admit outright they should go to jail for hosting the content that the reporters found.

Between this and the terrible use of Facebook to exploit Marine Corps females, it's going to take a lot more gardening in South Dallas before I believe Zuckerberg is actually interested in ethical behavior more than perceived image. YMMV.

dublinben2• 9 years ago

I wonder if Facebook will report themselves to the police as well, since they're the only ones in this situation who are distributing these images.

TallGuyShort2• 9 years ago

That's not true. The owners of the routers and switches between Facebook and its users are also distributing child porn.

roywiggins2• 9 years ago

Sounds like they did.

"We also reported the child exploitation images that had been shared on our own platform. This matter is now in the hands of the authorities."

ikeboy• 9 years ago

The title is misleading. It's not entirely clear from the article what kinds of images were not removed, but the article does make clear some of the ones reported were not child abuse images.

It sounds like some of them were regular images of children, but with suggestive comments attached. Now, that's still against facebook's TOS, but not illegal.

Edit: title has been changed to reflect source title better.

djsumdog4• 9 years ago

pages explicitly for men with a sexual interest in children
images of under-16s in highly sexualised poses, with obscene comments posted beside them
groups with names such as "hot xxxx schoolgirls" containing stolen images of real children
an image that appeared to be a still from a video of child abuse, with a request below it to share "child pornography"

That's what I thought. None of the above things are illegal (well, maybe the last one. IANAL). The 2nd one was common on Reddit's /r/jailbail subs before they shut them down.

They most likely do violate Facebook's terms of service, but with their massive userbase, I bet these pop up and get removed faster than their staff can deal with them.

artursapek• 9 years ago

How hard would it be for Facebook to automatically detect this kind of stuff by keywords? BBC shouldn't have to be doing this. They have how many thousands of employees? Why on earth are they complicit in this kind of stuff staying online?

I wonder if we'll see the #PizzaGate tin foil hat people being vindicated the same way that Vault 7 has vindicated the "1984" people.

Latty4• 9 years ago

How hard? Extremely. It turns out that people work around keywords extremely quickly, creating new euphemisms, acronyms and mispellings to avoid it. Eventually (or already) they will use such common terms that you get such a high false positive rate it makes the rest of the site useless.

People always make out like this is an easy problem, it's not. The best we can do without false positives is probably hashes - something already done (there are databases of hases of known images, and I think I remember there being a hash collision that got tons of sites reported for some normal image at some point, so even that isn't foolproof), and those are trivial to work around by modifying images in tiny ways.

Human moderation has huge cost and means that you have to give humans eyes on private content, which can be abused in itself, so you have a catch-22 there.

tristor• 9 years ago

"Facebook's rules forbid convicted sex offenders from having accounts.

But the BBC found five convicted paedophiles with profiles, and reported them to Facebook via its own system. None of them were taken down."

This tidbit in the article didn't so much surprise me exactly as pop out as something that seems unreasonable upon consideration even if it looks like a good idea on its face.

Key points for me regarding this is that recidivism rates are greatly affected by successful integration back into the community after serving time, and sex offenders have the most difficulty integrating already. While there's certainly reasonable concerns around the risk of these people accessing Facebook, by the admission of the company itself it's a new type of influential global community. By banning sex offenders outright they're being othered in a way which is a significant impediment to being reintegrated into society.

That aside, at least in the US, you can end up a sex offender for life for things as innocuous as stopping on the side of a long stretch of highway to empty your bladder. There's also no unified worldwide or national database of sex offenders to check against, nor are their uniform standards between states and nations on what qualifies. This means it's both difficult for Facebook to enforce this rule at all and practically impossible to do so in a way which would only apply to people convicted of offenses we can agree qualify (molestation, rape, sexual assault).

I find it extremely odd that Facebook would willingly put itself in the position to police accounts this way and that there'd be public support for the idea of banning people outright without any basis in their activity on Facebook.

aaron-lebo2• 9 years ago

While there's certainly reasonable concerns around the risk of these people accessing Facebook, by the admission of the company itself it's a new type of influential global community. By banning sex offenders outright they're being othered in a way which is a significant impediment to being reintegrated into society.

What do you mean? Facebook's business speak about being a new global community means what? People aren't being "othered" because they can't have Facebook accounts. The ability to have a Facebook does not have anything to do with getting back into society - lots of people do it (abstain from Facebook) voluntarily. On the other hand, the access a predator would have to children is a very real danger and is presumably dangerous to someone who has problems with that.

Do you have any stats for this? This gets thrown up from time to time, but how many people are sex offenders cause they took a piss on the side of the highway?

This means it's both difficult for Facebook to enforce this rule at all and practically impossible to do so in a way which would only apply to people convicted of offenses we can agree qualify (molestation, rape, sexual assault).

This information is readily available. Presumably Facebook can run the same check the BBC did. If they've been unfairly convicted that's not Facebook's issue, it is the courts, but Facebook does a have a very simple qualifier in most places.

Sorry if I come off as a little aggressive...it is just that in many of these cases there are real victims here and it isn't the sex offenders, it is the innocent children (or other) whose lives they have destroyed. There are cases were people are accused unfairly, but most cases are not that way (it is a major charge that is taken seriously). Sometimes people do things which are destructive to others, and there's only so much empathy when rights are restricted as a result.

Just don't personally feel that sorry for a sex offender who can't get on Facebook. If they've been wrongly convicted, they've got bigger problems, if not, then it's a privilege not a right.

megous4• 9 years ago

How about policing actual current behavior and not some historical act that the person was already punished for. You don't need to care for former sex offenders. But you might care for reducing recidivism. Driving people off the biggest surveillance platform in the world may actually hinder the ability to catch them doing it again.

ue_3• 9 years ago

Just don't personally feel that sorry for a sex offender who can't get on Facebook.

How do you feel about other criminals who have been convicted of a crime, then let out? Sure, Facebook isn't really the issue here, but why are they facing impediment to being included in society? Are murderers OK to do it? How about people who steal things?

Facebook's policy is clear; the question is whether it's a just policy, especially with their aim of building a 'global community' and the almost necessity of having a social life that has some online component nowadays in order to lead a happy life.

Facebook on its own is no big issue, but if other providers follow suit, I can't help but think it is unjust or at lesat unfair.

ploggingdev• 9 years ago

Assuming all the images reported by the BBC were what they say they were, Facebook handled the case very poorly, especially the part where they reported BBC journalists to the police. The least Facebook could have done was to acknowledge the journalists' role in being proactive about reporting abuse, have a human review the images and delete whatever was illegal or against their ToS.

This incident raises the question of how a company like Facebook should deal with people reporting content. There is no concept of community moderation in FB where specific users are moderators and can deal with reports of abuse. Maybe facebook should consider community led moderation efforts until the bots are ready to take over. Community led moderation efforts (not specifically dealing with reports of abuse) generally work very well for large communities where the company building the product does not have the bandwidth to deal with moderation. Some examples that come to mind : reddit, stackoverflow.

Latty4• 9 years ago

Community moderation just won't work with Facebook - the whole point of FB is that it's gated communities. Your options are to only moderate the public areas (which may catch some, but not the vast majority of the content they are talking about here), allow moderators to access private areas (which leads to invasion of privacy that is a different problem - imagine the "strangers can sign up for moderation and see your private stuff!" headlines), or only allow moderation over content they can see, which means not catching this kind of stuff, as the only people in those communities are a part of it (or investagative journalists, I guess).

pimmen• 9 years ago

Just like Stackoverflow though, you would have to use achievements or something in order to give users more influence over time, lest we have a rogue troublemaker deleting everything.

But what are those achievements going to be based on for a site where the number one use case is to keep up with friends and relatives? How many people shared your image? How many people reacted to it? Reddit and Stackoverflow have topics they discuss, and it's easy to know what's a contribution to that discussion. My baby picture might be a big contributor to the conversation between my relatives, it might be very distracting for my colleagues.

jauer• 9 years ago

have a human review the images and delete whatever was illegal or against their ToS.

Except the BBC didn't "report" it (provide a link to it), they "distributed" it (spread the actual content). Online hosts/services have to report distribution or they are liable: 18 U.S.C. § 2258A / https://www.law.cornell.edu/uscode/text/18/2258A

todayiamme2• 9 years ago

One of the more Orwellian moments I've experienced on Facebook happened when it didn't allow me to send an image link to my sister in a private chat because the string had sex written in it.

We're both adults and have been using Messenger to talk to each other every day for a few years now, but nope Facebook wouldn't relent. It even froze my ability to message her from my smartphone app for a short amount of time.

This amuses me more than it annoys me, because someone at Facebook decided that it's a good idea for a messaging product to censor private conversations between adults. What on Earth led them to believe that my messaging app should be my nanny?

EDIT: Here's a screenshot of the dialog: http://oi65.tinypic.com/33wt9br.jpg

MaxfordAndSons2• 9 years ago

That's bonkers. When was this, and what country or countries were you both in?

FWIW though, it's also possible she unknowingly had some sort of anti-obscenity filter turned on in her settings.

todayiamme2• 9 years ago

She's an American citizen and was in the US at the time. I was in Cambodia.

Nope, pretty sure that wasn't the case. We aren't aware of any such setting. What's really funny is that the context wasn't sexual at all. It was a medical conversation with a family member. I'd gotten the HPV vaccine earlier in the day and we were talking about it.

The image in question was from an online medical resource, and it didn't flag the image itself as I ended up sharing it directly with her anyway. I think it got flagged because the string had the letters sex in it. Or, if someone else had flagged it for some reason elsewhere... Who knows?

therealmarv• 9 years ago

yep, Facebook messenger censors! It also does not allow you to send links e.g. to warez sites... I'm sure it censors even more stuff.

owly• 9 years ago

Facebook is obviously not policing the content on their platform very well. Though it does sound like the FBI could easily run a sting operation and take down the perpetrators who are sharing. Then force Facebook to be text only. ;)

freehunter6• 9 years ago

The crime happening on Facebook should make it far easier for the FBI to track down the real people behind it.

This is something I don't understand... the NSA is listening to everything we do, watching everything we do, but when it comes time to find criminals we still struggle. Why? If they have all of our phone conversations and emails and social media profiles and the CIA owns Facebook and the FBI doesn't need warrants and all these doomsday scenarios about how the US is a police state, why do crimes even happen? And when they do, why does it take good old fashioned police work to solve them? And why do some still go unsolved?

If Facebook is tracking everything you do across the entire web, if they know who you are even if you don't have a profile, if they're invading your privacy that much, and if they're hosting illegal content, why can't they find the real names of these people? Why can't the FBI just issue a warrant for the data and Facebook hands over the people's names and home address?

Either law enforcement doesn't want to enforce the law, or the agencies don't actually have the capabilities we've been told they do. If they had the capability, you'd think they would actually use it. But when someone is threatening to shoot up a school on Twitter, it's not NSA wiretaps that catch him, it's users reporting him to Twitter and Twitter reporting it to the police. Where's the breakdown here? What happened to ECHELON?

kajecounterhack• 9 years ago

I work on spam and abuse (but not at Facebook). Also IANAL.

CP is radioactive / considered contraband. You cannot download or resend it to anyone under any circumstances. It sounds crazy but it breaks the law -- BBC should have sent links instead / used platform reporting. By downloading to their hard drive and emailing it, they engage in proliferation.

"But didn't their browser already download it? When they saw it?" Yeah. But they had to define sharing such images as being illegal, so...I don't know how laws work.

When you detect CP, AFAIK you must report it (...ironically sending this copy is OK though I'm not sure what the full mechanism looks like) and immediately delete it from your servers. You can keep an irreversible hash to do future detection.

Also fwiw our org has some special contractor who has to identify this stuff. None of eng has to look at it, thank heavens.

oliwarner• 9 years ago

A lot of people here seem to be focusing on the fallout, not the problem. Have you ever reported something to Facebook?

A while ago, a random guy friended my sister and started posting nonsense on her wall. I checked his profile and immediately found a dozen groups full of 5-10yos in swimming costumes interlaced with softcore and Photoshop'd merges of the two. Unpleasant, illegal and against the community rules. I reported it.

In another case a friend liked a video of a chap visibly getting a blowjob at the top of a mountain. Sure, why not... But that's hardcore content that's just started auto-playing at my workplace. Reported.

In both cases I got messages back saying the moderators actively disagreed with my reports and that the content would stay.

The problem seems to be that Facebook hired teenage boys and paedophiles to moderate.

Not shocked the BBC got the same results.

Facebook criticised over handling of reports of child exploitation content

Related Stories