WhatsApp moderators can read your messages if someone you talk to reports them

https://gizmodo.com/whatsapp-moderators-can-read-your-messages-1847629241

by celdon25 • 5 years ago

115 56 5 years ago

From the article "WhatsApp can read some of your messages if the recipient reports them."

Is this surprising? Any third party can read part of an e2e encrypted communication if one of the participants forwards it.

contravariant2• 5 years ago

Yeah it's not really too surprising, except that maybe the scale of what gets shared with Facebook is a bit unclear.

I'm not too sure at what point the artificial intelligence program gets involved though.

artiszt• 5 years ago

'a bit unclear' like in having got it half right like in simply unknown ?!

barbazoo3• 5 years ago

But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.

It's not just when a recipient reports them it seems but also when they have been flagged by their algorithm. If that were true, the claim that the conversation is e2e encrypted simply cannot be true, unless the algorithm runs on the client.

spywaregorilla• 5 years ago

Is there any particular reason to believe its not running on the client?

whoisjuan• 5 years ago

It sounds to me that there’s actually an algorithm between the report and the moderator to control the volume of manual moderation.

What you’re describing doesn’t work with E2E encryption. I really doubt it works that way.

KaiserPro3• 5 years ago

just think about the sheer number of people that report stuff.

given that facebook has less than 1k moderators, do you honestly think that they'd just let the moderators sift through everything manually?

obviously you'd classify stuff first, checking against known images is easy. Classifying new images is a lot harder, plus the ethics of training and labelling a dataset for accurate detection is pretty hard, also almost impossible to do legally.

I suspect the next best thing is detecting nudity and age of the subject, and taking the hit that you're going to prioritise a lot of malicious reports, rather than genuine.

TekMol• 5 years ago

The problem here is that the third party controls the software on both ends of the communication. And that software can send the messages to this party without the participants knowingly triggering it.

The article says that by reporting a user, the software on the site of the reporting user silently sends data to WhatsApp. The reporting user does not know what data is sent.

bilal4hmed2• 5 years ago

its not silent, when you report messages there is a prompt that tells the reporter what is happening

take a look https://twitter.com/WABetaInfo/status/1435221936888483847

ectopod2• 5 years ago

There seem to be two things happening.

When a user reports a post it is (unsurprisingly) forwarded to the moderators.

Additionally, there is some kind of AI CSAM detector, which automatically forwards posts.

In both cases, it also forwards the previous five messages from the thread to the moderators.

barbazoo3• 5 years ago

Correct me if I'm wrong but unless the "AI CSAM detector" is running on the client, it simply cannot be e2e encrypted.

spullara2• 5 years ago

It looks like the AI stuff applies to the groups content which is not E2E.

Jtsummers• 5 years ago

Seated at computers in pods organized by work assignments, these hourly workers use special Facebook software to sift through streams of private messages, images and videos that have been reported by WhatsApp users as improper and then screened by the company’s artificial intelligence systems.
Instead, WhatsApp reviewers gain access to private content when users hit the “report” button on the app, identifying a message as allegedly violating the platform’s terms of service. This forwards five messages — the allegedly offending one along with the four previous ones in the exchange, including any images or videos — to WhatsApp in unscrambled form, according to former WhatsApp engineers and moderators. Automated systems then feed these tickets into “reactive” queues for contract workers to assess.

From the actual ProPublica report. If their published understanding is correct, E2EE is not broken, but rather end users who are one of the ends of E2EE are sending the decrypted content to be moderated. The AI bit is a filter to reduce the amount of content passed on to human moderators.

From near that second quote:

Artificial intelligence initiates a second set of queues — so-called proactive ones — by scanning unencrypted data that WhatsApp collects about its users and comparing it against suspicious account information and messaging patterns (a new account rapidly sending out a high volume of chats is evidence of spam), as well as terms and images that have previously been deemed abusive.

That part is AI driven, but my reading is that the moderators do not get access to the encrypted data (the actual messages) only the behavior patterns, and from that make a determination of what to do.

nerdponx• 5 years ago

"AI" running on every client can automatically flag messages and send them to moderators.

Most can agree that violent imagery and CSAM should be monitored and reported; Facebook and Pornhub regularly generate media scandals for not moderating enough. But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.

slim• 5 years ago

E2E is useless if the software is not opensource. Especially if you don't trust the vendor

RcouF1uZ4gsC• 5 years ago

But WhatsApp moderators told ProPublica that the app’s artificial intelligence program sends moderators an inordinate number of harmless posts, like children in bathtubs. Once the flagged content reaches them, ProPublica reports that moderators can see the last five messages in a thread.

It seems like whatever the AI decides is questionable gets sent to moderators automatically, also revealing the thread history (at least partially) as well.

spullara2• 5 years ago

I think the AI was applied against non-E2E encrypted data like groups.

bilal4hmed2• 5 years ago

this says that when you report a message, it sends over the last 5 messages

https://twitter.com/WABetaInfo/status/1435221936888483847

The pro publica article is pure click bait

KaiserPro3• 5 years ago

Its all about signal to noise.

THose who are claiming that they are running Machine learning at the edge, think about this:

how on earth can they trust the data coming from it?

We all know that ML is hard, we also know that to get accurate classification requires serious horsepower.

None of this can be run on phone hardware without people noticing.

If you were to design a system for handling reported data, who's hard constraint is _human_ eyeball time, would you run ML at the edge? No.

You'd make it so the "report this message" sent the last n lines and the attachment, and then run the ML optimised for accuracy (not speed or memory) on your hardware that you control and trust explicitly.

I would have thought all of this is obvious, given that facebook is all about avoiding spending money on human moderators. Edge ML is not going to help you do that.

znpy• 5 years ago

The sadness of my life is that I'm probably bound to be repeating this until I let out my final breath:

If you're not running the service yourself and taking care of end-to-end encryption, somebody can (and probably will) read your messages. Actual privacy of messaging does not exists today. Don't even bother looking for that.

EDIT: just to clarify, it might just be that somebody can access your data for a legitimate purpose (say an operator of the service) and then leak/sell the data.

This is the simplest thing that can happen and that DID happen. The united arab emirates paid a Twitter employee from UAE to leak data about anti-UAE-government from Twitter. People died because of this.

Uberphallus3• 5 years ago

TL;DR: WhatsApp is E2E encrypted, but that doesn't mean there are only 2 ends.

jaywalk2• 5 years ago

No, that's not what the article is saying. There are two ends, but if one of the ends decides to forward the content to WhatsApp, then WhatsApp can read it. Which isn't shocking or breaking E2EE in any way.

sigjuice3• 5 years ago

How does one confirm how many “ends” are there in a conversation? If WhatsApp secretly set themselves (or another third party) up as an “end” in every conversation , that would also not break break E2EE.

Uberphallus3• 5 years ago

There are two ends, but if one of the ends decides to forward the content to WhatsApp, then WhatsApp can read it.

Translation: there are two ends, but one of the ends can send it to yet another end. That sounds like there are more than two ends.

skinkestek• 5 years ago

Can those who claimed that WhatsApp was infallible because it was E2E-encrypted take a step back and see that incentives matters as much as technology?

Please?

betwixthewires2• 5 years ago

When you need moderators in private chat something is messed up socially and technologically.

If you and I are texting, privately, and you say something to me I don't like, I'll tell you., or I will stop talking to you. That's healthy social interaction. If I report it instead, that's not healthy social interaction, and it appears to me the tech companies designing these solutions to problems that were solved a decade ago are more than happy to oblige because it gives them an excuse to surveil and censor. They're encouraging unhealthy social interaction because it serves their ends to do so.

If an instant messaging application has moderators it's not an instant messaging application.

rchaud• 5 years ago

2 billion people use Whatsapp, some of them for purposes that are not as pedestrian as yours.

Private Whatsapp groups can include thousands of people and the topics can be anything from retail discount alerts, to COVID updates, to political organizing. That absolutely merits some kind of reporting capability, particularly if criminal activity is being conducted or is about to be conducted.

betwixthewires2• 5 years ago

Alright well I'm not a WhatsApp user, so I have a couple of questions:

In these large groups that obviously need moderation, are the moderators appointed in some way by WhatsApp, or are they like classical rooms in that the founders of the room pick the moderation team.

Also, do one to one rooms have reporting functionality and who picks the moderator that it is reported to?

Anunayj• 5 years ago

Some people have started using Whatsapp for Telemarketing (message based, not voice). I frequently use the report option to report them.

So I feel like there are reasons to have that.

1st1• 5 years ago

Hm, and just recently WhatsApp management mocked Apple, bragged about their encryption, and said how they would never do anything like that.

https://twitter.com/wcathcart/status/1423701473624395784?s=2...

beermonster2• 5 years ago

They were critical of the client side scanning. So implementation as opposed to the goal. And also a marketing opportunity to take a pop at Apple due to their hubris over privacy and security.

beermonster2• 5 years ago

I read this as a user, whom has received and decrypted an e2ee message, can report it. This will send it, e2ee, to WhatsApp HQ. no breakage here. E2ee is no panacea.

blitzar• 5 years ago

Exactly. Your e2e message is encrypted from your end to the recipents end. They then decrypt the message and forward it to someone else.

Nothing new here. I really dont get what the expectation of these things are for people. Are they not aware, someone can forward a message to a thrid party, people can and do screenshot text and forward it. Writing a message, and sending it encrypted to someone else doesnt protect you from what they might do with the message they now have.

ziml772• 5 years ago

Boggles my mind that people see this as Facebook reading their encrypted messages. The person you sent the message to chose to forward the message to Facebook and are even told that this is happening when they report the message.

You can't stop someone who knows what a message says from sharing the contents of that message without physically restraining and gagging them. They can send a screenshot out, take a photo of the screen, write down the message on paper, or just memorize it and tell others.

dancemethis2• 5 years ago

Boggles mine that people believe proprietary platforms blindly. It can't be proved that there's no backdoor added after the initial consulting for OpenWhisper.

Gee, seeing how they defended Whatsapp tooth and nail time and time again, I'm inclined to not be surprised if it was added _during_.

A4ET8a8uTh0• 5 years ago

The part that aggravates me is that alternatives have their own issues. I managed to get my mom from whatsapp to signal, but when ( I wanted to say if, but that is wishful thinking ) it turns out it is no longer viable I know I will have a hard time migrating her. And even if I do manage to move her, it will be a painful process of re-learning new system for her.

TrianguloY• 5 years ago

When you press the report button, a confirmation message is shown explicitly saying that the last 5 messages will be sent to WhatsApp without the other user knowing. You can then accept or cancel.

You can also copy the whole conversation and send it to the police, or post them on a forum, etc.

WhatsApp moderators can't read your messages, but they can read what the person you talk to send them. E2E encryption means that only the ends should be able to decrypt and read the contents, but what they do with them is another different topic.

wolverine876• 5 years ago

WhatsApp moderators can't read your messages

How do you know?

dancemethis2• 5 years ago

There is no proof their OpenWhisper implementation wasn't tampered with. And chances are against users, since it's proprietary software.

bearcobra• 5 years ago

Alex Stamos twitter thread on the original ProPublica story has some interesting points about the tradeoffs when building these kinds of systems. https://twitter.com/alexstamos/status/1435285445336715265

I don't feel like it's unreasonable for a company to have a system where a user can say "Hey someone is sending me something unwanted using your service" and for that company to use technology to sort those complaints for humans to review and action appropriately based on their terms of service.

thinkingemote• 5 years ago

If you were to ask a (remote) contact "please delete this conversation as it is confidential" how can you be sure they have deleted it?

stevage• 5 years ago

I don't think this is as complicated or hard to understand as this journalist is making out. Any content in a messaging system can be shared by either party with third parties, even if it's encrypted end to end. Hitting the "report" button is just one example of that, sharing the last few messages with WhatsApp moderators.

lsh123• 5 years ago

Is your email private when recipient can take a screenshot and report it to the police? Is your snail mail private if recipient can take it to the police? The end to end encryption (or the envelope you put your snail mail into) only means that nobody but you and recipient(s) can read the content. It doesn’t mean that recipients can’t share said content with 3rd parties.

WhatsApp moderators can read your messages if someone you talk to reports them

Related Stories