Facebook Really Is Spying on You, Just Not Through Your Phone's Mic

https://www.wsj.com/articles/facebook-really-is-spying-on-you-just-not-through-your-phones-mic-1520448644

by asclepi • 8 years ago

750 448 8 years ago

So I was trying to remember the buff old guy from Avatar, Stephen Lang. Once I found him, I noticed that I am suddenly being recommended videos on YouTube that include him.

Okeydokey, machine-learning, moving on.

slededit2• 8 years ago

The claim of the article is that it would be technically infeasible - and they go into how difficult it is to interpret context. However simple keyword matching would be more than enough.

Facebook has a lot of compute resources, but they wouldn't have to use it. Your smartphone is more than fast enough to do simple speech recognition. The accuracy rate wouldn't have to be that high - you won't get mad if you see an ad for a misheard keyword.

monocasa• 8 years ago

Wouldn't that destroy your battery life?

Crespyl• 8 years ago

I don't necessarily believe that FB was literally listening to the phone mic, but it is the case that the FaceBook app was a notorious battery hog.

slededit2• 8 years ago

Radioshack sold a voice recognition chip in the 80s[1] that was a simple 8-bit microcontroller. If you are willing to slip on the accuracy and false positives you can do recognition with very little computation.

[1] http://21stdigitalhome.blogspot.ca/2013/06/vcp200-voice-reco...

jklinger410• 8 years ago

Does the Assistant listening for OK Google destroy your battery life?

Your phone reads sensor data as a base state.

testvox• 8 years ago

The Facebook app does destroy the battery life of certain phones at least. That is the only reason I've actually seen people uninstall it for.

callmeal• 8 years ago

It does. Uninstall FB and see how long you have to go between recharges. In my case it's every day with FB, and every 3 days without.

wuliwong• 8 years ago

I was thinking the same thing. Also not claiming to have any real idea of as to what they are actually doing but certainly having access to everyone's phone gives them a ton of distributed computing power for free.

StanislavPetrov• 8 years ago

I was thinking the same thing. Also not claiming to have any real idea of as to what they are actually doing but certainly having access to everyone's phone gives them a ton of distributed computing power for free.

Only if by "everyone" you mean people foolish and vapid enough to use Facebook and give them access to your phone.

gcb04• 8 years ago

It would be extremely easy to implement.

Ad is easy. You don't have to understand context. Just listen for a thousand or so keywords related to products that are paying you. Then if detection happens apply some rudimentary sentiment analysis on the surrounding phrase and that's all you will ever need.

if you have a couple millions for me to start a small team we can offer this as a service next month or two.

imhoguy2• 8 years ago

The last thing to install on ones phone is FB app - technically one gives up all privacy. If you need to use FB then stick to browser and clear browser data afterwards.

justinhj• 8 years ago

I also have Wechat :)

harryf2• 8 years ago

It's potentially not just the app. The Facebook SDK is embedded in probably 50%+ of _all_ the apps we use e.g. In top 10 most popular cocoapods - https://libraries.io/search?order=desc&platforms=CocoaPods&s... - the FBSDKCoreKit - https://libraries.io/cocoapods/FBSDKCoreKit

tzahola• 8 years ago

But that’s open source and can be audited.

vinchuco3• 8 years ago

Sometimes it even comes pre-installed, for your convenience ;)

lanius• 8 years ago

When I was still on Facebook, I used a 3rd-party Facebook app alternative. I forget which one exactly, but I distinctly remember being impressed that it barely needed any permissions.

whyagaindavid• 8 years ago

You mean tinfoil for Facebook?

jcfrei2• 8 years ago

You also give up most of your privacy if you buy a smartphone with an OS made by Apple or Google.

briandear• 8 years ago

Citation needed if you're going to include Apple in that group.

Here's Apple's privacy policy: https://www.apple.com/legal/privacy/en-ww/

And yes, it's followed relentlessly.

excalibur• 8 years ago

Data brokers run personal information through an algorithm before uploading so it’s not identifiable, Facebook says, but it still can be matched with Facebook account information.

You keep using that word...

MentallyRetired2• 8 years ago

Well, if you had known what you were looking for you would have seen it written on my dorm room window.

Karunamon8• 8 years ago

Similar Guardian article with no paywall: https://www.theguardian.com/technology/2017/nov/09/facebook-...

That said, this is a quite silly conspiracy theory. Believing the "recording" theory requires you to believe that Apple and Google both are in cahoots with Facebook to give them a rootkit-like API for mic and location access that override all of the OS-level controls and warnings about when that hardware is in use, and hide it from the data and battery usage stats on the phone.

This API, when (not if) found, would be a watershed moment for privacy legislation directed at all three companies. Little to gain, potentially the whole farm to lose.

vinchuco3• 8 years ago

this is a quite silly conspiracy theory.

Seems like you put conspiracy theories in two boxes. So what's on the other box?

Karunamon8• 8 years ago

Theories with evidence to support them.

CaptSpify2• 8 years ago

requires you to believe that Apple and Google both are in cahoots with Facebook to give them a rootkit-like API for mic and location access that override all of the OS-level controls and warnings about when that hardware is in use, and hide it from the data and battery usage stats on the phone.

You mean like when they gave Uber special access to grab screenshots even when the Uber app wasn't running? Yeah, totally impossible to believe.

Karunamon8• 8 years ago

You mean the access that had a completely legitimate reason to be used and was trivially located by way of reading the app's manifest files?

minimaxir• 8 years ago

It's worth nothing that the reason security researchers haven't just intercepted the traffic from the Facebook apps to see if its transmitting voice data is because the apps use Certificate Pinning, which prevents the SSL traffic from being decrypted using the SSL certificate generated by mitmproxy/Charles.

In light of that restriction, what might be interesting is looking at the amount of data transferred by the Facebook app with/without the microphone/location services enabled. (this is a data project I have in the pipeline)

jimrandomh• 8 years ago

Note that while certificate pinning does make reverse-engineering harder, it's also a legitimate security feature; without it, anyone who controlled a CA (including most major governments) would be able to forge a certificate and use it to spy on users.

Certificate pinning is a hurdle to reverse engineering, but a surmountable one, at least on Android. Since the app is running on a phone where you may potentially have root, you can pick it apart with a debugger and see the traffic before it leaves the phone. This is technically challenging, but it is something that people do sometimes.

Drdrdrq• 8 years ago

Or you can find and replace CA file/string with your own and then do mitm. There is virtually no way app developers can defend against such hacks on any platform owned by user (so, everywhere except on Apple devices).

guelo• 8 years ago

It's funny how all the security measures for "your safety" always seem to reduce user's transparency and control.

bergie• 8 years ago

I think Android has features for detecting whether the device is rooted. Facebook could then disable spying, in effect Volkswagening the situation

cozzyd• 8 years ago

You could vary the complexity to encode the sounds you input (silence,pure tones, white noise, psychocoustically weighted white noise, etc.) to prove that that they're transmitting them.

threeseed2• 8 years ago

Facebook recording voice data and transmitting to their servers in the US is not only highly illegal in most parts of the world it's also unacceptable to basically everyone. At some point it would have been leaked internally.

The idea that Facebook is doing this is just ridiculous.

TaylorAlexander2• 8 years ago

Surely no large organaization has ever been found to be doing illegal or unwelcome things.

nocman• 8 years ago

^ actually, IMHO the fact that you assume that "The idea that Facebook is doing this is just ridiculous." is what is ridiculous.

Some companies do "highly illegal" things all of the time. It all comes down to the fact that whoever is in charge:

1) doesn't hold to a moral system that restrains them from doing said illegal things (or at least doesn't hold to one consistently)

and

2) thinks they can do said illegal things without getting caught, or if they are caught, thinks they'll be able to recover reasonably well from any punishment (if there is any) that is handed down.

Personally, I do not know whether Facebook is recording/transmitting data like this, but I guess if I found out they were, I would not be surprised -- given the things the company has done in the past, and the things their leadership has said and done in the past.

code_duck3• 8 years ago

What if they just analyze the audio locally for keywords, and then transmit that?

Spooky234• 8 years ago

Most of the analysis done about this controversy boils down to Facebook not listening because they can buy it otherwise obtain data that is more effective.

When you read their responses to this controversy, pay more attention to what they don’t say. Whomever aggregates your viewing habits by listening to audio from your TV may be listening, for example, Facebook just Hoovers up the data.

trophycase• 8 years ago

Leaked internally... to Facebook employees...

mattrices• 8 years ago

You could also look at the variance in amount in different environments for different sensors (noisy/not noisy), (complex image/flat color).

skybrian• 8 years ago

Chrome is moving away from certificate pinning [1].

Even if it's an issue, a security researcher could recompile Chromium or Firefox with certificate pinning turned off and test with that.

[1] https://groups.google.com/a/chromium.org/d/msg/blink-dev/he9...

sofaofthedamned2• 8 years ago

The browsers are moving away from certificate pinning. The apps aren't, as they control both ends. If I made a Facebook replacement today i'd do it too.

dancek• 8 years ago

Couldn't someone just patch the binary to accept a different certificate? Or for that matter, read the decompiled bytecode and figure out if the app is listening? Or run a patched Android that logs all microphone API calls?

I suspect I'm overlooking something, as surely some security researcher would have done some of this already.

sofaofthedamned2• 8 years ago

Yup, patching the binary works fine.

imhoguy2• 8 years ago

Wouldn't it be possible to crack an app (decompile etc) to disable Certificate Pinning? What comes to my mind also is a debug to obtain TLS master-secret then one can decrypt encrypted stream post-factum (Wireshark accepts master-secrets).

omni• 8 years ago

Yes, and there are tools to do exactly that. https://github.com/sensepost/objection

doh2• 8 years ago

While certificate pinning is annoying, it's just an obstacle not a roadblock. I haven't met a security research that mentioned this to be a prohibitive feature.

There are couple ways out, from revers engineering the binaries, through jailbroken/rooted phones, running apps in simulators, etc.

tomc19852• 8 years ago

You can't beat certificate pinning with offline analysis?

usmannk• 8 years ago

It's very feasible to reverse engineer the Android APK for Facebook and replace the cert with your own.

https://serializethoughts.com/2016/08/18/bypassing-ssl-pinni...

http://blog.dewhurstsecurity.com/2015/11/10/mobile-security-...

pdpi• 8 years ago

Facebook already has plenty of issues with people not trusting them. Lying about this and then being caught red handed would be devastating from a public opinion perspective. Even if you assume a completely amoral team that cares about nothing but ad revenue, do you genuinely think they’d risk something as dangerous as that? Especially given how much you can still achieve with the data that is known to be collected?

jimjimjim• 8 years ago

in the current climate, getting caught doing anything just means you have to slime your way out of it, rename/rebadge the activity and then carry on when the attention goes somewhere else.

is uber still the most hated company or has the magnifying glass moved onto somewhere else?

numbsafari• 8 years ago

What makes you think that user trust isn’t one of the things they’re willing to break in order to move fast?

smt883• 8 years ago

I've met a lot of people outside HN who believe FB is listening to their conversations through the mic. I haven't yet met one who has stopped using it as a result.

I think you wildly overestimate how angry users get about privacy violations. For examples, Target, Yahoo, Home Depot, and Equifax have not been screamed into rubble.

pmoriarty2• 8 years ago

How devastating would it be, really?

It would likely be a blip in the news, and then people would move on, as usual.

Remember the Sony rootkit fiasco? People still buy Sony, and most people probably either never heard of that incident, don't remember, or don't care. Buying whatever the new Sony gizmo of the day is is more important to them.

Microsoft has had endless spyware fiascos, and people still routinely buy Windows, as long as they can play their games or run Office, that's all that matters to most of them.

Then there have been scandals like Enron, where the execs knew that they were doing something that was clearly illegal, and that their company really would be devastated if what they did was ever revealed. These "smartest people in the room" did it anyway.

Corporate history is full of just such deceptive and destructive practices. I'm not sure I'd put Facebook above that sort of thing, a priori.

dvfjsdhgfv• 8 years ago

Uber broke almost every moral rule and still everybody's using it.

falcolas2• 8 years ago

being caught red handed would be devastating from a public opinion perspective

For what it's worth, this public opinion backlash has not appeared with other companies: "Of course we're not working on leaked project [x]". "Look at project [x] we're working on!"

Even Facebook's own under-disclosed psychological experiments have been largely forgotten; Facebook has suffered few if any long-term ill effects from it.

z3t4• 8 years ago

Their customers love it, and with customers I mean advertisers. For users to leave there must be somewhere to leave to. And there needs to be another business model that does not depend on marketing. Maybe selling actual hardware, with built in social media.

Spooky234• 8 years ago

Hubris is powerful. Look at the dumb behavior exhibited by Uber as an example.

yAnonymous• 8 years ago

How much worse would being caught lying be compared to straight up admitting it? I'd say it's worth the risk for them.

ransom1538• 8 years ago

"which prevents the SSL traffic from being decrypted using the SSL certificate generated by mitmproxy/Charles"

There are some shady companies out there [1] that use the mic to listen to what shows are being played real time. [TVs in the US are on all the time] (Check out their customer list). These companies need this pinning.

[1] https://www.audiblemagic.com/

1001101• 8 years ago

Different (esp. older) versions might be another variable that would be of interest to a researcher.

feelin_googley• 8 years ago

Is there any reason that pinning should not be optional?

In other words, any reason that the user should not be able to "disable" it? (Use own root CA.)

Consider that in this case, the data being transferred to Facebook belongs to the user.

Is it unreasonable for a user to require that they be able to see what data is being transferred before they agree to transfer it?

fjsolwmv5• 8 years ago

Enterprises do this on PC --- they have to be able to MITM all their user's traffic. Not sure how mobile devices are covered.

As for transparency, look to GDPR and friends to see what rights are being declared.

JetSpiegel• 8 years ago

Considering that certificate pinning protects hacked user devices from making insecure communications, it would completely defeat the point if the user could disable it.

twostorytower• 8 years ago

Certificate pinning is relatively easy to disable on jailbroken iOS devices and rooted Android phones. I'm not highly technical and I did it myself to sniff Snapchat traffic 2 years ago. Any half competent security researcher should be able to do this trivially.

natch3• 8 years ago

If the speech recognition is done on the device, the amount of data could be very small indeed. If I was to do this, what I would do would be to defer the heavy lifting of speech recognition (if it caused any significant load on any of the chips) and perform it in the background, later, when the device is known to be plugged in to power. That way this kind of work would not be impacting the battery. I don't think looking at the amount of data is going to tell you much unless they have done things in a spectacularly stupid way.

PeterisP• 8 years ago

Speech recognition is quite compute-intensive and the load of running it on the device would be obviously detectable - not necessarily by random users, but definitely by any security researcher who'd care to do so.

Also, storing the data until it's plugged in would require an unusually large amount of storage, and that would be detectable.

yorby2• 8 years ago

why not inspect the traffic locally before it gets encrypted?

bitwize3• 8 years ago

This is why I like Caffè Nero's loyalty cards. They are little pieces of cardstock with nine coffee cups printed on them. You get a stamp every time you order. Once all nine cups are stamped, redeem the card for a free coffee. There's no PII on the card and your name isn't going through an additional computer (on top of the standard 17 a day, per Cereal Killer).

More places should do this.

tomcooks• 8 years ago

This is the kebab shop business model in Europe, is it really that rare to find simple loyalty cards with stamps where you live (assuming USA)?

bitwize3• 8 years ago

This is the USA we're talking about, where if an unethical way to wring out more dollars doesn't have a law explicitly forbidding it (and there are laughably few such laws here), it WILL be used against you.

I know of a few places that have simple stamp loyalty cards. Oftentimes they are mom and pop shops, or chains still finding their footing. Most cards are electronic and want a name and phone number associated with them.

VyseofArcadia• 8 years ago

Loyalty stamp cards are pretty common, but most chains (Starbucks etc.) have loyalty apps instead of loyalty cards.

pouetpouet• 8 years ago

Every pizza shop seems to have one in France. I've seen hairdressers with stamp cards in Germany.

justinclift• 8 years ago

Sounds super easy to make your own stamp then. ;)

bitwize3• 8 years ago

Yeah, you'll probably be able to get away with that about once per location, give or take.

EADGBE6• 8 years ago

I've seen a ton of custom hole punches being used in this case more than stamps.

s73v3r_• 8 years ago

Your fingerprints are all over that card! All I have to do is wait for one of those cards to hit their trash, and I'll have a pretty good idea of how long it takes you to fill one up, and so how much coffee you drink.

paulcole5• 8 years ago

I put my Caffè Nero loyalty card in the same pocket as my phone (which runs the Facebook app).

I have no way of proving this but I'm pretty sure they used my camera to read the card because the next thing I knew I was seeing ads for Starbucks (a competing coffee chain). I have never said Starbucks out loud and suddenly I am seeing ads.

sp00ls4• 8 years ago

Highly unlikely. More likely they're using location tracking or a shared data source to know that you like that specific coffee place.

EADGBE6• 8 years ago

That's silly. That probably costs them $0.XX to make.

Loyalty apps/datasets literally print money in comparison.

newnewpdro• 8 years ago

Am I alone in being horrified by the reality that many people today consume most if not all of their news and entertainment holding a camera and microphone on proprietary communication systems running software controlled by the same entities producing the content?

How is this not the greatest threat to democracy in the entire history of democracy?

jadedhacker• 8 years ago

It's just another step in the long slow boiling of the pot. The mass media already are nearly uniform in their interpretation of events with the exception of how they inflame cultural grievance. The American public is the most heavily propagandized in the world.

https://medium.com/@caityjohnstone/no-there-will-not-be-any-...

Edit: removed a section of this comment because it seemed too much to me.

distances• 8 years ago

Here's an example of someone that has encased himself inside of corporate propaganda so completely that it physically surrounds him at work, at home, and even his inner thoughts:

Interesting piece, but was the rent of that place $4800 per month? How does that make any sense?

SCdF• 8 years ago

I am coming around to the idea that this particular grievance-- at least the camera and microphone bit-- is the "terrorists" of privacy.

That is to say: it's an absolute bad thing and is a concern. But because it's so easily visualisable and conceivable, the actual threat is blown way out of proportion, and much less sexy threats that are much more important are ignored.

You're more likely to die slipping over in your bathtub than being killed by terrorism, etc.

I guess the "slipping in your bathtub" for privacy would be the consumerist / capitalist / deregulated culture that allows, normalises and incentivises trading away privacy for convenience with no real pushback. Which in turn actively discourages keeping your privacy, because it's like: why play some constrained rule-set that disadvantages you and no one else plays by?

pmoriarty2• 8 years ago

If in the 1980s, during the height of the Cold War, you had predicted that in 30 years virtually everyone would voluntarily be wearing tracking devices that pinpointed their location 24 hours a day, and voluntarily carried recording devices which could at least in principle record everything they and anyone they talked to said, you'd be branded a conspiracy theorist or a believer in sheer science fiction.

Today it's no longer a paranoid fantasy but a reality, yet even those who aren't in denial about it are mostly not choosing to opt out of the surveillance.

Synaesthesia• 8 years ago

There’s nothing democratic about corporations. They’re essentially tyrannical in their internal structure and unaccountable to the public.

jackjeff2• 8 years ago

I find it amazing that people even notice. I don’t have an ad blocker on all my browsers, and except for YouTube and interstitial ads that Chrome will soon block, most ads are simply not registered by my brain.

zxcb1• 8 years ago

Are you sure? Everyone says that they are “immune” to advertising (or propaganda), and yet here we are where the collective result shows that it works, surprisingly well too. Another theory could be that people just get the causal order wrong:

ad display => subconscious influence, nudge => eventual product purchase, mention etc => recognized ad => spooky feeling

jackjeff2• 8 years ago

Yes. I know it works. Why would these companies spend so much money if it was not. And in addition they have good metrics to know which campaigns work. But this flies in the face of my personal experience... Aside from YouTube, I can’t remember when was the last time I saw and ad and what it was. Must have been weeks ago.

Shikadi• 8 years ago

I find it amazing that so many people find it amazing that other people are different.

username223• 8 years ago

I keyed in my phone number so I could get loyalty points.

Do this: lie. 234-567-8901 works at a lot of US stores. Sometimes you can even use it for gas discounts.

EADGBE6• 8 years ago

I prefer 867-5309.

titzer• 8 years ago

Hey Facebook, if you got nothing to hide, show us the code.

We showed you our friends, our relationships, our interests, our intimate and disarmed states, our rants, and probably half of the websites we visited. (in retrospect, that was dumb)

Oh, your tin can and strings might show? Competitors might get ideas? Please.

Until then, not a fan of you, not clicking on your ads, and generally avoiding your site. In fact I think I'll start deconstructing my profile as soon as I can muster the courage to choke back my gag reflex.

Sincerely, A growing group of mugged social network burnouts.

bubblethink2• 8 years ago

Which code ? Server side code is unreasonable to expect. For client code, you can just use the browser. It may not be as convenient or full featured as the app, but it'll do the job if you absolutely must use FB. The situation is much better with stuff like FB which is not that essential. It's more problematic with Google Maps, where you can't get turn by turn navigation unless you use the app.

kibwen• 8 years ago

Server side code is unreasonable to expect.

Honest question: why would it be unreasonable for us to expect server-side code to be open-source? Facebook's value lies in its brand and its infrastructure, not in its code, so there's no risk of upstarts taking Facebook's code and standing up a clone (which, even with the code, is way easier said than done).

ghettoimp• 8 years ago

Agreed. I've only been using facebook through my phone's web browser and (at least for what I do) it's totally fine. Is there something the app gives you? Maybe for folks who use messages or want notifications...?

kennydude2• 8 years ago

They are VERY pushy about installing the app which is quite fishy. Now all photos have banners saying "Photos look better in the Facebook app" etc

alphakappa• 8 years ago

How does viewing the code give you confidence that the app is running the code you saw?

beisner• 8 years ago

You can pretty easily compare compiled binaries.

criddell• 8 years ago

Why couldn't you decompile the apps to see what it's doing?

hmhrex2• 8 years ago

Reply All did an episode on this that was really good: https://www.gimletmedia.com/reply-all/109-facebook-spying#ep...

The thing I thought was interesting is just how adamant people are about FB spying via mic.

They also had an update in their year-end episode: https://www.gimletmedia.com/reply-all/113-reply-alls-year-en...

Both worth a listen.

smt883• 8 years ago

Something to note as you listen is that people convinced they're being spied on are mostly amused by it. They don't sound outraged or frightened at all.

hmhrex2• 8 years ago

Which is a little surreal. I think I get the same way though. It really freaks me out, but there's not much I can do about it, so I'll laugh it off. I don't use social media (although I'm still on Reddit), and I still feel this way.

falcolas2• 8 years ago

just how adamant people are about FB spying via mic

I'm of the opinion that it's easier for your average Jane/Joe to believe (and maybe even preferable to believe) that someone is listening and responding to your words than a computer piecing together a picture of you from unrelated clues via some nebulous "machine learning algorithm".

Anybody can listen to your words and advertise to you based on them. It is, on the other hand, not feasible for a human to look at a stream of unrelated posts and figure out that you're pregnant.

_sdegutis2• 8 years ago

I might have believed that too, except I know about data profiles and ML, I’m a programmer and can think like that too, and yet I’m still convinced by the examples my wife and our friends personally encounter that there’s no better explanation than that FB is spying on us via the mic and using it to target ads to us.

tomc19852• 8 years ago

Well, Target was doing just that a few years ago, using patterns they noticed among purchasers.

Aunche• 8 years ago

A lot of it is confirmation bias as well. People will remember when an ad is creepily relevant, but they don't remember the dozens of times they were completely irrelevant. It's like when my friends made me watch Stranger Things, it felt like Stranger Things references suddenly started to appear everywhere on the internet.

freshhawk• 8 years ago

I think that's definitely the class of behaviour we're seeing. In general people don't think in terms of second order effects (or anything more abstract). Even if they do understand the concept, they will still prefer to believe something simpler.

Telling people that doing A and B leads to C which leads to D which makes E more likely to happen is just a bunch of gibberish that can't be right because who can you blame?

tim3332• 8 years ago

Humans tend to anthropomorphize - "my printer hates me" etc. It's simpler to think the thing listens than figure facebook included javascript tracking code along with the like button on some independent website that they visited an hour back. Even I have a job figuring what script did what.

kevstev• 8 years ago

My beef was more with google, but my isolated instance of this was pretty damning. I started watching Meet the Press after the election. It airs on Sunday mornings. One day though, it was pre-empted by a golf tournament, I forget which, maybe the British open. I never ever watch golf. I have been golfing maybe once, never search for it, never talk about it, have zero interest in it.

I think I got distracted, or maybe since my show wasn't on, I just decided to go shower, and left it on for a bit. The next week, I start getting notifications about golf on my phone.

There are two possibilities here- Verizon (my cable provider) is making data available on what I watched to google, or google is using my microphone to pick up what I am watching on TV. I don't know which is more likely, but VZ and google having a partnership like that and keeping it secret seems unlikely.

colordrops4• 8 years ago

There isn't strong evidence either way that Facebook is or isn't listening in on the mic. Considering they spy on everything else, and they've exhibited plenty of sociopathic behavior, it's reasonable to assume they listen to audio surreptitiously until proven otherwise.

circadiam• 8 years ago

I'm glad you're not a judge

madeofpalk13• 8 years ago

There isn't strong evidence either way that Facebook is or isn't listening in on the mic.

Is there _any_ evidence that Facebook is spying using the mic? Surely we have to start there, right?

ppbutt2• 8 years ago

Funny how they're adament that Facebook doesn't utilize microphone (while they still don't really know themselves). Why is it laughable that Facebook is using both the FB Pixel, microphone, and other technologies to spy?

Spooky234• 8 years ago

I found that episode annnoying because it implies that the red line is microphone tracking.

They buried a lede — the story laid out a case that Facebook doesn’t need your audio. Isn’t that a bigger story?

madeofpalk13• 8 years ago

They explicitly talk about that in the episode though - the reality of Facebook buying all this credit card and shopping history data might actually be creepier than them listening to your mic.

alanh• 8 years ago

It’s a wonderful podcast. +1 to the recommendation of that episode.

cryptonector• 8 years ago

Step 1: don't use Android. Step 2: don't use smartphones at all. Step 3: black-hole FB and others at your router at home.

I personally skip step 2 because I'm addicted to being able to quickly search for things, read blogs. But even so: a) don't login to any websites, b) clear website data often, c) minimize use of apps, d) recall that when you browse the web from an app you're using the same cookie jar and other state as the main browser, and that you're letting the app track you.

paulie_a• 8 years ago

I think this is the exact opposite approach to take, it is a cat and mouse game to avoid being tracked, and you will always be the mouse.

If anything, go out of your way to ruin the creepy tracking, analytics and bullshit metrics. Add as much noise as possible to their databases.

MentallyRetired2• 8 years ago

Drove by a scion iM in a parking lot one time.. and I said out loud "Scion iM? What the heck is that?" -- ads on IG and FB for Scion iM when I got home 20 minutes later. No search, no associative info, no dealer info... I simply said it.

I've had quite a few of those, and usually I can trace it back to me googling something, etc. But this time, nada.

Johnny5553• 8 years ago

I'd blame this on confirmation bias - there must be dozens if not hundreds of times you use or mention a product every day that you don't notice ads for, but the rare time you do, it reinforces the belief that they are reacting to what you said.

tudelo• 8 years ago

Well you surely can't deny that SOMEONE is listening? Google or Facebook, do some tests and you will find it's true. Confirmation bias is a possibility but from what I have seen and experienced I just can't believe that is always the case.

mywittyname2• 8 years ago

I don't have a FB account, but my wife's account absolutely delivers her ads intended for me. I've not been able to isolate exactly how it is doing it, but the dragnet is absolutely wide enough to capture a household.

She's received ads for Civic Type R (she hates cars), Senior Java Developer (she works in a totally unrelated field), cooking tools (I do all the cooking), and tons of other things. It creeps me out.

SilasX4• 8 years ago

It could very well be, but do you know what kind of evidence you would have to see to conclude that FB is listening? Otherwise you're just making a fully general argument against the possibility of this (or any such scheme) happening.

Side anecdote: One weekday after I vacationed in Tahoe, I saw a BART (subway) ad for Tahoe, and I was like, "oh, great , probably because I just came back from ... wait, that's not possible!"

digitaltrees• 8 years ago

Why don’t you run an experiment, say something totally random that you know you’ve never searched and is out of the range of normal interests you have and see what happens. I have done that on more than one occasion and it confirms we are being recorded. It’s not just Facebook.

paulcole5• 8 years ago

Did you consider the possibility that they were running ads for a car that most people hadn't heard of yet (i.e. it was newly released)?

marrone12• 8 years ago

They say in the article they track your location. I think it's more likely the ad happened from that.

chimmy_chonga• 8 years ago

Ghostery found 39 trackers on that article. The irony is strong today.

cheeze• 8 years ago

Doesn't Ghostery sell data to advertisers too?

whoisjuan• 8 years ago

They probably do.

herogreen2• 8 years ago

Privacy Badger is a good alternative.

christophertino• 8 years ago

We don't sell any data to 3rd parties. You can also see our source code here https://github.com/ghostery/ghostery-extension

akouris• 8 years ago

I think that one should consider the possibility of FB group indirectly accessing this data.

They don't have to collect information only directly from their FB/Instagram/WhatsApp apps: what they can do is buy information from other companies that publish thousands of "free" apps on appstores.

You have to wonder how so many of these free apps seem to sustain themselves since GDN advertising does not seem to be profitable enough.

FB group should be obligated to disclose whether they are buying information from these kinds of third parties.

More importantly they should disclose whether the price they pay is illogical, effectively making them silent partners in an indirect scheme to access your camera/mic information, while at the same time maintaining the allegation that "we do not access your mic through our apps".

1812Overture4• 8 years ago

They buy the info from massive data brokers, who buy data from other large and small data brokers, who buy the data from app makers, services, etc. User123@gmail.com expressed interest in buying kitty litter isn't exactly sensitive information so there's probably not much in the way of auditable logs maintained, probably impossible to determine the original provenance of the data in many cases.

thiagocsf• 8 years ago

I find it hard to take this article without a truckload of salt, given the Wall Street Journal is owned by News Corp.

Murdoch has money to make by gaining leverage over FB.

I haven’t had a Facebook account in almost 4 years and I block their traffic on my router, so I have no love for them. But I seriously doubt this story’s motives.

spinchange• 8 years ago

FWIW (admitedly not very much, anecdotally from another rando HNer) The newsroom side of the Journal is among the vanguard of quality journalism and has largely remainined that way post-Murdoch. The editorial pages on the other hand are extremely opinionated and seemingly in a completely different reality than the newsroom.

odammit• 8 years ago

Paywall?!

I assume they are doing it through toilet cams.

Is this a rehash of the Reply All episode?

https://www.gimletmedia.com/reply-all/109-facebook-spying

matthberg• 8 years ago

http://archive.is/iCnWG non paywalled link.

skjerns2• 8 years ago

thanks!

XMorbius• 8 years ago

I think people really underestimate the power of ad retargeting and advertising analytics. Take in some location information and cross-reference it with friends lists and their product searches and you can explain 90% of these occurrences without voice data.

ntxy• 8 years ago

"How to limit the amount of data Facebook and advertisers are collecting about you"

on wsj.com privacy badger and ublock origin go apeshit..

brandontreb5• 8 years ago

I did quite a bit of Jailbreak/tweak dev in the past, and I was curious if you could just hook into AVAudioRecorder and show an alert any time it was invoked.

So, I did this sort of thing years ago when I wrote a tweak for the InPulse smartwatch (later became Pebble) https://github.com/brandontreb/inPulseNotifier .I was able to hook into the system messaging, forward it to a custom bluetooth stack (sending it to the watch) and forward the message up the stack to be displayed by the system.

It would stand to reason that the same sort of process would be effective for catching Facebook invoking audio recording. Once you hook into the AVAudioRecorder's interface, you could theoretically observe the following:

1. Open the Audio Recorder app and hit Record - An alert should show to prove your tweak is working.

2. Open the Facebook app. If you receive a similar alert at some point, you could at least prove that FB is invoking the audio recorder at some point without the user's expressed permission.

Am I crazy or could this test actually work?

sine• 8 years ago

It's possible Facebook could be using an exclusive method to access hardware more directly, much like how Uber had access to restricted developer debugging tools which allowed them to record the screen even when the app was closed.

https://thehackernews.com/2017/10/uber-screen-record-iphone....

brandontreb5• 8 years ago

It is, however when building a mobile substrate tweak, you have visibility / access to the headers of every single system class. One could theoretically hook into any number of audio recording mechanisms (assuming they knew where to look ;) )

bluesign5• 8 years ago

yeah but this would be obvious (can be seen easily) as it will require some "entitlements"

willstrafach3• 8 years ago

I have checked. Facebook does not do this.

tectonic• 8 years ago

...and I just deleted the Uber app.

gcb04• 8 years ago

If you want to get paranoid... Maybe it can detect jailbreak and do nothing. or even better, detect jail break, use it to detect if there is hooks into the audioRecord interface, if no hooks, record even more with it's new found powers :)

madeofpalk13• 8 years ago

But Facebook doesn't have any extra, special entitlements.

nudpiedo• 8 years ago

you mean introduce a traceable side effect in the underlying dll/system-api? Sure that could work (many debuggers do that), but perhaps they are just not using the same API, or just find another way to stream the data without go through the same interfaces (idk, perhaps through browser APIs or they keep recording all time and just send portions of data which is locally inspected)... it is a good challenge and certainly observing the interruptions hardware could be the right way to go.

In the other hand that is a considerable effort for someone who does not usually work with this part of the stack... would you be able to introduce this changes in an android OS?

brandontreb5• 8 years ago

See my response to the other comment about them using private APIS. Basically, we would have to try and guess which APIs they were accessing under the hood.

But you are right, this would def be a considerable effort for someone not in the jailbreaking space. I would love to hack it up, but unfortunately haven't dabbled in JB dev since 2011.

That's why I posted the comment to HN. In hopes it might inspire someone in that space to build it. Might also be worth jumping in the theos IRC channel. For someone with the toolchain already set up and a jailbroken iOS device, the code is actually pretty trivial.

bluesign5• 8 years ago

This works for sure, but only on jailbroken devices.

On the other hand, facebook can check (at least on IOS) easily if the device is jailbroken and behave differently.

You can also patch binary and inject some code, (probably swizzle AVAudioRecorder methods) for the same effect.

In this case, Facebook can check binary integrity, and change behavior accordingly.

So this is kind a cat and mouse game.

brandontreb5• 8 years ago

Interesting. So you are suggesting they may have already considered this and may have some defensive programming around it?

lucideer• 8 years ago

On the other hand, facebook can check (at least on IOS) easily if the device is jailbroken and behave differently.

I'm not 100% sure what's involved in jailbreaking iOS, but I'm pretty sure on a rooted Android you could put measures in place to "fake" results for any root checks the Facebook app would run. You could patch any APIs Facebook could use to make such checks.

conradev3• 8 years ago

It's not a cat and mouse game if you put this detection in a process that Facebook does not have access to.

perfectstorm• 8 years ago

what we really need is a Facebook whistleblower.

wycy• 8 years ago

If I were Facebook and I were trying to surreptitiously record users via the microphone, I think I would do it by using lower-level hardware APIs rather than high-level Cocoa APIs.

Disclaimer: I don't really know a) if there is some other way to interface with the mic or b) what I'm talking about in general.

mattnewton2• 8 years ago

I am not an expert in this at all, but I would think they would need special permission from apple; many of these undocumented APIs get your app auto rejected.

conradev3• 8 years ago

The Microphone access switch in Privacy settings is not just to make users feel better – it enforces that the app has zero access to the microphone. If someone has reason to believe that's not the case, they should report it to Apple Security.

The tricky bit is when users give microphone access to the app (i.e. for video recording functionality), but want to verify it's only being used then.

nugi• 8 years ago

That is the marketing statement, yes. But the technical implementation is somewhat more complex and possible to bypass than your boilerplate comment suggests.

wtvanhest• 8 years ago

My theory is that a different 3rd party app is listening and that FB/Goog are buying the data without even knowing the 3rd party app is listening. Some of the coincidences could be frequency illusion, but I really don't think so. Some of the coincidences are just too strange.

gcb04• 8 years ago

And that might very well be siri.

Which explain the race to voice assistant, that are used maybe once or twice a year, yet everyone invest millions. from cortana to Echo.

jageen• 8 years ago

I am wonder, can we not just use wireshark and see that is any audio packet send to Facebook or not.

I am not network guy so just asking and seeking for valid explanation.

pliny• 8 years ago

How can you know if a packet contains audio?

ndesaulniers• 8 years ago

As a thought experiment, such content would likely be encrypted. The request size can give away the content type, but speech-to-text could be done on device, making it harder to guess the contents based on request size (assuming the identified speech would be significantly smaller compressed relative to audio).

edf133• 8 years ago

What if...

- The audio packets aren't recorded from the Fb App?

What if...

- The audio packets aren't then sent to Fb?

Fb only buys this data + integrates = problem solved

intrasight• 8 years ago

The only "app" I use to access Facebook and other social media is a well locked down browser. IMHO if you use apps, then your a schmuck and a sitting duck ;-) :-)

MR4D• 8 years ago

Obligatory link: https://www.wsj.com/articles/SB121962391804567765

Key quote (from 1999 - Scott McNealy): "You have zero privacy anyway. Get over it."

Darn - I hate when they're right.

wpasc• 8 years ago

Am I alone in not really caring about my privacy that much? I understand many people care about theirs, and I fully respect their right to their privacy. With that said, I could care less how much data Google/Facebook acquire on me. For now, all it is for more targeted advertisements.

My tune would obviously change if that data were used in more malicious ways. But as long as it is advertising targets, I personally don't care.

jonathanjaeger• 8 years ago

You're not alone. If we're talking unintended consequences of privacy invasions, I'm more inclined to not like something like letting people add you to groups without your permission (e.g. when people were outed by being added to a gay choir group on FB many years ago).

I might be biased because I run ad campaigns on FB and the data is helpful (more efficient than Snapchat or Twitter where they have less data on you).

seangrant• 8 years ago

The fear comes from exactly that. People aren't worried about targeted advertising. People are worried that swathes of their data isn't "theirs" anymore and can suddenly be used maliciously. Sure maybe not by Facebook. But Facebook's customers? The customers to those customers?

psyc• 8 years ago

Of course not. Nobody I know IRL cares about their privacy, save one or two people who also happen to be generally paranoid (I don't mean it's paranoid to worry about Facebook, though). From the looks of the web, people I don't know are far, far less concerned about their privacy.

nabla9• 8 years ago

Are you aware that 'your data' can involve your reactions and small actions like mouse moments and that all that data can be used to steer your emotions and behaviour without you noticing it.

Then there is the the combined effect of everyones data that is threatening.

sverige• 8 years ago

My question is whether I am alone in actually caring very much about my privacy.

I don't consider myself to be paranoid, but I have never been willing to share details of my life with strangers. I recognize that any online activity is subject to surveillance, but I do what I can to minimize sharing that. There's a lot you can do along those lines, with really fairly minimal effort -- though I consider "minimal" to include not having a facebook account and not having any photos of myself on the internet, for example, which I know from having these conversations in the past is for many people some insurmountable hurdle.

Anyway, why would I want strangers to know the details of my personal preferences and tastes? There's no benefit to sharing it as far as I can tell. That is what has always stumped me when people say they don't care about their privacy.

1812Overture4• 8 years ago

Not just online activity. Bank/credit card transactions. Utility/tax information. Postal information. Subscriptions to anything. DMV records. A detailed profile of you is for sale to anyone who cares to pay. Probably less detailed than most people, since you're careful, but enough to be spooky.

rlayton2• 8 years ago

Would you be upset if Facebook used information about you to choose what to display on your feed? Probably not. What about if they were simply experimenting to see if they could make you happier or more depressed by carefully choosing that information?

AlexandrB2• 8 years ago

My tune would obviously change if that data were used in more malicious ways. But as long as it is advertising targets, I personally don't care.

Diamonds are not actually forever, but data is. There's little you can do to prevent Facebook from using the data you've already given them in malicious ways tomorrow. Throw in an economic crisis or major war and it's almost a certainty that they will do so due to desperation or legislation.

jjeaff5• 8 years ago

But how can you be sure that Facebook will keep your data safe? Experian showed us that you cannot trust these data companies to keep your data out of the hands of nefarious people.

MayeulC• 8 years ago

Well, besides the fact that privacy violations are (should be?) against the law and unethical, there are a few considerations you could have.

The one that springs to my mind right now is: Knowledge is power; which is also true in that case: the more an entity knows about you, the more power it has over you. And not only blackmail, but recent hints (AI-powered election meddlings, addictive user interfaces, etc.) have proven that your instinct can be, and will be used against you, whether you are conscious of it or not.

And of course, there are problems linked to physical security (if the wrong person can see you're spending a week abroad, your house might be broken into, or worse).

Let's not dwell on the ethics of financing a company that sells your personal data.

Private data is by definition not supposed to be made public, it is sensitive. Treat it like an attack surface: the more there is out there, the more likely it is that something can be used against you.

groby_b• 8 years ago

If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him.

Once your data is collected, you have no control over it any more. Are you sure there's a valid retention policy, it is actually working and enforced, there are no secret agreements with e.g. the national security apparatus, and it is absolutely secure from malicious employees?

That is the big issue around data collection. Even if you're fine with it being collected now, you might not want a recording of this at a later point. For many different reasons. But a big one being that it's really easy to twist your words against you.

davidedicillo• 8 years ago

1) FB doesn't listen to you. 2) Why are people getting so upset for seeing relevant ads? Ads is what pays for many of the services we use for free, would you rather see obnoxious irrelevant ads?

AlexandrB2• 8 years ago

In my experience most targeted ads are both creepy and irrelevant. It's quite an achievement. After shopping for custom USB keys online I saw custom USB key ads for months. Long after I had already found a supplier for them. The experience is not unlike being followed around a mall by a really pushy salesman after you glance at a toaster; because now he thinks all you want to buy is more toasters.

If you bring this up with someone who works in adtech, the stock response is that we need even more tracking to fix it. It's like the adtech version of "no true Scotsman".

Edit: That's not to mention what a huge waste of money this must be for the client buying this ad space. After ordering something from an Adafruit-like electronics shop, I saw ads for them around the web for quite a while. Often trying to sell me the very thing I had just ordered.

How did I find them in the first place? Second or third page on Google search.

dllthomas• 8 years ago

It's like there's an assumption of increasing marginal utility...

crsmith2• 8 years ago

A couple of weeks ago I search for (on Google) and played (on YouTube, logged into my account) the Meow Mix song to play it for my 4 year old.

Less than an hour later, Instagram showed a Meow Mix advertisement on my wife's account, on her phone.

We have no animals. We have never had animals. She probably (though maybe not) has never logged into Instagram or Facebook on my browser.

It was too much of a coincidence.

bigger_cheese• 8 years ago

I'm not surprised. Facebook track browsing habits use it to share advertisements with your friends.

I've shared this story on HN before this was what I experienced from 2016ish:

I saw an ad buried in my facebook feed to "buy Gallium and Bismuth metal in Australia" I thought it was an oddly specific ad so I made a joking post about it - turned out several of my friends were seeing the same ad. A common friend we all shared who is a high school science teacher spoke up. He explained his class was studying the periodic table and he had purchased samples of Bismuth and Gallium online to show to the class.

I'm absolutely convinced only reason my friends and I saw that ad was because we all shared a friend who was searching for this stuff online.

That level of surveillance is really creepy to me...

bluesign5• 8 years ago

Probably tracked by your IP address.

1812Overture4• 8 years ago

IP address, or if you have any shared accounts you could have a cookie identifying you as the same user. Ads also target connections of people who're interested in a product so you could get the ad from being friends/following someone who watched the videos. The ads will find you.

crsmith2• 8 years ago

Even if FB/Instagram knows my wife and I share an IP address or there were both FB/Instagram and Google cookies on the same browser where I searched for Meow Mix, how would Instagram know my Google/Youtube history?

ipython• 8 years ago

Definitely by IP address. I proxy all traffic for a family member in Central Europe and hilariously enough I get ads for “local” nightclubs that are thousands of miles away from me (east coast US).

What’s funnier is when google decides that it needs to localize my search results for that country. So there is a lot of tracking that assumes all traffic from a single residential IPv4 address is somehow correlated.

6ak74rfy• 8 years ago

I hate Facebook, but I don't think the example you gave is accurate. Both your searches would only be recorded by Google. And why would Google share any of it with Facebook?

The only way the data could have gone over is through a malicious browser extension.

ppbutt2• 8 years ago

Any other source or way to get around the WSJ paywall?

jf• 8 years ago

The best way to get around the WSJ paywall is to subscribe to the WSJ

zamber• 8 years ago

NetGuard https://play.google.com/store/apps/details?id=eu.faircode.ne... allows for blocking traffic for apps per-domain and per-app.

AFWall https://play.google.com/store/apps/details?id=dev.ukanth.ufi... patches iptables to block domains used by apps that are not allowed network access.

I did Android QA for a VPN/Proxy app. Had to benchmark battery usage and verify that we upgraded to SPDY properly. A lot of rooting, and tcpdumping. Eventually we got a WiFi network set up by sysadmins that mirrored traffic to one Ethernet socket. Plugged it in on a local SSH-accessible machine and I could tcpdump the whole WiFi traffic without fiddling with devices.

Facebook and Instagram are awful network hogs. Both send a lot of packets every 5-10s when the screen is on to Facebooks tracking domains.

I removed both and haven't looked back since I saw that.

Google and other tracking companies of course are not better, hence the apps mentioned at the top.

This is not a viable solution long-term though. They will find ways to eventually circumvent these restrictions.

Now I'm thinking about setting up the same thing I had at work at home, just to see what kind of crap goes through my network daily.

You're on Windows 10? Guess what Microsoft does. Install Wireshark and check for yourself how much MS domain hits you get. Not to mention HTTP traffic from random apps and/or websites that circumvent privacy protection domain lists.

The implication here is that you're the product but not for that company or another. You're for all of them, ISP included. The more crap leaks on your network the more they know and sell.

As a side note, I have no idea why DNS traffic still goes in pretty much plaintext...

Have you heard about tracking data exchanges?

The overall problem is that everyone is spying on you on multiple levels because there's money to be made on profiling. Facebook just grabs the headlines. Apps listening to you are scary but the fact is you don't have to speak to be heard by them.

nathanken• 8 years ago

I've disabled auto update for all apps on my android. However, Facebook and Messenger still updates automatically. This itself raises suspicion of Facebook's control over my phone.

aylmao3• 8 years ago

1. Some big features in Facebook/Messenger already exist in the app, and are just enabled/disabled remotely. This is pretty common in most big apps, it's how they can control rollouts. For example, the Snapchat design update: not everyone got it at the same time because the code was living in the app and they gradually enabled it for each user.

2. Wouldn't surprise me if part of the app were heavily reliant on things that can be updated remotely. Chunks of big apps will sometimes be just views fetching some web components. Facebook created React Native, and iirc it can be updated dynamically, like a web site.

When you have lots of people working on an app you have a high probability of introducing bugs. I worked at a company that shipped a faulty update; it looked OK to users, but it was essentially DDoS-ing the servers. Having to wait for the App Store to approve your app to fix things like that is annoying and costly, so people tend to look for alternatives.

antiviral2• 8 years ago

I find this statement from Facebook in the article to be oddly worded, as if it was carefully constructed by a lawyer to avoid getting caught in a lie:

"Facebook does not use your phone's microphone to inform ads or to change what you see in News Feed."

OK... so they are saying they don't use your microphone to target ads. But how about precisely enumerating how FB uses your microphone?

Do they use it for any purpose other than helping you communicate during a call?

Do they try to infer any persona information about you, which can then be used indirectly to make money from your data?

I too have had odd coincidences where eerily relevant ads show up after I have had a conversation. If only FB was more transparent about what they do, I might not be so paranoid about it.

linuxftw• 8 years ago

I think you're on to something here about the indirection.

I too have witnessed the uncanny ads, but not even from my own phone (I don't have Facebook on my phone). A friend mentioned a particular restaurant I have never been to, been near, or searched for.

I can only assume my friend's conversation was geo-tagged either by my phone (android, no non-system mic access), or the data was combined on the server end to place both of us at the same place / same time, and used his recording to market to me.

I'd also like to note, the 'amount of bandwidth needed' is almost nothing by today's standards. Not to mention it can wait to transmit that data until on wifi. 8khz audio (telephone quality) is just kilobytes per second. A reasonably unsophisticated algorithm could trim the audio for an highs and lows (IE, statistically, sound below or above some dB threshold is trimmed because it won't be useful) and uploaded. We're talking about just a few kb per conversation.

Of course, the Facebook app is a memory, storage, and cpu hog (IMO), so I don't think it's unreasonable that given today's modern phone hardware some word recognition software may be present on devices themselves.

chemeng• 8 years ago

I wonder whether there is another mechanism at play. Often when I have conversations with friends, it's likely that either I or they search a term, or have done so recently. Couple that with the social graph and location tracking, and it's very easy for Facebook/Google to target an ad knowing when I interacted with another person and assume that I may have interest in the term that was recently searched by them.

SeanBoocock• 8 years ago

Or consider another scenario: your friend went to said restaurant and paid for something; read an article about the restaurant; viewed/interacted with an ad for the restaurant; liked a social media post for the restaurant; or was geo-tagged at or around the restaurant. Any of the above would be sufficient to associate your friend and the restaurant, and I'd wager targeting your social graph after you've interacted with a product/service would have a good ROI.

antiviral2• 8 years ago

You don't need to transmit the audio. Just transcribe to text, remove any common or stop-words (a, an, the) and transmit what's left over in encrypted form to cover their tracks.

giancarlostoro2• 8 years ago

A creepier one is before I proposed to my fiance I would now and then look up rings. One time I literally said "I need to look up some rings again" and went on Instagram not short after and BAM! There it was an advertisement for rings. It's too damn creepy of a coincidence, and I rarely ever believe in coincidences (sure enough more times than not they're not - at least in my experiences). There's been other moments of things I hadn't looked up or bought that I mentioned verbally and damn it, sure enough an ad on Instagram. It's downright witchcraft. Maybe I'll start documenting the phenomena every time it happens, maybe they'll auto take me out of the A/B testing for this by scraping HN. Ok I needa take off my tinfoil hat... Then again, could Google (Android is on my Phone) sell more detailed data to Facebook? I am going to heavily consider finally rooting my phone and installing a ROM on it.

chenning• 8 years ago

I agree with this. I don't think these companies need to tap your phone to figure out how to choose an Ad for you. They can do an excellent job just knowing that someone is at your house looking up something specific. I suppose one could argue that your privacy is really only worth as much as the person standing next to you values theirs. People just need to understand that their personal privacy decisions really aren't so personal after all.

tim3332• 8 years ago

Obviously facebook uses the microphone if you shoot video or make a call from the app, so the wording is not that odd.

IshKebab• 8 years ago

Yes in America while you are writing a status they use your microphone in a Shazam-like way to suggest things to put in your status. Apparently anyway. Not in America so can't test it.

code_duck3• 8 years ago

I’ve wondered that, too. Their denial is suspiciously specific. In a personal experience that made me wonder if FB was listening through my phone, the item was a friend suggestion.

eeeeeeeeeeeee2• 8 years ago

I've had discussions with people about things I've never searched for or purchased -- discussions around specific companies, and then the same day I start receiving advertisements for those companies, only on Facebook (don't even get Google ones). I have a hard time believing they are re-assembling this from purchase history or any other metadata they have that is not real time. I'm still looking for a simple explanation for this!

Facebook is not a trustworthy company, so I have a hard time believing them about this. But it's why I don't install ANY Facebook applications on my phone anymore and that includes Messenger and Instagram, or any other company they purchase.

wgerard4• 8 years ago

The simple explanation and (maybe scary) truth is that we're a lot more predictable than we think.

Remember when Target got into hot water for outing a pregnant teenager based on an assortment of items she bought [1]? That was based on items like scent-free soap, cotton balls, and vitamin supplements.

That was 6 years ago and (no offense to Target) done by a company that isn't nearly as adept at doing that kind of targeted advertising or inference.

For example, Facebook might know (hypothetically) that you're a young adult in the tech field who lives in Brooklyn and spends a lot of time looking at photography or you're friends with a lot of amateur photographers. So they show you ads for high-end cameras from the most popular vendors, knowing that you're likely to develop an interest in photography if you haven't already.

That's just an extremely simple example, you can imagine that there are far more subtle indicators about someone's interests or likely interests (like the cotton balls for pregnancy).

The timing is almost certainly a coincidence. It's far more likely they show you those ads all the time and you're just noticing them when the coincidence occurs because you're on high alert now for things like this.

1: https://www.forbes.com/sites/kashmirhill/2012/02/16/how-targ...

fny2• 8 years ago

The point of the article is you should be even more paranoid about what Facebook has without your microphone.

With the amount of data that's floating out there about you and your friends, these coincidences are way, way more likely, and are very reasonable if you think about it.

Your friend told you about some restaurant, and then an ad shows up on Facebook.

Why is your friend telling you about it? Maybe he went there recently enough? Maybe he gave the restaurant a review? Maybe the restaurant has a list of people who've been to the restaurant before and is using Facebook ad manager to target those people and their friends? It's on his mind, so maybe that's causing him to leave enough breadcrumbs behind to make Facebook have an inkling of his interest, and you're his friend so Facebook knows to perhaps nudge you as well. And maybe Facebook actually has some idea that you and that person are meeting (yay geotracking!)

Now add a prediction system driven by deep learning models that are terrifyingly good at finding signal with a decent dose of probability (think of all the conversations you've had that didn't result in eery advertising), and you've got yourself a frightening reality where a company doesn't need your thoughts to make a decent guess about what you're thinking.

Frankly it might not even be this complicated. Maybe your network searching for or purchasing things is enough for Facebook to guess you're a decent candidate for the ad too, and you guys just happen to meet up on the same day.

debt• 8 years ago

It seems to be pretty basic. They access the photos within the photo library and the location of the device(both of which you've most likely granted Facebook to access).

FB then cross reference with any other devices nearby and any identified objects within the photos are cross referenced with their ad inventory.

mirimir• 8 years ago

As my meatspace identity, I don't worry about this stuff very much. Indeed, it would be an identifier for me to do so. In meatspace, I'm just a regular guy. Mirimir, he's the privacy freak.

Anything that I want to keep private, I do as a compartmentalized persona. Separate hardware. Separate LAN. Separate Internet connection path. No overlapping Internet activity or interests.

exolymph• 8 years ago

Hell yeah, OPSEC!

aussieguy123• 8 years ago

I just revoked the location permission on android, facebooks app still works fine

natch3• 8 years ago

I've seen stories where people could not explain why they were suddenly seeing certain ads in their feeds, and they thought Facebook had recorded some keywords from a conversation.

For example one guy had a buddy who had recently purchased a certain motorcycle, and all of a sudden he started seeing ads for that motorcycle.

But... really there's a simpler explanation than the microphone. Although of course it doesn't by itself rule out the microphone being used.

Facebook can just see when you are in the same location as some other people, and see what things those other people are into, and then signal whatever ad networks that you might also be a prospect for those things. Visit your buddy and see his new bike? Start getting ads for the same bike. No audio needed, just location services and some posts on FB from your buddy about his motorcycle. And there are other sensors beyond that. A lot of things are possible once the user has granted permission for use of various inputs.

Also if it is the microphone as the story suggests it could be in some cases, the evidence for it being any one particular app is thin. There are other apps that get granted microphone access by users all the time, and some of them should be looked at, not just Facebook. Not to defend Facebook here, but the net should be cast wider than just one app, even if the ads are appearing on Facebook, which itself is perfectly capable of gleaning interest information from multiple sources including other ad networks fed by other apps.

akkartik• 8 years ago

Already said a few months ago: https://www.wired.com/story/facebooks-listening-smartphone-m...

ainiriand4• 8 years ago

In my humble opinion, we are missing something here. I am not a Facebook user since a long time. But something really interesting happens whenever me or my girlfriend are talking about a particular product.

She gets related ads in the web wherever she goes. She has Facebook installed in the phone while I don't.

But the interesting thing is that this is not happening to me. Never. I do not get ads for scuba diving suits if we speak about it. She does. How can we explain this?

jjeaff5• 8 years ago

Sounds like she is just more context aware. I skim past ads and never notice them. Some people might take note if the ad is something they had recently discussed.

ainiriand4• 8 years ago

Well, I pretty much adblock everything... I need to keep thinking a bit more about this.

sexy_seedbox• 8 years ago

Does she scuba dive as well? Or she may have possibly mentioned that you scuba dive in a message to a friend? Or somebody posted a photo of you related to scuba diving without you knowing?

ainiriand4• 8 years ago

We don't scuba dive. But some people in her family does... Now it kind of makes sense. Removes tinfoil hat

alexandercrohde• 8 years ago

If somebody wants to definitively answer this, there's no reason you couldn't just say random advertising related words in front of your phone (1 per day) and check how many come up in your search. There are basic statistical methods to establish if the results can be explained by chance.

This would have the upside of not requiring any reverse-engineering.

remir• 8 years ago

Yeah. For example, if you don't own a cat, you could just talk about cats and brands of cat food in front of your phone for a while. If you end up with a bunch of ads for cat products, then that would be seriously weird.

madeofpalk13• 8 years ago

In saying that, Twitter thinks I:

    * Own a cat, dog and other animal
    * Have between $100k- $999k liquid investible assets
    * Have a net worth between $1 and $1m
    * Am highly affluent
    * Am a high spender
    * Am a frugal spender
    * Own a house
    * Have multiple families

Yet none of these are true (well, I guess apart from the 'has income' demographic I'm in).

I know Twitter isn't known for being an advertising powerhouse (esp. compared to Facebook), but I wouldn't take too much stock in Facebook serving me up irrelevant ads.

I moved to the other side of the world 2 months, updated my "living" location on Facebook and have been tagged at multiple locations in my new city, yet Facebook still serves me ads for buying an iPhone or Car back in my home town.

https://imgur.com/a/HIkic

delroth• 8 years ago

Wouldn't this still be subject to confirmation bias? As in: you'd be more likely to notice advertisement that is related to a word you pronounced recently, since you would be training your brain to look carefully for these words.

ng-user• 8 years ago

I don't have a link right now (youtube though) but I saw a couple do this with 'cat food' and it ultimately resulted in a brigade of ads despite not owning a cat.

nelsonic• 8 years ago

@ng-user The "cat food" couple video (linked to from the Wired Article: https://www.wired.com/story/facebooks-listening-smartphone-m... ...) is: https://www.youtube.com/watch?v=U0SOxb_Lfps

ggg9990• 8 years ago

The notion that Facebook is secretly recording your microphone is beyond idiotic and shows how ignorant the general population is (and consequently how vulnerable). It is the fantasy of an aspiring but untalented screenwriter.

Facebook isn’t spying on your microphone because they don’t have to. They know enough about you to monetize the shit out of you from things that are out in the open. When the populace trusts Facebook and Google with nearly their entire digital lives, and the DOJ lets these giants acquire the rest without a fight, why would they need to resort to clumsy subterfuge?

foobaw2• 8 years ago

It's simply impossible for FB to record audio.

When I worked in the OEM industry, there were rigorous standards and compliances that had to be met to ship our phones. If an app was invoking audio recording without the proper permissions, this would be a huge red flag. Google would never approve the phone to be shipped as it would be breaking their CDD.

Also, if audio data was being transmitted using some obscure APN that does not use mobile data or Wi-Fi, OEMs could still easily detect these from the modem side, no matter how encrypted the data is. After all, the app is still JUST an app, in the system folder with all the other apps like Candy Crush. Unless this specific audio recording feature was built into the Android framework, I will say it is 100% impossible.

Note: this is just for Android. I have no idea how iOS works.

mkonecny2• 8 years ago

It's simply impossible for FB to record audio.

You should check your sources again - that statement is false.... You only have to look at the release notes from the dev preview on Android P today to see that was possible for any background app to access the camera and microphone at any time:

https://developer.android.com/preview/behavior-changes.html#...

Android P strengthens privacy by limiting the ability of background apps to access user input and sensor data. If your app is running in the background on a device running Android P, the system applies the following restrictions to your app: Your app cannot access the microphone or camera.

foobaw2• 8 years ago

It says that apps in the "background" cannot access user input and sensor data. This means that even if the app gains permission in the foreground, it won't have access when the app is running in the background.

Note that this use case was extremely rare anyway, but I know Google did this simply to ease people's minds (I know several people on the Android framework team).

harryVic• 8 years ago

You have to give permission if you want to share pictures, videos, etc. You can't then control when the app decides to use the mic privilege.

sinemetu11• 8 years ago

On desktop I use a custom hosts file redirecting all facebook and instagram traffic to localhost. Can the same be done on my iphone?

A "like" or "instagram this" widget is all they need to track basically everything you do even if you don't have fb.

dav43• 8 years ago

Signed up for Facebook Workplace trial and let it expire. They’ve now shared our details with third party implantation consultants who are approaching us. Not great.

jaaames• 8 years ago

I know that's a typo, but let this comment age a few years and it may not be.

kylelibra• 8 years ago

What is Facebook going to do about GDPR?

vadimberman2• 8 years ago

Facebook works directly with six data brokers, all of which allow you to opt out from their sharing of your personal data, everything from your email to your purchase history.

And that sort of explains everything.

I kept wondering how the info that keep to the Google-verse (search, Gmail) makes it to Facebook. Now I know.

Does it mean that Google collaborates with these data brokers? While it doesn't harm them directly, it seems like a myopic thing to do, arming a company that may undercut your sole major source of income. Yes, I imagine they take it directly from the device, but doesn't Google have control over the Android internals?

Also, I am in somewhat unique position. Being a Microsoft zealot, I still carry a Windows phone (v8.1) with slowly dying services. I almost don't use Facebook yet I still get these too relevant ads, mostly according to what I google on my desktop.

gvurrdon• 8 years ago

How much use do you make of Bing, if you don't mind me asking? Is it a case of mainly using it (as you are a Microsoft fan) but occasionally needing to refer to a Google search?

vadimberman2• 8 years ago

I am more used to Google which I use on the desktop, but when on the phone, I use both interchangeably.

When it comes to the US and global content, there is virtually no difference in the results. Google is much better in the local content and the knowledge graph results though, as well as the maps. Video search is better in Bing.

halayli• 8 years ago

I have suspicion that at least instagram is listening. I started saying words that I've never looked up for in the past, nor have I ever saw an ad related to them like 'dog walker', 'babies'.. and over the course of minutes I started seeing ads for both. Not sure how to explain this.

habosa2• 8 years ago

I work at Google and I'm a long time hobbyist Android developer. I have a lot of non-technical friends ask me about this.

What I always say is that this would not be possible. To constantly listen for audio, process it, and upload the results would use so much data and battery that you'd notice. Only a plugged in laptop/desktop could get away with this.

Am I wrong? Would it even be technically feasible for Facebook to listen all the time?

sattoshi• 8 years ago

"OK Google" seems to handle it.

habosa2• 8 years ago

I'm pretty sure it's optimized to listen for a single hotword and then initiate the network connection to process what comes next.

What people accuse Facebook of doing would require constant processing of the audio stream for random interesting tokens. Either process on device and then send metadata (more battery, less network) or send the raw stream and process on the server (less battery, more network).

flukus• 8 years ago

To constantly listen for audio, process it, and upload the results would use so much data and battery that you'd notice.

A few years ago I uninstalled the official facebook app because it was using all of my battery and a fair bit of data, even when I wasn't actively using it. The battery usage record in the settings indicated the facebook app was the cause. After removal it felt like I'd bought a new phone, everything was running so much faster. I recommended the change to friends who saw similar results.

I don't know if it was maliciousness or incompetence behind the battery usage, but it would seem that you can get away with it largely unnoticed.

AndrewUnmuted• 8 years ago

There's no reason the application must always be listening. Facebook could do their voodoo to determine when during the day you are most likely to be in a social atmosphere, and begin listening to the microphone only during that range of time.

Processing and uploading doesn't have to happen all at once. They could analyze/process the audio while you're asleep. The power drain would be negligible - or completely unnoticeable if your phone is charging while you sleep. Data obtained from these processes could be uploaded in chunks, hidden inside the photos and videos you upload from the Facebook app.

So, in my opinion, not only is this all possible, it'd be not too difficult for Facebook to hide it from the general public.

pdkl956• 8 years ago

would use so much data

You shouldn't assume that and actually do the math. GSM-HR (an early 90s codec that is fast to encode and designed to be low-power) combined with a trial noise gate will only require[1] uploading <100 kB/hr.

Only a plugged in laptop/desktop

Audio compression existed before MP3/AAC. GSM-HR only uses 5.6 kbit/s, and was designed to be encoded on early (GSM) cellphone embedded CPUs/DSPs. Modern devices have several orders of magnitude more CPU/power/bandwidth available.

all the time

It's only necessary to record when voice is actually present. A trivial noise-gate[2] will cut the recording time down to only a few percent of the time at most. Obviously a better filter should be possible.

[1] https://news.ycombinator.com/item?id=16346416

[2] https://en.wikipedia.org/wiki/Noise_gate

edf133• 8 years ago

This "I work at Google" thing... doesn't give instant knowledge on all areas.

There are many ways this could be covered, as many have said here (Above & Below). Noise gates... other Apps listening for key phrases etc.

E.g.

A Fitness App could be installed with a noise gate + setup to listen for specific fitness related terms... that triggers a simple pules to say person A is interested in that topic....

Another App does the same for topic B... and the list goes on...

That's just off the top of my head. (I don't work at Google)

stordoff2• 8 years ago

I've tested it on my own phone (Lumia 920), and a background audio recording uses about a half to a quarter as much power as the Gmail (IMAP) app - it's pretty negligible. Compressing it to send over the wire might use more (I was recording to WAV), but with the right choice of codec (e.g. GSM-HR as mentioned below) I can't see it adding all that much. A common complaint is that the Facebook app uses a lot of battery anyway, so the costs of audio recording probably _could_ (I doubt they _are_) be buried in that.

product506• 8 years ago

I know we are all really concerned about privacy but we need to take a step back here for a second. Facebook will continue to show you ads - that will never stop. By following what the author mentions in his article, all that will happen is that you will see less relevant and useless ads. Would you rather less ads which are useful or those which are completely disconnected from what you want? I have discovered some really nice SMB stores via FB ads - that would have never happened otherwise.

Good luck to the author as now he will see generic AT&T and Galaxy S9 ads. Privacy has its costs and one should make an informed decision eitherways.

jeena• 8 years ago

It might be just me, but I rather see irellevant ads like on TV which don't have that much power influencing my decisions on what to buy and what not to buy than a really targeted ad which makes it almost impossible to resist to spend my money on something I really don't need but just want.

product506• 8 years ago

In that case, you should follow the author's steps. I personally love the ads which FB and Google show - they are well targeted and look nice and some of them are very influential which I think is a good thing. Otherwise, I will still see ads but they aren't that helpful and just take up space.

pdkl956• 8 years ago

Would you rather less ads which are useful or those which are completely disconnected from what you want?

You're making the common mistake of assuming "targeted advertising" means targeted to what you want. The point is to allow marketing to target specific groups. Any overlap with your interests is just a coincidence.

product506• 8 years ago

Sorry I don't get you. I am specifically talking about allowing FB to target you based on your interests - so that you see more relevant ads.

epx• 8 years ago

I uninstalled FB from my phone because it cut the battery life in one third, even if not used at all. One can imagine how busy it was behind the curtain to need all these joules...

ShorsHammer• 8 years ago

For all the perhaps unfounded paranoia here and elsewhere, still yet to see a major manufacturer offer hardware camera/mic switches.

Call me cynical but surely there's some top-down pressure about adding a simple $0.07 component to $1000+ devices.

Torai2• 8 years ago

Full article here:

http://archive.is/dghhy

tontonius• 8 years ago

The real MVP is always in the comments

Torai2• 8 years ago

Thank you!

madmax96• 8 years ago

Honest question:

If this behavior disturbs us so much, why does anyone continue using Facebook?

lucozade• 8 years ago

I'd suggest it's because 'us' is a very small, and not exactly unbiased, sample of 'anyone'.

jokoon• 8 years ago

I have an android 6, from archos, which is relatively clean in term of vendor layer.

I regularly go in the storage part of apps, and despite the fact that I don't use many apps, they still manage to generate megabytes of data, which are not clean by the "clean cache" functionality.

I'm not using it for texts and calls, I only use it with wifi, but still, I have low trust in the android ecosystem.

megamindbrian2• 8 years ago

I wish they would disable distractions while driving.

mmariani• 8 years ago

It’s really too easy to tap the “I’m not driving” button. We have a global cell-phone-use-while-driving epidemic going on that’s taking way too many innocent lives. I wish tech companies would start sharing phone use data while driving. That’s when people will start to care about the lives of others and the rule of law, the moment they see their premiums going up things will get better rather quickly.

k3a• 8 years ago

GNU founder Stallman has a big page with many other reasons against using Facebook. You should read it too.. https://stallman.org/facebook.html

Facebook Really Is Spying on You, Just Not Through Your Phone's Mic

Related Stories