Google listening in to your room shows importance of privacy defense in depth

https://www.privateinternetaccess.com/blog/2015/06/google-chrome-listening-in-to-your-room-shows-the-importance-of-privacy-defense-in-depth/

by arto • 11 years ago

178 121 11 years ago

privateinternetaccess.com

uptown• 11 years ago

Type this into your Chrome address bar to see the extension status: chrome://voicesearch/

nakedrobot2• 11 years ago

NaCl Enabled Yes Microphone Yes Audio Capture Allowed Yes Current Language en-US Hotword Previous Language en-US Hotword Search Enabled No Always-on Hotword Search Enabled No Hotword Audio Logging Enabled No

What now?

AdmiralAsshat• 11 years ago

If you're running Chromium: Re-install with the opt-out flag.

If you're runnning Google Chrome: Weep.

thestepafter• 11 years ago

Go to chrome://settings/

Uncheck: Enable "Ok Google" to start a voice search.

anaptdemise2• 11 years ago

I have never had the "OK Google" voice search option enabled and chrome://voicesearch/ shown below. I deleted the hotword file specified as the extension path and also `rm rf` directories under the "Extensions/lccekmodgklaepjeofjdjpbminllajkg" specified as the Shared Module path. I also changed owner of the directory "lccekmodgklaepjeofjdjpbminllajkg" to root and the permission to read only. After restarting chrome, the extension/module still shows up in the chrome://voicesearch/ and Enabled in the chrome://extensions/ when --show-component-extension-options is used. Further, chrome://extensions/ shows "Hotword triggering" as allowed in Incognito Mode. Unchecking the box is automatically re-enabled.

  Google Chrome	43.0.2357.124 ()
  OS	Mac OS X
  NaCl Enabled	Yes
  Microphone	Yes
  Audio Capture Allowed	Yes
  Current Language	en-US
  Hotword Previous Language	en-US
  Hotword Search Enabled	No
  Always-on Hotword Search Enabled	No
  Hotword Audio Logging Enabled	No
  Field trial	Install
  Start Page State	No Start Page Service
  Extension Id	nbpagnldghgfoolbancepceaanlmhfmd
  Extension Version	0.0.1.4
  Extension Path	/Applications/Google Chrome.app/Contents/Versions/43.0.2357.124/Google Chrome Framework.framework/Resources/hotword
  Extension State	ENABLED
  Shared Module Id	lccekmodgklaepjeofjdjpbminllajkg
  Shared Module Version	0.3.0.5
  Shared Module Path	<user path redacted>/Application Support/Google/Chrome/Default/Extensions/lccekmodgklaepjeofjdjpbminllajkg/0.3.0.5_0
  Shared Module State	ENABLED

Mithaldu2• 11 years ago

This is pretty funny. He complains that Chromium managed to "bypass this audit-then-build process", by downloading stuff afterwards, while ignoring that this happening already shows that the audit process is completely useless since it failed to recognize and reject the code that would do this.

There's a TSA joke somewhere in this.

fweespeech2• 11 years ago

The audit process assumes the upstream maintainers aren't malicious actors attempting to actively bypass the process:

After upgrading chromium to 43, I noticed that when it is running and immediately after the machine is on-line it silently starts downloading "Chrome Hotword Shared Module" extension, which contains a binary without source code. There seems no opt-out config.

They don't have the resources to do a full audit on all the packages:

https://www.debian.org/security/audit/

Due to the sheer size of the current Debian release it is infeasible for a small team to be able to audit all the packages, so there is a system of prioritizing packages which are more security sensitive.

lfender6445• 11 years ago

if you goto chrome://settings/content it appears you can disable mic + camera access for all pages

rasz_pl2• 11 years ago

I bet this is for something else, probably html5, just like Flash has mic/camera access permissions.

markbnj• 11 years ago

I imagine that will break hangouts... not that hangouts on the desktop isn't already broken.

rasz_pl2• 11 years ago

I wonder if European Commission would be interested in adding this to their investigation, couple of hundred million dollars should be enough penalty for violating users privacy.

deoptimo2• 11 years ago

I agree with the sentiment, but I don't think a competition investigation is the appropriate forum to deal with this. There should be a fine, perhaps through a class action or criminal conviction.

I certainly hope the penalty for illegally wiretapping hundreds of millions of people is more than $1 per head. I value the privacy of my conversations (and life) way more than that.

Besides, if I opt-in to this service, why should everyone who ever walks into my home or office be presumed to have made the same choice? What if I am a doctor or lawyer who is not legally allowed to make that choice?

deoptimo2• 11 years ago

Why the down vote?

gress• 11 years ago

A couple of hundred million dollars is a trivial write off for google. It should be a 5% of revenue.

mondoshawan4• 11 years ago

Can we stop with the witch hunts and the desire to maim folks over minor infractions like this?

Consider that you're one of the developers that wrote this feature. You try very hard to make sure your users privacy rights are respected. Normally your work is strong and you catch all of the corner cases, but this one you missed. You've fixed it upstream, but folks are demanding 5% of the company's bottom line because of a mistake /you/ made. The code isn't even used unless the user ticks a box to turn it on in the first place, which is even verifiable with a cursory use of system monitoring tools like fuser, lsof, etc.

You've been marked as costing a company a major stake of their income. You'll likely never live that down.

If you don't like the company's behavior, /don't use the software/. Simple, clean, and effective in large numbers -- more so than regulation action. Chromium is an open source first browser, we as a community actually have a hand in its development. If you don't like this, fix it.

Oletros10• 11 years ago

Downloading a binary blob is violating user privacy?

ape4• 11 years ago

I always assumed the purpose of Chrome was to spy on us. So I don't use it.

derptron• 11 years ago

Wow, is there really no way to disable this? I guess I'm going back to Firefox.

jonknee5• 11 years ago

Of course there is a way to disable this, it's in the settings with all the other settings.

plorkyeran• 11 years ago

Where? You can turn off voice-activated search, but that doesn't actually disable the extension.

feld2• 11 years ago

Is there a GPO to control this setting in Windows?

edit:

This might work

http://www.chromium.org/administrators/policy-list-3#AudioCa...

jmnicolas• 11 years ago

I think this is the last straw for me : enough is enough.

I need to sit and reflect a bit on this, but I'm contemplating abandoning every bit of non free software I currently use (and there's a lot of it since I'm using Windows and Android).

kiba• 11 years ago

It appears that we're stuck in a tradeoff regarding software:

Libre, high quality, and user friendly. Pick two.

mrj• 11 years ago

Firefox is open, high quality and user friendly.

fixermark6• 11 years ago

I think we're all missing the point of the article.

The author isn't claiming that free software is immune to this either (think about the flaw hiding in plain sight in OpenSSH for years). The author is saying that if you want to protect yourself from image / audio capture without your consent, make sure you can physically deactivate those features on your hardware in a way that software cannot physically re-activate them, i.e. mechanical switches.

metric102• 11 years ago

Has anyone actually confirmed that Chrome is continuously sending audio back to Google? I highly doubt that this is the case. Instead, the plug in knows how to recognize "OK Google" all by itself. Once activated, then it starts sending audio data.

IF it where really listening even when inactive, then people would be complaining about it sucking up bandwidth and data allotments.

51Cards• 11 years ago

I was thinking something similar... that would be a LOT of data being sent back continuously while Chrome was running, which for many people is 24/7. If you had an office of Chrome users the entire office pipe could be eaten up by what is essentially X number continuous VOIP calls.

metric102• 11 years ago

Right. There's no way this would go unnoticed on any enterprise or government network, and once noticed it would end the use Chrome in those environments. In fact it would have unimaginably huge consequences. It would mean huge numbers of CEOs and government officials in all countries where being bugged, all dependent on Google securely keeping millions of audio streams private. This doesn't pass the giggle test.

Guvante• 11 years ago

It would be the worst possible implementation of the feature for a laundry list of reasons.

6d6b732• 11 years ago

First of all, they need to send only data when there is something to listen, so above a specific threshold. They can also compress the data. Second, I'm sure that if Google can handle traffic that's generated by youtube they can handle voip traffic.

DennisP2• 11 years ago

It's probably not actively listening to all users. But if it has the capability, then it can be activated upon request by law enforcement or the NSA.

jonknee5• 11 years ago

Any computer with a microphone (almost all!) has this capability.

declan2• 11 years ago

This happened 12 years ago when the FBI tried to eavesdrop on conversations taking place in a car with OnStar or a similar device. Agents wanted to remotely activate the car's microphone; I wrote about the case here: http://news.cnet.com/Court-to-FBI-No-spying-on-in-car-comput...

The 9th Circuit said "no," but the court's reasoning wasn't based on privacy concerns. The reasoning was that companies can only be forced to comply with wiretaps when the order would cause a "minimum of interference" (and the FBI's tap would have disabled the call-an-operator feature if there were an emergency, which exceeded the "minimum of interference" threshold).

This is not unique to Google, which has done a better job than just about any company I can think of at fighting off overly broad surveillance demands; see my post from two years ago for examples: https://news.ycombinator.com/item?id=5725899

Why could Apple, Microsoft, or Samsung not be compelled -- let's assume an actual court order exists -- to deliver a software update to a specific user that allows FBI agents remote access to that device microphone? Or AT&T? Or Verizon? We know from recent history that AT&T is hardly likely to put up a fight.

I wrote more about the outer limits of the Feds' surveillance authority here: http://www.cnet.com/news/how-the-u-s-forces-net-firms-to-coo... Excerpt: "Precedents were established a decade or so ago when the government obtained legal orders compelling companies to install custom eavesdropping hardware on their networks..." And earlier: "In 1977, the U.S. Supreme Court ruled that surveillance law is a "direct command to federal courts to compel, upon request, any assistance necessary to accomplish an electronic interception..."

In terms of a hierarchy of privacy protection, I trust technology > courts > Congress > DOJ oversight > FBI.

tzgur8• 11 years ago

good point. However, don't trust them to care about your privacy too much. This is never the case.

Mithaldu2• 11 years ago

I just tried the feature out on my windows pc with chrome, and chrome only reacts to ok google when you have a new, empty tab, or the google search page open and active as the main tab. Additionally i checked with procmon what network activity chrome was making, and while it starts sending stuff AFTER "ok google" is activated, it doesn't send any between me saying it and chrome confirming it.

The theory that it's a small local plugin is also affirmed by the fact that my cellphone can do "ok google" without any sort of network, and is sometimes tricked into activating by audiobooks that make noises completely unlike "ok google".

BostonEnginerd• 11 years ago

I'm not sure of the use of the feature if it requires you to touch your keyboard and mouse in order to open up a new window to use it.

j423• 11 years ago

It's not about consistently being bugged though--I see two troubling implications to this;

a) Government A decides target B has valuable communications, and uses this audio capture functionality as an attack vector (ie, a MiTM server modifies the chrome binary blob request slightly to a version where chunked audio is sent back to a control server).

b) (more likely) This binary blob contains a voice recognition algorithm, which can of course detect the phrase "ok, google." Imagine they wanted to detect other phrases, like "drugs" or "travel." Small modifications could easily allow an arbitrary list of "hot" terms to be targeted. Then no audio is even sent back from the user's computer--a small flag in your google account is simply set attaching your profile to "high risk" terms overheard, and databased, where it could later be queried by law enforcement.

It's troubling because there's no transparency, and if you spend a little time brainstorming about the ways this could be used maliciously (most likely by a nation-state) there are many possibilities...

lern_too_spel• 11 years ago

This is true of any auto-updating software, including your operating system and all evergreen browsers.

The problem with this blog post is that the author gets in a tizzy about what could happen, not what is actually happening. What could happen has not been affected by the recent Chromium screw-up, nor is it specific to Chromium.

valas• 11 years ago

What sort of transparency would satisfy you in this case?

mindslight• 11 years ago

c) (even more likely) The algorithm released in the blob is much simpler than the one locked in their servers, and it has many false positives. The audio from false positives is archived to help improve the simplistic algorithm.

jonknee5• 11 years ago

It can definitely detect OK Google on its own, here's what it looks like when you say "OK Google" when not connected to the internet (and it only shows up after saying it):

http://i.imgur.com/CCTGbsk.png

Oletros10• 11 years ago

Has anyone actually confirmed that Chrome is continuously sending audio back to Google

I would change it to:

Has anyone actually confirmed that Chrome is continuously listening?

tremon2• 11 years ago

Tell you what: you get me the source code, and I'll tell you what it's doing.

I'd say it's pretty much a given that it's continuously listening. How else would it know when to respond to an "ok, google" audio cue?

azakai• 11 years ago

The problem is that it cannot perfectly recognize "OK Google". There are going to be false positives, things that sound like "OK Google" to the plugin, but are not "OK Google". And it will send those recordings, plus what said after them, to Google servers.

codys• 11 years ago

Has anyone actually confirmed that Chrome is continuously sending audio back to Google?

That's not relevant. The problem is that they _could_ do that (for some subset of users, potentially) and in most cases no one would even notice it.

mgiuca• 11 years ago

More importantly, the plugin does not even run unless you opt in to hotwording (by checking the check box in settings). The open source Chromium code makes sure of this. So you do not need to take our word for it.

Please see my statement here for details: https://code.google.com/p/chromium/issues/detail?id=500922#c...

Furthermore, you are right that if you turn on the "Ok Google" setting, the plugin will start listening to your microphone, but will not send audio to Google servers unless it hears an "Ok Google".

turk-• 11 years ago

How does he know Chrome is transmitting ALL conversations that it hears? His arguments aren't valid:

"(Ok, so how does it know to start listening just before I’m about to say ‘Ok, Google?’)"

This could easily be achieved offline.

The same argument could be made for Siri, a wiretapping device which you carry with you all the time. In fact wiretapping your phone would be much more effective then wiretapping a computer browser application.

Before making such accusations he should present some solid data, like network traffic from an idle chrome application during conversations (with and without saying "Okay Google"). If an idle chrome application was always transmitting data to google, he would have a solid argument.

dreamfactory24• 11 years ago

listening doesn't necessarily mean recording and recording doesn't necessarily mean sending

fstutzman• 11 years ago

Agreed - there is a lot of speculation here.

In the U.S., implementation of a wiretapping scheme like this would be a significant civil and criminal violation. Google is already under a FTC consent degree for privacy violations, which would make it doubly egregious. So I'll give Google the benefit of the doubt - and assume there are privacy-protective mechanisms designed in to the system.

In the age of ambient technology, where anything can be recorded, the onus is now on developers to create systems that internally suppress mass privacy violation. The "Privacy by Design" approach (disclosure, I have an evangelist of PbD) can provide solid guidance as to how to build privacy-protective mechanisms (e.g. data minimization, data scrubbing) into ambient technologies.

jonstokes2• 11 years ago

Not surprised. I was a die-hard android user, but I kept having stuff like this happen to me, over and over again: http://www.reddit.com/r/technology/comments/2kwbl2/im_convin...

It also happens to my android-using friends. I've become convinced that Android phones are listening all the time so that they can figure out what we're about to search for and what to advertise to us.

Given that this has been my (admittedly anecdotal) experience with Android, I wouldn't be surprised at all if Google was trying to take this type of thing to the desktop with Chrome.

I love Google and have historically just not cared about my privacy as far as they're concerned, but I'm getting more creeped out as this kind of stuff becomes more pervasive.

Oletros10• 11 years ago

Can you provide any proof of that?

No, Android phones are not listening all the time

jonstokes2• 11 years ago

Nope, I have no proof, and of course I'd be happy to learn that they aren't listening. I just hypothesize that they are based on my experience, but I've not tried to test it in any systematic way.

fixermark6• 11 years ago

Targeted advertising is actually quite good at separating you the person from you your marketing footprint. Unfortunately, the ways it does this are hard to explain, but let me give an example.

A friend of mine was convinced that Google was data-mining his Gmail because it recommended a flight on a related search atop the dates he was considering visiting a friend (if I recall correctly, he looked up driving directions and Google suggested "You could get there faster if you took <FLIGHT> on <EXACT DATE HE AND HIS FRIEND HAD BEEN TALKING ABOUT FLYING>").

I punched in the same directions as him in an incognito browser session and... Got the same flight suggestion. Then I looked at the nearby flights and found that the suggested one was really just the cheapest one that week. Turns out his friend and he were planning to meet on a day they happened to be free... Because it was near a holiday, so a lot of people were free, so airlines factored that into their pricing model, so Google recommended a flight because it was a popular flight that a lot of people wanted to take between those two destination points.

It's far, far more likely that targeted advertising is working because we are not the special snowflakes we believe ourselves to be. The correlation algorithms can guess a lot from a few data points when they have billions of correlations to sort through.

Too2• 11 years ago

You might not have entered your old friends in your computer ever but they might have entered you. Just "invert" this relation and its quite reasonable to assume you know them also. Still spooky though.

bhauer2• 11 years ago

I want my desktop operating system to offer fairly fine-grained control of permissions I selectively grant to processes/applications. I would like the ability to easily revoke Chrome's ability to use my audio inputs, and then—if the use case comes up, such as a WebRTC conference—I can grant permission either on a one-time basis or until I revoke. This would be the operating system controlling the application's capability.

I'm guessing a rough approximation is possible on some operating systems. Given the sprawling management infrastructure in Windows, I wouldn't be surprised if it has some "policy" framework in place that allows devices to be declared off-limits at a process granularity. The missing piece, then, is a viable user interface on top of that.

I'm not asking for something akin to the simplified permissions model of mainstream sandboxed mobile operating systems. Not set-and-forget; and certainly not all-or-nothing ("accept these required permissions or don't install the app.") Rather, something quite a bit finer grained and with the necessary infrastructure to have the OS prompt for privileged access if the application wants something I've disallowed, in a manner akin to Windows UAC prompts for admin credentials.

Imagine starting Chrome one day to have your operating system prompt you, "Chrome would like access to audio input 1 (microphone). Allow for now, permanently, or deny?"

ionised2• 11 years ago

An HIPS can do much of this on Windows. It takes some training though.

I use the free one that comes with Comodo Firewall, but I am unaware of any free, quality, stand alone alternatives.

spdustin• 11 years ago

Okay, I'm going to put on my tin foil hat for a bit here.

Think of the corporate boardrooms with Chromebox for meetings, listening in even when not actively used for meetings. An exec at the Better Business Bureau [0] who chose Chromebox because they were excited to, "[reduce] the time [they spend] ... worrying about security concerns," is discussing the growing complaints the BBB has received about a competitor to a company owned by Google. He says, "Ok, Google owns their primary competitor, and they may have insight to offer us."

Wait, that's just my tin foil beanie. Let me put on the tin foil balaclava.

The U.S. Department of State [1] is in an all-hands-on-deck crisis meeting over a deeply divisive political situation involving a first-world ally. Chrome is updated with the eavesdropping feature (remember, it's just my tin foil that's making me choose that word, I know it's hyperbole), and it's already been "deployed to production immediately, bypassing cumbersome testing." Someone in the meeting says, "OK, Google News has been trending a lot of stories about this issue." Sensitive things are then said about this ally, things that are now being heard by an enemy of the state, because they were able to use their previously embedded network sniffers to capture and forward interesting network traffic.

It's frightening that a feature is enabled by default, and difficult to disable, that could capture sensitive conversations without the knowledge of the parties speaking because they innocently started a sentence with, "OK, Google." Certainly this violates wiretapping laws?

Let's pile on. Hospitals and medical centers are using this too, according to the Chrome for Work pages. A doctor says, "Ok, Google had a lot of results about new HLA-B27 research," when discussing a patient's arthritic concerns, while proceeding to outline the patient's symptoms and how treatment should proceed and now we're looking at a potential HIPAA Privacy Rule violation.

As I type this, I look over at my Amazon Echo, and I'm reminded of something I heard once. If you're not paying, you're not the customer, you're the product. Is that hypocritical of me to accept my Amazon Echo but not the behavior of Google Chrome?

[0] https://www.google.com/work/chrome/resources/customer-storie...

[1] https://www.google.com/work/chrome/resources/customer-storie...

fixermark6• 11 years ago

Yes it is. ;) The author wants us to remember that we live in a world that has these technological possibilities.

That doctor's phone should have had a mechanical switch to disable the microphone, is the author's point.

izzydata• 11 years ago

My only audio recording device is my webcam and it is incapable of being in use without the light being on as far as I am aware. So how would it send them audio data without the audio device realize it is being used?

noir_lord4• 11 years ago

My only audio recording device is my webcam and it is incapable of being in use without the light being on as far as I am aware.

You assume that the light isn't controlled by the same software, that's not always the case, the FBI has admitted this.

https://grahamcluley.com/2013/12/webcam-spying-without-turni...

I take the view that anything under software control or the control of a chip I can't open is suspect, I've taped the webcam on laptops and physically disconnected built in microphones (I use a headset, built in ones suck).

I'm not really happy about my Nexus 4 at all either, I think my next phone will be a dumb mobile.

The sad thing is that I try to avoid paranoia with this stuff but the threat landscape is so large it's practically a full time job staying up to date with whats going on.

w8rbt• 11 years ago

One thing to keep in mind. If your friend has a Google device using Chrome, and you are close to them (in the same room) it hears your voice as well.

Oletros10• 11 years ago

Any source for this?

alfiedotwtf• 11 years ago

Physics

Oletros10• 11 years ago

has the author really tested if Chromium is listening?

Downloading a binary blob is very bad, but the accusations that author makes wihouth a single proof is more FUD than anything

fixermark6• 11 years ago

It's irrelevant to the argument. The author is making an appeal to the slippery slope (even if it's not happening right now, and even if Google is not doing it.... It could easily happen tomorrow and anyone could do it. Shutter your cameras and make sure your microphones can be physically deactivated).

bduerst3• 11 years ago

Isn't an argument based solely on FUD essentially a slippery slope?

anaptdemise2• 11 years ago

There is also a bug report from a year ago. https://code.google.com/p/chromium/issues/detail?id=381747

zobzu2• 11 years ago

Note that all android phones have that issue. Also all windows phones and soon windows 10.

Oh and smart tvs. it is a real problem though

McGlockenshire2• 11 years ago

all android phones have that issue

Citation needed.

"Ok Google" functionality is an expressly required opt-in. I had to go out of my way to turn it on.

zobzu2• 11 years ago

google phones (nexuses) pretty much turn it on as u hit next next next on install thats actually pretty similar the chrome tvs also warn you usually - but nontech ppl dont notice. next next.

jimhefferon• 11 years ago

This is why there needs to be a switch on all computers to physically turn the microphone off.

How is that hard?

fixermark6• 11 years ago

Physical switches are the most common point of mechanical failure. It translates to real-world lost revenue in terms of returns and repairs to include them at all.

... but yes, they should still be included. ;)

nulltype2• 11 years ago

There's a light for the camera, they should just add one for the microphone.

Too2• 11 years ago

Same for wifi, camera and speakers. Preferably a real physical lid for the camera, the switches are in most cases just connected to an io pin and could potentially be overridden. Even if they are not its just very discomforting to have a camera constantly pointing at you, I feel weird when i see a lens even when I know the power is totally cut.

For speakers it has happened to me a few times that something has started to play loud sounds and I can not mute it because windows is locked up somehow or im in fullscreen or whatever.

neumino• 11 years ago

When you’re installing a version of GNU/Linux like Debian or Ubuntu onto a fresh computer, thousands of really smart people have analyzed every line of human-readable source code

If this was true, Debian would have not build/release this version of Chromium. The author is living in the past or in another dimension. Some projects are complex, and it's hard/impossible to read/understand everything for a single human being.

wslh• 11 years ago

What happen if you play an audio file saying "Ok Google" ?

Google listening in to your room shows importance of privacy defense in depth

Related Stories