VPN by Google One security assessment

https://research.nccgroup.com/2022/12/09/public-report-vpn-by-google-one-security-assessment/

by campuscodi • 4 years ago

189 127 4 years ago

research.nccgroup.com

bri3d4• 4 years ago

The actual pentest findings here are pretty boring, but the architecture details are quite interesting, as are the identified risk models for the attempt at "anonymity."

The fundamental anonymity model Google have employed is that the VPN tunnel is authenticated using "blind tokens" which are signed by an authorization service "Zinc." The Zinc service validates OAuth credentials passed in from the client app to verify a user's subscription, then signs the blind token created by the client app to authorize the user's session with the actual VPN backend "Copper." Ostensibly, this will separate the user's Google identity from the VPN identity since the VPN service only knows that it got a valid, signed user token, not what user that token belongs to.

Ultimately, as pointed out in the report, this is IMHO not that useful. Google could still trivially re-correlate traffic inside of the VPN service by using packet inspection combined with a crafted (or accidental!) plaintext sidechannel from an authenticated Google client application (unencrypted DNS requests to a special subdomain, port/IP knocking style requests to a specific list of IPs in a specific order, etc.).

Also, if there's timestamped logging in both the Zinc and Copper services, the attempt at blinding between the two sides of the system is also quite meaningless since the flow to Zinc and the flow to Copper could just be back-correlated back into a user identity using timing by a Google employee with access to logs.

jeffbee2• 4 years ago

It seems very silly to analyze the Google VPN with Google as your privacy adversary. The point of the VPN is to hide your traffic from 9th-circle privacy adversaries like local telephone companies.

It also seems pretty silly to mix in interesting findings in a big PDF sprinkled among other items that are, speaking generously, debatable matters of opinion, like whether programs should be stripped or not.

bri3d4• 4 years ago

I agree with this - VPNs protect you from only the ends of the chain, while centralizing the middle, and it's rather foolish to think that any VPN construct will reliably blind traffic and identity from the VPN provider. Depending on your threat model this is a good thing or a bad thing overall.

However, this "privacy" goal is something that a huge number of VPN providers, including Google, claim as a goal and market heavily against, so it's fair to assess against it IHMO.

As for the second part, yeah, that's pentest reports in a nutshell. A few nuggets of information combined with checklist-satisfying irrelevant findings from automated tooling.

Izmaki3• 4 years ago

One such "interesting finding" is that the VPN software is run with admin privileges, including the parts that deal with concepts which would not need admin privileges, such as debug logging, because this increases the attack surface for a hacker who already has limited control of the machine, to potentially elevate their privileges, if they manage to find vulnerability.

This is one of their medium findings. This report smells like "Guys, we MUST find something, give me something, anything!"... :P

dfc• 4 years ago

What does "9th-circle privacy adversary" mean?

zoklet-enjoyer2• 4 years ago

Yeah, I just use it so my employer can't see what websites I'm browsing

anon84873628• 4 years ago

I assumed people are mostly worried about Google being compelled to de-anonymize the traffic (for a government).

encryptluks22• 4 years ago

I pay for Mullvad and also have Google One. I use Google One for times that I have to connect to an unknown WiFi network but don't care about if Google knows it is me. I use Mullvad when I want some confidence in not being tracked.

chiefalchemist• 4 years ago

Why not Mullvad all the time?

class4behavior• 4 years ago

but don't care about if Google knows it is me

This is not what privacy primarily is about, though. Taking Cypher's choice is also not some profound path.

While privacy is indeed a measure protecting one against becoming a victim or a target of malicious intent. First and foremost, however, privacy determines who is in control and has power over you through (access to) knowledge related to you. It's a human right that shapes all aspects of any society, not just some best practice like locking one's door.

The problem is people don't think or care much about far away ills, that's why it is too easy for us to dismiss and normalize the degradation of our rights.

rasz• 4 years ago

The fundamental anonymity model Google have employed is that the VPN tunnel is authenticated using "blind tokens" which are signed by an authorization service "Zinc." The Zinc service validates OAuth credentials passed in from the client app to verify a user's subscription, then signs the blind token created by the client app to authorize the user's session with the actual VPN backend "Copper." Ostensibly, this will separate the user's Google identity from the VPN identity since the VPN service only knows that it got a valid, signed user token, not what user that token belongs to.

In video form for those not keeping up: https://www.youtube.com/watch?v=y8OnoxKotPQ

World1774• 4 years ago

The fundamental anonymity model Google have employed is that the VPN tunnel is authenticated using "blind tokens" which are signed by an authorization service "Zinc." The Zinc service validates OAuth credentials passed in from the client app to verify a user's subscription, then signs the blind token created by the client app to authorize the user's session with the actual VPN backend "Copper." Ostensibly, this will separate the user's Google identity from the VPN identity since the VPN service only knows that it got a valid, signed user token, not what user that token belongs to.

In case you needed even more information on how this works, Google provides on their page [1]

The blinding algorithm employed was first described by Chaum in 19826, and is commonly referred to as ‘RSA Blind Signing’. The goal is to never use the same identifier in the Authentication server and the Key Management Service. To accomplish this, the client generates a token, hashes it using a Full Domain Hash, and combines it with a random value and the server’s public signing key to produce a blinded token. That blinded token is then signed by our authentication server. When the client wants to connect to the VPN, it can unblind the blinded token and its signature using the random value only it knows. The unblinded token and the signature are then verifiable by our Key Management Server.

This is an easy way to understand how this works. This actually does not sound like the start-of-the-art for cryptography. In their example, the client has to provide the secret, which, then corresponds to a hash that was encrypted using this secret key along with Zinc's public key. Zinc knows who provided it the blinded token that it signed. On revelation of this secret key, Zinc could immediately determine who is the owner.

Tornado Cash bypasses this leak of information by using a zero knowledge proof involving a Merkle path [2]

[1] https://one.google.com/about/vpn/howitworks

[2] https://github.com/tornadocash/tornado-core

jmole• 4 years ago

FTA [1]: "the only piece that links the authentication server to the data tunnel server is a single, public key, used to sign all blinded tokens presented during a limited period of time"

It sounds like the only information Google could produce is: "yes, this traffic came from our VPN, here is a list of the 20,000 users who logged in during this key rotation period"

Am I missing something?

World1774• 4 years ago

This actually does not sound like the start-of-the-art for cryptography.

I'm wrong. Review the paper Google provided as a source for more information. [1] There are also lectures available online from Ronald Rivest, [2] one of the researchers that helped to invent/discover the RSA cryptosystem. (If you are not familiar with how RSA works)

[1] https://www.hit.bme.hu/~buttyan/courses/BMEVIHIM219/2009/Cha...

[2] https://www.youtube.com/watch?v=v4qaxSKTVlA

skybrian• 4 years ago

Although it might not be very reassuring to people on the outside who treat Google as a monolithic threat, internal controls like this do make insider attacks more difficult, by making them more visible to internal security and audits.

In particular, it sounds like it would be difficult to do it via log analysis alone; it would have to be a more active attack and may require code changes. Either changing code in source control or somehow swapping in a different binary can be pretty noisy (assuming there are good controls over the build and deploy process).

You might compare with having good accounting. You can't just trust people when they say, "we have strict accounting" because they could very well be lying, but nonetheless, not having proper accounting is a red flag.

yakak• 4 years ago

It seems interesting to me that even if we consider Google as a monolith, the authorities with abilities to compel Google are not monolithic and have varied rights and varied legitimacy. Quite a few groups that could subpoena logs could not compel active actions like wire tapping, etc.

Izmaki3• 4 years ago

Writing that report must have been hell with - I bet - all the attention from Sr. management.

sillysaurusx6• 4 years ago

The actual process of writing those reports is generally pretty laid back. It was hell mostly because it consumes all your time. Only about 50% of your time is spent hacking, if that. The rest is poured into that damn report.

It goes through multiple rounds of review, and every small detail is caught and corrected. Which is how it should be. It’s just the opposite of fun.

The report is literally the product. The implications of that didn’t sink in until a few months on the job.

rfoo• 4 years ago

Not to mention you HAVE to put some really lame recommendation here.

"Application binary is not obfuscated", seriously, wtf?

tptacek16• 4 years ago

Nobody spends 50% of their time writing reports. If anything, it's the exact opposite: people do their entire doc the last Friday of the test. It's enough of a problem that carefully scoped tests sometimes include doc days in the SOW. I used to get in arguments with the person we hired to run one of our offices for demanding that people stay late on Fridays to finish docs that customers weren't even going to look at until the following week.

This "multiple rounds of review" thing is news to me too.

To answer the original comment: the ordinary process for a project like this with a public report is, you deliver the whole project as if there wasn't a public report, with an internal report, and then the public report is based on the internal one (probably post-remediation). So actually delivering the project is usually no different than any other project.

bink4• 4 years ago

I'm going to stay away from the tire fire of accusations between you and the former NCC boss. But I do want to say that having run a pen testing firm for many, many years these claims aren't hard to believe. It does depend on what you consider to be time spent working on a report, however.

50% time actually hacking even sounds kinda high depending on how the organization is structured. We had some engineers that just absolutely sucked at writing documentation but were wizzes at hacking so they got away with passing some of the documentation off on others (dramatically increasing the other engineers workloads, but sometimes that's ok). We also had some folks that were just great at formatting and dealing with client requests for docs and they could take workload off of engineers and make everyone happy.

An engagement ranged from 3 days to 6 weeks, with the average around 3 weeks. A typical internal pen test would be 2 weeks onsite followed by a week for writing the report and handling feedback (and travel and meetings and blah blah blah). It wasn't unusual for a client to request that all references to X be changed to Y or the word "should" be changed to "shall" or whatever other thing client management cared about. That time can add up quickly and make your job miserable.

Every single pen testing firm goes through a phase where they realize they are "wasting" weeks of effort on writing their reports. And they all decide they're going to automate as much as they can to improve their profit margins and employee happiness. They all usually come up with a snarky name for the new tool that does this automation. Most of the time the generation tool saves pain when it comes to formatting tables, table of contents, glossaries, etc., but wastes time wrestling with client expectations that their report be unique or focus on unusual areas. Then the engineer has to figure out how to modify the auto generated report without breaking any future changes to finding severity or writeup.

Some people would also consider time spent presenting the report to a board of directors as time spent "working on a report" and I wouldn't really disagree. It's not unusual to send an engineer and a manager to do a presentation at a remote site after the engagement finishes. That'll include 1-2 days of travel time and the time the group spends trying to "up sell" the client on further engagements.

It's been some years since I've been in that field but back in my day a break down of a 3 week engagement could look something like this:

First week is the fun part. You probably spend 30-40 billable hours on actual discovery, scanning, testing, hacking. And if you're smart you spend 3-5 hours on the early design of the report. The client might want a focus on the API or on WiFi or whatever and you arrange things so the report is organized properly early on and the team can input their findings as needed (or input them to the automated system which might not have a dedicated category for this focus area).

Much of the second week could be considered part of writing the report. You've ideally root'd most of the infra in the first week and now you're using tools and manual inspection to find every single vuln you can in the networks. You're using automated tools to find common mis-configurations and missing patches. Those tools are (hopefully) connected to your automated report generating system somehow so you don't need to format them manually. But there's always new stuff on each engagement that'll need some custom write-ups and severity determination will need to be adjusted depending on the client's security posture and risk profile.

The third week is 1-2 days getting the report ready, though that can extend further for unique engagements. Then there's submitting it to a tech editor, submitting it to management, submitting it to the client and dealing with corrections from all of them. It's extremely rare for a client not to have comments and changes that need to be addressed and that's another 2-8 hours.

Pen testing is difficult -- the travel, the presentations, the sales, the report writing. I'm not saying what this commenter is claiming is true, but the job is not how most people imagine it to be. I'll also say that that's not a criticism of NCC at all. All firms I've worked for, managed, or talked with are like this.

wglb• 4 years ago

I worked at Matasano for two years and did many tens of assessments. None even remotely approached 50% of the time spent on the assessment.

I left there before the Litany of Failure was completed. I did security assessments on my own and wrote something similar to Litany of Failure (I think it is better) and the thing is that you should be documenting as you go along so it is not such a painful crush at the end.

svet_0• 4 years ago

Really boring report, nothing of real essence. Either G's product is bullet-proof or the research quality from NCC has deteriorated.

nibbleshifter3• 4 years ago

The quality of work from NCC has always been really inconsistent. It also depends on "which part of NCC" you were looking at.

Aachen2• 4 years ago

It also depends on "which part of NCC" you were looking at.

For the security consultancy part of NCC, it's not like I catalogue and re-check their findings so probably this is biased, but the only report I remember is the one from Keybase where they failed to notice that the claimed end to end encryption trusts the server to deliver the right keys. This was tested together with some other people on HN and packet capturing (one theory was that it checks the third-party websites like reddit/HN/... proofs, and that it's user error if you don't have any, but no, not even that).

I was really surprised by both Keybase getting something so fundamental wrong (they claim some blockchain magic verification which you can do on the command line, but the app doesn't have a blockchain client and no manual fallback either, so it's never verifying anything and instead fully trusts the centralized Keybase-operated proprietary servers) and by NCC not noticing this problem. Someone I knew from the security stackexchange site and whom I admire greatly took part in the audit, but of course they never replied (not even declining to comment) when I emailed them with a question about how this verification works (at that point, I still felt like I must be missing something so this email wasn't phrased accusatorily).

I don't have a bad impression of NCC in general and we all make mistakes, but yeah that's the example that stuck in my mind.

jokowueu• 4 years ago

I hope they audit outline next

xnx3• 4 years ago

Google One VPN is a big deal, and potentially as big a nightmare for shady data brokers as the iOS cookie apocalypse was for Facebook and others. Google One was available to paid subscribers on Android, then on Windows, and Mac. Now some Pixel owners can use it. Imagine if they turn it on for all Chrome OS users, or all Chrome mobile users, or all Chrome users period. You can have your suspicions about Google, but I'd trust them over any other tech company or VPN provider.

WanderPanda• 4 years ago

Can you elaborate why you trust them over Apple?

xnx3• 4 years ago

Probably a tie with Apple. I like that Apple isn't trying to sell my attention (yet), but I do prefer the higher level of openness from Google (Android, Chrome, open source contributions, etc.)

dathinab2• 4 years ago

Maybe due to Apple acts in/wrt. China.

People somehow end up thinking Apple cares deeply about their private but AFIK Apple only cares about privacy because it makes them money. Or more specifically the image it creates helps them in selling phones.But the moment the _long term_ cost outweighs the benefits they also have shown to bail. (I highlighted long terms because there had been case where short term it was more expensive because it e.g. involved legal cases, but especially this legal cases are a grate image profit form them as people will use it like "don't worry see they didn't even brake their security to help the police with <insert arrested very bad person>").

Now that doesn't mean Apple is bad to some degree it's a win win for the user: They get better privacy and Apple gets more users. But it doesn't mean you can fully trust Apple (or IMHO any Company).

syrrim• 4 years ago

Its a service that costs them money. Unless it gains them money somehow, they won't hand it out for free.

hrrsn• 4 years ago

Google One is already a paid service. This is just Google catching up to iCloud Private Relay.

YetAnotherNick• 4 years ago

Internet’s bandwidth has to literally grow by something like 70% as everything would need two hops if it was to be implemented. Also it would be spammer’s dream, as IP is the number one factor that is used in spam detection.

jmclnx• 4 years ago

“With VPN by Google One, we will never use the VPN connection to track, log, or *sell* your online activity.

This can be expanded to: "And by using Google VPN, only Google can see your search history and WEB Sites you visit. Apple will not see anything they do not need to see." :)

quotemstr2• 4 years ago

Some of the findings are ridiculous. To highlight the lack of binary obfuscation as a threat (albeit a "low-severity" one) is absurd. Kerckhoff's principle [1] applies. Security through obscurity (or obfuscation) is no security at all. To highlight the lack of binary obfuscation (which, in its maximal form, would be the client being open source software) as a security threat is the "demand" for security vulnerabilities exceeding the "supply".

If you, as a business, make money from identifying security vulnerabilities in applications, then you have every incentive to invent vulnerabilities where not exist. And if you're a client of such a service, then no matter how conscientious you are, no matter how much attention you pay to security, someone, somewhere, will be able to claim that he's discovered a vulnerability if he's able to make arbitrarily hostile assumptions about the underlying platform.

In the limit: "Attack: Program fails to defend against an attack in which Intel specially and secretly recognizes this program's machine code and sends the contents of its memory space to the NSA. Recommendation: Program ceases to exist.". Give me a break!

[1] https://en.wikipedia.org/wiki/Kerckhoffs%27s_principle

smoldesu2• 4 years ago

If binary obfuscation is a threat, then every iPhone and Android handset ships with a pretty broken threat model. I don't disagree with what you're saying, but binary fuzzing is pretty much par for the course on these matters.

quotemstr2• 4 years ago

Huh? The system libraries of Android are not especially obfuscated

BeefWellington• 4 years ago

    To highlight the lack of binary obfuscation as a threat (albeit a "low-severity" one) is absurd. Kerckhoff's principle [1] applies.

The report doesn't call lack of binary obfuscation a "low-severity" threat. They mention it as an informational finding. Informational is not Low severity and the rest of your rant seems based on this misunderstanding.

You're not wrong in the general case about pentest firms having an incentive to make severity up; you're just wrong about that applying to this particular case.

oars• 4 years ago

What is Google's long term strategy with this product? Or will it be another thing they deprecate in a few years...

SXX• 4 years ago

I guess it would be super easy to verify whatever Google VPN have any privacy in it at all. Just start using their VPN to agressively parse Google own services and look if it get's your VPN account banned.

javajosh• 4 years ago

Starting on page 13, where they talk about the broadcast receiver issues, the report mentions "tiktok libraries" several times. What's going on there? Is Google's VPN solution dependent on tiktok in some way?

ywoski• 4 years ago

TikTok is the name of Google's internal (and convoluted) framework for first party Android apps.

They have some internal libraries developers are supposed to use when handling intents that prevent common mistakes developers might make.

VPN by Google One security assessment

Related Stories