WireGuard Servers Running from RAM

https://mullvad.net/tr/help/wireguard-servers-running-from-ram/

by Sami_Lehtinen • 4 years ago

207 71 4 years ago

est• 4 years ago

Reminds me of old days when ed2k servers are entirely running on RAM to avoid raids.

mbreese2• 4 years ago

I read this differently than intended. For the life of me, I couldn’t figure out why RAIDs were so troublesome. I mean, yes, they can be a little slow, so running in RAM would be much faster.

Then I got it. Raid. Not RAID. Made so much more sense.

taneq• 4 years ago

That's why it's more reliable, because you don't need to be spraying bug spray on it. nods

smileybarry• 4 years ago

Maybe they just really don't like 25-man raids in WoW.

BonoboIO2• 4 years ago

Good old times.

SV_BubbleTime3• 4 years ago

I get the immediate appeal of “open source servers that run from RAM”… but it’s the same thing as always, you just have to believe your VPN provider.

There is just no saying that they don’t swap gear out for the auditors or for the disk less servers or whatever it is.

I like that they say the right things. But I think the current models of Facebook and Google show just how much money there is in data, so it’s just faith that the VPN provider won’t fall to temptation.

c_o_n_v_e_x2• 4 years ago

Wouldn't there be a clear performance difference between a RAM based and disk based (even if it's NVMe SSD) machine? If so, isn't that something you can routinely/programmatically authenticate?

dietr1ch• 4 years ago

Would it be clear from 100km+ away?

I think that low latency can act as a proof of being close and fast, but once the apparent network latency goes up how do you know that server times are low if the server can claim the time was spent on the network?

dundarious2• 4 years ago

I doubt there would be any detectable difference. Logging to disk can easily be done out of the hot loop, and I doubt any service with disks has any relevant memory in swap/page file.

The point of the RAM-only server is not that it ensures that everything is operating from RAM. Everything is operating from RAM already even on servers that have attached disks. The point is that RAM-only means there is no intentional or even unintentional logging to non-volatile memory or storage. Think of it as a physics enforced capability system (no disk is physically connected).

TakeBlaster162• 4 years ago

Forwarding packets doesn't need much if any disk i/o, so the server will be running mostly from RAM either way, even if the hard drive isn't physically unplugged.

gruez• 4 years ago

I was going to say that theoretically this was something that trusted computing can solve, but then I realized even if you could guarantee that the server was behaving as advertised (ie. diskless, no log), there's nothing preventing the provider from putting a middlebox in front of the server to log all packet flows. From there it's fairly trivial to deanonymize every connection through traffic analysis.

sanroot99• 4 years ago

I bet some kind of blockchain based trust can solve some issues ,for eg , hardware ownership delegated with blockchain trust .It will add p2p ownership/rent of hardware

oceanplexian• 4 years ago

The provider doesn’t have to do anything. Their upstream transit provider probably has a span port plugged directly into the NSA. Which is what was happening with Room 641A over a decade ago.

On one hand, they won’t be able intercept encrypted VPN traffic, but on the other, they are definitely collecting IP addresses, timestamps and traffic heuristics. With enough resources to pick out patterns and bulk collection on both sides, encryption and logging might not even matter.

glowiesvpn• 4 years ago

"fall to temptation" implies they were legitimate actors in the first place.

Most VPNs are run by small and relatively non-transparent private companies. It would be trivial and in fact the obvious thing to do for law enforcement and similar agencies to setup VPN companies and advertise them to their targets.

ISPs are actually more transparent and their business model does not fundamentally relies on exploiting customer data.

TakeBlaster162• 4 years ago

ISPs are actually more transparent and their business model does not fundamentally relies on exploiting customer data.

To be fair this applies to VPNs too. Their customers give them money, and that should make their business sustainable without needing to spy on people.

To be honest I trust Mullvad way more than I trust my ISP not to sell my DNS queries.

Shared404• 4 years ago

ISPs are actually more transparent and their business model does not fundamentally relies on exploiting customer data.

And yet they do, largely because many people _can't_ vote with their dollars. I can move which VPN provider I use. Also, VPN's are a big benefit on filtered wifi if that's something you have to deal with.

LeoPanthera• 4 years ago

You are right.

But.

Mullvad is more trustworthy that most, and possibly more trustworthy than all other public VPN providers. They have a good history, and are regularly audited: https://mullvad.net/en/blog/tag/audits/

Don't get me wrong, you still have to believe them. But they're easier to believe than others.

patrakov3• 4 years ago

Well, I have a blog post regarding audits: https://patrakov.blogspot.com/2020/01/vpn-privacy-policies-a...

In short: auditors audit providers against their own privacy policies. If there is a loophole, the result, with some auditors, might still be a legitimate "pass" - e.g. because exfiltration is not logging, and the privacy policy says only "we don't log".

P.S. Mullvad's privacy policy falls into the same bucket: "we never store any activity logs of any kind". Sending data to a third party who stores it would be 100% compliant. Mullvad, please fix the wording.

COGlory• 4 years ago

Yup. But you can use crypto or mail cash to Mullvad without your name attached, and then just multihop to them as an endpoint, and it's about as secure as a VPN can be.

SV_BubbleTime3• 4 years ago

I like the idea. But it’s pointless if the connections do log - or leak data in some way.

You are mostly connecting to the VPN with a same or similar IP. So right there; you are back to having to trust they aren’t logging or leaking.

If they are, it doesn’t matter Mull is missing your name, you are already correlated from ISP to endpoint in this case.

Likewise, if they aren’t logging, what does it matter if they have your name and account info?

hajhatten• 4 years ago

I may be wrong. But OVPN is the only one I know that has actually been proven in court not to store any user activity.

https://www.ovpn.com/en/blog/ovpn-wins-court-order

m3adow• 4 years ago

PIA (Privateinternetaccess) had a similar case: https://torrentfreak.com/private-internet-access-no-logging-...

This was before they were bought by some shady company though, so I wouldn't rely on this policy being unchanged anymore.

BonoboIO2• 4 years ago

The bad things is, the past does not predict the future ( would be awesome ^^ )

They did not log, but maybe they are now. I don’t say that ovpn logs, but the 3 letter agencies can infiltrate a company.

panick21_2• 4 years ago

Did you read Mullvad plan for System Transparency? They are doing a lot so that they can actually verify where the server is running.

Foxboron• 4 years ago

So you, the user, can verify what the server is running :)

Disclaimer: I work on this.

INTPenis• 4 years ago

Mullvad is pretty much the only VPN provider saying the right things and doing the right things. They actually publish open source code and articles backing up their statements.

Ontop of that many of us here in Sweden know people at Mullvad and can attest to them being conscious of privacy issues.

rekoil2• 4 years ago

many of us here in Sweden know people at Mullvad and can attest to them being conscious of privacy issues

Me for example, I can vouch for them if that matters to anyone, but it's the same problem isn't it, if people can't trust Mullvad on their word, why should they trust two random HN users they've never met or conversed with before?

Trust in IT systems is super complicated.

hatware• 4 years ago

you just have to believe your VPN provider

Do you have a better plan...?

mbreese2• 4 years ago

Remember — (from their own article announcing these servers):

Running the system in RAM does not prevent the possibility of logging. It does however minimise the risk of accidentally storing something that can later be retrieved.

This doesn’t mean you can’t be logged. Running in RAM just means that any system level logging is transient and largely accidental. But if there were a need to specific logging, data could always be sent to a different node with disks. From Mullvad’s point of view, there is a reliability benefit to having diskless nodes. But from a privacy point of view, your access could still be logged, if required. But it would probably require more “active” monitoring than “passive”/accidental logging.

https://mullvad.net/en/blog/2022/1/12/diskless-infrastructur...

badrabbit2• 4 years ago

Not if the whole os is a VM in ram, still though, network logging can happen by middleware.

preisschild• 4 years ago

Sure you can. For example, with rsyslog you can just send all logs over the network to a logging server (which has disks).

matja• 4 years ago

Even without network logging, the hypervisor can still access the memory of the VM, parsing the kernel process and paging structures to read process memory, which can store transient logging information.

Things like AMD SEV/SEV-ES are designed to mitigate that aspect, the implementation of which can be proven to end-users of the VM by remote attestation.

voltagex_• 4 years ago

https://github.com/mullvad/system-transparency seems to include the stboot mentioned in the article.

rekoil2• 4 years ago

Would be pretty bad transparency if the code providing it wasn't transparent haha.

aborsy4• 4 years ago

Running entirely in RAM is same as running from an encrypted disk, with keys available only to whoever can dump the RAM.

A better solution would be something like Apple’s private relay.

Also, either you trust the provider or you don’t.

stingraycharles• 4 years ago

Apple’s private relay is just CloudFlare WARP with some additional IP blocks + locality built in, not?

I wouldn’t bet on it being super secure in case law enforcement comes after you, for example.

mwint2• 4 years ago

Here’s the whitepaper: https://www.apple.com/privacy/docs/iCloud_Private_Relay_Over...

They do two hops, first to an Apple-controlled server, then to the “second relay” which is operated by Cloudflare in a lot of cases. Encryption is terminated at the second relay.

So Cloudflare sees the content (or whatever is visible in a TLS stream), and Apple sees your real IP, but neither can know both without collaboration.

aborsy4• 4 years ago

Not exactly. WARP hides your content and IP from anyone but the destination website. The website can still track the user. Apple mixed it with own relay such that neither Apple nor Cloudflare can track the IP and content simultaneously.

It’s mostly an anti-tracking feature. But also now government needs cooperation from two companies.

On another note, WARP is a VPN. But Mullvad is preferred to WARP, because Cloudflare most likely logs connections for some time.

viraptor• 4 years ago

Not really equivalent. There are possible attacks based on: key generation process, stored data correlation, access patterns, etc. You're much safer if you don't store anything in the first place.

jeroenhd• 4 years ago

I would trust real TOR much more than Apple's pseudo-TOR. They control all the entry and exit nodes so correlation attacks are quite viable. Might as well chain two VPNs if you want a TOR-like experience without slowdowns. Bonus feature is that you can rotate providers.

aborsy4• 4 years ago

Of course, but Tor speeds are KB/s or low Mb/s in my experience.

agsamek• 4 years ago

Sweden is going to join NATO soon. So do not expect VPNs operating from their teritory and under their law to be any different then the rest of Europe, as it was before.

alrlroipsp2• 4 years ago

How would you expect Sweden joining NATO to change privacy laws for that country?

shapefrog• 4 years ago

Sweden is a 14 eyes country already.

desperadovisa• 4 years ago

Meh, I generally agree with the laws of my country and of the EU, so I don’t really care if they can obtain a warrant to wiretap me. I prefer to live in a society where law enforcement can be effective (with checks!) than one where bad things can happen with zero negative consequences.

Mave83• 4 years ago

Running from memory is no big deal. You gain all and everything like a normal system and can still log persistent to some lokal disk or a remote central log service as it is quite common.

For example, all Ceph storage nodes bootet with croit.io run in RAM and have no OS installed. But you still have all logs and everything available right out of the box.

panick21_2• 4 years ago

Read what they are actually doing, its not just running from ram.

api• 4 years ago

A trick I've used in the past for secrets on cloud providers: store them in "/dev/shm". Requires init after boot from a trusted source of course.

It's absolutely not foolproof but reduces the odds of them ending up on a SAN somewhere where they might be found by someone scanning free space or gaining direct access to an iSCSI bus or similar.

LinuxBender• 4 years ago

Adding to this for completeness sake, if using tmpfs such as the default /dev/shm mount, one must ensure that swap is either encrypted or disabled as tmpfs is swap backed and data not being actively accessed/written can end up in swap on disk.

ajvs• 4 years ago

I think the only thing that would give more trust is a remote attestation setup similar to GrapheneOS[1] since it's the only way to prove that servers are running the software they publicly say they are.

The configuration would have to be run at the hypervisor level, so that logging couldn't simply occur by nesting the server in a VM then logging that.

[1] https://attestation.app

seanw444• 4 years ago

I love that Mullvad is doing these public-facing experiments.

infogulch• 4 years ago

Lets say, hypothetically, that there did exist some trustworthy VPN provider, one that behaved in a way that you would approve (where, if you had unlimited time and access to the internal operation of the provider to evaluate it, then you would eventually be convinced that it's "good", as good as reasonably, humanly possible). So assuming it exists, what would it look like? What would you see that would be positive or negative signs?

WireGuard Servers Running from RAM

Related Stories