Tenement Farming and Cloud HSMs

by jandeboevrie • 2 years ago

40 19 2 years ago

tptacek2• 2 years ago

I don't think the term "HSM" really carries much meaning, and if you're familiar with the space, you probably won't find it too shocking to hear someone claim that Amazon has put more cryptographic expertise to bear on KMS than has been embedded in any of the well-known rackmount HSMs.

greggsy• 2 years ago

I’m still not totally convinced that the HSMs are tamper evident. I’ve read (ok perused) the compliance reports, and they just say ‘yeah we have HSMs and they’re secure’, but what assurances can consumers have with the integrity of the hardware, short of being able to walk into a datacentre (never going to happen), install their own product, and lock the rack with their own key?

It’s unreasonable, expensive, and probably unnecessary and irrational for the vast majority of customers, but it still niggles at me from time to time.

alfanick• 2 years ago

I lately dealt with some infrastructure issue, involving HSM. Literally "we have HSMs and they're secure", except they don't work, and audit logs are non-existing. And if you enable them, they sometimes work (mostly not), and give output in some undocumented binary format. Not fun experience.

j16sdiz• 2 years ago

With rack mount HSM, I can monitor for physical access with security camera.

When it is on cloud, who knows what?

lxgr2• 2 years ago

The obvious solution is to use a cloud security camera! It might even be showing a real server rack somewhere.

Whinner• 2 years ago

Can you trust the security camera isn’t breached and not showing a false stream?

latchkey2• 2 years ago

Pretty much the plot of every single bank heist movie.

nickf• 2 years ago

...and many of the cloud providers with HSM-backed services (Azure KeyVault, AWS KMS) don't give you any verifiable proof. At least with GCP you can download the attestations and verify them with the HSM vendor. With Azure you pay more for HSM-backed keys with no evidence the HSM exists or is in use. Trust, but verify.

_1tan2• 2 years ago

I am personally involved in procurring HSMs for regulatory reasons. I would be more than happy to deploy an alternative solution that's demonstrably better suited for threats in todays typical cloud environments and fight it out with the regulator. The author sadly doesn't answer that question. Anyone having an idea?

tptacek2• 2 years ago

What's the reg you're up against that says your need an HSM?

_1tan2• 2 years ago

E.g. this: https://www.bsi.bund.de/EN/Themen/Unternehmen-und-Organisati...

I am aware of industry talks of lobbying on behalf of HSM manufacturers that led to these requirements and that's just sad.

616c• 2 years ago

Also CAB Forum has started to require private keys in HSM only for code signing certs.

awaythrow999• 2 years ago

Pretty sure ETSI demands it for trusted service providers https://portal.etsi.org/TB-SiteMap/ESI/Trust-Service-Provide...

lxgr2• 2 years ago

How about confidential computing, e.g. AWS Nitro enclaves, Intel TDX etc?

HSMs make most sense when they’re performing high-level operations (“Is this credit card CVC valid?”); when used as signature or decryption oracles (“Hi, I’m a trusted application server, now sign this email!”) their security gain rapidly diminishes.

Sometimes they get used for key storage alone (“Hi, I’m an application server booting up, give me the RSA signing key for account x!” or even worse “Hi, I’m an application server booting up, give me the key wrapping all the user keys in our database!”), with obvious implications.

Getting an HSM vendor to implement your use case can get very expensive; confidential computing lets you do it yourself, i.e. draw a much larger “trusted” box in your architectural diagram than otherwise feasible.

pclmulqdq• 2 years ago

I have been working on using cloud FPGAs as replacements for HSMs for this reason. They are inherently single-tenant when you use them, and if you do everything right, you can have them cryptographically shred their internal state when they are reprogrammed. You don't have as many physical security promises (but you don't own Amazon's HSM anyway, so I would argue that FIPS 140-3 is not relevant from the physical security angle), but they are a lot more isolated from other users and from meddling by Amazon insiders than Amazon's Cloud HSM is.

The drawback is that they are very expensive, so you have to be really paranoid to want to use them, and they are new enough and complicated enough that you have to trust us and our auditors to have done everything right.

latchkey2• 2 years ago

As someone who has previously hosted cloud FPGA's, this is very interesting.

they are a lot more isolated from meddling by Amazon insiders than Amazon's Cloud HSM is.

I'm curious how you can make this claim.

tarasglek• 2 years ago

The design pattern of maximal separation of thing that holds private keys from thing that needs access to secrets is great. That's my main lesson learned from this article.

I think this space is ripe for significant evolution.

Authors point re hashicorp vault being marginally better than checking keys into source control hits home. Eg often infra is one stolen ssh key away from having code committed that can select * from vault. That also comes with crappy provenance that doesn't integrate well with gitops.

My hopeful view is that we can incrementally fix this bottom-up:

1. ssh should use the mobile phone biometrics as HSM and expose that via ssh-agent

2. From there simple projects should use sops to commit their ssh encrypted secrets to git. I wrote https://github.com/tarasglek/github-to-sops to make it easy to share secrets this way. One can even use sshd private keys to separate secrets for infra from dev ones.

3. For more complex projects one can loop aws kms type hsms.

4. For more complex orgs we then need to borrow some cleverness from crypto ecosystem as they actually made a lot of progress in various tricks to derive trust/ownership...but instead of stupid blockchain one should keep using git.

5. This then can be extended down to database level mods where one could extend row-level ACLs with signature-based provenance checks in 4

Note I am in no way a security person. Just have a desire for someone to make this problem go away for us mere mortals.

namibj• 2 years ago

Termux-API lets you access the Android HSM with a "how recent was the last unlock" limit attached to the key and a screen it can pop up to let you "unlock" without first locking.

Via PKCS11 libtergent.

Tenement Farming and Cloud HSMs

Related Stories