Put your SSH keys in your TPM chip

https://raymii.org/s/tutorials/Put_your_SSH_keys_in_your_TPM_chip.html

by type0 • 3 months ago

100 115 3 months ago

Or put them in a $2 FLOSS Gnuk token/smart card that you can carry with you and still have strong password protection and AES encrypted data at rest with KDF/DO:

https://github.com/ran-sama/stm32-gnuk-usb-smartcard

lights0123• 3 months ago

I would love a world where I could put all my API keys in the TPM so malware couldn't gain persistent access to services after wiping my computer. This would be so easy if more providers used asymmetric keys, like through SSH or mTLS. Unfortunately, many don't, which means that stealing a single bearer token gives full access to services.

There's also the TPM speed issue. My computer takes ~500ms to sign with an ECC256 key with the TPM, which starts to become an issue when running scripts that use git operations in serial. This is a recurring problem that people tend to blame on export controls: https://stiankri.substack.com/p/tpm-performance

convolvatron2• 3 months ago

apologies for asking this question here instead of actually doing the research, but it always seemed to be that while putting keys in a secure environment would help against leakage of the private bits, there really isn't a great story around making sure than only authorized requests can be signed. is this a stupid concern?

guipsp• 3 months ago

It is not a stupid concern, butt there is architecture around making sure you can't just save a request for later and replay it

justincormack• 3 months ago

Yubikey can require touch, and Secretive for Apple Secure enclave can require touch with fingerprint id. Some people disable these, it depends exactly on your use case.

lokar4• 3 months ago

In some cases there is a work-around for bearer tokens. If they allow key/cert login to generate the token (either directly, or via oath), and the token can be generated with a short lifetime, you can build something pretty safe (certainly safer then having a not-expiring, or long TTL token in a wallet).

rehevkor5• 3 months ago

For Yubikey, this guide is worth looking at: https://github.com/drduh/yubikey-guide ("Community guide to using YubiKey for GnuPG and SSH - protect secrets with hardware crypto.")

Liskni_si11• 3 months ago

It's also a bit outdated. OpenSSH supports FIDO2 natively, so all this gnupg stuff is unnecessary for ssh. One can even use yubikey-backed ssh keys for commit signing.

And the best thing is that you can create several different ssh keys this way, each with a different password, if that's something you prefer. Then you need to type the password _and_ touch the yubikey.

knorker7• 3 months ago

This assumes that the server is running a recent enough OpenSSH. Configured with this enabled. For Linux servers, sure. For routers, less obviously so.

kemotep• 3 months ago

This is the sk-ed25519 kind of keys correct?

These work flawlessly with the KeepassXC ssh-agent integration. My private keys are password protected, saved securely inside my password vault, and with my ssh config setup, I just type in the hostname and tap my Yubikey.

TacticalCoder• 3 months ago

I'd say this is more up to date:

https://www.stavros.io/posts/u2f-fido2-with-ssh/

We've got private Git repos only accessible through ssh (and the users' shell is set to git-shell) and it's SSH only through Yubikey. The challenge to auth happens inside the Yubikey and the secret never leaves the Yubikey.

This doesn't solve all the worlds' problem (like hunger and war) but at least people are definitely NOT committing to the repo without physically having access to the Yubikey and pushing on it (now ofc a dev's computer may be compromised and he may confirm auth on his Yubikey and push things he didn't meant to but that's a far cry from "we stole your private SSH key after you entered your passphrase a friday evening and are now pushing stuff in your name to 100 repos of yours during the week-end").

ajross4• 3 months ago

Or, alternatively, don't. Stuff in a TPM isn't for "security" in the abstract, it's fundamentally for authentication. Organizations want to know that the device used for connection is the one they expect to be connecting. It's an extra layer on top of "Organizations want to know the employee account associated with the connection".

"Your SSH keys" aren't really part of that threat model. "You" know the device you're connecting from (or to, though generally it's the client that's the mobile/untrusted thing). It's... yours. Or under your control.

All the stuff in the article about how the TPM contents can't be extracted is true, but missing the point. Yes, you need your own (outer) credentials to extract access to the (inner) credentials, which is no more or less true than just using your own credentials in the first place via something boring like a passphrase. It's an extra layer of indirection without value if all the hardware is yours.

TPMs and secure enclaves only matter when there's a third party watching[1] who needs to know the transaction is legitimate.

[1] An employer, a bank, a cloud service provider, a mobile platform vendor, etc... This stuff has value! But not to you.

Liskni_si11• 3 months ago

TPMs can be useful to you as an individual if you're trying to protect against an evil maid attack. Although I think Linux isn't quite there yet with its support for it. The systemd folks are making progress though.

SAI_Peregrinus• 3 months ago

That only helps if you set a strong password as your TPM PIN. Otherwise its hardware-bound with no access control, and just as susceptible to evil maid attacks as storing the keys directly in a file.

ajross4• 3 months ago

evil maid attack

So does a pass phrase though, with significant less complexity and fragility.

Again, the linked article and responses here are making IMHO a pretty bad mistake with threat model analysis.

red_admiral2• 3 months ago

TPM isn't for "security" in the abstract, it's fundamentally for authentication.

Which is what SSH keys are for?

The advantage of this approach is that malware can't just send off your private key file to its servers.

ajross4• 3 months ago

The advantage of this approach is that malware can't just send off your private key file to its servers.

The use case is ssh keys! If malware can run an ssh command on the remote host, it doesn't need to steal your key, it can just install itself there. Or add its own keys to the access, etc... At best, you'd have to detect and fix that sort of thing with auditing and control, something that's isomorphic to the "third party" requirements I was mentioning.

To repeat the third time: this is all terrible threat model analysis. TPMs do not have value for individuals managing access between trusted devices. TPMs are for third-party validation.

tiberious7264• 3 months ago

TPM isn't for "security" in the abstract, it's fundamentally for authentication

What on earth do you think I make my users present keys for???

You know all those guides saying "you should never copy an ssh private key over the network. Make a new one for each device" that every idiot dev ignored? Now I can enforce that.

doubled112• 3 months ago

Yes, this would stop people from asking for my key when they choose the wrong one for a new AWS EC2 instance.

Not a chance. It is my key.

realusername• 3 months ago

A big warning is that a lot desktop motherboards (at least consumer oriented ones) wipe the TPM when you update the BIOS.

Well no thanks, that risk is much higher than what this is worth.

wang_li4• 3 months ago

I had a friend tell me once that his yubikey is more secure than my authenticator app on my phone because my phone has this giant attack surface that his yubikey doesn't. Yet the yubikey has an entire attack surface of the computer it is plugged into. Which is largely the same or worse than my phone's.

I'm wondering why that doesn't apply here. The TPM holds the key to the cipher that is protecting your private keys. Someone uses some kind of RCE or LPE to get privileged access to your system. Now it sits and waits for you to do something that requires access to your SSH keys. When you do that you are expecting whatever user prompts come up, the malware rides along this expectation and gets ahold of your private SSH keys and stores them or sends them off somewhere. I'm not even positive that they need high degree of privileges on your box, if they can manipulate your invocation of the ssh client, by modifying your PATH or adding an ssh wrapper to something already in your path, then this pattern will also work.

What am I gaining from using this method that I don't get from using a password on my ssh private key?

systd-basiliskd3• 3 months ago

The promise of HSM, TPM and smart cards are that you have a tiny computer (microcontroller) where the code is easier to audit. Ideally a sealed key never leaves your MCU. The cryptographic primitives, secret keys and operations are performed in this mini-computer.

Further promises are RTC that can prevent bruteforce (forced wait after wrong password entry) or locking itself after too many wrong attempts.

A good MCU receives the challenge and only replies with the signature, if the password was correct. You can argue that a phone with a Titan security chip is a type of TPM too. In the end it doesn't matter. I chose the solution that works best for me, where I can either only have all keys in my smart card or an offline paper wallet too in a fireproof safe. The choice is the user's.

lokar4• 3 months ago

And (unlike on your computer or phone), the HSM/TPM has its own CPU/memory and firmware, it's in control from the start of boot.

wang_li4• 3 months ago

For SSH to use your keys a calculation has to be done using your private key and then send the results back to the remote site so it can validate that you got the results that prove you have your private key. The TPM and your yubikey do not do this calculation. They allow software on your computer to access the private key in plaintext form, perform this calculation, and then send the result (and then presumably overwrite the plaintext key in RAM). If your system has been compromised, then when this private key is provided to the host based software, it can be taken.

Liskni_si11• 3 months ago

They can use the key as long as they can access your computer, but they shouldn't be able to get the secret key out of the TPM or Yubikey and use it elsewhere while your computer is off. That's the main point of HSMs.

hart_russell• 3 months ago

Also a Yubikey requires you to physically push it to sign. So an attacker needs to have physical access.

wang_li4• 3 months ago

I know the title says "in your TPM chip" but the method described does not store your private key in the TPM, it stores it in a PKCS keystore which is encrypted by a key in your TPM. In actual use the plaintext of your private ssh key still shows up in your ssh client for validation to the remote host.

The recommended usage of a yubikey for ssh does something similar as otherwise your key consumes one of the limited number of slots on the key.

knorker7• 3 months ago

my authenticator app on my phone

Depending on which authenticator app (or maybe applies to all?), that data either is, or can be, backed up.

A yubikey cannot be cloned.[1]

the malware rides along this expectation and gets ahold of your private SSH keys and stores them or sends them off somewhere.

Ah, this is where your misunderstanding lies. No, the crypto operation runs ON the TPM or yubikey. The actual secret key NEVER lives in RAM. (ehem, after it was imported, if importing is the method by which it was generated)

[1] You know what I mean. Of course in principle it can be. But not like a phone where it can literally be sent via scp.

zackify• 3 months ago

I'd rather have them in 1password because I use too many different computers and it is perfect for that and with ssh agent forwarding

finaard2• 3 months ago

Anything pkcs#11 you can proxy. I'm using that on some systems - I have an old notebook with a nitrokey hsm at home. It binds pkcs11-proxy to a local wireguard interface, so I'm registering systems I want to be able to use those keys to that notebooks wireguard. They still need a pin for unlocking a session as well.

drum55• 3 months ago

Seems a little pointless, your keys can't be stolen but they can be instantly used by malware to persist across anything you have access to. The keys don't have any value in their own right, the access they provide does.

Nextgrid2• 3 months ago

The idea with HSM-backed keys is that even in case of compromise, you can clean up without having to rotate the keys. It also makes auditing easier as you can ensure that if your machine was powered down or offline then you are guaranteed the keys weren't used during that timeframe.

jamiesonbecker2• 3 months ago

Rotating keys is easy with the right software. (I work @ Userify) Agree with the auditing point

Token-based keys, to tptacek's point, is that they can be a giant pain once you start scripting across fleets.

lxgr4• 3 months ago

That's still an improvement. In sophisticated attacks, attackers might well store stolen credentials and use them at a later, more opportune time.

Of course a real secure attention sequence would be preferable, such as e.g. requiring a Touch ID press on macOS for keys stored in the Secure Enclave. Not sure if TPM supports something similar when a fingerprint sensor is present?

knorker7• 3 months ago

Without presence test (e.g. yubikeys touch) it's certainly not perfect. But it does close some real world attacks. Like the key can only be used while your laptop is on. (assuming laptop, here).

And keys cannot be stolen from backups.

Or stolen without your knowledge when you left your laptop unguarded for 5min.

Not every attacker has persistent undetected access. If the key can be copied then there's no opportunity for the original machine's tripwires to be triggered by its use. Every second malware runs is a risk of it being detected. Not so, or not in the same way, with a copied key.

spwa42• 3 months ago

I guess you could implement that on android.

Borealid2• 3 months ago

TPMs support setting a PIN without which a key cannot be used.

The PIN can be an arbitrary string (password).

tegmentum• 3 months ago

I wish they wouldn't do that with the naming. It's confusing as hell to call it a PIN (Personal Identification Number) if it's actually a password.

palata11• 3 months ago

Though if your computer is compromised, then the malware can read the password, right?

lxgr4• 3 months ago

On macOS, you can now also move them into your Secure Enclave without any third party software! Previously discussed here: https://news.ycombinator.com/item?id=46025721

lokar4• 3 months ago

You can probably combine the yubikey with a TPM:

Keep a CA (constrained to your one identity) with a longish (90 day?) TTL on the TPM. Use it to sign a short lived (16h?) keys from your TPM, use that as your working key.

palata11• 3 months ago

But then why not use the Yubikey directly?

lokar4• 3 months ago

If you just need to authenticate a couple times, you would. For example, if you are just using the cert to get a couple oath tokens.

But, if you are making a lot of x509 authenticated calls directly, then the speed and not needing to touch the key are important. Or if you need to ssh to 10,000 hosts quickly, things like that.

hypeatei2• 3 months ago

Didn't Tailscale try to do something similar but found out quickly that TPMs 1) aren't as reliable as common wisdom makes them out to be, and 2) have gotchas when it comes to BIOS updates?

I can't find it now, but I believe someone from Tailscale commented on HN (or was it github?) on what they ran into and why the default was reverted so that things were not stored in the TPM.

EDIT: just saw the mention in the article about the BIOS updates.

tiberious7264• 3 months ago

If you run into the link to this, is love to read it. Proper, modern, pcrphase binding with a signing key should remove these firmware update issues irt the raw pcr value changing

hypeatei2• 3 months ago

Yep, found the relevant links:

https://github.com/tailscale/tailscale/issues/17622

https://news.ycombinator.com/item?id=46532666 (direct comment link, more discussion on the issue in the parent)

red_admiral2• 3 months ago

We don't want that in our shell history

This may be bash-only, but a space before the command excludes something from history too.

capitainenemo2• 3 months ago

Only if you do something like this... export HISTCONTROL=ignorespace

Personally I like this which reduces noise in history from duplicate lines too. export HISTCONTROL=ignoreboth:erasedups

JoeBOFH• 3 months ago

While you aren’t wrong, I haven’t seen a distro that defaults to bash not have this enabled by default for a long while.

rkeene2• 3 months ago

We created Keeta Agent [0] to do this on macOS more easily (also works with GPG, which is important for things that don't yet support SSH Signatures, like XCode).

Since it just uses PKCS#11, it also works with tpm_pkcs11. Source for the various bits that are bundled is here [1].

Here's an overview of how it works:

1. Application asks to sign with GPG Key "1ABD0F4F95D89E15C2F5364D2B523B4FDC488AC7"

2. GPG looks at its key database and sees GPG Key "1ABD...8AC7" is a smartcard, reaches out to Smartcard Daemon (SCD), launching if needed -- this launches gnupg-pkcs11-scd per configuration

3. gnupg-pkcs11-scd loads the SSH Agent PKCS#11 module into its shared memory and initializes it and asks it to List Objects

4. The SSH Agent PKCS#11 module connects to the SSH Agent socket provided by Keeta Agent and asks it to List Keys

5. Key list is converted from SSH Agent protocol to PKCS#11 response by SSH Agent PKCS#11 module

6. Key list is converted from PKCS#11 response to gnupg-scd response by gnugpg-pkcs11-scd

7. GPG Reads the response and if the key is found, asks the SCD (gnugpg-pkcs11-scd) to Sign a hash of the Material

8. gnupg-pkgcs11-scd asks the PKCS#11 module to sign using the specified object by its Object ID

9. PKCS#11 module sends a message to Secretive over the SSH Agent socket to sign the material using a specific key (identified by its Key ID) using the requested signing algorithm and raw signing (i.e., no hashing)

10. Response makes it back through all those same layers unmodified except for wrapping

(illustrated at [2])

[0] https://github.com/KeetaNetwork/agent

[1] https://github.com/KeetaNetwork/agent/tree/main/Agent/gnupg/...

[2] https://rkeene.org/tmp/pkcs-sign.png

tptacek• 3 months ago

This is a neat trick that people have been doing with Yubikeys for a long time, but from an operational security perspective, if you have a fleet rather than just a couple of hosts, the win is only marginal vs. short-lived keys, certificates, and a phishing-proof IdP.

nyrikki• 3 months ago

Lots of ways to establish a persistent presence with a short time life key, especially if it is in env or a file it is trivial to find.

In theory the Linux kernel keyring would help here, even with a tsm or in conjunction with it.

Unfortunately as the industry abandoned the core Unix permission system (uid/gid) all of these methods just get a devfs[null] bind mount.

Only process that also support the traditional co-hosting model like nginx and Postgres do.

We would need nonce keys to gain no value from kernel memory or hardware storage.

gempir• 3 months ago

The integration of the ed25519-sk keys is just so easy and similar to normal ssh keys, so the upgrade is way easier.

You just need to tighten your sshd config, you can even add a "touch required" of the Yubikey to the sshd config. Has been in debian stable since like 11 I think?

So it's super friendly to integrate and very secure, as you need to physically be on your pc, have your yubikey and have your exact pc. So that's a lot of factors.

jcalvinowens• 3 months ago

This could make real sense for ssh host keys, since they need to be used without presence and they're generally tied to the lifetime of the machine anyway.

I saw a write up where someone successfully got sshd to use a host key from a fido2 yubikey without touch, but I can't find it...

As far as "TPM vs HSM", it is soooo much simpler to make a key pair with a fido2 hardware key:

  ssh-keygen -t ed25519-sk -O resident -O verify-required -C "your_email@example.com"

You can get them for <$30.

tiberious7264• 3 months ago

The authors of both this article and ssh-tpm-agent (disjoint set) really need to learn about pcrphases and the signing keys therefor: https://github.com/Foxboron/ssh-tpm-agent/issues/15

samhclark• 3 months ago

Do you have any more info you could add about that topic, or a direction to point me? As far as I know, (systemd-)pcrphase is for measured boot, but I'm not sure how that interacts with signing keys.

As someone who stores my SSH keys in my TPM, and has struggled with picking the right PCR values for Secure Boot in the past, I'm interested in learning more.

asveikau• 3 months ago

I've always felt that having your keys tied to specific hardware will just lock you out when the hardware breaks. Maybe I'm overstating the risk. It's a good idea to have multiple methods and test the backups.

atoav• 3 months ago

I mwan a key isn't that long. Convert it to a QR code, print it out and stick it somewhere safe.

auraham• 3 months ago

I have the same concern with all hardware used for storing keys and secrets for crytpo.

SomeHacker44• 3 months ago

Agree. This is why I do not use Passkeys, and I have 4 physical Yubikeys tied to every system I secure with them.

deepsun2• 3 months ago

Cloud-based passkeys are okayish (1pass, bitwarden), as they are available on multiple devices.

However not all devices play well with it, e.g. iOS and Android don't ask 1pass for the passkey. I also couldn't make it ask NFC for my hardware Yubikey with passkeys, but maybe I just did something wrong.

qurren2• 3 months ago

I still don't understand what the hell passkeys are. Weren't passwords and {hardware keys | authenticator apps} enough?

I don't think average Joe is going to understand these passkeys either.

palata11• 3 months ago

I don't get this. Why don't you use your Yubikeys as passkeys then?

jagged-chisel• 3 months ago

The general consensus has been that you create a key pair per client computer that you use. If one is stolen (say your laptop), you login from your desktop and revoke the stolen key. If the hard drive fails, you login from another client.

I don’t see much difference between that and storing the key on a TPM. If you have one key and you lose access to that key, then you lose access to the server.

Point: you need a backup key anyway.

jamiesonbecker2• 3 months ago

One key per device is exactly what we recommend too. Private keys should always be protected as much as possible within that device and should never leave that device.

Just paste all of your devices' public keys into your authorized_keys file and leave a comment at the end for what device it's for. in Userify, it literally goes right into your nodes' authorized_keys file almost verbatim. (disclaimer: I work at https://Userify.com)

And then, if you leave your token or laptop at the airport or whatever, just remove that key right from your phone and it'll take effect in seconds across all the nodes/instances (if you're using Userify) or you can just write a quick for-inline-sed loop to remove it from your authorized keys everywhere.

pseudohadamard• 3 months ago

It's unlikely to be inside the TPM. The way pretty much all TPMs store data is to use a key inside the TPM to encrypt data stored outside the TPM, because the TPM is a repurposed smart card with barely any storage or capabilities outside of DRM. Bitlocker is an extreme example of this, but things like Fapi_CreateSeal()/Fapi_Unseal() also store the sealed item outside the TPM even though they appear to be using the TPM for storage. So what you do is the same as what Bitlocker does, use the TPM's storage root key (SRK) to seal some master key (in Bitlocker terms the VMK) and the master key seals the encryption key used, which is also sealed with some user-entered emergency-access password or whatever that still gives you access if the TPM dies.

OTOH for SSH use if you lose the key you just create a new one, it's not like you've lost the only copy of your Bitlocker key.

hparadiz• 3 months ago

I feel like this is great for state actor level security but way too much for everyday use.

dboreham• 3 months ago

Where "everyday" means security from what kind of attacker?

bob10292• 3 months ago

I'd consider storing (generating) them in AWS KMS. It's $1/key/month and you don't have to worry about hardware failures, etc. Each key must have a separate policy attached which controls who it can be used by and how. It is possible to create keys the root account cannot touch. If you have anything running on EC2, it's an extremely compelling option because you can authenticate directly via IMDSv2 tokens and IAM roles, avoiding the need for any kind of secret strings.

palata11• 3 months ago

Not sure I get that. If you generate it "in the cloud", doesn't it mean that someone else (the cloud) has access to it?

bob10292• 3 months ago

It really depends on what you are trying to optimize for.

If you are doing something illegal or controversial with the key, then yes it would be foolish to store it in the cloud.

If your main concern is it becoming compromised due to a local exploit or physical breach, then I'd argue it is a strong option.

whalesalad• 3 months ago

A big warning is that a lot desktop motherboards (at least consumer oriented ones) wipe the TPM when you update the BIOS.

that's not good

aprilnya• 3 months ago

Termius integrates TPM/Secure Enclave keys nicely on Windows Mac iOS etc as well, which is nice because then you can have a key on your phone (so if you lose your laptop you won’t get locked out of your servers)

Put your SSH keys in your TPM chip

Related Stories