Stealing unencrypted SSH-agent keys from memory

https://www.netspi.com/blog/entryid/235/stealing-unencrypted-ssh-agent-keys-from-memory

by lelf • 12 years ago

81 38 12 years ago

yock• 12 years ago

I've upvoted this because I want to see what really smart people have to say about it. That said, I've always been of mind that if someone gains unauthorized root access to my box, then nothing on it can ever be trusted again. This includes things in memory. In other words, this feels obvious to me, once pointed out.

Is there something novel about this that I'm missing?

hemancuso• 12 years ago

No. If you have the ability to run code as root it's game over. Even if the agent re-encrypted the key in memory with its own temporary key it's just a layer of obfuscation that can easily be overcome.

tlrobinson• 12 years ago

I agree with you, but in case you missed the author's rationale:

"However, this causes the attacker to have to wait for the target to type in their passphrase. This might be hours, days, or weeks, depending on how often the target logs out. This is why obtaining the SSH key from memory is vital to pivoting to other machines in a speedy fashion."

I'm not sure how you would solve this problem while providing the functionality ssh-agent provides, aside from perhaps a HSM or something.

agwa7• 12 years ago

Anyone keeping SSH keys in an ssh-agent process for weeks on end is doing it wrong. You can use the -t option to ssh-agent to specify a maximum lifetime for identities added to the agent. I would set it to less than a day.

MichaelGG• 12 years ago

Make sure nothing can run as root? Perhaps by using a VM system. You could run ssh-agent on another machine and make key requests over an internal network.

Also you can get a cheap "HSM" by using a smartcard.

peterwwillis• 12 years ago

The problem of a root user inspecting the memory of arbitrary other users is solved with MAC like SELinux. https://security.stackexchange.com/questions/7801/keeping-se...

But if you can exploit a specific user's program, you can (usually) inspect the memory of other programs managed by that user. You can actually create security policies so secure that a user who exploits the sshd program can't read from the same user's ssh-agent process memory, but that's not practical for most people.

agwa7• 12 years ago

It's not that novel. If an attacker gets root it's game over.

That said, the article is a good reminder of the risk of keeping decrypted keys in a long-running ssh-agent process.

zobzu3• 12 years ago

nothing's novel - but consider this for example:

- if its a server and you copy your private key onto it its likely to be compromised one day and the key reused even thus it was encrypted.. since its decrypted in memory when used. plus keystrokes could be captured. if you use this key elsewhere, all other servers are also compromised now..

- you can keep the key in a hardware token, then even if a laptop is remotely compromised your key - and thus servers you're not currently accessing - are safe. its also a little harder to hijack a connection or silently ask the key (generally its not silent with a token)

e403• 12 years ago

So, as someone needing to run unattended scripts across a wide variety of hosts, what's the best practice for managing SSH keys? I've been using ssh-agent. Is there a better way?

agwa7• 12 years ago

Make sure that the SSH keys are tightly restricted in the authorized_keys file so they can only run the commands that need to be run unattended. If possible, also restrict them by source IP address. Here's an example authorized_keys file that I have in production for doing unattended backups:

  command="/usr/local/backups/backup_server /etc/snapshot_backup_list",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty,from="10.70.0.0/16" ssh-rsa AAAAB...

I wouldn't worry unduly about protecting the keys themselves - since they need to be accessible for unattended operation, there's not much you can do to prevent them from being accessible to an intruder.

e403• 12 years ago

This is hard, since there is such a large number of them. It would be very painful to maintain this list. For a while I had the keys expire in ssh-agent, but that didn't work because you can't extend the expiration before you run a set of unattended commands. That is, if it's 11.5 hours into a 12 hour expiration, and I need to run commands that will span the expiration, then I'm screwed, since 1) I can't query when the keys will expire, and 2) I can't extend it another 12 hours (from now, not from the expiration).

I feel like ssh-agent is essential to my life but it's badly designed and I have this nagging feeling it's insecure (and articles like this feed this fear).

graylights• 12 years ago

Not really a solvable problem. You want your machine to have the keys to run scripts but don't want your compromised machine to have the keys to run scripts. The best you can do is restrict what the keys can do or use authentication that doesn't work unattended.

If you're just doing batch jobs, then you could have the script remove keys from ssh-agent when it's done. At a certain point you have to presume the integrity of your machine. Otherwise your password can just be keylogged as you're unlocking key.

ambrice2• 12 years ago

So, is there a big advantage to running ssh-agent vs unencrypted 0400 keys in your ~/.ssh directory?

agwa7• 12 years ago

Yes. ssh-agent prevents non-root users, including the user running ssh-agent[1] from being able to extract the private key. That's much more secure than storing unencrypted keys in ~/.ssh.

[1] ssh-agent uses some tricks, like making itself setgid, and/or using prctl(PR_SET_DUMPABLE, 0) on Linux to make its memory undumpable.

0x0• 12 years ago

And on OSX, it integrates with the keychain as well!

Canada3• 12 years ago

That won't prevent your key from being stolen if you leave it on a VM controlled by someone else.

Your key belongs on your personal device, and agent forwarding enables you to behave as if your key is everywhere you need it to be.

ipsin• 12 years ago

The technique of using gdb to attach to ssh is pretty old.

Here's an example from 2005, which was presented (iirc) at Defcon as well: http://www.blackhat.com/presentations/bh-usa-05/bh-us-05-boi...

In that case, SSH-Jack would just piggyback on existing (user-level) ssh connections, which is also pretty serious, though that's not as exciting as stealing keys.

zobzu3• 12 years ago

personally i find it more exciting than steal the key because harder :)

then again this as been done since the beginning of times sadly, or excitingly, i dont know :)

any tool that promotes awareness of this inherent issue is good anyway IMO

void-star2• 12 years ago

Yep. The general technique of scraping memory for private keys and certificates a pretty well-known and established. Particularly in forensics and malware circles.

I don't point it out to engage in the "who was first" thing. But to point out that this is very much an applied attack in the real world. Real attackers (includes "forensics analysts", incase you don't consider them "attackers" too) have been using this technique in malware as well as countermeasures/investigations for quite a while, now.

See: http://www.trapkit.de/research/sslkeyfinder/

https://github.com/emonti/yara-ruby/blob/master/samples/sslk...

http://volatility-labs.blogspot.com/2013/05/movp-ii-21-rsa-p...

EDIT: actually a much earlier discussion is from '98 by none other than Shamir

https://www.cs.jhu.edu/~astubble/600.412/s-c-papers/keys2.pd... [PDF]

Canada3• 12 years ago

This is exactly why SSH has agent forwarding.

Do not copy your private key to other systems.

ssh -A is your friend.

m-app• 12 years ago

Well, it prevents your private key from being compromised but an attacker with root on the intermediate box can still authenticate to other servers using your SSH agent socket.

If you really want to do secure SSH hopping, use the ProxyCommand directive with `netcat` in your SSH config.[1]

[1] http://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_Ju...

Canada3• 12 years ago

Nice. Thanks for sharing

nullc• 12 years ago

I wish SSH had a narrow kind of delegation capability... E.g. if instead of ssh-agent having access to the key, it used the key to sign a tuple like {time-limit, [host-ip], ephemeral_pubkey}, then forgot the private key, and used the ephemeral key to actually authenticate to hosts— but they'd only accept it from the listed host ip set and during the specified time limit.

Obviously if the attacker has access long enough they'll capture the passphrase and private key eventually, but it would be nice to narrow the window.

Right now the best you can probably do is use a gpg smartcard with the ssh-agent/gpg-agent shim stuff, (random tutorial here: https://github.com/herlo/ssh-gpg-smartcard-config/blob/maste...)

cnvogel• 12 years ago

Your scheme could probably be implemented using certificate based SSH: Your users log onto a CA which gives out user-certificates with a specified lifetime, they copy that to their desktop-machine(s), or maybe add it to a ssh-agent like intermediate key-storage and can do their days work.

Tomorrow, they'll have to get new certs (automatically, after authenticating to the CA using appropriate means).

Here's a tutorial showing cert-based ssh auth: http://neocri.me/documentation/using-ssh-certificate-authent...

Obviously with secure key storage (smart-card) prohibiting accidental loss of the private key, and a working cert-revocation infrastructure it's not necessary in the first place.

icebraining• 12 years ago

That's essentially Mozilla Persona, but for SSH instead of the web. The people at Mozilla are in fact already discussing such possibilities: https://hacks.mozilla.org/2013/04/mozilla-persona-for-the-no...

FedRegister• 12 years ago

I wish SSH had a narrow kind of delegation capability... E.g. if instead of ssh-agent having access to the key, it used the key to sign a tuple like {time-limit, [host-ip], ephemeral_pubkey}, then forgot the private key, and used the ephemeral key to actually authenticate to hosts— but they'd only accept it from the listed host ip set and during the specified time limit.

So Kerberos? You can do that with Kerberos and the GSS-API in OpenSSH already.

Stealing unencrypted SSH-agent keys from memory

Related Stories