Signing .jars is not worth the effort

https://quanttype.net/posts/2020-07-26-signing-jars-is-worthless.html

by nurettin • 6 years ago

64 90 6 years ago

quanttype.net

based22• 6 years ago

https://en.wikipedia.org/wiki/Key_server_(cryptographic)

based22• 6 years ago

https://stackoverflow.com/questions/3307146/verification-of-...

https://issues.apache.org/jira/browse/MNG-6026 Extend the Project Object Model (POM) with trust information (OpenPGP, hash values)

https://issues.apache.org/jira/browse/MNG-5814 Be able to verify the pgp signature of downloaded plugins against a trust configuration

CamouflagedKiwi3• 6 years ago

I've thought this about Maven Central before (not familiar with Leiningen, but it seems it's trying to do a similar thing).

Maven Central has PGP signatures for all uploaded artifacts - but they are in fact useless, because anyone can create a PGP key that claims to be (say) maven-releases@google.com and upload that to a keyserver. There appears to be no mechanism by which a consumer can know whether the signing key should be trusted, so an attacker uploading a malicious artifact can easily upload a malicious signature with it.

I'm not going to argue with any of the criticisms of PGP in the linked article, but they don't seem hugely relevant to the problem here; the fundamental trust problem is much deeper than "GPG has janky code" (and it's not like there aren't any other options at all).

hasenpfote2• 6 years ago

Mh i thought your answer is ridiculous. Why would someone enforce/use a pgp signature with public key servers without a proper challenge?

I know it from my visited 'network of trust' but apparently there is no real challenge involved in uploading your key.

Thats just really really shitty.

the_pwner2242• 6 years ago

The keyserver is intentionally designed to be a write-only (no delete/edit) database where anyone can upload their keys. GPG's target market contains people living under highly oppressive governments, where getting a 'please verify your identity' email could get you killed/etc. Given this requirement, it must be possible for anyone to upload a key claiming they are asdf@example.com.

Now it turns out having a globally accessible, unauthenticated, write-only database is a really stupid idea, and there was a piece submitted on HN within the past year about this. Similarly at some point as a demonstration someone uploaded ~50 GPG keys for Linus Torvalds (or something like that) as another demonstration of this.

The reason this is OK is that you have out-of-band verification. When I upload my key for asdf@example.com, I will also put a page on example.com saying that my public key has fingerprint XXXXX. This is the exact same thing that happens with SSH; the first time you connect to another computer it shows you the key fingerprint and asks you whether it is as expected, as discussed in another top level comment here.

So if you want to verify a signature on a file I sent you and you find 50 public keys on the keyserver claiming to be me, you can still very easily figure out which key is mine by looking at the keys' fingerprints. Or instead of publishing it on the WWW I can tell you the fingerprint with another more private/secure method, such as a piece of paper at a dead drop. Or the network of trust can be used: you already have Alice's public key on your computer, and she can sign my key on the keyserver. So when you find 50 keys on the keyserver claiming to be for asdf@example.com, you know the one signed by Alice is the real one.

chrisseaton6• 6 years ago

Isn't the idea that you form your own trust network? For example meeting people in-person at conferences and signing each other's keys, and extending trust that way?

CamouflagedKiwi3• 6 years ago

Conceptually you could do that, if you were willing to only use dependencies from people you trusted that closely. That can only be a very, very tiny minority of people using Maven.

renewiltord3• 6 years ago

There's a human in the loop when you get your Maven Central account that lets you publish your jar under some prefix.

CamouflagedKiwi3• 6 years ago

That's orthogonal to the PGP signing though. That verifies that you control the domain for the Maven group id but doesn't need any PGP keys (it's more akin to the process of verifying DNS records for SSL certificates).

WatchDog• 6 years ago

Maven central will let you sign artifacts with any published key you like, but one thing you can do in theory, is verify that new releases are signed by the same key as a known good release. I am not aware of anyone actually doing this though.

yarg• 6 years ago

That approach is still vulnerable to what amounts to a MITM attach: a bad actor can simply provide the legitimate versions for whatever period he deems required to build trust.

Uwqye2134trf2• 6 years ago

When I work on embedded Linux stuff I sign my packages.

Shipping hardware as opposed to software allows secure deployment of pre-shared keys which can be trusted.

Even if someone hacks our automatic updates server (not too unlikely, it's some shared hosting), devices we have sold won't trust the modified packages because 512-bit ECDSA signature won't match the public key they have pre-deployed.

beefhash• 6 years ago

Out of curiosity:

1. Why ECDSA?

2. Why a 512-bit prime for the curve?

Uwqye2134trf2• 6 years ago

1. Only two asymmetric algorithms are widely supported and almost universlly recommended, RSA and ECDSA. Compared to RSA, ECC needs smaller keys for same security.

2. I did that couple years ago already, I think it actually was 521 bits, the best one recommanded by FIPS at that time. The hardware has no relations to FIPS, it's not even _that_ expensive and should contain no secret data. I just saw no reasons not to implement the best security available: development time is not affacted by the count of these bits.

MaxBarraclough14• 6 years ago

As far as I know, nobody ever verifies the signatures in a systematic way.

If you'll forgive a semi-relevant ramble about SSH:

The situation seems similar with SSH. As far as I can tell, just about everyone goes with the approach of trust-blindly-on-first-use. I've made a habit of manually checking the public-key when I SSH into a new EC2 instance using PuTTY. It is not obvious how to do this. Existing tooling just isn't set up with it in mind.

The fingerprint reported on the EC2 system-log uses a different hash function than PuTTY uses. (This is PuTTY's fault. It's behind OpenSSH.) I was able to get an answer on ServerFault, but I was surprised no-one had asked the question before. [0]

With some lesser-known distros it seems to be outright impossible, as the fingerprint is not written to the system-log at all. Presumably the distro maintainers never check SSH fingerprints.

Still less relevantly: the trust-blindly-on-first-use antipattern is known euphemistically as TOFU, for trust on first use. This term can also refer to where the public key is manually verified by the user on first use. [1] Very unhelpful for a term-of-art to be so ambiguous.

[0] https://serverfault.com/q/996828/

[1] https://en.wikipedia.org/wiki/Trust_on_first_use

forty• 6 years ago

SSH certificates avoid this issue and are used in real life. We make them with hashicorp vault, it works well and is really convenient.

GordonS• 6 years ago

Huh, I had no idea there was even such a thing as SSH certificates!

Presuming you mean X.509 certificates, is this part of the standard spec (e.g. should work with OpenSSH etc)? Do you know if it works with PuTTY?

MaxBarraclough14• 6 years ago

Have to admit I'm not very familiar with SSH certs.

I'm generally just spinning up a fresh Ubuntu instance on EC2, so I have an entirely 'vanilla' SSH set-up. Small scale, so I don't especially mind the ritual of verifying the fingerprint. It's annoying though that it takes a few minutes for the EC2 system-log to be populated with the 'true value' of the fingerprint.

kccqzy• 6 years ago

Big companies that care about security use short-lived certificates for SSH, not simple private/public keys.

still_grokking• 6 years ago

I would recommend using the original tooling.

The ssh CLI shows the fingerprint, and explicitly asks:

  $ ssh example.com
  The authenticity of host 'example.com (1.2.3.4)' can't be established.
  RSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
  Are you sure you want to continue connecting (yes/no/[fingerprint])?

MaxBarraclough14• 6 years ago

That's roughly what I've ended up doing. [0]

Both OpenSSH and PuTTY ask you to confirm the fingerprint, but PuTTY's fingerprint scheme is MD5/hex, whereas modern versions of the OpenSSH client use SHA256/Base64.

[0] https://serverfault.com/questions/996828/#comment1296514_996...

closeparen4• 6 years ago

The instance itself needs to get its SSH host key signed, or upload its fingerprints somewhere, on first boot. This is easy enough to write, but making it any more secure than plain old TOFU is tricky.

MaxBarraclough14• 6 years ago

making it any more secure than plain old TOFU is tricky.

To refer again to my complaint about this term: this isn't meaningful. It's not clear whether you mean trust blindly on first use (total failure to cryptographically verify the identity of the server) or manually check SSH fingerprint on first use (secure but inconvenient).

Both get called TOFU. One meaning of the term is a valid means of establishing a secure channel, and the other is cybersecurity negligence.

I'm not just being obtuse here, I really don't know which you mean. Manually checking fingerprints on first use seems like a pretty solid approach to SSH security, if you can trust the client to actually perform the check. Short-lived keys are the obvious improvement that this approach doesn't really support. Is that what you meant?

laegooose• 6 years ago

Thank you for asking serverfault question and writing about it here.

When I started to work with EC2 and SSH few years ago I encountered this problem. However I was overwhelmed with the stuff I had to do for actual business and had no energy to investigate.

Next question is, how do we protect against this attack without waiting for few minutes and making all those clicks to find the fingerprints every time we launch an instance?

MaxBarraclough14• 6 years ago

Someone mentioned SSH certificates. My understanding is that it's the 'proper' solution, but is more work to set-up. Whether it's worth it will presumably be a question of scale.

ryanlol3• 6 years ago

I've made a habit of manually checking the public-key when I SSH into a new EC2 instance using PuTTY. It is not obvious how to do this. Existing tooling just isn't set up with it in mind.

This is a not necessarily useful if you use key auth. The attacker can’t perform an useful MITM attack on the connection no matter what key you accept from the server. At best the attacker could let you log into their server, but presumably you’d notice that you’re logged into the wrong server.

I guess there are a few corner cases where this might help, such as blind scp uploads.

MaxBarraclough14• 6 years ago

The attacker can’t perform an useful MiTM attack on the connection no matter what key you accept from the server.

True, thanks to public key crypto.

At best the attacker could let you log into their server

This is another way of saying the machine you end up trusting could be under the control of an attacker. So yes, that's a serious concern.

presumably you’d notice that you’re logged into the wrong server

Not if it's a fresh Ubuntu instance. Even if not, I'd expect SSH security to answer that definitively, on principle. SSH shouldn't be any sloppier than HTTPS.

ptx3• 6 years ago

Surely the attacker can serve up an SSH server implementation that forwards the connection to the intended server while recording and/or modifying the decrypted traffic as desired? To the user everything would appear as normal (expect perhaps for some additional latency) but their data would be exposed to the attacker.

If there were no way to perform a MITM attack because you would notice that you were connected to the wrong server, the whole system of keys and fingerprints would be pointless. So would the CA system for HTTPS.

vbezhenar2• 6 years ago

Here's an approach that I'm using with small VPS providers. I'm using web remote access to install Linux (from pristine distribution ISO). That web remote access uses HTTPS, so I can be sure that all input/output is E2E protected from provider to my computer. After I install Linux, I boot it first time and access it using that HTTPS remove console. Then I'm running something like ssh-keygen -l -f /etc/ssh/ssh_host_ed2519.pub -E md5 (I might be wrong about this command, check mans before use, also it does not require root, so login under ordinary user) and I can see md5 hash of server ssh public key. I write it down (or rather print-screen), connect with Putty and verify it. I believe that this method provides as much chain of trust as possible.

MaxBarraclough14• 6 years ago

That's the approach I use with Linode. Unlike EC2 there's no browser-viewable system-log that shows the public key, but it can be retrieved using a web-based terminal session.

To find the fingerprint in PuTTY's obsolete format, the command I use is equivalent to the one you've given. It was provided in the ServerFault answer I linked:

     ssh-keygen -l -E md5 -f /etc/ssh/ssh_host_ed25519_key.pub

I don't like comparing fingerprints by eye, but PuTTY doesn't make it easy to do otherwise. This issue doesn't arise with PowerShell's OpenSSH client, where I can just copy the fingerprint (in the new format) to the clipboard. (It lacks features like mouse support though, so I prefer to use PuTTY.)

To find the fingerprint in the new format for OpenSSH, I believe the command is:

    ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub

> I believe that this method provides as much chain of trust as possible.

Yes, can't do better than that. If your HTTPS session with your VPS provider has been compromised, it's all over anyway.

MertsA• 6 years ago

What's absolutely incredibly frustrating is that it's absurdly common to ignore SSH fingerprints even if they're staring you in the face. I just spun up a Fedora 32 cloud image on RamNode over the weekend and the default image oh-so-helpfully displays the SSH fingerprints as part of the MOTD so you can verify it just looking at the login prompt on console. Except apparently the script that generates that runs before the SSH keys are regenerated on the first boot of the image, so the fingerprints displayed don't match the keys on disk. Either no one else has noticed this bug yet or nobody has been bothered enough to fix it.

There is a solution though, SSHFP records for the host. I just set them up after catching that issue just because of how frustrating blindly trusting on first use is. Now I can SSH to my servers even without anything for them being in known_hosts and it checks for SSHFP records and verifies the fingerprint and DNSSEC protects the SSHFP records.

tptacek2• 6 years ago

SSHFP requires you to DNSSEC-sign your zones, which virtually nobody in the technology industry does, for a variety of reasons, some involving security.

A more straightforward solution that doesn't require you to hitch your wagon to a moribund infrastructure overhaul is to use SSH certificates. SSH certificate infrastructures have other benefits beyond resolving the introduction problem; for instance, they're easier to manage than keys once set up, and, perhaps most importantly, they make it possible to issue short-lifespan keys based on single signon, which is much safer than having everyone register a canonical personal key.

atlgator• 6 years ago

Java applets are largely dead at this point but they do still exist in small corners of the IE world. Signing those jars is paramount and the signature is validated by the browser.

johannes1234321• 6 years ago

"Java Web Start" still exists and there are at least two programs I use from time to time based on that technology. One being a custom thing, the other openstreetmap editor jOSM.

chrisseaton6• 6 years ago

"Java Web Start" still exists

I'm afraid it's been dead for a while now.

Java Web Start (JWS) was deprecated in Java 9, and starting with Java 11, Oracle removed JWS from their JDK distributions.

https://openwebstart.com/

upofadown• 6 years ago

What's with the random shot at PGP at the end? PGP is perfectly fine for this particular application. None of the stuff that the article talks about has nothing to do with the particular tool used to sign the releases.

deepersprout• 6 years ago

PGP is bad and needs to go away

how should we sign git commits?

the_pwner2242• 6 years ago

There's nothing special about PGP that makes it good for signing git commits, and plenty of alternatives exist. Signify is a simple tool to do signing, used by OpenBSD: https://news.ycombinator.com/item?id=9708120. Minisign is also an alternative, though it doesn't seem as popular: https://jedisct1.github.io/minisign/. These may not be well integrated into git, but aside from 'everyone already uses and integrates with PGP,' there's no real reason stopping the usage of these other tools.

The two things that PGP does are pretty simple/straighforward. What makes PGP bad is that PGP itself it is really complicated, and the way it is integrated into email etc. is also complicated and carries a lot of historical baggage. But if you just want to make a public/private keypair and then use it to sign & encrypt data, that's pretty easy (or as easy as writing any crypto code can get). Signify, Minisign, and Age are clean, simple implementations.

Age does encryption: https://news.ycombinator.com/item?id=21895671

Of course this is sort of a https://xkcd.com/927/ situation. PGP is already used by everyone so why switch?

tptacek2• 6 years ago

Minisign is designed to be compatible with signify.

jacques_chester• 6 years ago

It's possible to sign with TLS certs, just not well-known.

https://stackoverflow.com/questions/50150318/sign-git-commit...

monoideism2• 6 years ago

The perfect is the enemy of the good.

-- Voltaire

OK, if you're going to downvote this, please tell me how you plan to convince git and Github to replace PGP with signify? Github in particular has invested significant time building up their PGP support.

Personally, I would be fine with signify, I've used it in the past and I like it, and I think people are right when they say we should move toward more focused, Unix-style cryptographic tools - for greenfield projects.

But that doesn't we should abandon all current uses of PGP, particularly when it's working as well as it is with git and Github. There's absolutely nothing wrong with it. It does what it's supposed to.

It took a long time to get PGP supported by Github, and now you're going to want them to change it?

Edit: If people want to add support to Git for signify, and lobby Github to support it, I'd be in favor. But strongly, strongly opposed to removing PGP support.

baybal2• 6 years ago

The author made a fundamental mistake of judgement.

So far, what he says amount to an advocacy against digital signing of code as such.

Miikka Koskinen says "digital signing doesn't work, so, lets decide to use something even more broken." Just that alone makes me think he has no say on this.

derefr• 6 years ago

Nah. He said that the current infrastructure for signing JARs is broken, making supplying signed JARs pointless. He didn’t say that this is the way things should be.

He advised, at the end, fixing the system; but it is implicit in his tone that he doesn’t expect anyone to try—it’s been broken this long with nobody caring, so why would that change now?

And yes, he advised relying on other mechanisms for verification, for the time being. Because those mechanisms provide nonzero (if small) security, while the current infrastructure for JAR signing provides zero security.

(Compare: “if calling the police does nothing, at least carry pepper spray.” This advice does not imply that pepper spray is better than effective police response; only better than an ineffective one. It also does not imply that one should not seek police reform. It only suggests what one should do while the condition of “ineffective police response” still holds.)

At the end, he says:

I’ve written this post in part to be proven wrong. I’m eagerly waiting for posts from y’all about how you do, in fact, systematically verify the signatures.

One would presume that, if he were proven wrong, and the digital signatures Of JAR files actually did anything, he would retract this post and post one giving the opposite advice.

trishankkarthik• 6 years ago

This is why you should use a well-designed system such as The Update Framework (TUF) that aims to make security as usable as possible:

[1] https://www.python.org/dev/peps/pep-0458/ [2] https://theupdateframework.io/

compsciphd2• 6 years ago

Here's an aside / funny thing about jar signing on blurays. The Blu-Ray spec only really enforces jar certificate verification for encrypted blurays (i.e. AACS). non AACS blurays (i.e. decrypted), ignore the certificate chain. i.e. a self signed certificate that uses its own key to sign the jars works fine per the spec.

The only reason people can take blurays (with java menus) and make them reliably playable when decrypted (in real players, ignoring things like VLC which can easily ignore the cert chain) is because the bluray spec tells them to explicitly ignore the certificate chain. This enables decryption programs to rewrite the java bytecode (and resign the jar's with a self signed certificate) removing any protection mechanisms (think screenpass) that are in the java code itself.

While I sort of understand the reason for it (you want to be able to test discs before mastering without encryption and without the need for signing), it really killed what could have been an effective security model. Basically the standard strong security vs convenience dilemma.

AnthonyMouse• 6 years ago

I half suspect they do things like that on purpose, because the alternative is to create demand for players that disregard the restrictions, and by that point they're also going to disregard all of the other ones and let the user fast forward through commercials etc.

So if they let the thing they don't want people to do be hard but not too hard, it satisfies demand from the people determined to do it instead of creating a market to solve the problem which would make it more convenient.

compsciphd2• 6 years ago

I don't know. perhaps, but they've shown a large willingness to force the hardware players to enforce security mechanisms that can't really be avoided nicely (ex: cinavia, there are ways to avoid it, but very few on disc based players)

philips• 6 years ago

A solution I am working on for these sorts of software supply chain attacks uses transparency logs. Take a look here (warning, alpha): https://github.com/transparencylog/btget

Essentially the binary transparency log acts like a notary service. It appends the cryptographic digest of a URL to an append only log. The append only log cannot be rewound without detection by clients. And clients verify the contents they receive against the entry in the log.

Two nice properties for the issues outlined in this post:

1. Hosting providers (or third parties) can add this sort of log without needing developers to do things like manage key material or 2FA tokens 2. It is complementary to good upload authentication or signing systems

In fact, Go is using transparency logs for source libraries. I think similar systems should be used for binaries in all other language ecosystems as well.

I would love to see a world where major source code distributions start running transparency log servers as part of their own file host integrity protections. Imagine maven, github, npm, etc all running these sorts of services for all uploads and using them by default in their clients. Users would have additional confidence that the v0.4.3 package they downloaded to their system two weeks ago is the same one their colleague got or the CI/CD system got.

ithkuil2• 6 years ago

Some time ago I made a fun hack where I abuse the Go transparency log to store immutable url->fingerprint mappings of arbitrary urls:

https://github.com/mkmik/getsum

Example:

    $ getsum https://github.com/bazelbuild/bazel/releases/download/3.4.1/bazel-3.4.1-installer-linux-x86_64.sh
    9808adad931ac652e8ff5022a74507c532250c2091d21d6aebc7064573669cc5

now, if somebody ever replaces that file on github, getsum will fail with an error.

(I was too lazy to make getsum actually download the file if the checksum matches, but that should be trivial; for now just do "getsum $url && wget $url").

This works by creating synthetic Go modules that embed "facts" about URLs inside generated Go code, which gets then downloaded by getsum and parsed using the stdlib "go/parser" package. Since all Go modules are recorded in the Go "sumdb" public transparency log, once this module has been generated it cannot ever change. (Getsum uses a constant "version" for the synthetic packages, thus facts are immutable).

I'm confident this works because it piggy-backs on the stable production transparent logs infrastructure of a high-profile project managed by a high-profile company. On the other hand the Go transparency logs admins could ban my "getsum.pub" domain if the generated traffic bothers them. This was for me just a quick way to show to some colleagues the potentials of transparency logs etc, and of course to have fun.

dependenttypes• 6 years ago

(I was too lazy to make getsum actually download the file if the checksum matches, but that should be trivial; for now just do "getsum $url && wget $url").

Please don't. Instead check the hash of the local file. A malicious server could serve a different file for getsum and for wget (bonus point: you won't have to download it twice)

lmm• 6 years ago

As far as I know, nobody ever verifies the signatures in a systematic way.

I systematically check the signatures on the jars of my dependencies. See e.g. https://github.com/m50d/tierney/blob/master/free/keys.proper... . If these artifacts were signed with different keys, I would notice (my CI builds would fail). There are still points of failure (e.g. if you could subvert the maven-pgpverify-plugin itself), but security is about increasing the cost of attacks.

It’s hard to find the public keys for the library maintainers. Sometimes they upload them on the keyservers, sometimes not.
There’s no established way of communicating that which public keys should be trusted. If there’s a new release and it has been made with a new key, your best bet is to e-mail the maintainer and ask what is up.
It’s hard to get any security benefits from the signatures in practice.

This is making perfect the enemy of good. Any solution to this problem would have to start with a signature mechanism. The signatures that currently exist make it a little bit harder to fake a library. If you want to make it harder, check signatures on jars you depend on, ask maintainers to publish their keys to keyservers and communicate which keys should be trusted, demand explanations when keys change.

Better to light a candle than curse the darkness. "x is incomplete, therefore it should be replaced by a completely new thing" is a total fallacy - and, strangely, seems to be used only when attacking PGP.

Signing .jars is not worth the effort

Related Stories