STARTTLS Everywhere

by peter_tonoli • 8 years ago

190 99 8 years ago

djrogers• 8 years ago

Except it’s not like let’s encrypt at all. This appears to test your mail server for starttls functionality, but it doesn’t issue certs or anything like that.

schoen6• 8 years ago

I think the title of the submission is mis-editorialized. The comparison that my colleagues working on this project make is that it's like HTTPS Everywhere, or even more like the HSTS preload list, rather than like Let's Encrypt. I don't think you'll find the comparison to Let's Encrypt on the site itself.

(Edit: in addition to the site tester that you noticed, there is also a public policy list and some forthcoming tools to enforce STARTTLS on the client MTA side when delivering e-mail, preventing downgrade and MITM attacks.)

However, you can also use Let's Encrypt to make this more useful by getting a publicly-trusted certificate for the TLS service on your mail server, and then listing your mail server with this list!

The introductions my colleagues posted about this today are at

https://www.eff.org/deeplinks/2018/06/announcing-starttls-ev...

https://www.eff.org/deeplinks/2018/06/technical-deep-dive-st...

which will hopefully give a clearer explanation of what the service is meant for.

devereaux4• 8 years ago

I'm sorry, but the problem is more complicated than that.

SSL is not mandatory on either port 25 or 587, and SSL can not be made mandatory if you follow the RFCs (it can be made mandatory if you tweak your MTA, but you may lose some mail). Advertising for SSL over DNS means you trust the DNS records - which you shouldn't without DNSSEC. Even with it, there can be workarounds that in practice will allow MITMs.

The only real solution is making SSL mandatory, and doing SMTP over SSL as in the good old days of using stunnel on port 465 to decrypt, then netcat to forward the output to the MTA running on localhost:25

But that is not standard. Maybe the efforts would be better invested by changing the standards to have a port where SMTP can not happen at all without SSL - just like port 465 was, over 10 years ago.

dang• 8 years ago

Ok, we've changed the title above to the hopefully least controversial part of the article's original title. (Submitted title was "STARTTLS Everywhere: HTTPS Everywhere, but for SMTP".)

schoen6• 8 years ago

Per the comment by djrogers, I think the submission title is mis-editorialized and doesn't describe what this project is meant for very clearly.

teilo• 8 years ago

Yeah, my mail server cert is issued by LetsEncrypt. So this has nothing to do with actually encrypting your SMTP.

jlgaddis2• 8 years ago

"STARTTLS Everywhere: HTTPS Everywhere, but for SMTP"

peter_tonoli3• 8 years ago

Thanks for the suggestion - Done :-)

move-on-by2• 8 years ago

From my understanding, this is pretty much the same thing as the HSTS preload list[1], except for SMTP servers instead of HTTPS servers. It has two main goals. First, encourage cert validations. If you don't validate a cert, it means nothing. Second, help prevent MITM downgrade attacks. If your server is on the list, it means it should always use a secure cert. If it isnt, then don't connect. Same thing as HSTS preload list.

[1] https://hstspreload.org/

peter_tonoli3• 8 years ago

Under 'about', the rationale for STARTTLS is listed, which includes, but isn't exclusive to the HSTS like list:

  - Increase STARTTLS adoption
  - Increase the number of mailservers that actually validate certificates
  - Maintain a STARTTLS Policy List to help prevent downgrade attacks on email services.

tialaramex6• 8 years ago

You write that "If you don't validate a cert, it means nothing". I should clarify that in this situation you get opportunistic encryption. I think you understood that, but the way it's phrased isn't clear. The situation is that we get secrecy, but we don't actually know who we're talking to since we didn't validate the cert. This genuinely is better than nothing.

If we were talking to a bad guy then we're screwed. But if we are talking to our intended recipient, even if their cert is bogus, our connection is protected from bad guys on the path. The bad guys can't choose later to access that communication after all, their only opportunity is to guess we're doing opportunistic encryption and intercept the original connection to play MitM, and if they guessed wrong their hand is shown by doing that.

This situation is unpalatable even for nation state adversaries like the NSA, because they would prefer not to be seen to be meddling. Even for something like the FSB, which scarcely cares if it's seen to be meddling, it does force them to make a decision they'd rather leave until after the fact. MitM everybody (and be known to do it) or leave them alone and maybe regret that later.

move-on-by2• 8 years ago

If we were talking to a bad guy then we're screwed. But if we are talking to our intended recipient, even if their cert is bogus, our connection is protected from bad guys on the path.

You acknowledge that you don't know who you are talking to. That alone proves my point. If the cert is bogus/unvalidated then its trivial for the bad guy to intercept and supply their own bogus cert instead. You are correct that its encrypted, but that means nothing when you do not know who you are talking to. It could even be a string of bad guys all capturing and injecting another bad cert. Encryption without validation means nothing.

they would prefer not to be seen to be meddling

Time and time again ISPs have been caught injecting javascript/cookies/html inside unencrypted traffic. If ISPs are willing to do that, then why should nation states be afraid to? The point of 'bad guys' is that they can't be trusted to not do bad things.

Besides the point that bogus/unvalidated certs do nothing, it would be easier to just perform a downgrade attack if the client isn't doing any validations. Downgrade to unencrypted and easily see everything. That is why they are creating a list of services that should only allow valid encrypted connections.

all_blue_chucks• 8 years ago

SRTTLS is vulnerable to downgrade attacks. It's better than nothing, but if someone wants to intercept it they can.

devereaux4• 8 years ago

Yes, because SSL is optional on port 25 and 487.

But you can chose to disregard the RFCs and disallow servers that will not encrypt to send you any mail. No downgrade attack then. It requires some manual changes. It may cause you to not receive email from some servers.

You can also only accept mail on port 465, which in practice is used for SMTP over SSL. You will receive even less mails.

Cf another reply I made today about that: https://news.ycombinator.com/item?id=17397500

belorn• 8 years ago

465 used to be deprecated since 1998 when STARTTLS was created, and it was only kept in use since older Microsoft applications (including Entourage v10.0 and its successor, Outlook for Mac 2011) did not support STARTTLS.

However in January this year, the proposed RFC 8314 reinstate the registration of port 465 for implicitly encrypted mail submission. I think it is a bit early to close down port 25, but a great idea to make sure 465 is correctly configured if it isn't already.

eklitzke• 8 years ago

This is far too little, too late:

* The SMTP RFC says that mail servers MUST NOT require STARTTLS to receive mail. Postfix (and I imagine most other production grade SMTP servers) has an option to require STARTTLS anyway, so if you really want STARTTLS you can already require that clients have it enabled, despite the braindead standard.

* STARTTLS ensures that mail was encrypted only in the final hop, from the last server to your server. That usually means it transited the public internet encrypted, but it definitely does not assure it.

There are interesting email security efforts afoot, notably the draft standard called "SMTP Require TLS": https://tools.ietf.org/html/draft-ietf-uta-smtp-require-tls-... . Unfortunately the reality is that the internet mail infrastructure evolves at an incredibly glacial pace. The entire SMTP protocol would benefit greatly from the adoption of an SMTP/2 protocol, rethought with modern security practices in mind.

devereaux4• 8 years ago

I fully agree. It will not achieve anything. FYI, "SMTP Require TLS" was running on port 465 many years ago.

With many ISPs (and even some VPS) now blocking port 25, it could have been an opportunity to migrate naturally towards the "new" unencumbered port.

schoen6• 8 years ago

Please see my reply downthread:

https://news.ycombinator.com/item?id=17397816

ubernostrum• 8 years ago

The SMTP RFC says that mail servers MUST NOT

The email RFCs require supporting a lot of things that are now horrendously outdated, and forbid doing a lot of things that should now be best practices.

The RFCs should lose that battle.

axaxs2• 8 years ago

See, at surface value I agree. But at a deeper level I don't. The problem is too many broken protocols exist via RFC only (dns, smtp, etc). Rather than violate that, or force it into something it's not, we should be writing newer, better protocols. The friction against this is huge, mainly by those who capped out at the previous RFCs, though. Email(well, smtp) itself is so outdated and broken, I can't believe it's still so widely used.

cm218710• 8 years ago

How did these committees think that optional, downgradable encryption was preferable to a standalone, encrypted only port (465)? Were they trying to save server ports, like if it was a precious resource? Any design decision I have seen regarding email since 2000 defies common sense. Like I heard most SMTP implementations do not validate certificates. What good is an unvalidated certificate? SPF is treated as indicative only or ignored.

We are managing to replace http1.1 with http2, it will take time but it is on the way. I am not even aware that an actual SMTP2 draft protocol that would solve all these design flaws (unverified sender, unencrypted).

And SMTP has a benefit http didn’t have. Most people access their emails through webmails, smartphones or enterprise outlook/exchange servers. For webmails and exchange only the server needs to be updated, and for smartphones, their short life ensures that older versions are pretty much all retired within 5 years. In addition, a handful of big players (google yahoo apple msft) have such a concentration of recipient accounts (retail users) that they can force a change on the market with the threat of your mail being classified as junk by them. So we could achieve a pretty quick transition.

JdeBP2• 8 years ago

The same has been pointed out about Web browsers using SRV DNS lookups to find servers.

* https://news.ycombinator.com/item?id=17395600

SPF, of course, should be used everywhere so that we reach the point of complete balkanization and collapse, and people realize that we can (and in fact in some ways already have) switch wholesale to different systems, faster. (-:

* http://jdebp.eu./FGA/smtp-spf-is-harmful.html

* https://news.ycombinator.com/item?id=10405864

lvh2• 8 years ago

Apparently 465 is only for mail submission, not MTA-MTA -- but of course that still leaves the obvious questions of a) why? b) if it must be a different port, why not make that TLS-only?

mailr3• 8 years ago

I'm surprised to see the opposition to self-signed certificates from the EFF. Pushing people towards centralized certificate authorities seems to be anti-freedom. Maybe there is a decentralized approach instead.

pde32• 8 years ago

We spent a lot of time thinking about this before we started building Let's Encrypt. In the case of the Web we didn't think there was a way to make self-signed certificates workable for anything because older browsers will always error on them. In the case of mailservers, self-signed certificates work for enabling TLS, and that's great, but you're extremely vulnerable to MITM attacks. So this project offers a way to prevent the MITMs, if you want it.

If there aren't major technical obstacles we might be willing to take pull requests for STARTTLS Everywhere that allow mailservers to announce self-signing policies, but it hasn't been a priority thus far because LE certs are so easy to get and are slightly more authenticated.

cm218710• 8 years ago

The problem with STARTTLS is a mitm downgrade attack. Self-signed certs are exposed to the same kind of mitm attack.

Mail is very sensitive communication. It is reasonable for the EFF to worry the risk of evedropping. Some websites are still sending passwords by email!

chrismorgan2• 8 years ago

This doesn’t seem to be working properly. Domains that I try are saying “couldn’t connect”, e.g. https://starttls-everywhere.org/results/?fastmail.com

mailr3• 8 years ago

I think there is an error in their SPF configuration. "v=spf1 ip4:67.212.170.242 -all" But verification connections come from a different address.

chrismorgan2• 8 years ago

I’m confused. This is nothing to do with SPF (it’s failing to connect in the first place), and what domain is that SPF record for anyway?

marcbradshaw• 8 years ago

The starttls-everywhere IP connects, tries a bunch of commands which don't look at all like a regular mail connection. It cycles through a bunch of tls settings and never sends an email. In short, it got itself blocked by anti-spam measures.

xyziemba• 8 years ago

It's worth also mentioning MTA-STS [1], currently a draft. It provides a mechanism for domains to declare a STARTTLS requirement via a file at https://<domain>/.well-known/mta-sts.txt. It's backwards compatible and opt-in. There seems to be some testing already going on [2].

[1] https://tools.ietf.org/html/draft-ietf-uta-mta-sts-21

[2] https://mta-sts.gmail.com/.well-known/mta-sts.txt

Boulth• 8 years ago

Yep, Google.com has the record too but I haven't seen them scanning my mta sts file...

mike-cardwell5• 8 years ago

You need to have a correctly formatted DNS TXT record at _mta-sts.example.com first. They'll check for that, and only if it exists will they bother attempting to fetch your mta-sts policy from https://mta-sts.example.com/.well-known/mta-sts.txt

cm218710• 8 years ago

I am curious as for why they would use https for that. It sounds like something that should be better stored in DNS. Unencrypted but your MX entries are not more encrypted.

UnrealIncident• 8 years ago

I assume the problem is that most domains don't have DNSSEC. So if you can MITM the SMTP stream you can MITM the DNS request.

znpy• 8 years ago

This is interesting, I have been wondering for a while if I could disable plaintext email delivery on port 25 and just leave encrypted submission on port 465.. Would I be losing mail ?

kalleboo• 8 years ago

Gmail says 89% of inbound email is encrypted (up from 60% 2 years ago). So theoretically you'd lose 11% of your mail. https://transparencyreport.google.com/safer-email/overview?h...

flashmob5• 8 years ago

Yes, unfortunately you'd be losing email. Port 465 has been deprecated a while back (1997).

I guess that's the problem - there is no practical way to disable plaintext email from the start. You may always need to accept the connection, see if the client will STARTTLS, disconnect if they do not and hope they don't re-try and keep hammering your servers with the same message, the error message "hey, i'm not accepting plaintext" will most likely get ignored.

ivanr• 8 years ago

Additionally, MTA-STS has recently been approved as Proposed Standard, so we may see increased support for it in the near future. [1]

The key advantage of MTA-STS is that it can be deployed quickly, without almost any disruption of the already-deployed infrastructure. Of course DNSSEC and DANE already solve the same problem, but MTA-STS is the designed by and for people who don't want to use DNSSEC.

The EFF's efforts are complementary to MTA-STS. As schoen mentioned elsewhere on this thread, in the early days it's probably useful to have a preload list. (That said, I am not sure why they feel it necessary to cast MTA-STS in negative light: "[...] so the sender will never know the recipient supports STARTTLS." [2] They should explain that there MTA-STS is trust on first use and has a memory effect, exactly like the well-accepted HTTP Strict Transport Security.)

We added support for MTA-STS testing to Hardenize just today, along with a blog post that explains the background and explains how to deploy it. [3] (Disclaimer: Hardenize founder.)

[1] https://www.ietf.org/mail-archive/web/ietf-announce/current/...

[2] https://www.eff.org/deeplinks/2018/06/technical-deep-dive-st...

[3] https://www.hardenize.com/blog/mta-sts

achillean2• 8 years ago

Out of ~6.5 million SMTP servers [1] on the Internet roughly 2.7 million support STARTTLS [2]. However, 1 million of them also support SSLv3 and ~200,000 still allow SSLv2.

[1] https://www.shodan.io/search?query=port%3A25

[2] https://www.shodan.io/report/PIHm43GP

peter_tonoli3• 8 years ago

And how many just use self-signed certificates?

achillean2• 8 years ago

I would assume that most of the certs w/ a chain length of 1 are self-signed so around 1.6 million (out of the 2.7 million).

jedberg• 8 years ago

I find it odd that the tester gets mad if it can't connect to the naked domain, even when the naked domain has MX records.

Other than that this is really cool!

It made me sad though -- I checked all the domains that I used to run mail for at some point in my career, and not one of them passed all the tests. When I managed email, I always made sure that our servers met all the current standards.

kroeckx2• 8 years ago

I'm still using a self-signed certificate because I'm using DANE / DNSSEC. I don't know of any tool that can properly help me automate the rollover of a let's encrypt certificate.

J-Kuhn• 8 years ago

certbot?

You can edit the cron job for certbot and add a script that should be called when certificates were renewed.

rossy• 8 years ago

You can put a hash of your public key in the TLSA record (TLSA 3 1 1). Unlike the full certificate hash (TLSA 3 0 1,) the public key hash shouldn't change when your LE certs rollover.

See: https://community.letsencrypt.org/t/please-avoid-3-0-1-and-3...

tialaramex6• 8 years ago

Note that if doing this you may need to explicitly tell Certbot (or whatever client you use) not to keep replacing your key pair when renewing certificates.

If you set things up to involve CSRs this problem goes away because the CSR binds to a key pair, just don't change the CSR unless your DNS names change. But if you use CSRs, Certbot forces you to take charge of ensuring the renewal schedule etcetera, since it has no way to be sure if the same CSR can be used next time.

Mail servers can do more here to help automate this. Also, maybe we can imagine that TLS-ALPN-01 (a forthcoming Let's Encrypt proof of control method) could work with SMTP STARTTLS. That would let a mail server take responsibility for getting its own certs (port 25 is on the short list of "Authorized ports" for the Ten Blessed Methods) by telling its own TLS implementation "Hey, when asked for ALPN, offer this extra ALPN feature and repeat everything I tell you" to prove it is really who it says it is.

I think the above could work, but it would need effort from Let's Encrypt and TLS-ALPN-01 itself isn't finished yet.

cm218710• 8 years ago

Perhaps I miss your point but if you use one of the supported DNS providers [1] can't you use certbot with DNS authorization and export the key and certificate to your mail server? Otherwise the acmev2 protocol isn't horrible to implement if you want to integrate it with a custom infrastructure.

[1] https://certbot.eff.org/docs/using.html#dns-plugins

kroeckx2• 8 years ago

There are multiple ways to set it up: - Reuse the key, set up the TLSA record using the public key. Certbot really doesn't seem to support this in an automated way. It requires that you generate a CSR, but there doesn't seem to be a way to renew this automatically. - Generate a new key for each certificate, which is what certbot. You would first need to add both the old and the new certificate to DNS, wait for the TTL of the old to expire, change to the new certificate, remove the old from DNS. There doesn't seem to be a tool to do this. - Set up the TLSA record to say you trust the CA.

mtgx• 8 years ago

The main problem with this is that STARTTLS is not anywhere near good enough, but if it sees high adoption, nobody may bother with something better in the future because they'll all think "Mission Accomplished."

dwheeler• 8 years ago

There is never something that is "good enough" to solve all problems. In addition, stronger approaches are often harder to adopt (which is why they often don't get adopted). This effort to encourage STARTTLS everywhere provides reasonable protection against bulk passive monitoring, at very low effort. I think encouraging the use of TLS for all email traffic is a good idea.

aorth• 8 years ago

Cool. This tool made me realize that my domain's DNS records were messed up. Everything worked most of the time, but sometimes people wouldn't receive my emails! It turns out I was missing the latest MX and SPF records from my provider (Kolabnow).

snvzz• 8 years ago

Doesn't support IPv6. Can't connect to IPv6-only mail servers.

mike-cardwell5• 8 years ago

Does anybody that runs a large mail system have any stats about how much mail is delivered over SSLv2 and SSLv3 nowadays?

mike-cardwell5• 8 years ago

States "Couldn't connect to grepular.com. We couldn't find any MX records for grepular.com! Did you make a typo or enter the wrong email domain? Make sure to enter the part of your email address after the @!"

Then proceeds to show results for the MX anyway. With a single error: "Does not use a secure TLS version Error: Could not establish connection with hostname %!s(MISSING)"

Seems a bit broken to me.

the_zeroth_law5• 8 years ago

Thanks for catching that! We definitely still have some bugs to work out.

soetis1• 8 years ago

Since proper email transport encryption is finally getting some momentum with DANE and MTA-STS, I'm surprised that STARTTLS Everywhere doesn't mention them.

DANE is being recommended (mandated?) by the European Union, and is on the rise. MTA-STS is backed by Gmail, Microsoft and many others, which is likely to give it traction.

the_zeroth_law5• 8 years ago

We do mention both: see https://www.eff.org/deeplinks/2018/06/technical-deep-dive-st...

To sum up from that post, we think STARTTLS Everywhere is a stop-gap measure until DNSSEC is fully deployed, and STARTTLS Everywhere can act as a preload list for MTA-STS (to prevent DNS downgrade attacks).

auscompgeek• 8 years ago

Hmm. X.509 subject names aren't meant to be case-sensitive, right? I just realised one of the domains I manage have MX records in all-caps, and the cert checker fails on that, even though Gmail's SMTP servers have valid certs.

tialaramex6• 8 years ago

X.509 subject names are human readable text and probably you shouldn't try to reason about them beyond the simplest exact bit-for-bit comparison.

Where the subject name is the name of a something on the Internet it should use the Internet's defined "Subject Alternative Name" (SAN) mechanism, rather than trying to squeeze into the X.500 system's hierarchical directory. For compatibility, and to save the X.509 subject being empty which confuses some software, you may write one of the DNS names or IP addresses into the X.509 Subject's Common Name (CN) field as human readable text, but you should always write all DNS names and IP addresses into SANs.

The dnsName SAN is defined like a DNS record, so it's case-insensitive and (if it's an IDN) Punycoded, it's also deliberately defined with a single encoding that is too narrow for anything much beyond an actual (Punycoded if appropriate) DNS name, to avoid people trying to write "extended ASCII" characters into this field by mistake.

the_zeroth_law5• 8 years ago

Yep. This looks like a bug in STARTTLS Everywhere, and we're working on it!

wila2• 8 years ago

My email server passed, but the test mentioned, it could be better by adding it to the EFF's STARTTLS policy list [1].

The only problem being that that list is currently practically empty.

So while I was considering adding that email server which I maintain, when I saw the items on the list it makes me reconsider.

The only email servers currently on the list are : google, yandex, yahoo, icloud, outlook, comcast, eff, qq and facebook.

So while it might be an opportunity to get your server on the list early, personally I worry more about what it will break as to what it will help.

In conclusion, it looks like a great initiative, but it will have to be in production for a while before I would recommend anybody to add their server to that list.

[1] https://dl.eff.org/starttls-everywhere.policy.json

the_zeroth_law5• 8 years ago

One of the nice features of the policy list is that you can put your server on it in testing mode. In testing mode, any failure in TLS negotiation is logged and reported, but the message is sent over the insecure communication. So it shouldn't cause any breakage.

See https://github.com/EFForg/starttls-everywhere/blob/master/RU... for more info.

wila2• 8 years ago

OK, that's interesting and does indeed help in taking away that concern.

Another question if I may.

The site mentions:

the STARTTLS Policy List gives mail servers another point of reference to discover whether other mail servers support STARTTLS

Is there any email provider, email server component or email client that already uses this policy list?

Because that's another thing I miss from the FAQ.

kseistrup• 8 years ago

The checker concludes that because my server doesn't allow TLSv1.0 and TLSv1.1 it “Does not use a secure TLS version”, when in fact it does allow/require TLSv1.2 (and the checker does connect via TLSv1.2 later on). Is that an intentional conclusion or a bummer?

the_zeroth_law5• 8 years ago

I'm an EFF technologist, and we're looking into this!

exabrial• 8 years ago

I'm not a big fan of STARTTLS, I'd rather just have implicit TLS (All or nothing) from the get go.

jgowdy• 8 years ago

I appreciate what they're trying to do, and it may improve the status quo, but we've learned that the push away from implicit SSL/TLS and towards STARTTLS was wrong. Using one insecure aspect (DNS) to note that you SHOULD be able to do TLS with my mail server isn't a great solution.

We need to revisit the STARTTLS vs implicit TLS debate in light of the obvious vulnerability and overhead of starting with plain TCP connections and then hopefully signalling towards security. HTTPS is obviously implicit TLS and it works great. We know STARTTLS has issues. Can we not keep going down the STARTTLS road for email, while going down the implicit TLS road for other things?

johnp_• 8 years ago

Implicit TLS is already favored for Email submission and access per RFC8314: https://tools.ietf.org/html/rfc8314

The problem is partly because we don't have an assigned port for MTA2MTA implicit TLS. Otoh DANE also already provides a way to have mandatory TLS for MTA2MTA traffic.

STARTTLS Everywhere

Related Stories