Let's Encrypt and Nginx – State of the art secure web deployment

https://letsecure.me/secure-web-deployment-with-lets-encrypt-and-nginx/

by llambiel • 10 years ago

290 84 10 years ago

feylikurds3• 10 years ago

Ewwww, that renewCerts.sh is pretty crappy. Who the hell is going to check the /var/log/letsencrypt/renew.log everyday to see if renewing failed?

Could not they do something nicer with systemd and email?

pfg4• 10 years ago

The default behaviour of cron is to email the user if a job finishes with a non-zero exit code, which seems to apply here in case of renewal failure.

feylikurds3• 10 years ago

But is not the default account that it would email root? I run Debian and almost never log in as root. Would all admin sudoers receive the email?

a-priori• 10 years ago

This sounds like a job for Dead Man's Snitch.

https://deadmanssnitch.com/

diakritikal• 10 years ago

Just using caddy server seems a lot simpler...

qrv3w• 10 years ago

+1 for Caddy. I was using NGINX a long time, and wrote some similar scripts to make certificates for my web apps. After I switched to Caddy I had a 6x smaller config file, no more ln -s, and HTTPS without every having to think about it!

sbarre• 10 years ago

I'd never heard of Caddy, and it looks great! Thanks for giving me something to tinker with this weekend! :-)

tlrobinson• 10 years ago

Neat, I like that when started with no arguments/config it just serves the files in the current directory, but then you can customize it from there.

I have "alias webserver='python -m SimpleHTTPServer'" in my shell config, but I think I'll switch to Caddy.

callahad• 10 years ago

For local development, consider looking into devd (https://github.com/cortesi/devd). It's a single binary that supports things like livereload, network throttling, routing, and reverse proxying.

aroch• 10 years ago

You do need to restart Caddy in order to renew your cert, FWIW

mholt• 10 years ago

Not true - Caddy has always renewed certificates automatically, and the latest version (0.8.2) renews without restarting. Relevant change: https://github.com/mholt/caddy/commit/11103bd8d68ed9d8dcd2fc...

zenlikethat• 10 years ago

Agreed, Caddy is great.

ivan_ah6• 10 years ago

Isn't running this as a `@daily` CRON job too much? I thought Let's Encrypt certs were good for 3 months? Why not @monthly or months 0,2,4,6,8,10 ?

teraflop• 10 years ago

If something breaks, you might as well find out about it as soon as possible. That way you have the full 90 days to figure it out at your leisure, instead of 60 or 30.

ivan_ah6• 10 years ago

Right. Also, I just saw that `letsencrypt-auto renew` will only issue new certs if < 30 days left on current cert.

sschueller• 10 years ago

I just use https://github.com/lukas2511/letsencrypt.sh/ single bash script.

Add a config.sh and setup nginx alias. Then just add domains to the domains.txt and have the script run via cron daily.

Finished

X-Istence• 10 years ago

This is the script I use too. I have a hook that automatically restarts nginx, which fires only if a cert has changed. Very simple. Works very well.

emilevauge• 10 years ago

https://github.com/containous/traefik now has native Let's Encrypt support ;)

ivan_ah6• 10 years ago

The --webroot option doesn't work for my setup, so I need to shutdown nginx for 2-3 seconds and use the --standalone option. I set this as a CRON job that will run every two months. It's not elegant, but it's done.

Here's the modified script using certonly and the --force-renew flag.

    #!/bin/bash
    # Force-renew the "Let's Encrypt" certificates for a given domain
    # Run this as root as a BI-MONTHLY cron job
    export DOMAINS="yourdomain.com,www.yourdomain.com"
    export LOGFILE="/var/log/letsencrypt/renewal_yourdomain.log"

    echo "Stopping nginx temporarily to renvew certificates for $DOMAINS ..."
    service nginx stop

    echo "Calling /opt/letsencrypt/letsencrypt-auto certonly --standalone --force-renew -d $DOMAINS"
    if ! /opt/letsencrypt/letsencrypt-auto certonly --standalone --force-renew -d $DOMAINS > $LOGFILE 2>&1 ; then
        echo "certonly call failed, restarting nginx"
        service nginx start
        echo "LOG info:"
        cat $LOGFILE
        # TODO: email administrator...
        exit 1
    fi

    echo "certonly call succeeded, restarting nginx"
    service nginx start

Note: don't run this as a daily cron job since this has --force-renew...

schoen5• 10 years ago

Do you ever get problems with the socket still being in use after nginx is shut down?

ivan_ah6• 10 years ago

Not on the N=1 times I've run the script, but will look out for this in the future.

ran290• 10 years ago

I'm curious: why doesn't webroot work for your setup?

ivan_ah6• 10 years ago

A dynamic script is handling all requests, so there is no "webroot" directory where you can put stuff for them to appear under /

schoen5• 10 years ago

It scared me to see that the author recommended running

  curl http://nginx.org/keys/nginx_signing.key | sudo apt-key add -

(This adds a key or keys downloaded over an unauthenticated http connection to one's Debian keyring, allowing whatever keys the network sends back to authenticate any future package updates.) I wrote to the author with a note expressing my concern.

icebraining3• 10 years ago

Unfortunately, it seems there's no secure way to fetch the key. The nginx team recommends one checks the "web-of-trust" to check if the key is signed by others.

rajivm• 10 years ago

At the least though it could be https.

rlpb2• 10 years ago

Stick it on a keyserver, and then ask gpg to fetch it from that keyserver with the full fingerprint. Assuming that your instructions that include the fingerprint are secure (which they have to be, else the instructions could root your box anyway), then that should be reasonable.

This does assume that gpg verifies that the key retrieved matches the ID requested, which I assume it does. Otherwise that'd be quite a serious bug.

dublinben• 10 years ago

Based on the PGP pathfinder here[0], it is likely this is a valid key. I'm only a few signatures away from this nginx signing key.

[0]http://pgp.cs.uu.nl/

llambiel• 10 years ago

I agree with your concern. As a temporary "workaround" I've updated the article with a SHA256 checksum of the key. I'll chase the Nginx team for serving this key over https

schoen5• 10 years ago

Thanks for the update!

click1703• 10 years ago

We should really be verifying the fingerprint of the key itself, even if it is served over HTTPS.

click1703• 10 years ago

What bugs me is how prevalent that has become.

I'm looking at you Jenkins![0]

[0] https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenki...

goodplay• 10 years ago

The rust project also advocates this method of installing software on their download page. To their defense, however, they do offer gpg signatures for their tarballs, even if you need to dig around for a bit to find them.

Also, anyone suggesting that this method of downloading and installing software is secure due to its use of HTTPS is incredibly reckless.

tokenizerrr• 10 years ago

Jenkins at least serves the key over HTTPS. Would you instead prefer they did not offer packages for your package manager at all? I sure wouldn't, I really appreciate those packages for easy upgrades.

lamby• 10 years ago

Note that it also does not limit the key to simple nginx packages - if I take control of that repo I could trick you into installing my version of, say, base-files or bash, etc.. :(

e12e• 10 years ago

Also, while apt-key might handle random input fine, I'm concerned with anything that goes "curl http://example.com | sudo ...". In this special case, "apt-key add -" should avoid most problems - but I'd still prefer verifying the (possibly untrusted) gpg-key as a normal user, and then elevating to add what appears to be the correct key, and a valid gpg-key via apt-key add.

tbrock• 10 years ago

Lets encrypt fixes the encryption problem sure but does anyone else feel that all we really needed was really great documentation on what to do instead of an intrusive set of scripts?

icebraining3• 10 years ago

No, because you need the automation to make the short expiration times bearable, and having those short expiration times is more secure.

Besides, the official script is just one part of the project; the others are (1) free certs and (2) a standard protocol, which you can use with other tools.

doublerebel3• 10 years ago

Yes, the API documentation is lacking especially with what we've gotten used to from Swagger markup and Stripe's API doc style. I have scoured for such an easy breakdown and found none. As a result I actually just implemented a new, clear, client for LetsEncrypt and have been documenting as I go.

It's made me think we should have a Swagger or API Blueprint of the spec on github that everyone can keep up to date. What do you think?

pfg4• 10 years ago

Are you referring to the server-side API the client is communicating with, or the internal API the client exposes?

The former is documented in the ACME specification[1], currently being worked on by the IETF. There are many low-level ACME libraries for basically every language[2], and a pretty decent guide on writing your own client as well[3].

[1] https://ietf-wg-acme.github.io/acme/

[2] https://github.com/letsencrypt/letsencrypt/wiki/Links#librar...

[3] https://github.com/alexpeattie/letsencrypt-fromscratch

IgorPartola• 10 years ago

I have been happy with https://github.com/lukas2511/letsencrypt.sh. I am trying to get it packaged for Debian/Ubuntu and either get it into Debian-proper or at least host the repo myself to make it easier to use for the common case. Since nginx reloads the cert on a SIGHUP it makes it really easy to have zero downtime renews.

As for getting notified if something goes wrong I use the following in my crontab:

    10 5 * * *  root    test -e /usr/local/bin/letsencrypt.sh && /usr/local/bin/letsencrypt.sh -c > /dev/null

letsencrypt.sh outputs errors to stderr, so any errors will be sent to the root account. To get that working, do:

    apt-get install postfix
    echo 'postmaster:     root' > /etc/aliases
    echo 'root:           igor@example.com' >> /etc/aliases
    newaliases

Problem solved.

smithclay2• 10 years ago

Recently went through a similar setup, but used Docker and some existing h2-friendly images. Think it's a nice way forward for deploying to production environments.

Wrote about the process here: https://clay.fail/posts/hip-http2-using-docker/

mikewhy• 10 years ago

I recently built docker-gen-letsencrypt[1]. It's the same concept as what you're using, but fully automated for getting certs.

[1] https://github.com/mikew/docker-gen-letsencrypt

smithclay2• 10 years ago

That's awesome, will be updating the site to use your image this weekend. Like the support for docker-compose and the staging servers, too.

mrits• 10 years ago

Interesting choice of cryptos. My latest client isn't letting us use anything besides GCM right now.

velox_io• 10 years ago

Thanks for the info on the headers, I can't believe they've issued certs for over a million domains!

Here's my notes on setting up LE on IIS if anyone one is interested, it's done by using Powershell/ Package manager.

//1. Install (you will get some security prompts) Install-Module -Name ACMESharp

Import-Module ACMESharp

Initialize-ACMEVault

New-ACMERegistration -Contacts mailto:somebody@example.org -AcceptTos

//2. Request the challange, this is for a website currently running on IIS. 'WebSiteRef ' refers to the name of the site within IIS

New-ACMEIdentifier -Dns demo.velox.io -Alias demo Complete-ACMEChallenge demo -ChallengeType http-01 -Handler iis -HandlerParameters @{ WebSiteRef = 'Demo' }

Submit-ACMEChallenge demo -ChallengeType http-01

//3. Create & download the certificate

New-ACMECertificate demo -Generate -Alias demoCert

Submit-ACMECertificate demoCert

Update-ACMECertificate demoCert

Get-ACMECertificate demoCert -ExportPkcs12 "C:\Users\USER\desktop\demoCert.pfx"

You can now install this on your server.

andersonmvd• 10 years ago

Look at how many lines we need to secure tls connections on nginx. We need better defaults.

jzelinskie• 10 years ago

"State of the art" and "cron" should probably never be in the same article.

homero• 10 years ago

Can someone tell medium? They're still buying comodo certs for their custom domains

iancarroll• 10 years ago

They probably do not want to deal with LE's rate limiting and shorter renewal periods.

ruslo• 10 years ago

  > sed -i 's|PasswordAuthentication yes|PasswordAuthentication no|g' /etc/ssh/sshd_config

Will not work if string is commented out:

  > grep PasswordAuthentication /etc/ssh/sshd_config
  # PasswordAuthentication yes
  > sed -i 's|PasswordAuthentication yes|PasswordAuthentication no|g' /etc/ssh/sshd_config
  > grep PasswordAuthentication /etc/ssh/sshd_config
  # PasswordAuthentication no

sleepychu• 10 years ago

I'm really not a fan of this domain grab to write a single article with no(t a lot of?) new information aimed at selling services from a single host. You're not the only person guilty of this but it feels quite misleading like the article is coming from a 3rd party.

alexpeattie• 10 years ago

You can create a more hardened setup by using a 4096 bit RSA key:

  /opt/letsencrypt/letsencrypt-auto certonly --rsa-key-size 4096 --server https://acme-v01.api.letsencrypt.org/directory -a webroot --webroot-path=$DIR -d $DOMAINS

...and using the secp384r1 curve for ECDHE key exchange:

  # in your nginx.conf
  ssl_ecdh_curve secp384r1;

Arguably, the real state of the art is to use an ECDSA certificate. Let's Encrypt recently started supported them, they offer a equivalent level of security to RSA at much lower bit lengths (a 384 bit ECDSA key is considered equivalent to a 7680 bit RSA key) and a few recent TLS vulnerabilities (like DROWN) have targeted implementation details of RSA.

lorenzhs• 10 years ago

4096 bit RSA keys offer very little additional security (2048 is plenty for at least the next few years, and with a certificate that's valid for 90 days, there's practically no risk - you can rotate the key rather easily if something bad comes along), but has a fairly big impact on performance and battery life, especially on mobile devices.

realusername• 10 years ago

Here is also my config if anyone is interested (also A+ on ssllabs.com): https://gist.github.com/alex-min/158f35f604b24e163ae9, feel free to copy it. (or suggest improvements !)

I also recommand https://sslcatch.com which sends you a warning email if your certificate is about to expire. I have a crontab to renew it but this can be also helpful just in case.

nickpsecurity3• 10 years ago

It's a nice tutorial. The title tripped me out, though: a common webserver + HTTPS + free certificate on Windows/Linux is "state of the art secure web deployment?" I'd hate to see what passes for average or (shudders) ancient.

In my mind, I'm seeing "state of the art" being more like a combo of Ur/Web for apps, robust implementation of OP2 web browser for client, lighttpd rewritten in Haskell, HTTPS component written in SPARK or Rust, all running on GenodeOS or CheriBSD in isolated partitions, C parts compiled with CompCert extended with Softbound + CETS, anti-fuse FPGA doing I/O offloading/mediation, and hardware done in Bluespec. That is state of the art with probably badass results. This submission is... more run of the mill. Immediately useful, though. :)

Let's Encrypt and Nginx – State of the art secure web deployment

Related Stories