Nginx, TLS, and Let's Encrypt

The TLS/SSL Config for Of-Mu.org Using Let’s Encrypt

Description

This is a guide about how I setup and maintain the TLS/SSL certificates for this website, of-mu.org.

As of 2017-10-14 05:18:35 PST/PDT this site was getting an A+ rating from SSL Labs using the configuration shown here.

You can rerun your own SSL Labs test on this site using the link below:

https://www.ssllabs.com/ssltest/analyze.html?d=of-mu.org

Here is my screenshot of boastful joy:

Nerd Pride!

References, Attributions, and Thanks

So, how did I do this? Basically, I followed this guide on the Arch Linux wiki:

https://wiki.archlinux.org/index.php/Let's_Encrypt

To be clear, Linux is a passion for me and Arch Linux is my distro (or “Linux distribution”) of choice.

This is all only possible thanks to the EFF, Let’s Encrypt and the people over at Arch Linux. Of course there is the larger FOSS movement as well. Without the contributions of countless unnamed people my site would neither function in it current form nor be so well protected with TLS/SSL certificates.

Thanks to all of you for your fine and noble work.

The Instructions (or Show Me Already)

This is Arch Linux, therefore I installed certbot like so:

pacman -S certbot

Warning: If you want to know what to click on next, the rest of this guide is likely going to be a disappointment for you, so please continue with that caveat in mind. This is all done via the command line (in bash no less).

I am not running a standard nginx config (and didn’t have luck with the nginx + certbot plugin), so I performed the manual install. Many might find this “the hard way” but so often for me, “the hard way” seems to be the easiest way and the experience tends to result in a deeper understanding of things. Such is life.

The manual install requires that I have a place to serve HTTP content to certbot. I created a place to do this with nginx in the file /etc/nginx/conf.d/letsencrypt.conf :

location ^~ /.well-known {
  allow all;
  alias /var/lib/letsencrypt/.well-known/;
  default_type "text/plain";
  try_files $uri =404;
}

I then restarted nginx and ran the certbot like so:

systemctl restart nginx
certbot --manual

Then certbot required me to confirm that I control the domain by adding several keys (as plain text files) to the following location : /var/lib/letsencrypt/.well-known/

To put it another way, certbot won’t simply give out new TLS/SSL certificates for a domain to anyone. It wants to have a reasonable degree of certainty you own this domain (or are in control of it) by having you put certain files in certain places on the site which contain exactly what it tells you to put in them. If you can prove to certbot you control the webserver it will grant the certificate. There is more to it than this of course and this is only one requirement, but a key one. If you cannot host the files as asked, you cannot get your free TLS/SSL certificate.

The certbot program talks to a server side program. It is this remote server side tool (which certbot is talking to) that will make the remote HTTP requests to your domain and if your web server sends the correct responses certbot considers this to be verification that you are allowed get a certificate for this domain.

If one is wondering how to get the certbot data into the files it expects, may I recommend the venerable echo command for this? For example, you may put the VALUE in the file named FILENAME using something like this:

echo "VALUE" >> /var/lib/letsencrypt/.well-known/"FILENAME"

Once this is done, I replied (via the shell script prompting) that each file is updated and certbot confirms this via remote HTTP request using its server side tool.

It will have at least one test for each domain name (or sub-domain) being registered. It is also important to be sure all domains are being correctly resolved via DNS before attempting this or it will fail.

After successful answering all challenge/responses certbot will download your new TLS/SSL certificates to local files!

TLS/SSL Cert Auto-updates

Now, these certificates expire in 90 days. This is wonderful for two reasons - first, it keeps you updating your keys making any comprised keys expire sooner than if you had year long keys. Second, it requires that we constantly return to our security configurations. Security has never been and will never be: “set it and forget it”. Such thinking is, to me at least, the antithesis of security. If you are not checking, probing, and documenting the efforts to circumvent your security, let’s be honest - do you really know for certain that you are secure? If one had security issues, how does one know (either way) without checking?

So, how did I decide to update the certificates. I automated it! (I can hear the groans about “set and forget” hypocrisy already but I assure you I have other tools to perform checks as well.)

To get the automated updates working, I create a systemd service to do this named /etc/systemd/system/certbot.service:

[Unit]
Description=Let's Encrypt renewal

[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --quiet --agree-tos --post-hook "/usr/bin/systemctl restart nginx.service"

Now systemd has built in cron like capabilities which allow me to schedule this job. As an aside, I initially resisted the move from init scripts for service control. However, systemd has demonstrated its value to me countless times and I now appreciate being “forced” to evolve and learn new tools. The systemd init replacement does solve numerous of problems with init scripts and is actually easier to work with after I learned how. I have come to appreciate systemd as a very useful and improved tool over init scripts.

Enough gushing about my favorite programs and back to the TLS/SSL work!

I then created the timer named /etc/systemd/system/certbot.timer:

[Unit]
Description=Once daily renewal of Let's Encrypt's certificates

[Timer]
OnCalendar=daily
RandomizedDelaySec=2h
Persistent=true

[Install]
WantedBy=timers.target

Finally, I enable the timer like so (not the service - the timer controls the service not the other way around):

systemctl enable certbot.timer
systemctl start certbot.timer

To see if it is running one can use:

> systemctl list-timers
NEXT                         LEFT     LAST                         PASSED       UNIT                         ACTIVATES
Sun 2017-10-15 11:35:56 PDT  7h left  Sat 2017-10-14 11:35:56 PDT  16h ago      systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.serv
Mon 2017-10-16 00:00:00 PDT  19h left Sun 2017-10-15 00:00:00 PDT  4h 3min ago  logrotate.timer              logrotate.service
Mon 2017-10-16 00:00:00 PDT  19h left Sun 2017-10-15 00:00:00 PDT  4h 3min ago  man-db.timer                 man-db.service
Mon 2017-10-16 00:00:00 PDT  19h left Sun 2017-10-15 00:00:00 PDT  4h 3min ago  shadow.timer                 shadow.service
Mon 2017-10-16 01:41:49 PDT  21h left Sun 2017-10-15 01:19:21 PDT  2h 44min ago certbot.timer                certbot.service

Yup, it’s running! :)

Configuring Nginx and Best Practices

Now, as an individual human-being (I am still human and there is still only one of me, right?) with many responsibilities, it is difficult to stay up to date on all the best cryptography practices. It can be very helpful to have resources to assist with something like this and one which I have found incredibly effective is located here:

https://mozilla.github.io/server-side-tls/ssl-config-generator/

Those wizards over at Mozilla are giving away the magic for free again! What ever will they do next? (Hint: probably keep giving away cool things for free).

Generic TLS/SSL Recommendations from the Site Above

When I told the site above my version of nginx and openssl it recommended the following configuration:

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    # Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    # certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
    ssl_certificate /path/to/signed_cert_plus_intermediates;
    ssl_certificate_key /path/to/private_key;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;


    # modern configuration. tweak to your needs.
    ssl_protocols TLSv1.2;
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
    ssl_prefer_server_ciphers on;

    # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
    add_header Strict-Transport-Security max-age=15768000;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;

    ## verify chain of trust of OCSP response using Root CA and Intermediate certs
    ssl_trusted_certificate /path/to/root_CA_cert_plus_intermediates;

    resolver <IP DNS resolver>;
 }

Specific Examples for Of-Mu.org

The following is an example of how the nginx server block appear is/was configured for this site (of-mu.org at the time the test was run):

### Hugo site for of-mu.org

  server
  {
    listen 443 ssl http2 default_server;
    ssl_certificate /etc/letsencrypt/live/of-mu.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/of-mu.org/privkey.pem;
    ssl_trusted_certificate /etc/letsencrypt/live/of-mu.org/chain.pem;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;
    ssl_protocols TLSv1.2;
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
    ssl_prefer_server_ciphers on;
    add_header Strict-Transport-Security max-age=15768000;
    ssl_stapling on;
    ssl_stapling_verify on;
    server_name of-mu.org;

    #access_log  logs/host.access.log  main;
    location /
    {
      root /var/of-mu.org/public/;
      index index.html;
    }
  }

There is also a 301 HTTP to HTTPS redirect in another section like so:

 server
{
  listen 80 default_server;
  listen [::]:80 default_server;
  return 301 https://$host$request_uri;
}

That is it! That is how I got an A+ from SSL Labs. (Google.com was only an A today so I get a prize or something for this right? If so, I like ice cream - a lot. Just a hint if anyone was thinking about it.)