Certbot

Certbot
Image shows:	Certbot
Summary
Title:	Certbot
Description:	Certbot, from the Let's Encrypt project of the EFF, provides free and automated SSL certificates for QualityBox.
More
Notes:	For non-public sites, we can still generate certificates using DNS
Test:	Test on SSL Labs.com
Example:	See File:Certificate grade.png

Certbot is a tool that allows you to manage the Transport Layer Security (TLS) of your Webserver.

Let's Encrypt

We used to run certificates from StartSSL because they offer free one-year certificates. However, today we upgraded to using 'LetsEncrypt' and our certificates are both more secure and easier to manage. Instead of a "B" grade, we now have "A" grade security.

Certbot (code) is a fully-featured, extensible client for the Let's Encrypt CA (or any other CA that speaks the ACME protocol) that can automate the tasks of obtaining certificates and configuring web servers to use them. This client runs on Unix-based operating systems. It requires root access and is beta software.

Until May 2016, Certbot was named simply letsencrypt or letsencrypt-auto, depending on install method. Instructions on the Internet, and some pieces of the software, may still refer to this older name.

Certbot website at EFF.org (the Electronic Frontier Foundation).

Service

Using our Ansible role, we can install the certbot client. Then we can install as many certificates as needed; plus setup an automated job which will renew them.

Ansible has a letsencrypt module however, using it is a two-step process. We opted instead to create a more robust Ansible playbook to install and automate certificates.

Current Setup for Meza

cd ~/src/ansible-certbot
# edit host file as needed, setup DNS as needed
# check if the user you expect to use can SSH to the host
ssh centos@demo.qualitybox.us
# activate ansible
source ~/bin/ansible/hacking/env-setup
ansible --version
# edit letsencrypt.yml as needed to identify the host for the play
ansible-playbook -i hosts letsencrypt.yml
# or if you need to use a different user for SSH
ansible-playbook -i hosts letsencrypt.yml --user=root

# if installing by package (default), then certbot is installed "normally". If by source, it's in /opt/certbot
## on the target host
# see if there are any certificates
certbot certificates
# stop HAProxy to make sure the host is accessible, but don't install into Apache, just use standalone
systemctl stop haproxy
certbot certonly --standalone -d demo.qualitybox.us
# then run meza deploy to concatenate the cert files into a .pem file installed into HAproxy
# or if you have to do it manually
rm -f /etc/haproxy/certs/meza.key
rm -f /etc/haproxy/certs/meza.crt
DOMAIN='demo.qualitybox.us' sudo -E bash -c 'cat /etc/letsencrypt/live/$DOMAIN/fullchain.pem /etc/letsencrypt/live/$DOMAIN/privkey.pem > /etc/haproxy/certs/$DOMAIN.pem'
# edit HAProxy to use the certificate
vim /etc/haproxy/haproxy.cfg 
systemctl restart haproxy
systemctl status haproxy
# Now you should be able to access $DOMAIN with a green lock icon

With HAProxy

Like Lighttpd, Pound, Ejabberd and other servers, HAProxy expects a single .pem file that has the combined contents of the fullchain.pem and privkey.pem.

Fortunately, certbot offers a --pre-hook, a --post-hook and a --renew-hook that we can use.

There is even an experimental plugin if you want to go that route, but it's not necessary.

Although a cron like certbot renew --quiet --no-self-upgrade will work to renew certs, it's not going to install them. So, a better approach is to modify haproxy and also setup a renewal script. We have to modify the certbot configuration for (each) certificate. Notice how we specified the port at 54321, which we'll use in Haproxy: cat /etc/letsencrypt/renewal/demo.qualitybox.us.conf

# renew_before_expiry = 30 days
version = 0.25.1
archive_dir = /etc/letsencrypt/archive/demo.qualitybox.us
cert = /etc/letsencrypt/live/demo.qualitybox.us/cert.pem
privkey = /etc/letsencrypt/live/demo.qualitybox.us/privkey.pem
chain = /etc/letsencrypt/live/demo.qualitybox.us/chain.pem
fullchain = /etc/letsencrypt/live/demo.qualitybox.us/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = f47c655802900ba026fb42e0bef8acd7
http01_port = 54321
authenticator = standalone
installer = None
pref_challs = http-01,

Important parts of the Haproxy configuration. More detail

frontend www-https
        bind *:443 ssl crt /etc/haproxy/certs
        reqadd X-Forwarded-Proto:\ https
        acl letsencrypt-acl path_beg /.well-known/acme-challenge/
        use_backend letsencrypt-backend if letsencrypt-acl

[snip]

backend letsencrypt-backend
        server letsencrypt 127.0.0.1:54321

#!/bin/sh
# instead of manually creating a list like this
# declare -a arr=("demo.qualitybox.us" "freephile.qualitybox.us")
# loop through a dynamic list of directories in 'live'
for SITE in $(ls -D /etc/letsencrypt/live)
do

        # move to correct let's encrypt directory
        cd /etc/letsencrypt/live/$SITE
#       echo -e "working in the /etc/letsencrypt/live/$SITE directory\n"

        # cat files to make combined .pem for haproxy
        cat fullchain.pem privkey.pem > /etc/haproxy/certs/$SITE.pem
#       echo -e "created /etc/haproxy/certs/$SITE.pem\n"
done

# reload haproxy
systemctl reload haproxy
# echo -e "reloaded haproxy\n"

# use crontab -e as 'root' to setup cron to renew expiring certificates 
30 2 * * * /usr/bin/certbot renew --renew-hook "/root/bin/renew.sh" >> /var/log/certbot.log

On Amazon

If you're one of the millions of Amazon Web Services customers, you may be tempted to use their lookalike service called AWS Certificate Manager.

Don't.

AWS certificates can't even be copied between regions. AWS certificates can't be used on your own servers. AWS certificates have annoying limits. They use email in whois to validate (not going to work if you have domain privacy turned on)!!! Amazon should have just joined the rest of the world in supporting LetsEncrypt. Fortunately, LetsEncrypt can be used everywhere, including AWS.

On Cloudflare

Cloudflare offers "Keyless SSL" and "Universal SSL". Although these are excellent signs that Cloudflare is doing good work, it remains true that origin traffic may not be encrypted at all leading to a huge trust problem.

The best way for Cloudflare customers to encrypt fully, is to use Certbot. Cloudflare support echoes word-for-word what Let's Encrypt says in their community forum: How to get a Let's Encrypt certificate while using CloudFlare

tldr; Use the --webroot-path option with the certonly --preferred-challenges="dns", but you'll need to manually intervene. Optionally, if you just turn off the proxying while you issue the certificate, you can use TLS-SNI and HTTP-01 challenges (and then turn proxying back on.) For more advanced usage, check out the docs where they describe pre and post validation hooks

With the pre-hook and post-hook options, you can script the conditions needed to authenticate and renew. certbot renew --pre-hook "service nginx stop" --post-hook "service nginx start"

The --renew-hook only runs when a certificate has been successfully renewed, so use this script to do things like concatenate the fullchain and

Resources

1. https://letsencrypt.org/getinvolved/ Get Involved with Lets Encrypt
   1. https://letsencrypt.org/getting-started/ Getting Started
   2. https://github.com/letsencrypt/letsencrypt Code on GitHub
   3. https://letsencrypt.readthedocs.org/en/latest/ Docs
2. Apache docs
3. Ubuntu Server Guide - Certificates and Security
4. Ansible role for LetsEncrypt

FAQ

How do I obtain a new certificate?

1. setup the SSL virtual host in Apache, and be sure to include some SSLCertificateFile and SSLCertificateKeyFile directives using the "snake-oil" certs.
   e.g.
   SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
   SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
   
2. Then run certbot
   /opt/certbot/certbot-auto --apache -d example.org,www.example.org,wiki.example.org --dry-run
   /opt/certbot/certbot-auto --apache -d example.org,www.example.org,wiki.example.org

What certs do I have?

sudo /opt/certbot/certbot-auto certificates

How do I see more info about my certificates?

You can view the 'Security' pane in Chrome Developer Tools, but for quicker access from the command-line, use openssl

sudo openssl x509 -in /etc/haproxy/certs/demo.qualitybox.us.pem -text

Using keytool you can also quickly inspect the certificate of a host:

sudo keytool -printcert -sslserver equality-tech.com

How do I expand a cert to include some sub-domains? (wildcard certs are not supported)

sudo /opt/certbot/certbot-auto certonly --cert-name example.org --expand -d example.org,www.example.org,wiki.example.org,example.com,www.example.com,wiki.example.com

How do I cleanup my old/test certificates?

sudo /opt/certbot/certbot-auto revoke --cert-path /etc/letsencrypt/live/baz.example.org/cert.pem

sudo /opt/certbot/certbot-auto delete --cert-name baz.example.org

Electronic Frontier Foundation https://eff.org

Secure Sockets Layer = secure (encrypted) underpinning for HTTP

Domain Name System. An Named Host to IP Address lookup system

Transport Layer Security

HyperText Transfer Protocol = the foundation of data communication for the World Wide Web. See HTML