Ansible, LetsEncrypt!, NGINX, and ActionHero

If you don’t already know LetsEncrypt! is an awesome project which aims to bring free HTTPS certificate to every site on the web. HTTPS makes everything safer and more secure by protecting your information and browsing history while it is in transit from your computer to the server. Traditionally, getting an HTTPS certificate was a confusing and expensive process. It was also something of a racket, as certificate providers rarely provided a technical service per-se, just a guarantee that they were keeping their certificates safe, and that your website’s certificate, which was based off of theirs, was also safe. HTTPS trust goes something like "I trust DigiCert.com, and DigiCert.com says that site.com is safe… so I guess I trust site.com!"

Anyway, you can read more about how LetsEncrypt! works on their site.

SwitchBoard.chat is a small web site I’m running (based on ActionHero of course) which allows your team to send and receive SMS messages to a centralize place, share messages and address books, and generally makes working with SMS for your team easier. As SwithBoard.chat is a small service right now, the front-end runs entirely on one server… and is a great candidate for a basic LetsEncrypt! HTTPS certificate.

You can read the SwitchBoard.chat launch announcement here.

The fine folks at the EFF have created CertBot, an easier-to-use wrapper around the LetsEncrypt! command line tools. Ansible is a tool which helps you configure your servers (and deploy to them) automatically. We can combine both of these to make automatic HTTPS certificate generation a breeze!

An aside about LetsEncrypt!: Unlike a normal certificate authority, who grants certificates for 1,2, or 3 years at a time, LetsEncrypt! *only* grants 3-month certificates. They do this because they *want* to encourage automation and re-generation of certificates. This encourages folks to constantly be proving that they really do own the domain in question, and leads to a safer internet for all of us.

For this setup, we are going to set up our front-end server like this:

First, ensure your DNS records are pointing to the server.

We now have a chicken-and-egg problem. We want to run NGINX to serve out site (and validations for LetsEncrypt!) but we don’t have our certs yet, son NGINX won’t boot. So the first time we run this, we need to run a temporary web server, but every subsequent time, we’ll use Nginx.

To generate those certs, here’s what we do:

1# tasks/main.yml
2
3- name: install certbot dependencies
4  apt: name={{ item }} state=present
5  with_items:
6    - build-essential
7    - libssl-dev
8    - libffi-dev
9    - python-dev
10    - git
11    - python-pip
12    - python-virtualenv
13    - dialog
14    - libaugeas0
15    - ca-certificates
16- name: install Python cryptography module
17  pip: name=cryptography
18
19- name: download certbot
20  become: yes
21  become_user: "{{ deploy_user }}"
22  get_url: >
23    url=https://dl.eff.org/certbot-auto
24    dest=/home/{{ deploy_user }}/certbot-auto
25- name: chcek if we've generated a cert already
26  stat: path=/etc/letsencrypt/live/switchboard.chat/fullchain.pem
27  register: cert_stats
28
29- name: generate certs (first time)
30  become: yes
31  # become_user: '{{ deploy_user }}'
32  shell: "/home/{{ deploy_user }}/certbot-auto certonly --standalone {{ letsencrypt_domain_flags | join(' ') }} --email {{ letsencrypt_email}} --non-interactive --agree-tos"
33  when: cert_stats.stat.exists == False
34
35- name: generate certs (subsequent time)
36  become: yes
37  # become_user: '{{ deploy_user }}'
38  shell: "/home/{{ deploy_user }}/certbot-auto certonly --webroot -w /home/{{ deploy_user }}/www/switchboard.chat/current/public {{ letsencrypt_domain_flags | join(' ') }} --email {{ letsencrypt_email}} --non-interactive --agree-tos"
39  when: cert_stats.stat.exists == True
40
41- name: hup nginx
42  service: name=nginx state=reloaded

The variables look like

1# From group_vars/production
2deploy_user: "me"
3letsencrypt_email: "boss@switchboard.chat"
4letsencrypt_domain_flags:
5  - "-d switchboard.chat"
6  - "-d www.switchboard.chat"
7  - "-d api.switchboard.chat"

Here are the steps broken out:

• Install CertBot & dependancies (for Ubuntu/Debian)
• Check if we are running the first time (no cert on system yet) or a subsequent time
• Build our Install script.
• Reload Nginx

How does this work?

If we are running the first time we are telling CertBot to spin up a temporary web server using the ` — standalone` flag. If we already have a running web server (NGINX) e are telling CertBot to use ActionHero’s public directory to place it’s trust files. What CertBot does is generate some custom files (which look like /.well_known/{{gibberish}}) and then tells the LetsEncrypt! server to try and load those generated URLs. If it can, it knows that you own the DNS addresses in question, and it grants you the certificate you asked for! CertBot can also run its own web server for the domain tests, but since we are already running NGINX, we don’t need to!

In our Ansible server provisioning step, we’ll then set up and run our ActionHero project. You should configure it listen only on a local socket, IE:

1// from config/servers/web.conf
2exports.production = {
3  servers: {
4    web: function (api) {
5      return {
6        port: "/home/deploy/www/switchboard.chat/shared/sockets/actionhero.sock",
7        bindIP: null,
8        padding: null,
9        metadataOptions: {
10          serverInformation: false,
11          requesterInformation: false,
12        },
13      };
14    },
15  },
16};

… and then you can configure NGINX to load your ActionHero project as a backend:

Our NGINX.conf (and the Ansible role to manage NGINX) looks like this:

1# handlers/main.yml
2
3- name: restart nginx
4  service: name=nginx state=restarted
5
6- name: reload nginx
7  service: name=nginx state=reloaded

1# templates/production.conf.j2
2
3#user  nobody;
4worker_processes  2;
5
6error_log  /var/log/nginx/error.log warn;
7pid        /var/run/nginx.pid;
8
9
10events {
11  worker_connections 1024; # increase if you have lots of clients
12  accept_mutex on; # "on" if nginx worker_processes > 1
13}
14
15http {
16  include       mime.types;
17  default_type  application/octet-stream;
18  server_tokens off;
19  sendfile        on;
20  keepalive_timeout  65;
21  server_names_hash_bucket_size 64;
22  types_hash_max_size 2048;
23
24  gzip on;
25  gzip_http_version 1.0;
26  gzip_comp_level 9;
27  gzip_proxied any;
28  gzip_types text/plain text/xml text/css text/comma-separated-values text/javascript application/javascript application/x-javascript font/ttf font/otf image/svg+xml application/atom+xml;
29
30  log_format  main  '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for" $request_time';
31
32  server {
33      listen 80;
34      server_name _;
35
36      location /nginx_status {
37          stub_status on;
38          access_log on;
39          allow 127.0.0.1;
40          deny all;
41      }
42
43      location / {
44          rewrite ^(.*) https://www.switchboard.chat$1 permanent;
45      }
46  }
47
48  server {
49      listen       443;
50      server_name switchboard.chat;
51
52      ssl     on;
53      ssl_certificate      /etc/letsencrypt/live/switchboard.chat/fullchain.pem;
54      ssl_certificate_key  /etc/letsencrypt/live/switchboard.chat/privkey.pem;
55      ssl_prefer_server_ciphers On;
56      ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
57      ssl_session_cache shared:SSL:10m;
58
59      return 301 https://www.switchboard.chat$request_uri;
60  }
61
62  server {
63      proxy_redirect off;
64
65      listen       443 default_server;
66      server_name  _;
67
68      ssl     on;
69      ssl_certificate      /etc/letsencrypt/live/switchboard.chat/fullchain.pem;
70      ssl_certificate_key  /etc/letsencrypt/live/switchboard.chat/privkey.pem;
71      ssl_prefer_server_ciphers On;
72      ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
73      ssl_session_cache shared:SSL:10m;
74
75      access_log  /var/log/nginx/access.switchboard_chat.log  main;
76      error_log   /var/log/nginx/error.switchboard_chat.log;
77
78      client_max_body_size 10M;
79
80      location /primus {
81          proxy_http_version 1.1;
82          proxy_buffering off;
83          proxy_set_header Upgrade $http_upgrade;
84          proxy_set_header Connection "Upgrade";
85          proxy_set_header Host $host;
86
87          proxy_pass http://unix:/home/{{ deploy_user }}/www/switchboard.chat/shared/sockets/actionhero.sock;
88      }
89
90      location / {
91          root        /home/{{ deploy_user }}/www/switchboard.chat/current/public/;
92          expires 1m;
93          try_files /$uri/index.html
94                    /$uri.html
95                    /$uri
96                    @app;
97      }
98
99      location @app {
100          proxy_pass http://unix:/home/{{ deploy_user }}/www/switchboard.chat/shared/sockets/actionhero.sock;
101      }
102
103  }
104}
105

1# tasks/main.yml
2
3- name: ensure the nginx dir
4  file: path=/etc/nginx state=directory owner=root
5
6- name: ensure the nginx log dir
7  file: path=/var/log/nginx state=directory owner=nobody group=nogroup
8
9- name: ensure the default site is removed
10  file: path=/etc/nginx/sites-{{ item }}/default state=absent
11  with_items:
12    - enabled
13    - available
14  notify:
15    - restart nginx
16
17- name: nginx.conf
18  template: src=production.conf.j2 dest=/etc/nginx/nginx.conf
19  notify:
20    - reload nginx
21
22- name: install nginx
23  apt: pkg=nginx state=present
24  notify:
25    - restart nginx
26
27- meta: flush_handlers

You’ll notice that we are using `tryfiles` to attempt to have NGINX serve static files out of ActionHero’s public directory. While ActionHero _can service static assets for you, NGINX is simply better and faster at it. This also allows NGINX to continue serving assets if the ActionHero server is down for some reason, and you can use JavaScript on in your front-end code to render an appropriate error message to your visitors if a health check fails.

NGINX is also sourcing our HTTPS certificates from a strange place… This is where CertBot will place our certs. That directory is actually a collection of Symlinks which CertBot will update as needed as it renews them for us.

Now, every time you run Ansible, you will check if there are updates needed to your HTTPS certificates, and get new versions. HTTPS Automated.