Wrestling with Docker

For years I have been running a blog and other web apps on a VPS running Ubuntu 14.04 and Apache - a standard LAMP system. However, after experimenting with some apps - temporarily installing them and testing them, only to discard them, the system was becoming a total mess. Worst of all, various MySQL files were ballooning out in size: the ibdata1 file in /var/lib/mysql was coming in at a whopping 37Gb (39568015360 bytes to be more accurate).

Now, there are ways of dealing with this, but I don't want to have to become an expert in MySQL; all I wanted to do was to recover my system and make it more manageable.

I decided to use Docker. This is a "container system" where each app runs in its own container - a sort of mini system which contains all the files required to serve it up to the web. This clearly requires a certain amount of repetition between containers, but that's the price to be paid for independence. The idea is that you can start or stop any container without affecting any of the others. For web apps many containers are based on Alpine Linux which is a system designed to be as tiny as possible, along with the nginx web server.

There seems to be a sizable ecosystem of tools to help manage and deploy docker containers. Given my starting position of knowing nothing, I wanted to keep my extra tools to a minimum; I went with just two over and above docker itself: docker-compose, which helps design, configure, and run docker containers, and traefik, a reverse proxy, which handles all requests from the outside world to docker containers - thus managing things like ports - as well as interfacing with the certificate authority Lets Encrypt.

My hope was that I should be able to get these all set up so they would work as happily together as they were supposed to do. And so indeed it has turned out, although it took many days of fiddling, and innumerable questions to forums and web sites (such as reddit) to make it work.

So here's my traefik configuration:

 1defaultEntryPoints = ["http", "https"]
 2
 3[web]
 4address = ":8080"
 5  [web.auth.basic]
 6  users = ["admin:$apr1$v7kJtvT7$h0F7kxt.lAzFH4sZ8Z9ik."]
 7
 8[entryPoints]
 9  [entryPoints.http]
10  address = ":80"
11    [entryPoints.http.redirect]
12      entryPoint = "https"
13  [entryPoints.https]
14  address = ":443"
15    [entryPoints.https.tls]
16
17[traefikLog]
18  filePath="./traefik.log"
19  format = "json"
20
21
22# Below here comes from
23#   www.smarthomebeginner.com/traefik-reverse-proxy-tutorial-for-docker/
24# with values adjusted for local use, of course
25
26# Let's encrypt configuration
27[acme]
28email="amca01@gmail.com"
29storage="./acme.json"
30acmeLogging=true
31onHostRule = true
32entryPoint = "https"
33  # Use a HTTP-01 acme challenge rather than TLS-SNI-01 challenge
34  [acme.httpChallenge]
35  entryPoint = "http"
36
37[[acme.domains]]
38  main = "numbersandshapes.net"
39  sans = ["monitor.numbersandshapes.net", "adminer.numbersandshapes.net", "portainer.numbersandshapes.net", "kanboard.numbersandshapes.net", "webwork.numbersandshapes.net",
40 "blog.numbersandshapes.net"]
41
42# Connection to docker host system (docker.sock)
43[docker]
44endpoint = "unix:///var/run/docker.sock"
45domain = "numbersandshapes.net"
46watch = true
47# This will hide all docker containers that don't have explicitly set label to "enable"
48exposedbydefault = false

and (part of) my docker-compose configuration, the file docker-compose.yml:

 1version: "3"
 2
 3networks:
 4  proxy:
 5    external: true
 6  internal:
 7    external: false
 8
 9services:
10
11  traefik:
12    image: traefik:1.6.0-alpine
13    container_name: traefik
14    restart: always
15    command: --web --docker --logLevel=DEBUG
16    volumes:
17      - /var/run/docker.sock:/var/run/docker.sock
18      - $PWD/traefik.toml:/traefik.toml
19      - $PWD/acme.json:/acme.json
20    networks:
21      - proxy
22    ports:
23      - "80:80"
24      - "443:443"
25    labels:
26      - traefik.enable=true
27      - traefik.backend=traefik
28      - traefik.frontend.rule=Host:monitor.numbersandshapes.net
29      - traefik.port=8080
30      - traefik.docker.network=proxy
31
32  blog:
33    image: blog
34    volumes:
35      - /home/amca/docker/whats_this/public:/usr/share/nginx/html
36    networks:
37      - internal
38      - proxy
39    labels:
40      - traefik.enable=true
41      - traefik.backend=blog
42      - traefik.docker.network=proxy
43      - traefik.port=80
44      - traefik.frontend.rule=Host:blog.numbersandshapes.net

The way this works, at least in respect of this blog, is that files copied into the directory /home/amca/docker/whats_this/public on my VPS will be automatically served by nginx. So all I now need is a command on my local system (on which I do all my blog writing), which serves up these files. I've called it docker-deploy:

1hugo -b "https://blog.numbersandshapes.net/" -t "blackburn" && rsync -avz -e "ssh" --delete public/ amca@numbersandshapes.net:~/docker/whats_this/public

Remarkably enough, it all works!

One issue I had at the beginning was that my original blog was served up at the URL https://numberdsandshapes.net/blog and for some reason these links were still appearing in my new blog. It turned out (after a lot of anguished messages) that it was my mis-handling of rsync. I just ended up deleting everything except for the blog source files, and re-created everything from scratch.