The configuration of the ACME protocol with an HTTP authentication is quite simple. For this, a custom Treafik configuration file is needed instead of the default one. First, the entry points are configured:

defaultEntryPoints = ["https","http"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
  [entryPoints.https.tls]

Two entry points are defined, one for HTTP and one for HTTPS. HTTP requests are redirected to the HTTPS entry point so that all traffic is secured.

Then, some other parts of Traefik are configured:

[retry]

[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "vps123456.ovh.net"
watch = true
exposedbydefault = false

The retry setting enables request retries in the case of network errors. The default number of retries is the number of servers in the backend, minus 1. So if a backend is composed of only one server, no retry is done before replying with an error.

Then, the Docker configuration enables communication on the Docker bus with the endpoint and watch settings. It also sets the DNS name of the server. The exposedByDefault parameter indicates that containers are ignored by Traefik unless they have the traefik.enable label set to true. This setting must be added to the audio-encoder container configuration.

Finally, here is the Let's Encrypt configuration:

[acme]
#caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
email = "[email protected]"
storage = "acme.json"
entryPoint = "https"
OnHostRule = true
  [acme.httpChallenge]
  entryPoint = "http"

Ignore the caServer comment for now. An email associated to the domain name must be provided for notifications from Let's Encrypt. The storage setting indicates in what file the credentials are stored. The entryPoint setting indicate which entry point is associated to the ACME service. The OnHostRule setting allows us to automatically generate a certificate request when a new host rule is seen on the frontend. This allows us to automatically enable TLS when new server containers are started on a sub-domain. Finally, the ACME service is configured to use the http challenge.

This whole configuration file is available in the GitHub repository (https://github.com/PacktPublishing/Hands-On-Reactive-Programming-with-Python) of this book, in the audio-encode-server-2/traefik.toml sub-folder.

The Docker Compose file of the stack also requires some changes. First, the new files needed by Treafik must be added to the bound files of the reverse proxy container:

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.toml:/traefik.toml
      - ./acme.json:/acme.json

The first additional binding replaces the default configuration file with the one enabling TLS via Let's Encrypt. The second one is the file where the Let's Encrypt credentials will be stored. This file must be present before starting the stack:

$ touch acme.json
$ chmod 600 acme.json 

Then, the audio-encoder container must be exposed to Traefik:

    labels:
      - "traefik.enable=true"

The two other containers should not be exposed as frontends to Traefik, so both Minio and Traefik itself must have the enable label set to false:

    labels:
      - "traefik.enable=false"

Before the stack can be tested, the audio-encoder Docker image must be uploaded to the server. There are several ways to do this:

• The first option is to upload the application's code and generate the image directly on the server. The problem with this solution is that the server must have all tools needed by the application installed locally, which is precisely what Docker allows us to avoid!
• The second option is to upload the image file generated locally to the server. With this method, there is nothing new to install on the server.
• The third option is to use a public or private Docker registry and upload the audio-encoder image to it so that it can be retrieved on the server.

This last solution is fine if the image can be published publicly (for example on https://hub.docker.com/), or if a private registry is available in your company.

Let's use the second option, which is quite simple thanks to SSH:

docker save audio-encoder | bzip2 | pv | ssh user@host 'bunzip2 | docker load'

With this single line, a local Docker image is uploaded to the host server, and loaded in its local registry. Replace user with the name of the SSH user that logs in if authentication is done via login/password. The pv tool prints a progress bar for the transfer. If this tool is not available on your system, you can omit it.

Now, the whole application stack can be started with a single command:

$ docker-compose up -d

curl -X POST --data-binary @../audio-dataset/banjo1.mp3 https://vps123456.ovh.net/api/transcode/v1/flac/banjo1

The audio transcode service is now available on the internet, ready to scale, and served from HTTPS!

Now that the service works, the web interface of Traefik should be disabled or secured to prevent anybody from configuring it. Also, the Minio web interface may not be exposed publicly. An easy way to secure them while still being able to use them is to bind them on the localhost interface, and use SSH forwarding to access them remotely.

For this, the Traefik web interface port must be defined this way:

- "127.0.0.1:8080:8080"

And the Minio port must be defined this way:

- "127.0.0.1:9000:9000"

Then, local port forwarding must be configured when connecting via SSH to the server:

$ ssh -L 8080:localhost:8080 -L 9000:localhost:9000 user@server

Once the SSH connection is established, web interfaces are accessible on a web browser on the localhost.

A last note on Let's Encrypt's configuration: when testing the initial configuration, the certificate request often does not work. Several trials may be needed to fix access or configuration issues. The problem with this is that certificate requests are limited to 50 per week per domain. If this limit is reached, then no other certificate can be delivered for several days.

A staging environment is available to test a configuration before requesting real certificates. This can be done simply by not commenting the caSever line in the Treafik configuration file. The staging environment has no rate limit, but it generates certificates that are not signed by root entities. As a consequence, they are not recognized as valid certificates by HTTP clients. Once the configuration works, comment this line again, and re-create an empty acme.json file so that a valid certificate is requested.

Also, during this initial configuration step, it might be useful to add debugging traces in Traefik logs. Debug logs are enabled the following way in the configuration file:

debug=true

Then, the logs are visible via Docker:

docker logs audio-encoder_reverse-proxy_1