Halcyon Part 3: Postpone unpacking, let’s start in the clouds.

This post is written on-the-go and shows the actual experience. The repository has been updated afterwards. You can still follow along, just keen an extra eye open for unnecessary changes and errors. 🙂

We had a look at the docker-compose.yml configuration in part 2. Let’s get this thing going.

I’ll start with a fresh droplet on DigitalOcean (yes, that’s a referral link, my first one actually). Since we’re just testing for now, I’ll go with the cheapest $5 option.

1. Create the droplet
2. Add your SSH key (optional if you’re planning to use the web console)
3. Enter the hostname: halcyon-test-env-1
4. Hit the “Create” button

Logging in to the droplet

Let’s stick to the web console for now. Since we’re just testing, i’m also not doing anything about security. But remember, this is important when you’re running your server in the cloud. Also when running locally, but then you’re usually protected by the networks router/firewall as well. We’ll get into security later on.

Preparing your server for docker and docker-compose

We’ll follow the instructions on these pages to get docker and docker-compose up and running:

• https://docs.docker.com/engine/install/ubuntu/
• https://docs.docker.com/compose/install/

Cloning the repository

The repository: https://github.com/mklasen/halcyon

Run these commands:

git clone https://github.com/mklasen/halcyon.gitcd halcyon/
docker-compose up

You’ll see that compose will first download some images, and i’m pretty sure it’ll give us an error after we’re done, but let’s see!

Result:

Creating nginx-proxy ... 
Creating dns         ... 
Creating dns         ... error
WARNING: Host is already in use by another container

Creating nginx-proxy ... donece9b2e2148827de973f581d47df22b2d83abd24fb54be67b49283e2a05): Error starting userland proxy: listen tcp4 0.0.0.0:5Creating catchall    ... done

ERROR: for dns  Cannot start service dns: driver failed programming external connectivity on endpoint dns (014560ce9b2e2148827de973f581d47df22b2d83abd24fb54be67b49283e2a05): Error starting userland proxy: listen tcp4 0.0.0.0:53: bind: address already in use
ERROR: Encountered errors while bringing up the project.

Yup.. that’s definitely an error.. So, what’s running on port 53?

We’ll jump to google for something like “show ports used linux”, and that brings us to the command sudo lsof -i -P -n. That shows that port 53 is used by systemd-r. What’s that?

Freeing port 53

The previous link shows us how to free up port 53, let’s edit /etc/systemd/resolved.conf and set DNSStubListener to No;

nano /etc/systemd/resolved.conf

Hit cntrl+O to save and cntrl+X to exit the editor. Then we’ll enter reboot and restart the console.

The port should be free now, so let’s try running docker-compose up again.

We’re presented with some more errors, but these are fixable. Let’s change the docker-compose.yml file to make it suitable for cloud hosted servers. We’ll start by changing the 192.168.2.70 IP to {your_droplet_ip_address} and the DNS_Forwarder to 1.1.1.1. That brings us a bit closer.

Since we got some errors from the other containers as well, let’s specifically start the dns container by using the docker-compose up dns command.

Fixed it! Apparently the issue is with the quotes used. Removing them, like so DNS_A=*.hyc={your_droplet_ip_address} solved the issue.

Now, let’s restart the dns service and deattach with the command docker-compose up -d dns.

Then, we’ll fire some DNS requests from our own computer using commands like dig mklasen.hyc @{your_droplet_ip_address} and dig google.com @{your_droplet_ip_address}.

The first issue? We didn’t expose our DNS container. Back to the docker-compose.yml file to add expose arguments, like so:

    expose:
      - "53"

Add the lines above environment in the dns container, and then run:

docker-compose stop
docker-compose up -d dns

Now, let’s re-run our commands:

dig mklasen.hyc @{your_droplet_ip_address}
dig google.com @{your_droplet_ip_address}

And we’ve got results! Not the one we’re looking for yet, but.. we’re getting there..

The result of the command shows: “WARNING: recursion requested but not available”. When we look at the docs of our service we see an argument called ALLOW_RECURSION=any, let’s add that on our docker-compose.yml file and try again. Save the file, stop the container, restart it and fire the dig command. And.. look what we got?

➜  ~ dig mklasen.hyc @{your_droplet_ip_address}

; <<>> DiG 9.10.6 <<>> mklasen.hyc @{your_droplet_ip_address}
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14063
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;mklasen.hyc.			IN	A

;; ANSWER SECTION:
mklasen.hyc.		5	IN	A	{your_droplet_ip_address}

;; Query time: 14 msec
;; SERVER: {your_droplet_ip_address}#53({your_droplet_ip_address})
;; WHEN: Wed Mar 23 11:49:49 CET 2022
;; MSG SIZE  rcvd: 105

A working DNS service, that catches any .hyc request and returns it’s own IP address. And requests for any other domain are forwared and answered by Cloudflare’s 1.1.1.1 DNS server.

Making sure your server is still able to resolve DNS

Since we disabled the server’s own DNS service, we need to update a configuration file to make it use the dockerized DNS service.

Run sudo nano /etc/resolv.conf

Change 127.0.0.53 to 127.0.0.1

While we’re at it.. Let’s have a look at the nginx-proxy service?

Getting the nginx-proxy service up and running

docker-compose up nginx-proxy

The first one: “ERROR: you need to share your Docker host socket with a volume at /tmp/docker.sock”

is fixed by adding a volume, like so:

    volumes:
      - "/var/run/docker.sock:/tmp/docker.sock:ro"

That seems to resolve issues with the nginx-proxy service. Should we try starting all services again?

Let’s first stop everthing with docker-compose stop.

Run docker-compose up

The moment of truth

Let’s change our computer’s DNS server to {your_droplet_ip_address} and see if we can actually get the mail.hyc domain working!

But first.. let’s expose the nginx-proxy container for the public, do this by adding:

    expose:
      - "80"
      - "443"

to the nginx-proxy service.

Now, point your browser to “http://mail.hyc” (make sure that it’s http and not https) and we got a working interface!

See you in part 4!

State of the repository before this blog post: https://github.com/mklasen/halcyon/tree/9061a6e1b4561eb7c9a7c50fd09dd49614c2a678
State of the repository after this blog post: https://github.com/mklasen/halcyon/tree/d5af6dc70c50e1ab03c6afb092a6f73b5ddf6b29