I ended up using caddy-docker-proxy because the vanilla Caddyfile method was giving me some trouble with my internal docker hostnames. This container just lets you add some labels to your other containers in your docker compose file and then works out the proxy for you, so long as you’ve made the appropriate DNS records.
+1 for Avelon, my favourite Lemmy client!