Traefik with multiple Docker networks

Sometimes, mostly for smaller customers, we deploy staging and production environments on the same machine. Not an ideal setup, but doable. Thanks to Docker the different instances can live in isolation, except of course when one instance takes the server down, the other instance is also affected.

Deploying an application twice with the same docker-compose.yaml file does not work, because the service names need to be unique. And how would e.g. nginx know to with php-fpm instance to talk to? Thankfully, Docker has us covered.

Docker networks to the rescue! We can create separate networks in Docker which will isolate the different applications and their services from each other. Following along I will cover all the steps needed to configure Docker, Traefik, and an example application to make use of Docker networks.

First, let's define some global services in a global docker-compose.yaml file which is hosted somewhere on the server. Global services like Traefik or a MariaDB database should be deployed and upgraded independently of the application itself.

The docker-compose.yaml file for the global services looks like this:

version: "3.7"

services:
  traefik:
    image: traefik:2.9
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/opt/traefik/traefik.toml:/traefik.toml
      - /etc/opt/traefik/acme.json:/acme.json
    networks:
      - staging_default
      - production_default
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.rule=Host(`myhost.loc`)"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.routers.traefik.tls.certresolver=cert_resolver"

  database:
    image: mariadb:10.4
    restart: always
    volumes:
      - /var/lib/mysql:/var/lib/mysql
    networks:
      - staging_default
      - production_default

networks:
  staging_default:
  production_default:

In the docker-compose.yaml file we have 2 services defined: traefik and database - as well as 2 networks staging_default and production_default. The traefik and database service are part of both networks.

Since Docker Compose uses the current directory name as a prefix for service names or network names, we need to use a common prefix when launching the global services as well as when launching the applications. Otherwise, the services end up in different networks and can't "see" each other. Instead of using the current directory name, we can define the environment variable COMPOSE_PROJECT_NAME with some custom name:

export COMPOSE_PROJECT_NAME="customer"

Now, when running docker compose up Docker Compose will use customer as a prefix and e.g. generate the networks as "customer_staging_default" and "customer_production_default".

What does the docker-compose.deploy.yaml file from our application look like?

version: '3.7'

services:
  web:
    image: nexus3.loc/customer/sulu/nginx:${TAG:-latest}
    build:
      context: .
      dockerfile: docker/nginx/Dockerfile
    environment:
      - PHPFPM_HOST=phpfpm
      - PHPFPM_PORT=9000
      - NGINX_PORT=80
    links:
      - phpfpm
    volumes: &appvolumes
      - "/var/www/${TRAEFIK_ENV}/public/upload/:/var/www/public/upload"
      - "/var/www/${TRAEFIK_ENV}/var/log/:/var/www/var/log/"
      - "/var/www/${TRAEFIK_ENV}/var/uploads/:/var/www/var/uploads/"
    labels:
      - "traefik.enable=true"
      - "traefik.http.middlewares.sulu_${TRAEFIK_ENV}_https.redirectscheme.scheme=https"
      - "traefik.http.routers.sulu_${TRAEFIK_ENV}.entrypoints=web"
      - "traefik.http.routers.sulu_${TRAEFIK_ENV}.rule=Host(`${TRAEFIK_HOST}`)"
      - "traefik.http.routers.sulu_${TRAEFIK_ENV}.middlewares=sulu_${TRAEFIK_ENV}_https@docker"
      - "traefik.http.routers.sulu_${TRAEFIK_ENV}_https.rule=Host(`${TRAEFIK_HOST}`)"
      - "traefik.http.routers.sulu_${TRAEFIK_ENV}_https.tls.certresolver=cert_resolver"
      - "traefik.http.routers.sulu_${TRAEFIK_ENV}_https.tls=true"
      - "traefik.http.routers.sulu_${TRAEFIK_ENV}_https.entrypoints=websecure"

  phpfpm:
    image: nexus3.loc/customer/sulu/phpfpm:${TAG:-latest}
    build:
      context: .
      dockerfile: docker/phpfpm/Dockerfile
      target: php-prod
    environment:
      - "APP_ENV=${APP_ENV}"
      - "APP_SECRET=${APP_SECRET}"
      - "DATABASE_URL=${APP_DATABASE_URL}"
    volumes: *appvolumes

As you can see, we are using quite a few environment variables which are defined in our GitLab project as CI/CD variables. Those variables depend on the staging or production environment. That means the variable names are identical for the different environments and GitLab CI will take care to pick the "right" value.

The most important variables are TRAEFIK_ENV and TRAEFIK_HOST. TRAEFIK_ENV contains either "staging" or "production" as a value, depending on the active environment. TRAEFIK_HOST is the hostname that Traefik will use to route the requests to that container.

In our Gitlab CI pipeline configuration, we use the following logic for deploying the application:

deploy:stage:
  stage: deploy
  only:
    - tags
  environment:
    name: staging
    url: https://staging.loc
  script:
    - export TAG="${CI_COMMIT_TAG}"
    - export COMPOSE_PROJECT_NAME="customer_staging"
    - docker compose -f docker-compose.deploy.yml pull
    - docker compose -f docker-compose.deploy.yml stop || true
    - docker compose -f docker-compose.deploy.yml up -d
  allow_failure: false

Ihe COMPOSE_PROJECT_NAME environment variable is configured slightly differently here. We have to set it customer_staging to make it work as the default network name in Docker Compose is default.

The deployment configuration for production looks identical, except for the environment configuration and the COMPOSE_PROJECT_NAME. Both obviously point to the production setup.