Not nested `labels:` under `deploy:` for Docker Swarm deployment cause Traefik to not create a HTTP Route

Cloufish · 26 March 2023 12:54

I’m following the guide in Full docker example | Vikunja
With some changes:

HTTP Only
Using LoadBalancer
Other domain
With GlusterFS, but I don’t think that’s an issue
Deployed on Docker Swarm

If the the HTTP Route is not even created, then I think the issue is with Labels (but I really can’t figure out what is wrong). I tried many different tweaks, but I finally ended with following docker-compose.yml:

version: '3'

services:
  vikunja-api:
    image: vikunja/api:latest
    ports:
      - "80"
    environment:
      VIKUNJA_DATABASE_HOST: db
      VIKUNJA_DATABASE_PASSWORD: supersecret
      VIKUNJA_DATABASE_TYPE: mysql
      VIKUNJA_DATABASE_USER: vikunja
      VIKUNJA_DATABASE_DATABASE: vikunja
      VIKUNJA_SERVICE_JWTSECRET: testtest
      VIKUNJA_SERVICE_FRONTENDURL: http://vikunja.docker.local/
    volumes:
      - vikunja-volume:/app/vikunja/files
    networks:
      - web
      - vikunja
    depends_on:
      - db
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.vikunja-api.rule=Host(`vikunja.docker.local`) &&
(PathPrefix(`/api/v1`) || PathPrefix(`/dav/`) || PathPrefix(`/.well-known/`))"
      - "traefik.http.routers.vikunja-api.entrypoints=web"
      - "traefik.http.routers.vikunja-api.tls.certResolver=acme"
      - "traefik.http.services.vikunja-api.loadbalancer.server.port=3456"
      - "traefik.docker.network=web"
      - "io.portainer.accesscontrol.users=admin"
  vikunja-frontend:
    image: vikunja/frontend:latest
    ports:
      - "80"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.vikunja-frontend.rule=Host(`vikunja.docker.local`)"
      - "traefik.http.routers.vikunja-frontend.entrypoints=web"
      - "traefik.http.routers.vikunja-frontend.tls.certResolver=acme"
      - "traefik.http.services.vikunja-frontend.loadbalancer.server.port=80"
      - "traefik.docker.network=web"
      - "io.portainer.accesscontrol.users=admin"
    networks:
      - web
      - vikunja
    restart: unless-stopped
  db:
    image: mariadb:10
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
    environment:
      MYSQL_ROOT_PASSWORD: supersupersecret
      MYSQL_USER: vikunja
      MYSQL_PASSWORD: supersecret
      MYSQL_DATABASE: vikunja
    volumes:
      - vikunja-volume-db:/var/lib/mysql
    networks:
      - web
      - vikunja
    restart: unless-stopped
    command: --max-connections=1000


networks:
  vikunja:
    driver: overlay
    attachable: true
    name: vikunja
  web:
    external: true
    name: web

volumes:
  vikunja-volume:
    driver: glusterfs
    name: "gfs/vikunja-volume"
  vikunja-volume-db:
    driver: glusterfs
    name: "gfs/vikunja-volume-db"

I have other apps that are setup in a similar way. So It’s not “Infrastructure” Issue

No Logs in Traefik - because there’s no Route created of course
No Error Logs in the Containers - but I guess I’ll send them anyway:

vikunja-db:

2023-03-26 12:44:47+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.11.2+maria~ubu2204 started.
2023-03-26 12:45:04+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2023-03-26 12:45:05+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.11.2+maria~ubu2204 started.
2023-03-26 12:45:05+00:00 [Note] [Entrypoint]: Initializing database files
PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
To do so, start the server, then issue the following command:
'/usr/bin/mariadb-secure-installation'
which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.
See the MariaDB Knowledgebase at https://mariadb.com/kb
Please report any problems at https://mariadb.org/jira
The latest information about MariaDB is available at https://mariadb.org/.
Consider joining MariaDB's strong and vibrant community:
https://mariadb.org/get-involved/
2023-03-26 12:47:55+00:00 [Note] [Entrypoint]: Database files initialized
2023-03-26 12:47:55+00:00 [Note] [Entrypoint]: Starting temporary server
2023-03-26 12:47:55+00:00 [Note] [Entrypoint]: Waiting for server startup
2023-03-26 12:47:55 0 [Note] Starting MariaDB 10.11.2-MariaDB-1:10.11.2+maria~ubu2204 source revision cafba8761af55ae16cc69c9b53a341340a845b36 as process 99
2023-03-26 12:47:55 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-03-26 12:47:55 0 [Note] InnoDB: Number of transaction pools: 1
2023-03-26 12:47:55 0 [Note] InnoDB: Using generic crc32 instructions
2023-03-26 12:47:55 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2023-03-26 12:47:55 0 [Note] InnoDB: Using liburing
2023-03-26 12:47:55 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2023-03-26 12:47:55 0 [Note] InnoDB: Completed initialization of buffer pool
2023-03-26 12:47:55 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2023-03-26 12:47:57 0 [Note] InnoDB: 128 rollback segments are active.
2023-03-26 12:47:57 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2023-03-26 12:47:57 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2023-03-26 12:47:57 0 [Note] InnoDB: log sequence number 46656; transaction id 14
2023-03-26 12:47:57 0 [Note] Plugin 'FEEDBACK' is disabled.
2023-03-26 12:47:57 0 [Warning] 'user' entry 'root@53da6b6804d5' ignored in --skip-name-resolve mode.
2023-03-26 12:47:57 0 [Warning] 'proxies_priv' entry '@% root@53da6b6804d5' ignored in --skip-name-resolve mode.
2023-03-26 12:47:58 0 [Note] mariadbd: ready for connections.
Version: '10.11.2-MariaDB-1:10.11.2+maria~ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
2023-03-26 12:47:58+00:00 [Note] [Entrypoint]: Temporary server started.
2023-03-26 12:48:20+00:00 [Note] [Entrypoint]: Creating database vikunja
2023-03-26 12:48:20+00:00 [Note] [Entrypoint]: Creating user vikunja
2023-03-26 12:48:20+00:00 [Note] [Entrypoint]: Giving user vikunja access to schema vikunja
2023-03-26 12:48:20+00:00 [Note] [Entrypoint]: Securing system users (equivalent to running mysql_secure_installation)
2023-03-26 12:48:22+00:00 [Note] [Entrypoint]: Stopping temporary server
2023-03-26 12:48:22 0 [Note] mariadbd (initiated by: unknown): Normal shutdown
2023-03-26 12:48:22 0 [Note] InnoDB: FTS optimize thread exiting.
2023-03-26 12:48:26 0 [Note] InnoDB: Starting shutdown...
2023-03-26 12:48:26 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
2023-03-26 12:48:26 0 [Note] InnoDB: Buffer pool(s) dump completed at 230326 12:48:26
2023-03-26 12:48:26 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2023-03-26 12:48:26 0 [Note] InnoDB: Shutdown completed; log sequence number 46656; transaction id 15
2023-03-26 12:48:26 0 [Note] mariadbd: Shutdown complete
2023-03-26 12:48:26+00:00 [Note] [Entrypoint]: Temporary server stopped
2023-03-26 12:48:26+00:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up.
2023-03-26 12:48:26 0 [Note] Starting MariaDB 10.11.2-MariaDB-1:10.11.2+maria~ubu2204 source revision cafba8761af55ae16cc69c9b53a341340a845b36 as process 1
2023-03-26 12:48:27 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-03-26 12:48:27 0 [Note] InnoDB: Number of transaction pools: 1
2023-03-26 12:48:27 0 [Note] InnoDB: Using generic crc32 instructions
2023-03-26 12:48:27 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
2023-03-26 12:48:27 0 [Note] InnoDB: Using liburing
2023-03-26 12:48:27 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2023-03-26 12:48:27 0 [Note] InnoDB: Completed initialization of buffer pool
2023-03-26 12:48:27 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
2023-03-26 12:48:29 0 [Note] InnoDB: 128 rollback segments are active.
2023-03-26 12:48:29 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2023-03-26 12:48:29 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2023-03-26 12:48:29 0 [Note] InnoDB: log sequence number 46656; transaction id 14
2023-03-26 12:48:29 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2023-03-26 12:48:29 0 [Note] Plugin 'FEEDBACK' is disabled.
2023-03-26 12:48:29 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
2023-03-26 12:48:29 0 [Note] Server socket created on IP: '0.0.0.0'.
2023-03-26 12:48:29 0 [Note] Server socket created on IP: '::'.
2023-03-26 12:48:29 0 [Note] InnoDB: Buffer pool(s) load completed at 230326 12:48:29
2023-03-26 12:48:29 0 [Note] mariadbd: ready for connections.
Version: '10.11.2-MariaDB-1:10.11.2+maria~ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution

vikunja-frontend:

/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/default.conf.template to /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/50-injector.sh
info: API URL is /api/v1
info: Sentry enabled: false
info: started at 2023-03-26T12:44:54+00:00
/docker-entrypoint.sh: Launching /docker-entrypoint.d/60-ipv6-disable.sh
/docker-entrypoint.sh: Configuration complete; ready for start up

vikunja-api:

info: creating the new user vikunja with 1000:1000
usermod: no changes
2023/03/26 12:52:35 No config file found, using default or config from environment variables.
2023-03-26T12:52:49.652571572Z: INFO	▶ migration/Migrate 093 Ran all migrations successfully.
2023-03-26T12:52:49.652773755Z: INFO	▶ models/RegisterReminderCron 094 Mailer is disabled, not sending reminders per mail
2023-03-26T12:52:49.652900782Z: INFO	▶ models/RegisterOverdueReminderCron 095 Mailer is disabled, not sending overdue per mail
2023-03-26T12:52:49.653091697Z: INFO	▶ cmd/func25 096 Vikunja version v0.20.4
⇨ http server started on [::]:3456

If there’s anything I can provide please tell me!!!

kolaente · 26 March 2023 13:34

This looks like it should work. Does it show up in Traefik’s dashboard?

The port declarations seem unnecessary though, but I’m not sure if that’s a problem

Cloufish · 26 March 2023 13:49

Unfortunately not, as I mentioned, no Traefik Routing gets created

And that’s not the case with other Services that I have:

I even got inspired by you and launched my own gitea instance - and it works.
Here’re labels from gitea deployment:

      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.gitea.rule=Host(`gitea.docker.local`)"
        - "traefik.http.services.gitea.loadbalancer.server.port=3000"
        - "traefik.docker.network=web"
        - "io.portainer.accesscontrol.users=admin"

In vikunja docker-compose.yml only the necessary labels that I thought of were added

kolaente · 26 March 2023 14:31

Really weired. I use a config similar to yours on my deployments and it works without issues. Maybe enable debug logs in Traefik and check there?

Cloufish · 26 March 2023 14:58

Thank you so much for this suggestion. I managed to solve this because of you!!!

I enabled DEBUG log mode and noticed these logs related to Vikunja:

time="2023-03-26T14:41:04Z" level=debug msg="Filtering disabled container" providerName=docker container=vikunja-vikunja-api-ljh1277p3fj62clr0bcg4vvtm
time="2023-03-26T14:41:04Z" level=debug msg="Filtering disabled container" container=vikunja-vikunja-frontend-91zep3ia1b7uvxcvlpz7hiiej providerName=docker
time="2023-03-26T14:41:04Z" level=debug msg="Filtering disabled container" providerName=docker container=vikunja-db-vyjh63y6anyyru88tkxyktr4c

I then googled them and found this post → Traefik is filtering traefik docker container - #4 by cakiwi - Traefik v2 (latest) - Traefik Labs Community Forum

It seems that when deploying to Docker Swarm we need to nest labels under deploy:
I also deleted label for entrypoint and certificate, because it was in some way a duplicate of this label
- "traefik.docker.network={{traefik_network_name}}"
^ And I don’t know If its right or not, I’ll find out later xD

So my final labeling look like this:

    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.vikunja-api.rule=Host(`vikunja.docker.local`) && (PathPrefix(`/api/v1`) || PathPrefix(`/dav/`) || PathPrefix(`/.well-known/`))"
        - "traefik.http.services.vikunja-api.loadbalancer.server.port=3456"
        - "traefik.docker.network=web"
        - "io.portainer.accesscontrol.users=admin"
[...]

    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.vikunja-frontend.rule=Host(`vikunja.docker.local`)"
        - "traefik.http.services.vikunja-frontend.loadbalancer.server.port=80"
        - "traefik.docker.network=web"
        - "io.portainer.accesscontrol.users=admin"

kolaente · 26 March 2023 15:33

Awesome, glad you figured it out!