Database troubles when upgrading to 19

Support
1

Have Vikunja running via Docker and made a mistake when upgrading to 19. To be clear, the first mistake is making assumptions about how to upgrade instead of RTFM and making sure. The good news is I ssh’d in a made a copy of the entire vikunja folder first. I’m hoping that can help with a solution to the following problem I’ve created.

While I do sometimes manage containers on the command line, I also have portainer running and will do some stuff there. When 0.19 was announce I logged in to portainer and started with the vikunja db container. I selected the “recreate” button and had it pull the latest image. In hindsight I don’t know why I thought I needed to do this. The Maria db image had nothing to do with the new vikunja release. Anyway, I went and did this same step for the api, proxy, and frontend container.

At this point I relealise I only need to do the frontend and api containers :frowning:

When I was done doing all this recreating Vikunja was not happy. Database errors all around. I deleted all the new containers, and am trying to figure out if I can just recreate everything using the folder that I copied. If I just run docker-compose up -d from inside there, will it magically restore everything, or did my “pull the latest image” decision cause some rereperable harm? Maybe there’s something else I’m not thinking of?

Thank you for any help!!

2

Tried to start fresh with the backup folder. Below are some snippets from the log files. I deleted all containers in portainer, ssh’d in to server and deleted vikunja folder, then restored the backup copy I made. From inside the folder I ran docker-compose up -d

URL to access vikunja is http://repo1.local:8700. On that page is the following error…

Could not find or use Vikunja installation at "repo1.local:8700". Please try a different url.

From the proxy container logs

172.26.0.1 - - [05/Aug/2022:23:58:33 +0000] "GET /api/v1/notifications?page=1 HTTP/1.1" 502 157 "http://repo1.local:8700/namespaces" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15" "-",
2022/08/05 23:58:33 [error] 29#29: *20 connect() failed (113: No route to host) while connecting to upstream, client: 172.26.0.1, server: , request: "GET /api/v1/notifications?page=1 HTTP/1.1", upstream: "http://172.26.0.4:3456/api/v1/notifications?page=1", host: "repo1.local:8700", referrer: "http://repo1.local:8700/namespaces"

From the api container

2022/08/06 00:07:01 No config file found, using default or config from environment variables.,
2022-08-06T00:07:01.925262102Z: CRITICAL	▶ migration/Migrate 002 Migration failed: dial tcp: lookup db on 127.0.0.11:53: no such host

From the db container

2022-08-06 00:07:54+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.8.3+maria~jammy started.,
2022-08-06 00:07:54+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql',
2022-08-06 00:07:54+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.8.3+maria~jammy started.,
2022-08-06 00:07:54+00:00 [Note] [Entrypoint]: MariaDB upgrade information missing, assuming required,
2022-08-06 00:07:54+00:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade) required, but skipped due to $MARIADB_AUTO_UPGRADE setting,
2022-08-06  0:07:54 0 [Note] mariadbd (server 10.8.3-MariaDB-1:10.8.3+maria~jammy) starting as process 1 ...,
2022-08-06  0:07:55 0 [Note] InnoDB: Compressed tables use zlib 1.2.11,
2022-08-06  0:07:55 0 [Note] InnoDB: Number of transaction pools: 1,
2022-08-06  0:07:55 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions,
2022-08-06  0:07:55 0 [Warning] mariadbd: io_uring_queue_init() failed with ENOSYS: check seccomp filters, and the kernel version (newer than 5.1 required),
2022-08-06  0:07:55 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF,
2022-08-06  0:07:55 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB,
2022-08-06  0:07:55 0 [Note] InnoDB: Completed initialization of buffer pool,
2022-08-06  0:07:55 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.6.4.,
2022-08-06  0:07:55 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error,
2022-08-06  0:07:55 0 [Note] InnoDB: Starting shutdown...,
2022-08-06  0:07:55 0 [ERROR] Plugin 'InnoDB' init function returned error.,
2022-08-06  0:07:55 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.,
2022-08-06  0:07:55 0 [Note] Plugin 'FEEDBACK' is disabled.,
2022-08-06  0:07:55 0 [ERROR] Unknown/unsupported storage engine: InnoDB,
2022-08-06  0:07:55 0 [ERROR] Aborting

In case it helps, my docker compose 
version: '3'

services:
  db:
    image: mariadb:10
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
    environment:
      MYSQL_ROOT_PASSWORD: supersecret
      MYSQL_USER: vikunja
      MYSQL_PASSWORD: secret
      MYSQL_DATABASE: vikunja
    volumes:
      - ./db:/var/lib/mysql
    restart: unless-stopped
  api:
    image: vikunja/api
    environment:
      VIKUNJA_DATABASE_HOST: db
      VIKUNJA_DATABASE_PASSWORD: secret
      VIKUNJA_DATABASE_TYPE: mysql
      VIKUNJA_DATABASE_USER: vikunja
      VIKUNJA_DATABASE_DATABASE: vikunja
    volumes:
      - ./files:/app/vikunja/files
    depends_on:
      - db
    restart: unless-stopped
  frontend:
    image: vikunja/frontend
    restart: unless-stopped
  proxy:
    image: nginx
    ports:
      - 8700:80
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - api
      - frontend
    restart: unless-stopped
3

This looks like Vikunja tries to connect on the wrong port to the database. Does it work if you include the database port on the host?

4

Sorry for the beginner question, but where would I do that? Can I specify it in my compose file?

5

Yes, it should be enough to change this:

      VIKUNJA_DATABASE_HOST: db

to

      VIKUNJA_DATABASE_HOST: db:3306
6

Gave that a shot, but still get the same error. This time it’s accompanied by the following error

AxiosError: Network Error

There is also the following error from the proxy. I feel like there is a networking issue here but I’m at a loss as to how to confirm that. One of the screenshots has the IP’s for each of the containers. The database container continues to also spit out abort errors as mentioned earlier.


2022/08/08 21:14:37 [error] 29#29: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.80.1, server: , request: "GET /api/v1/info HTTP/1.1", upstream: "http://192.168.80.4:3456/api/v1/info", host: "repo1.local:8800", referrer: "http://repo1.local:8800/"

image
image920×754 49.3 KB

7

image
image1536×356 34.4 KB

8

Can you verify the database container is up and running and works as it should?

9

The container appears to be up and running, but I don’t think the db itself is in a good state. Below are the latest errors, which are similar to the original post. Going back to my original steps to upgrade (where I made mistake somewhere), I started with the db container in portainer and told it to pull the latest image. Not sure if that’s what caused all this.


2022-08-08 22:11:10+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.8.3+maria~jammy started.

2022-08-08 22:11:11+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'

2022-08-08 22:11:11+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.8.3+maria~jammy started.

2022-08-08 22:11:11+00:00 [Note] [Entrypoint]: MariaDB upgrade information missing, assuming required

2022-08-08 22:11:11+00:00 [Note] [Entrypoint]: MariaDB upgrade (mariadb-upgrade) required, but skipped due to $MARIADB_AUTO_UPGRADE setting

2022-08-08 22:11:11 0 [Note] mariadbd (server 10.8.3-MariaDB-1:10.8.3+maria~jammy) starting as process 1 ...

2022-08-08 22:11:11 0 [Note] InnoDB: Compressed tables use zlib 1.2.11

2022-08-08 22:11:11 0 [Note] InnoDB: Number of transaction pools: 1

2022-08-08 22:11:11 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions

2022-08-08 22:11:11 0 [Warning] mariadbd: io_uring_queue_init() failed with ENOSYS: check seccomp filters, and the kernel version (newer than 5.1 required)

2022-08-08 22:11:11 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF

2022-08-08 22:11:11 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB

2022-08-08 22:11:11 0 [Note] InnoDB: Completed initialization of buffer pool

2022-08-08 22:11:11 0 [ERROR] InnoDB: Upgrade after a crash is not supported. The redo log was created with MariaDB 10.6.4.

2022-08-08 22:11:11 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error

2022-08-08 22:11:11 0 [Note] InnoDB: Starting shutdown...

2022-08-08 22:11:11 0 [ERROR] Plugin 'InnoDB' init function returned error.

2022-08-08 22:11:11 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

2022-08-08 22:11:11 0 [Note] Plugin 'FEEDBACK' is disabled.

2022-08-08 22:11:11 0 [ERROR] Unknown/unsupported storage engine: InnoDB

2022-08-08 22:11:11 0 [ERROR] Aborting
10

Solved! Looking at the DB logs more I finally noticed it was complaining about running Maria version 10.8.3, when everything was on 10.6.4 when I told Portainer to pull the latest image. 10.6.4 isn’t available via docker anymore so I specific image: mariadb:10.6.8 and everything loaded up perfect.

Thank you Kolaente for making me look harder at the log files!!

1 Like