All data deleted when upgrading & can't restore

1.0 The Issue

After restoring a database dump from vikunja/api:0.20 using vikunja dump and vikunja restore my data is accessible. However, upgrading the API image to: vikunja/api:0.21 or vikunja/api:latest all my data disappears. And I get the following error:

An error occurred:
Error: Error while refreshing user info:

Please check if the api url is correct.

1.1 What I have tried

  1. Restore dump to vikunja/api:0.21.
"could not restore table data for table users: Error 1054 (42S22): Unknown column 'default_list_id' in 'field list'"
  1. Restore dump to vikunja/api:latest
 CRITICAL        ▶ db/Restore 2ca Could not find table definition for table lists
  1. Restore dump to vikunja/api:0.20
  1. After restoring dump to v0.20 (step 4), update the container to v0.21
Data is deleted.
  1. After restoring dump to v0.20 (step 4), update the container to latest
Data is deleted.

2.0 Setup

2.1 System details

Arch: x64
OS: Ubuntu-Server

2.2 Configuration


version: '3'

    container_name: vikunja_db
    image: mariadb:10
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
      MYSQL_DATABASE: vikunja
      - REDACTED/vikunja/db:/var/lib/mysql
      - vikunja_db
    restart: unless-stopped
    container_name: vikunja_api
    image: vikunja/api:0.20
      VIKUNJA_DATABASE_HOST: vikunja_db
      - REDACTED/vikunja/data:/app/vikunja/files
      - vikunja_db
    restart: unless-stopped
      - reverse_proxy
      - vikunja_db

    container_name: vikunja_frontend
    image: vikunja/frontend:0.20.5
    restart: unless-stopped
      - no-internet
      - reverse_proxy

    external: false
      name: reverse_proxy
    external: true

Restoring dumps with newer versions is not really supported. You should always restore the dump with the same version you’ve used to create the dump, then upgrade.

How did you verify your data is gone? Did you check the database?

To verify the data is deleted I ran:

MariaDB [vikunja]> select email from users;
Empty set (0.000 sec)
MariaDB [vikunja]> select * from tasks;
Empty set (0.001 sec)

I’m not sure how to proceed if updating just erases everything.

Also, testing from a blank DB and new container on v0.20 (I then created some sample tasks), if I upgrade that container to v0.21 or :latest, everything disappears within the web UI. Although, I’m still able to login and the database contains the tasks.

Is there anything in the logs during/after upgrading? Can you verify the data is present when you restore from dump on 0.20?

Vikunja won’t remove data on its own during upgrade, especially not erase everything.

:latest is 0.22 currently, not 0.21.

Yes, it visually shows when logged in and within the DB.

Logs dump:

info: creating the new user vikunja with 1000:1000
usermod: no changes
2024-01-06T21:01:24.164339292Z: INFO	▶ config/InitConfig 001 No config file found, using default or config from environment variables.
2024-01-06T21:01:24.16592887Z: CRITICAL	▶ migration/Migrate 003 Migration failed: dial tcp connect: connection refused
info: creating the new user vikunja with 1000:1000
usermod: no changes
2024-01-06T21:01:24.875265478Z: INFO	▶ config/InitConfig 001 No config file found, using default or config from environment variables.
2024-01-06T21:01:25.529797667Z: INFO	▶ migration/Migrate 0bf Ran all migrations successfully.
2024-01-06T21:01:25.530034112Z: INFO	▶ models/RegisterReminderCron 0c0 Mailer is disabled, not sending reminders per mail
2024-01-06T21:01:25.530149555Z: INFO	▶ models/RegisterOverdueReminderCron 0c1 Mailer is disabled, not sending overdue per mail
2024-01-06T21:01:25.530429663Z: INFO	▶ cmd/func27 0c3 Vikunja version v0.22.0
⇨ http server started on [::]:3456
2024-01-06T21:01:37.956022134Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 4.163721ms - REDACTED
2024-01-06T21:01:37.962102945Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 1.631768ms - REDACTED
2024-01-06T21:01:37.968221711Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 2.515491ms - REDACTED
2024-01-06T21:01:37.969698841Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 3.837035ms - REDACTED
2024-01-06T21:01:48.077535118Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 2.768922ms - REDACTED
info: creating the new user vikunja with 1000:1000
usermod: no changes
2024-01-06T21:01:58.848271041Z: INFO	▶ config/InitConfig 001 No config file found, using default or config from environment variables.
2024-01-06T21:01:58.872187002Z: INFO	▶ migration/Migrate 05a Ran all migrations successfully.
2024-01-06T21:01:58.872292157Z: INFO	▶ models/RegisterReminderCron 05b Mailer is disabled, not sending reminders per mail
2024-01-06T21:01:58.872348294Z: INFO	▶ models/RegisterOverdueReminderCron 05c Mailer is disabled, not sending overdue per mail
2024-01-06T21:01:58.872637911Z: INFO	▶ cmd/func27 05e Vikunja version v0.22.0
⇨ http server started on [::]:3456
2024-01-06T21:02:08.243234589Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 2.469436ms - REDACTED
2024-01-06T21:02:18.258205786Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 3.780879ms - REDACTED
2024-01-06T21:02:19.511888181Z: WEB 	▶ REDACTED  GET 200 /api/v1/info 866.6µs - REDACTED
2024-01-06T21:02:19.572497191Z: WEB 	▶ REDACTED  GET 401 /api/v1/lists/2/tasks?sort_by[]=position&order_by[]=asc&filter_by[]=done&filter_value[]=false&filter_comparator[]=equals&filter_concat=and&page=1 159.71µs - REDACTED
2024-01-06T21:02:19.572918587Z: WEB 	▶ REDACTED  GET 401 /api/v1/lists/2 93.083µs - REDACTED
2024-01-06T21:02:19.574279473Z: WEB 	▶ REDACTED  GET 401 /api/v1/lists/2/tasks?sort_by[]=position&order_by[]=asc&filter_by[]=done&filter_value[]=false&filter_comparator[]=equals&filter_concat=and&page=1 134.6µs - REDACTED
2024-01-06T21:02:22.80006886Z: WEB 	▶ REDACTED  GET 200 /api/v1/info 362.057µs - REDACTED
2024-01-06T21:02:26.308040251Z: WEB 	▶ REDACTED  POST 200 /api/v1/login 200.797316ms - REDACTED
2024-01-06T21:02:26.320349934Z: WEB 	▶ REDACTED  GET 200 /api/v1/user 1.575764ms - REDACTED
2024-01-06T21:02:26.366855913Z: WEB 	▶ REDACTED  GET 404 /api/v1/namespaces?is_archived=true&page=1 193.112µs - REDACTED
2024-01-06T21:02:26.376473895Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 9.865049ms - REDACTED
2024-01-06T21:02:26.390381276Z: WEB 	▶ REDACTED  GET 200 /api/v1/labels?page=1 23.669161ms - REDACTED
2024-01-06T21:02:26.420301581Z: WEB 	▶ REDACTED  GET 404 /api/v1/lists/2 130.87µs - REDACTED
2024-01-06T21:02:26.420527028Z: WEB 	▶ REDACTED  GET 404 /api/v1/lists/2/tasks?sort_by[]=position&order_by[]=asc&filter_by[]=done&filter_value[]=false&filter_comparator[]=equals&filter_concat=and&page=1 319.963µs - REDACTED
2024-01-06T21:02:26.435357091Z: WEB 	▶ REDACTED  GET 404 /api/v1/lists/2/tasks?sort_by[]=position&order_by[]=asc&filter_by[]=done&filter_value[]=false&filter_comparator[]=equals&filter_concat=and&page=1 113.871µs - REDACTED
2024-01-06T21:02:26.499532334Z: WEB 	▶ REDACTED  GET 200 /api/v1/avatar/admin?size=50&=1704574945783 133.137438ms - REDACTED
2024-01-06T21:02:31.67997309Z: WEB 	▶ REDACTED  POST 200 /api/v1/user/token 1.962685ms - REDACTED
2024-01-06T21:02:31.692776929Z: WEB 	▶ REDACTED  GET 200 /api/v1/avatar/admin?size=50&=1704574951150 1.617878ms - REDACTED
2024-01-06T21:02:31.693392553Z: WEB 	▶ REDACTED  GET 200 /api/v1/user 2.015405ms - REDACTED
2024-01-06T21:02:31.703958564Z: WEB 	▶ REDACTED  GET 200 /api/v1/avatar/admin?size=50&=1704574951167 1.379422ms - REDACTED
2024-01-06T21:02:36.365906068Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 2.07636ms - REDACTED
2024-01-06T21:02:47.148901746Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 3.706048ms - REDACTED
2024-01-06T21:02:57.243782952Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 3.780412ms - REDACTED
2024-01-06T21:03:07.357268059Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 3.914461ms - REDACTED
2024-01-06T21:03:17.547765885Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 1.533253ms - REDACTED
2024-01-06T21:03:27.702543636Z: WEB 	▶ REDACTED  GET 200 /api/v1/notifications?page=1 2.215366ms - REDACTED

The data is deleted regardless if the database was recovered using restore or a completely new on on v0.20 was created and upgraded to v0.21 or v0.22.

When I login via the web UI nothing shows up and I get the error as a toast “Error: not found”, which I assume to be related to this request which fails:

GET https://REDACTED/api/v1/namespaces?is_archived=true&page=1

Status: 404

Does it get deleted if you simply restart the stack?

You should always upgrade the frontend when you upgrade the api. The frontend tries to fetch all namespaces but namespaces were removed in 0.21.0.

It persists after docker restart ... and docker compose down && docker compose up -d.

I overlooked this. After upgrading both the api and frontend from v0.20 to v0.21 to v0.22 it worked! I have no idea why keeping the fronted as v0.20 resulted in the db erasing if the Vikunja was restored from a the CLI.

Either way, thank you for your help.

Yeah that still sounds strange. The frontend doesn’t even have the capability to erase data.

You might want to check out the backup guide for the future, now that everything works: What to backup | Vikunja