Importing tasks from Todoist

Hey guys,

Just found this amazing piece of software and it was exactly what I was looking for to move away from Todoist.

I have Vikunja setup through docker on a raspberry pi 4. Everything runs smooth.
I would like to import my tasks from Todoist to Vikunja, I know this is possible with the ‘todoist migrator’, I have setup the necessary flags in config.yml but I do not see any other change.

Could someone please guide me through the process of using the migrator?

Hi!

If don’t have any lists or tasks, there should be a button on the “overview” page.

If you already have lists or tasks you can access the migrator from the user settings (Top right arrow next to your name > Settings > Scroll down to “Migrate from other services to Vikunja”, click the button and follow the instructions). If you don’t see that button either, your migration is probably not set up correctly.

Thanks a lot for the reply and the great product!

I do not see the settings so I’m assuming the migrator isn’t set up correctly. I’m passing all of my configuration options as environmental variables in the docker-compose.yml file for ease of deployment.

I notice the environmental path for the todoist migrator is VIKUNJA_MIGRATION_TODOIST
In the code I notice there are 4 flags required by the migrator, I translated them to environmental variables and passed them through the docker compose file, namely -
VIKUNJA_MIGRATION_TODOIST_ENABLE
VIKUNJA_MIGRATION_TODOIST_CLIENTID
VIKUNJA_MIGRATION_TODOIST_CLIENTSECRET
VIKUNJA_MIGRATION_TODOIST_REDIRECTURL

and now I do see the option in the settings. However, when I click on it, I run into the following error -
{"error":"redirect_uri_not_configured"}

The url I used to setup the todoist app is the same as the frontend url, which is the same url I am passing for the redirecturl, say ‘https://tasks.example.com

What should the redirect url be in this case?

PS - I do have them setup in the config.yml file located at the root, but they don’t seem to be picked up by vikunja

Glad Vikunja is useful.

That should be something like <frontend url>/migrate/todoist. It need to match the one you configured in todoist when creating the application there.

This section of the example config might be helpful to you.

Have you mounted the config file into the container at one of the config file locations?

Thanks a lot for your help, Konrad. Vikunja is an amazing product!

I was able to get the migrator to work finally, needed to configure the redirect url in todoist’s app console exactly as you mentioned and the migration worked like a breeze.

I did not mount the config file directly, however I mounted the home directory and also specified the SERVICE.ROOTPATH environmental variable. I eventually passed all the configuration options needed as environmental variables in the docker-compose and things seem to be working fine!

Great to hear it’s working now!

I’m trying to set up the Todoist migration tool according to the instructions here but am getting the same {"error":"redirect_uri_not_configured"} error.

Here’s my config.yml:

  todoist:
    # Wheter to enable the todoist migrator or not
    enable: true
    # The client id, required for making requests to the todoist api
    # You need to register your vikunja instance at https://developer.todoist.com/appconsole.html to get this
    clientid: *************************
    # The client secret, also required for making requests to the todoist api
    clientsecret: *************************
    # The url where clients are redirected after they authorized Vikunja to access their todoist items.
    # This needs to match the url you entered when registering your Vikunja instance at todoist.
    # This is usually the frontend url where the frontend then makes a request to /migration/todoist/migrate
    # with the code obtained from the todoist api.
    # Note that the vikunja frontend expects this to be /migrate/todoist
    redirecturl: https://tasks.example.com/migrate/todoist

https://tasks.example.com is my frontend url, and it’s also the app service url I’ve set up in the Todoist app console. I also tried setting https://tasks.example.com/migrate/todoist as the app service url in the Todoist console.

Is there something I’m doing wrong here? It feels like I’ve followed the instructions correctly.

I’m shooting in the dark here, but perhaps the import could be failing because Todoist updated their API about 12 days ago:

I’m not sure what method Vikunja is using to communicate with Todoist…

UPDATE:
I got past the error. Needed to update the Oauth redirect field in the todoist dashboard. I was adding the redirect url in the “App service url” instead.

The import still didn’t work though. It just took me back to the “get started” page.

Yeah, I think todoist import is not working because Vikunja is still using the Todoist v8 sync API.

Todoist has published their new Sync API documentatation (v9) here.

I’ve just pushed an update so that the Todoist migrator now uses the v9 api: 6a97a214a3

While doing this, I also discovered a bug in the frontend which caused the auth code to not be passed down properly and fixed that as well: c09fbe9abe

Please update both the api and the frontend to unstable (once the CI released them, should take ~45min) and test if it works.

Hello @kolaente ,

thank you for building Vikunja!
Im currently trying to migrate from todoist and came across this thread.
Ive tried the migration feature with current unstable versions:

Frontend-Version: 0.20.2+150-3e4bbd58a3
API-Version: v0.20.1+66-530bb0a63c

but the migration doesnt go through. in api logs i see

2023-01-23T21:08:15.583655353Z: ERROR	▶ handler/Migrate de0 json: cannot unmarshal object into Go value of type []*todoist.project

ive got quite a few number of lists in todoist (my old wunderlist import), are there any limits to be aware of? Is there anything i can do to provide more debug information?

Best
Moritz

Can you enable debug logs and try again? There should be additional information in there.

Sure!
Logs start when initiating the migration in vikunja.
Im a bit puzzled by Got 0 done items for user 1 as i surely do have more than 0 items in todoist :wink:

Please LMK if there is anything else i can do to diagnose the issue.

Best
Moritz

2023-01-24T19:11:42.113009473Z: WEB 	▶ 10.244.0.103 fafd180c7b4da1fc8788c2c1ebcfd725 GET 200 /api/v1/migration/todoist/auth 213.828µs - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:42.131900703Z: WEB 	▶ 10.244.0.103 2eb55a0370e5198d38d19a09d91c5ec8 GET 200 /api/v1/labels?page=1 16.915159ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:42.16443636Z: WEB 	▶ 10.244.0.103 a79d02952334ba9031b2bd4a8df15255 GET 200 /api/v1/notifications?page=1 51.5636ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:42.182922426Z: WEB 	▶ 10.244.0.103 09ba642cefd9dba127d4d97b8ba839d5 GET 200 /api/v1/namespaces?is_archived=true&page=1 65.68539ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:42.192978479Z: WEB 	▶ 10.244.0.103 41871257a4b63368085a97297996902b GET 200 /api/v1/avatar/moritz?size=50&=1674587499709 93.084456ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:47.26305401Z: WEB 	▶ 10.244.0.103 85582b301adb5c321ce9cff879fadf88 POST 200 /api/v1/user/token 3.415045ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:47.378288209Z: WEB 	▶ 10.244.0.103 6d474691c81ccb2eb706d3e7af384c48 GET 200 /api/v1/user 1.561364ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:47.382632445Z: DEBUG	▶ initials/GetAvatar 0bd Serving initials avatar for user 1 and size 50 from cache
2023-01-24T19:11:47.382697498Z: WEB 	▶ 10.244.0.103 9baddfb726832542ad7682062875e60c GET 200 /api/v1/avatar/moritz?size=50&=1674587505035 6.858478ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:47.48422806Z: DEBUG	▶ initials/GetAvatar 0bf Serving initials avatar for user 1 and size 50 from cache
2023-01-24T19:11:47.4842871Z: WEB 	▶ 10.244.0.103 10b08e779c665ffef716048fb076ea2d GET 200 /api/v1/avatar/moritz?size=50&=1674587505150 2.85696ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.126506867Z: WEB 	▶ 10.244.0.103 5c5dc75eb9af7899ecbe4c6e4923797d GET 200 /api/v1/info 163.982µs - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.235025763Z: WEB 	▶ 10.244.0.103 90e1016359591cb36dacf55ec5ae880d GET 200 /api/v1/user 3.147842ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.553930511Z: DEBUG	▶ initials/GetAvatar 0c2 Serving initials avatar for user 1 and size 50 from cache
2023-01-24T19:11:48.553990622Z: WEB 	▶ 10.244.0.103 f88b228a3a01ca1ed04fc75f9cf2eb1a GET 200 /api/v1/avatar/moritz?size=50&=1674587506004 1.091745ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.562093356Z: WEB 	▶ 10.244.0.103 3b5ab06406e453acf752d2ef520c143c GET 200 /api/v1/migration/todoist/auth 2.382874ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.562821417Z: WEB 	▶ 10.244.0.103 1ec00e209af3ef760fe8affe87e9387c GET 200 /api/v1/labels?page=1 9.090374ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.57606261Z: WEB 	▶ 10.244.0.103 75aea5e159c2aa5c76f056fdc066605d GET 200 /api/v1/notifications?page=1 1.875963ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.581458241Z: WEB 	▶ 10.244.0.103 518659c05fc9dd4acd11277d2b569eb0 GET 200 /api/v1/namespaces?is_archived=true&page=1 21.285101ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:48.771617893Z: WEB 	▶ 10.244.0.103 846208e104aab882b5f1f1f7f0527f14 GET 200 /api/v1/migration/todoist/status 3.256539ms - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T19:11:49.42233258Z: DEBUG	▶ todoist/Migrate 0d4 [Todoist Migration] Starting migration for user 1
2023-01-24T19:11:49.660733425Z: DEBUG	▶ todoist/Migrate 0d5 [Todoist Migration] Got user token for user 1
2023-01-24T19:11:49.660784069Z: DEBUG	▶ todoist/Migrate 0d6 [Todoist Migration] Getting todoist data for user 1
2023-01-24T19:11:49.836454169Z: DEBUG	▶ todoist/Migrate 0d7 [Todoist Migration] Getting done items for user 1
2023-01-24T19:11:50.140561944Z: DEBUG	▶ todoist/Migrate 0d8 [Todoist Migration] Got 0 done items for user 1
2023-01-24T19:11:50.140649767Z: DEBUG	▶ todoist/Migrate 0d9 [Todoist Migration] Getting archived projects for user 1
2023-01-24T19:11:50.450852086Z: ERROR	▶ handler/Migrate 0da json: cannot unmarshal object into Go value of type []*todoist.project
2023-01-24T19:11:50.450942902Z: WEB 	▶ 10.244.0.103 9f1de69203b1c1428c71bba0f1763db2 POST 500 /api/v1/migration/todoist/migrate 1.029303711s - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36

Can you check the last unstable build? I’ve noticed and fixed a few things today.

the first 200 items are processed within seconds, after which things slow down a lot and after 90 minutes the migration failed with the following log output:

2023-01-24T22:09:31.064921643Z: DEBUG	▶ todoist/Migrate 1ed0 [Todoist Migration] User 1 has more than 200 done tasks or projects, looping to get more; iteration 1780
2023-01-24T22:09:33.147698044Z: DEBUG	▶ todoist/Migrate 1ed1 [Todoist Migration] Retrieved full task data for done task 4129002404
2023-01-24T22:09:33.147785681Z: DEBUG	▶ todoist/Migrate 1ed2 [Todoist Migration] User 1 has more than 200 done tasks or projects, looping to get more; iteration 1781
2023-01-24T22:09:33.828538384Z: DEBUG	▶ todoist/Migrate 1ed3 [Todoist Migration] Retrieved full task data for done task 4144724329
2023-01-24T22:09:33.828604601Z: DEBUG	▶ todoist/Migrate 1ed4 [Todoist Migration] User 1 has more than 200 done tasks or projects, looping to get more; iteration 1762
2023-01-24T22:09:34.335095824Z: ERROR	▶ handler/Migrate 1ed5 invalid character '<' looking for beginning of value
2023-01-24T22:09:34.33522579Z: WEB 	▶ 10.244.0.103 2f0d03161cdf31db11dea313d4df0db4 POST 500 /api/v1/migration/todoist/migrate 1h28m50.162560379s - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
2023-01-24T22:09:35.16322653Z: ERROR	▶ handler/Migrate 1ed6 invalid character '<' looking for beginning of value
2023-01-24T22:09:35.163338493Z: WEB 	▶ 10.244.0.103 012721960cfd06db013efbd6de03d637 POST 500 /api/v1/migration/todoist/migrate 1h30m52.927715224s - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36

UI doesnt show any entries, but i guess thats because none have been persisted yet / transaction has not been comitted.

not sure how many items i have in total, but the biggest list holds over 3k resolved tasks.

appreciating your help!

That looks like the api returned HTML where json was expected.

What version are you using? I’ve added another fix in fix(migration): todoist pagination now avoids too many loops · 682123a9c9 - api - Gitea which should speed up the pagination a lot.

It seems that due to the batch-size bug you fixed the process ran into some kind of timeout, causing html to be returned. Using your latest unstable the migration ran through in about 12 minutes - for ~3200 tasks.

Thanks :wink:

1 Like

Just wanted to update that import works for me now! Sorry I couldn’t test out the release earlier

@kolaente looks like another thing crept into the migration
I switched already to the unstable branch getting this same error.

2023-05-05T17:24:49.057707058+02:00: DEBUG      ▶ todoist/Migrate 1d3 [Todoist Migration] Starting migration for user 1
2023-05-05T17:24:49.359517501+02:00: DEBUG      ▶ todoist/Migrate 1d4 [Todoist Migration] Got user token for user 1
2023-05-05T17:24:49.359539313+02:00: DEBUG      ▶ todoist/Migrate 1d5 [Todoist Migration] Getting todoist data for user 1
2023-05-05T17:24:50.136112463+02:00: ERROR      ▶ handler/Migrate 1d6 json: cannot unmarshal bool into Go struct field reminder.reminders.is_deleted of type int64

Am I overlooking something?

EDIT: I had a glance at the code (api/todoist.go) where IsDeleted indeed is an Int64. Also I double checked the Sync API Reference (v8 & v9) from Todoist and they are also stating that is_deleted on the Reminders is an Integer. I have the feeling at some point the value from Todoist (either 0 or 1) is parsed into a boolean value?

That is indeed strange. Can you check the json value? If Todoist changed the value from being an int to a bool that’s an easy fix, we just need to avoid breaking this in the future again. Also it’s interesting because some Todoist api “things” have is_deleted as an int, and some as a bool.

On the other hand, we don’t actually use that property (for reminders anyway) so the best fix would be to flat-out remove the property so that the json parser ignores it?

Tried it with their Sync API v9 and indeed the reminders are returning boolean values for is_deleted :roll_eyes: contrary to their documentation.

    "reminders": [
        {
            "due": {
                "date": "2023-05-12T11:15:00",
                "is_recurring": false,
                "lang": "en",
                "string": "2023-05-12 11:15",
                "timezone": null
            },
            "id": "2449057233",
            "is_deleted": false,
            "item_id": "5703623136",
            "minute_offset": 30,
            "notify_uid": "28805462",
            "type": "relative"
        }
    ]

And I agree, if that part of data isn’t used, it certainly makes sense to strip it out altogether unless you plan to have it at some point in the future. But even then it could be re-added anyways.
I’m not knowledgable with Go, so I don’t know where to adapt that in the code :sweat_smile: