Typesense - 250 hits limit

Hello,

I’m running version 0.23.0 with Docker Compose and decided to try out the Typesense integration. After performing a full reindex using the vikunja index CLI command, the logs showed no errors, so I began testing.

In the Kanban view, the fast search can no longer find tasks by partial title names. While I can still search for task titles, I now need to use filtering options, such as filtering for “Test task” to find tasks with partial names. Additionally, I can search for comments on the Kanban board using fuzzy matching, like “hello123”, and the results are correct.

Another issue I’m encountering is that I have over 600 tasks. When I open the List, Gantt, and Table views, I receive a UI error, and the Docker Compose logs display the following:

vikunja  | 2024-07-01T09:47:34.569848488Z: WEB  ▶ 192.168.96.1  GET 304 /assets/ProjectList-qm15u6OH.css 4.132968ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T09:47:34.588363854Z: ERROR        ▶ handler/ReadAllWeb 266 status: 422 response: {"message": "Only upto 250 hits can be fetched per page."}
vikunja  | 2024-07-01T09:47:34.5884174Z: WEB    ▶ 192.168.96.1  GET 500 /api/v1/projects/4/tasks?sort_by[]=position&order_by[]=asc&filter_by[]=done&filter_value[]=false&filter_comparator[]=equals&filter_concat=and&page=1 2.364987ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36

Is there a configuration option to prevent this error in these views? I believe this is due to a Typesense limit per page that cannot be changed. I’m using Typesense typesense/typesense:26.0.

Thank you!

Does that happen directly when you open it, or only when you filter it?

Does it happen with the latest unstable build as well?

Hello,

I changed my docker compose to use image: vikunja/vikunja:unstable and got Version: v0.23.0-916-f6c041db5b

When I open it without filters it already shows the error


vikunja  | 2024-07-01T12:35:22.460725658Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/notifications?page=1 4.142426ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:22.461005522Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/avatar/adminbot.aluminallux?size=50&=1719837322443 4.533309ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:22.468517102Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/projects?is_archived=true&page=1 11.983838ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:22.475609476Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/labels?page=1 19.13622ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:22.485453048Z: ERROR        ▶ handler/ReadAllWeb a14 status: 422 response: {"message": "Only upto 250 hits can be fetched per page."}
vikunja  | 2024-07-01T12:35:22.485506586Z: WEB  ▶ 192.168.112.1  GET 500 /api/v1/projects/4/views/5/tasks?sort_by[]=position&order_by[]=asc&filter=&filter_include_nulls=false&filter_timezone=Europe%2FLisbon&s=&expand=subtasks&page=1 2.947533ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:22.488140367Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/projects/4 2.740664ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:22.6702688Z: WEB    ▶ 192.168.112.1  GET 200 /api/v1/notifications?page=1 1.288054ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:127.0) Gecko/20100101 Firefox/127.0
vikunja  | 2024-07-01T12:35:27.458400141Z: WEB  ▶ 192.168.112.1  POST 200 /api/v1/user/token 632.826µs - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:27.460940185Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/user 438.108µs - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:27.461915591Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/avatar/adminbot.aluminallux?size=50&=1719837327459 1.513892ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T12:35:27.465307618Z: WEB  ▶ 192.168.112.1  GET 200 /api/v1/avatar/adminbot.aluminallux?size=50&=1719837327461 1.803527ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36

Thank you!

Maybe I have to reindex again? Because now nothing shows on the Kaban view either

Tried to reindex using the unstable version but seems that I can’t access the container shell

$ docker exec -it vikunja sh
OCI runtime exec failed: exec failed: unable to start container process: exec: "sh": executable file not found in $PATH: unknown

If I go back to the stable version I can access the container shell but if I try to reindex I now get

/app/vikunja # /app/vikunja/vikunja index
2024-07-01T13:24:44.131056627Z: INFO    ▶ config/InitConfig 001 No config file found, using default or config from environment variables.
2024-07-01T13:24:44.166884686Z: INFO    ▶ migration/Migrate 05c Ran all migrations successfully.
2024-07-01T13:24:44.167161143Z: INFO    ▶ cmd/func4 05d Indexing… This may take a while.
2024-07-01T13:24:44.183861534Z: CRITICAL        ▶ cmd/func4 061 Could not reindex all tasks into Typesense: could not get all tasks: Error 1054 (42S22): Unknown column 'bucket_id' in 'field list'

Is there a change in the database structure beween the latest stable and the latest unstable?

This looks like a bug. I will take a look.

Yeah rolling back between versions is not really supported. You should restore a backup if you want to restore.

To answer your question about running the cli, check out this article in the docs: CLI | Vikunja

Thank you for your answer.

Just to sumarize.

Just moved to the unstable version and did a docker exec vikunja /app/vikunja/vikunja index

2024-07-01T15:02:22.161067276Z: INFO    ▶ config/InitConfig 001 No config file found, using default or config from environment variables.
2024-07-01T15:02:22.218714406Z: INFO    ▶ migration/Migrate 069 Ran all migrations successfully.
2024-07-01T15:02:22.238496018Z: INFO    ▶ cmd/func4 06a Indexing all tasks… This may take a while.
2024-07-01T15:02:25.350786948Z: INFO    ▶ cmd/func4 ebc Done!

All good with the Kanban view using the unstable version

Typesense api shows me the collection information

...
"name": "tasks",
"num_documents": 611,
"symbols_to_index": [],
"token_separators": []

The List, Gantt, and Table views have the error.

Let me know if you need more information testing or debugging something when you release a fix

Thank you!

1 Like

Did you change the service.maxitemsperpage config option by any chance? Config options | Vikunja

Yes I did to VIKUNJA_SERVICE_MAXITEMSPERPAGE: 1000.
To avoid using pagination on the API calls.

I am using the API to import data from Wekan to Vikunja and to speed coding a bit I changed the maxitemsperpage. It’s useful for a scenario where I am importing the comments from one side to the other.

Ok. So if I change it back to 50 I don’t get the error in the List and Gantt views. Still get an error in the Table view, but it’s a different error

vikunja  | 2024-07-01T16:38:07.426855462+01:00: WEB     ▶ 172.27.0.1  GET 200 /api/v1/projects/4/views/5/tasks?sort_by[]=position&order_by[]=asc&filter=&filter_include_nulls=false&filter_timezone=Europe%2FLisbon&s=&expand=subtasks&page=1 32.325955ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T16:38:08.159921312+01:00: WEB     ▶ 172.27.0.1  GET 200 /api/v1/notifications?page=1 2.034256ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:127.0) Gecko/20100101 Firefox/127.0
vikunja  | 2024-07-01T16:38:08.273761565+01:00: ERROR   ▶ handler/ReadAllWeb 292 status: 404 response: {"message": "Could not find a field named `positions.view_7` in the schema for sorting."}
vikunja  | 2024-07-01T16:38:08.273818007+01:00: WEB     ▶ 172.27.0.1  GET 500 /api/v1/projects/4/views/7/tasks?sort_by[]=index&order_by[]=desc&filter=&filter_include_nulls=false&filter_timezone=Europe%2FLisbon&s=&expand=subtasks&page=1 2.732815ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T16:38:12.374483181+01:00: WEB     ▶ 172.27.0.1  GET 200 /api/v1/projects/4/views/8/tasks?filter=&filter_include_nulls=false&s=&per_page=25&page=1 124.677459ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
vikunja  | 2024-07-01T16:38:14.260607114+01:00: WEB     ▶ 172.27.0.1  GET 200 /api/v1/notifications?page=1 1.096048ms - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36

I’ve pushed a change in 2982acc0ec to restrict the max length to 250 when using Typesense. That’s a limitation of Typesense, nothing we can do about it. With the change, the error should go away.