fix(medcat-trainer): Support uploading model packs with longer local filenames by alhendrickson · Pull Request #583 · CogStack/cogstack-nlp

alhendrickson · 2026-07-17T11:08:15Z

Issue

I run medcat trainer with postgres
I have a model pack on disk with a very long name
I try saving the model pack, it errors with this error

[medcattrainer] psycopg.errors.StringDataRightTruncation: value too long for type character varying(100)

Fix

For now, I've just updated the column size to handle longer ones. This felt like the easiest fix with the least impact.

The right thing to do is probably that we shouldnt use the file path of the file on the users disk at all here. Decided we can do this later in order to get this fixed today.

Details

Django FileField defaults to 100 characters if max_length isnt set
Only postgres use errors here. Sqlite doesnt actually check the constraints
model_pack = models.FileField(max_length=255, help_text='Model pack zip')

Logs

Two errors possible, if theres a metacat it fails first. Else it later then fails on CDB.

[medcattrainer] Internal Server Error: /api/modelpacks/
[medcattrainer] Traceback (most recent call last):
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
[medcattrainer] return self.cursor.execute(sql, params)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
[medcattrainer] raise ex.with_traceback(None)
[medcattrainer] psycopg.errors.StringDataRightTruncation: value too long for type character varying(100)
[medcattrainer]
[medcattrainer] The above exception was the direct cause of the following exception:
[medcattrainer]
[medcattrainer] Traceback (most recent call last):
[medcattrainer] File "/home/api/api/models.py", line 169, in save
[medcattrainer] concept_db.save(skip_load=True)
[medcattrainer] File "/home/api/api/models.py", line 330, in save
[medcattrainer] super().save(*args, **kwargs)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 902, in save
[medcattrainer] self.save_base(
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1008, in save_base
[medcattrainer] updated = self._save_table(
[medcattrainer] ^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1169, in _save_table
[medcattrainer] results = self._do_insert(
[medcattrainer] ^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1210, in _do_insert
[medcattrainer] return manager._insert(
[medcattrainer] ^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
[medcattrainer] return getattr(self.get_queryset(), name)(*args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/query.py", line 1873, in _insert
[medcattrainer] return query.get_compiler(using=using).execute_sql(returning_fields)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1882, in execute_sql
[medcattrainer] cursor.execute(sql, params)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 122, in execute
[medcattrainer] return super().execute(sql, params)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
[medcattrainer] return self._execute_with_wrappers(
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
[medcattrainer] return executor(sql, params, many, context)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
[medcattrainer] with self.db.wrap_database_errors:
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in exit
[medcattrainer] raise dj_exc_value.with_traceback(traceback) from exc_value
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
[medcattrainer] return self.cursor.execute(sql, params)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
[medcattrainer] raise ex.with_traceback(None)
[medcattrainer] django.db.utils.DataError: value too long for type character varying(100)
[medcattrainer]
[medcattrainer] The above exception was the direct cause of the following exception:
[medcattrainer]
[medcattrainer] Traceback (most recent call last):
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
[medcattrainer] response = get_response(request)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
[medcattrainer] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
[medcattrainer] return view_func(request, *args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/viewsets.py", line 125, in view
[medcattrainer] return self.dispatch(request, *args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 515, in dispatch
[medcattrainer] response = self.handle_exception(exc)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 475, in handle_exception
[medcattrainer] self.raise_uncaught_exception(exc)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 486, in raise_uncaught_exception
[medcattrainer] raise exc
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 512, in dispatch
[medcattrainer] response = handler(request, *args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/mixins.py", line 19, in create
[medcattrainer] self.perform_create(serializer)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/mixins.py", line 24, in perform_create
[medcattrainer] serializer.save()
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/serializers.py", line 210, in save
[medcattrainer] self.instance = self.create(validated_data)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/serializers.py", line 991, in create
[medcattrainer] instance = ModelClass._default_manager.create(**validated_data)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
[medcattrainer] return getattr(self.get_queryset(), name)(*args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/query.py", line 665, in create
[medcattrainer] obj.save(force_insert=True, using=self.db)
[medcattrainer] File "/usr/local/lib/python3.12/contextlib.py", line 81, in inner
[medcattrainer] return func(*args, **kwds)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/api/api/models.py", line 187, in save
[medcattrainer] raise FileNotFoundError(f'Error loading the CDB from this model pack: {self.model_pack.path}') from exc
[medcattrainer] FileNotFoundError: Error loading the CDB from this model pack: /home/api/media/mct2_model_pack_long_name_000000000011111111112222222222333333333344444444445555555555_RehltKC.zip
[medcattrainer] ERROR 2026-07-17 10:39:01,744 log.py l:253:Internal Server Error: /api/modelpacks/
[medcattrainer] Traceback (most recent call last):
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
[medcattrainer] return self.cursor.execute(sql, params)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
[medcattrainer] raise ex.with_traceback(None)
[medcattrainer] psycopg.errors.StringDataRightTruncation: value too long for type character varying(100)
[medcattrainer]
[medcattrainer] The above exception was the direct cause of the following exception:
[medcattrainer]
[medcattrainer] Traceback (most recent call last):
[medcattrainer] File "/home/api/api/models.py", line 169, in save
[medcattrainer] concept_db.save(skip_load=True)
[medcattrainer] File "/home/api/api/models.py", line 330, in save
[medcattrainer] super().save(*args, **kwargs)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 902, in save
[medcattrainer] self.save_base(
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1008, in save_base
[medcattrainer] updated = self._save_table(
[medcattrainer] ^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1169, in _save_table
[medcattrainer] results = self._do_insert(
[medcattrainer] ^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1210, in _do_insert
[medcattrainer] return manager._insert(
[medcattrainer] ^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
[medcattrainer] return getattr(self.get_queryset(), name)(*args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/query.py", line 1873, in _insert
[medcattrainer] return query.get_compiler(using=using).execute_sql(returning_fields)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1882, in execute_sql
[medcattrainer] cursor.execute(sql, params)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 122, in execute
[medcattrainer] return super().execute(sql, params)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
[medcattrainer] return self._execute_with_wrappers(
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
[medcattrainer] return executor(sql, params, many, context)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
[medcattrainer] with self.db.wrap_database_errors:
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in exit
[medcattrainer] raise dj_exc_value.with_traceback(traceback) from exc_value
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
[medcattrainer] return self.cursor.execute(sql, params)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
[medcattrainer] raise ex.with_traceback(None)
[medcattrainer] django.db.utils.DataError: value too long for type character varying(100)
[medcattrainer]
[medcattrainer] The above exception was the direct cause of the following exception:
[medcattrainer]
[medcattrainer] Traceback (most recent call last):
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
[medcattrainer] response = get_response(request)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
[medcattrainer] response = wrapped_callback(request, *callback_args, **callback_kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper
[medcattrainer] return view_func(request, *args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/viewsets.py", line 125, in view
[medcattrainer] return self.dispatch(request, *args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 515, in dispatch
[medcattrainer] response = self.handle_exception(exc)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 475, in handle_exception
[medcattrainer] self.raise_uncaught_exception(exc)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 486, in raise_uncaught_exception
[medcattrainer] raise exc
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/views.py", line 512, in dispatch
[medcattrainer] response = handler(request, *args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/mixins.py", line 19, in create
[medcattrainer] self.perform_create(serializer)
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/mixins.py", line 24, in perform_create
[medcattrainer] serializer.save()
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/serializers.py", line 210, in save
[medcattrainer] self.instance = self.create(validated_data)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/rest_framework/serializers.py", line 991, in create
[medcattrainer] instance = ModelClass._default_manager.create(**validated_data)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
[medcattrainer] return getattr(self.get_queryset(), name)(*args, **kwargs)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/models/query.py", line 665, in create
[medcattrainer] obj.save(force_insert=True, using=self.db)
[medcattrainer] File "/usr/local/lib/python3.12/contextlib.py", line 81, in inner
[medcattrainer] return func(*args, **kwds)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/api/api/models.py", line 187, in save
[medcattrainer] raise FileNotFoundError(f'Error loading the CDB from this model pack: {self.model_pack.path}') from exc
[medcattrainer] FileNotFoundError: Error loading the CDB from this model pack: /home/api/media/mct2_model_pack_long_name_000000000011111111112222222222333333333344444444445555555555_RehltKC.zip

[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
[medcattrainer] raise ex.with_traceback(None)
[medcattrainer] django.db.utils.DataError: value too long for type character varying(100)
[medcattrainer]

...

[medcattrainer] raise MedCATLoadException(f'Failure loading MetaCAT models - {unpacked_model_pack_path}') from exc
[medcattrainer] api.models.MedCATLoadException: Failure loading MetaCAT models - /home/api/media/
[medcattrainer] ERROR 2026-07-15 15:44:56,875 log.py l:253:Internal Server Error: /api/modelpacks/
[medcattrainer] Traceback (most recent call last):
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
[medcattrainer] return self.cursor.execute(sql, params)
[medcattrainer] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[medcattrainer] File "/home/.venv/lib/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
[medcattrainer] raise ex.with_traceback(None)
[medcattrainer] psycopg.errors.StringDataRightTruncation: value too long for type character varying(100)

…filenames

mart-r

Looks good!

alhendrickson added 2 commits July 17, 2026 11:00

fix(medcat-trainer): Support uploading model packs with longer local …

247d251

…filenames

revert: restore dev to sqlite

71d0fa8

mart-r approved these changes Jul 17, 2026

View reviewed changes

alhendrickson merged commit 00f91f7 into main Jul 17, 2026
11 checks passed

alhendrickson deleted the fix/trainer/model-pack-length branch July 17, 2026 12:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(medcat-trainer): Support uploading model packs with longer local filenames#583

fix(medcat-trainer): Support uploading model packs with longer local filenames#583
alhendrickson merged 2 commits into
mainfrom
fix/trainer/model-pack-length

alhendrickson commented Jul 17, 2026

Uh oh!

mart-r left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

alhendrickson commented Jul 17, 2026

Issue

Fix

Details

Logs

Uh oh!

mart-r left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants