Skip to content

Adding cache schema version - #1204

Open
EngCaioFonseca wants to merge 4 commits into
devfrom
cache_schema_version
Open

Adding cache schema version#1204
EngCaioFonseca wants to merge 4 commits into
devfrom
cache_schema_version

Conversation

@EngCaioFonseca

@EngCaioFonseca EngCaioFonseca commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Pull Request Change Description

This solves #1198 and adds the cache schema versioning as well as lay the groundwork for the follow up steps or making the cache persistent.

The cache schema is now versioned with Alembic. It runs on startup: a fresh cache gets stamped at the latest version, and an existing cache gets any missing migrations applied automatically. Base tables are still created with the raw CREATE blocks in db_init.py, so adding a new table works exactly like before. Alembic only handles changes to existing tables.

The first migration adds the labels column from #1189. Existing caches never got it, because CREATE TABLE IF NOT EXISTS doesn't touch a table that already exists. The migration adds the column and clears the old cached issues so they get re-collected with labels.

A few things added for the cache long-term:

  • build the repo_id indexes without locking the tables (CREATE INDEX CONCURRENTLY)
  • only flush the Redis broker when the cache actually changed instead of on every startup, and stopped it from also wiping user sessions
  • take a lock during init so two app pods starting at once don't step on each other
  • build the DB connection strings with psycopg2's make_dsn so passwords with special characters don't break them

This is also groundwork for making the cache persistent instead of rebuilt on every boot.

Generative AI disclosure

Please select one option:

  • This contribution was NOT assisted or created by Generative AI tools.
  • This contribution was assisted or created by Generative AI tools.

If AI tools were used, please provide details below:
- What tools were used? Claude - Opus 4.8; GPT - 5.6 Sol
- How were these tools used? Draft, review iteration.
- Did you review these outputs before submitting this PR? Yes.

Signed-off-by: Caio Fonseca <engcaiofonseca@protonmail.com>
@EngCaioFonseca EngCaioFonseca self-assigned this Aug 25, 2026
@EngCaioFonseca EngCaioFonseca moved this from Backlog to In Progress in Aspen Project Board Aug 25, 2026
@EngCaioFonseca EngCaioFonseca linked an issue Aug 25, 2026 that may be closed by this pull request
@MoralCode

Copy link
Copy Markdown
Contributor

#1189 made changes to the schema, so this may need to change to account for that

Comment thread 8Knot/cache_manager/db_init.py Outdated
Comment on lines +14 to +16
Our data model is fairly simple, so for now the overhead of proper
db migration tooling is mostly bloaty. We can return to this decision
in the future if necessary.
Our data model is fairly simple, so we do not use alembic. Schema
changes to existing tables are numbered SQL functions in MIGRATIONS
below, applied on startup. We can return to alembic later if needed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you elaborate on why we aren't using alembic? This seems like a very odd technical decision to me, especially if we are using SQL functions in its place

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, and I kinda agree, but in this we're not removing alembic, or touching it in this repo. I've updated the comment to explain the reasoning behind it. In summary, since it is a small fixed schema, as well as disposable/unlogged and rebuildable cache, it's easier and more efficient to not use it and it's the reason why we don't need it.

Signed-off-by: Caio Fonseca <engcaiofonseca@protonmail.com>
Signed-off-by: Caio Fonseca <engcaiofonseca@protonmail.com>
@EngCaioFonseca
EngCaioFonseca marked this pull request as ready for review August 30, 2026 23:51
Signed-off-by: Caio Fonseca <engcaiofonseca@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

Version 8Knot cache schema

2 participants