Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,10 @@
**/.project
**/.settings
**/bin

# macOS
.DS_Store
**/.DS_Store

# Local Kora sources/examples used for docs verification (never commit)
.kora-agent/
2 changes: 1 addition & 1 deletion mkdocs/docs/en/changelog/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -597,7 +597,7 @@ Added:
* Added [HTTP Client/Server logging masking](../documentation/http-server.md#configuration)
* Added HTTP Client & Server metrics enriched
* Added OpenAPI additional contract annotations for HTTP client/server
* Added annotation processor for [JDBC result set mappers](../documentation/database-jdbc.md#entity)
* Added annotation processor for [JDBC result set mappers](../documentation/database-jdbc.md#view)
* Added [Resilient Retry & Timeout](../documentation/resilient.md#retry) virtual thread support

Fixed:
Expand Down
700 changes: 552 additions & 148 deletions mkdocs/docs/en/documentation/cache.md

Large diffs are not rendered by default.

548 changes: 465 additions & 83 deletions mkdocs/docs/en/documentation/camunda7-bpmn.md

Large diffs are not rendered by default.

312 changes: 229 additions & 83 deletions mkdocs/docs/en/documentation/camunda7-rest.md

Large diffs are not rendered by default.

522 changes: 400 additions & 122 deletions mkdocs/docs/en/documentation/camunda8-worker.md

Large diffs are not rendered by default.

568 changes: 492 additions & 76 deletions mkdocs/docs/en/documentation/config.md

Large diffs are not rendered by default.

391 changes: 315 additions & 76 deletions mkdocs/docs/en/documentation/container.md

Large diffs are not rendered by default.

1,134 changes: 785 additions & 349 deletions mkdocs/docs/en/documentation/database-cassandra.md

Large diffs are not rendered by default.

392 changes: 312 additions & 80 deletions mkdocs/docs/en/documentation/database-common.md

Large diffs are not rendered by default.

846 changes: 652 additions & 194 deletions mkdocs/docs/en/documentation/database-jdbc.md

Large diffs are not rendered by default.

143 changes: 114 additions & 29 deletions mkdocs/docs/en/documentation/database-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,21 @@ agent:
use_when: "Use this file for Kora docs or implementation questions about Kora database migration modules for Flyway and Liquibase, migration configuration, startup behavior, and database integration; key triggers include FlywayJdbcDatabaseInterceptor, LiquibaseJdbcDatabaseInterceptor, FlywayConfig, LiquibaseConfig, JdbcDatabaseModule."
---

Modules to migrate the database along with the service launch.
Database migrations apply schema and reference data changes in a controlled order: they create tables, indexes, constraints, and perform other `SQL` operations required by a new application version.
In Kora, migration modules are bound to `JdbcDatabase` initialization through a `GraphInterceptor<JdbcDatabase>`: when the application starts, `JdbcDatabase` is created as a graph component, and the interceptor's `init()` runs migrations before the component is published to the rest of the graph.
If a migration fails, `init()` throws, so `JdbcDatabase` component initialization and the whole graph build (application startup) fail as well.
The interceptor's `release()` is a no-op: migrations are never rolled back or re-run when the application stops.

This approach is convenient for local development, tests, and small installations where the application runs as a single instance.
For environments with multiple replicas, choose a separate migration execution method in advance so migrations are not run simultaneously from every application instance.
Repositories do not create the database schema themselves: tables, indexes, constraints, and reference data must be created by migrations or by an external database preparation process.

## Flyway { #flyway }

Module for database migration using the [Flyway](https://documentation.red-gate.com/fd) tool.
During `JdbcDatabase` initialization, the module calls `Flyway.migrate()` with settings from the `flyway` section.
Migrations are run by `FlywayJdbcDatabaseInterceptor`, which is provided by `FlywayJdbcDatabaseModule`.
`Flyway` is wired to `SLF4J` (`loggers("slf4j")`), so migration output and the `FlyWay migration applied in ...` timing line (logged at `INFO`) appear in the application's normal logs.

### Dependency { #dependency }

Expand Down Expand Up @@ -38,13 +48,14 @@ Module for database migration using the [Flyway](https://documentation.red-gate.
interface Application : FlywayJdbcDatabaseModule
```

Requires [JDBC module](database-jdbc.md) dependency.
Requires the [`JDBC` module](database-jdbc.md) because migrations are executed through `DataSource`.
Applications usually include both modules: `JdbcDatabaseModule` creates `JdbcDatabase`, and `FlywayJdbcDatabaseModule` adds the migration interceptor.

### Configuration { #configuration }

Example of the complete configuration described in the `FlywayConfig` class (default values are specified):
Example of the complete configuration described by the `FlywayConfig` class:

===! ":material-code-json: `Hocon`"
===! ":material-code-json: `HOCON`"

```javascript
flyway {
Expand All @@ -57,13 +68,15 @@ Example of the complete configuration described in the `FlywayConfig` class (def
}
```

1. Whether database migration is enabled when the application starts. If `false`, migrations will not be executed.
2. Directory paths where migration scripts are located.
3. Whether to execute migrations within a transaction.
4. Whether to verify checksums of existing migrations before execution. An error will occur if they do not match.
5. Whether to allow mixing transactional and non-transactional SQL operations in a single migration. If enabled, the entire migration will be executed **without a transaction** to avoid errors in databases where certain operations cannot be run inside a transaction.
This setting is only relevant for databases that do not support executing certain operations within a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite.
6. Additional key-value configuration properties for `Flyway#configurationProperties`.
1. Enables migration execution during `JdbcDatabase` initialization (default: `true`). If set to `false`, the module skips the `Flyway.migrate()` call.
2. Paths to directories with migration scripts (default: `["db/migration"]`).
3. Executes migrations inside a transaction when supported by the database and the `SQL` operations themselves (default: `true`).
4. Validates checksums of already applied migrations before executing new ones (default: `true`). If checksums do not match, startup fails with an error.
5. Allows mixing transactional and non-transactional `SQL` operations in one migration (default: `false`).
If enabled, the whole migration is executed **without a transaction** to avoid errors in databases where some operations cannot run inside a transaction.
This setting is relevant for databases that do not support executing certain operations inside a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite.
6. Additional `Flyway` key-value properties (default: `{}`).
Use them to pass settings that do not have a separate Kora configuration option, such as `schemas`, `baselineOnMigrate`, `placeholderReplacement`, or `placeholders.*`.

=== ":simple-yaml: `YAML`"

Expand All @@ -77,17 +90,44 @@ Example of the complete configuration described in the `FlywayConfig` class (def
configurationProperties: {} #(6)!
```

1. Whether database migration is enabled when the application starts. If `false`, migrations will not be executed.
2. Directory paths where migration scripts are located.
3. Whether to execute migrations within a transaction.
4. Whether to verify checksums of existing migrations before execution. An error will occur if they do not match.
5. Whether to allow mixing transactional and non-transactional SQL operations in a single migration. If enabled, the entire migration will be executed **without a transaction** to avoid errors in databases where certain operations cannot be run inside a transaction.
This setting is only relevant for databases that do not support executing certain operations within a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite.
6. Additional key-value configuration properties for `Flyway#configurationProperties`.
1. Enables migration execution during `JdbcDatabase` initialization (default: `true`). If set to `false`, the module skips the `Flyway.migrate()` call.
2. Paths to directories with migration scripts (default: `["db/migration"]`).
3. Executes migrations inside a transaction when supported by the database and the `SQL` operations themselves (default: `true`).
4. Validates checksums of already applied migrations before executing new ones (default: `true`). If checksums do not match, startup fails with an error.
5. Allows mixing transactional and non-transactional `SQL` operations in one migration (default: `false`).
If enabled, the whole migration is executed **without a transaction** to avoid errors in databases where some operations cannot run inside a transaction.
This setting is relevant for databases that do not support executing certain operations inside a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite.
6. Additional `Flyway` key-value properties (default: `{}`).
Use them to pass settings that do not have a separate Kora configuration option, such as `schemas`, `baselineOnMigrate`, `placeholderReplacement`, or `placeholders.*`.

### Migration Files { #flyway-files }

By default, `Flyway` looks for migrations in `src/main/resources/db/migration`.
A regular migration file has a name like `V1__init_schema.sql`, where `V1` is the version and the part after the double underscore is the description.

```text
src/main/resources/db/migration/
V1__init_users.sql
V2__add_user_status.sql
```

Example of a simple migration:

```sql
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
name TEXT NOT NULL
);
```

When `Flyway` starts, it creates a service migration history table and applies only new versions.
If `validateOnMigrate` is enabled, already applied files must not be changed without a separate migration history repair process.

## Liquibase { #liquibase }

Module for database migration using the [Liquibase](https://www.liquibase.com/supported-databases) tool.
During `JdbcDatabase` initialization, the module obtains a connection from `DataSource`, creates a `Liquibase` instance, and calls `update()`.
Migrations are run by `LiquibaseJdbcDatabaseInterceptor`, which is provided by `LiquibaseJdbcDatabaseModule`.

### Dependency { #dependency-2 }

Expand Down Expand Up @@ -117,21 +157,22 @@ Module for database migration using the [Liquibase](https://www.liquibase.com/su
interface Application : LiquibaseJdbcDatabaseModule
```

Requires [JDBC module](database-jdbc.md) dependency.
Requires the [`JDBC` module](database-jdbc.md) because migrations are executed through `DataSource`.
Applications usually include both modules: `JdbcDatabaseModule` creates `JdbcDatabase`, and `LiquibaseJdbcDatabaseModule` adds the migration interceptor.

### Configuration { #configuration-2 }

Example of the complete configuration described in the `LiquibaseConfig` class (default values are specified):
Example of the complete configuration described by the `LiquibaseConfig` class:

===! ":material-code-json: `Hocon`"
===! ":material-code-json: `HOCON`"

```javascript
liquibase {
changelog = "db/changelog/db.changelog-master.xml" //(1)!
}
```

1. Path to [master file](https://docs.liquibase.com/concepts/changelogs/home.html) migration configuration
1. Path to the main [`changelog`](https://docs.liquibase.com/concepts/changelogs/home.html) file with migration definitions (default: `db/changelog/db.changelog-master.xml`).

=== ":simple-yaml: `YAML`"

Expand All @@ -140,16 +181,60 @@ Example of the complete configuration described in the `LiquibaseConfig` class (
changelog: "db/changelog/db.changelog-master.xml" #(1)!
```

1. Path to [master file](https://docs.liquibase.com/concepts/changelogs/home.html) migration configuration
1. Path to the main [`changelog`](https://docs.liquibase.com/concepts/changelogs/home.html) file with migration definitions (default: `db/changelog/db.changelog-master.xml`).

Unlike `Flyway`, the `Liquibase` module does not have an `enabled` setting: if the module is connected to the application graph, migrations run during `JdbcDatabase` initialization.
If a `Liquibase` migration fails, the module wraps the error in `IllegalStateException`, and application startup is interrupted.

### Migration Files { #liquibase-files }

By default, `Liquibase` looks for the main `changelog` file at `src/main/resources/db/changelog/db.changelog-master.xml`.
`Liquibase` supports different `changelog` formats, but an `SQL`-oriented project often benefits from keeping migrations as formatted `SQL`.
The main file can include such migrations with `include`.

```text
src/main/resources/db/changelog/
db.changelog-master.xml
changes/
001-init-users.sql
```

Minimal main `changelog`:

```xml
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd">

<include file="db/changelog/changes/001-init-users.sql"/>
</databaseChangeLog>
```

Example of an included migration in formatted `SQL`:

```sql
--liquibase formatted sql

--changeset app:001-init-users
CREATE TABLE users (
id BIGSERIAL PRIMARY KEY,
name TEXT NOT NULL
);
```

## Recommendations { #recommendations }

???+ warning "Recommendation"

**We do not recommend** using migration modules to run applications in an environment where with horizontal scaling
by increasing the number of working application replicas. Since this will lead to a migration call on each replica setup.
Also keep in mind that every restart of the application will also trigger migrations.
**Migration modules are not recommended** for running migrations on application startup in horizontally scaled environments
where the application runs with multiple replicas. Each replica will try to execute migrations during startup.
Also keep in mind that every application restart triggers the migration mechanism again.

In such cases, use the [Flyway Gradle Plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway) for local development,
run `Flyway` from code after database startup in tests,
use a [Kubernetes Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) for production Kubernetes environments,
or run migrations separately from `CI`.


In such cases we recommend using something like [Flyway Gradle plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway) for local development,
for tests use Flyway startup from code after database startup, for Kubernetes combat environment use [K8S Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/)
or migration from CI via [Flyway Gradle plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway).
Loading
Loading