diff --git a/.gitignore b/.gitignore index b5006b5..d57a34a 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,10 @@ **/.project **/.settings **/bin + +# macOS +.DS_Store +**/.DS_Store + +# Local Kora sources/examples used for docs verification (never commit) +.kora-agent/ diff --git a/mkdocs/docs/en/changelog/changelog.md b/mkdocs/docs/en/changelog/changelog.md index f25b0fb..98a824a 100644 --- a/mkdocs/docs/en/changelog/changelog.md +++ b/mkdocs/docs/en/changelog/changelog.md @@ -597,7 +597,7 @@ Added: * Added [HTTP Client/Server logging masking](../documentation/http-server.md#configuration) * Added HTTP Client & Server metrics enriched * Added OpenAPI additional contract annotations for HTTP client/server -* Added annotation processor for [JDBC result set mappers](../documentation/database-jdbc.md#entity) +* Added annotation processor for [JDBC result set mappers](../documentation/database-jdbc.md#view) * Added [Resilient Retry & Timeout](../documentation/resilient.md#retry) virtual thread support Fixed: diff --git a/mkdocs/docs/en/documentation/cache.md b/mkdocs/docs/en/documentation/cache.md index 6d12e4a..1e725a4 100644 --- a/mkdocs/docs/en/documentation/cache.md +++ b/mkdocs/docs/en/documentation/cache.md @@ -4,20 +4,22 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora cache module, cache annotations, Caffeine and Redis cache backends, cache key mapping, telemetry, invalidation, and async cache signatures; key triggers include @Cache, @Cacheable, @CachePut, @CacheInvalidate, CaffeineCacheModule, RedisCacheModule, CacheKeyMapper, LoadableCache." --- -Module for creating caches based on [Caffeine](https://github.com/ben-manes/caffeine) or [Redis](https://redis.io/docs/about/) -using both declarative-style annotations and using their imperative style. +The module provides typed caches for storing computation results and reusable data, +so expensive operations do not have to run on every access. A cache can be used declaratively through method annotations +or imperatively through an injected interface, with local `Caffeine` and external `Redis` available as storage backends. +Local `Caffeine` is useful for fast in-process storage, while `Redis` is suitable for a shared cache used by several application instances. For a step-by-step walkthrough before the reference details, see [Cache](../guides/cache.md) and [Multi-Level Cache](../guides/cache-multi-level.md). ## Caffeine { #caffeine } -Library-based implementation of [Caffeine](https://github.com/ben-manes/caffeine) for in-memory caches within the application. +Implementation based on the [Caffeine](https://github.com/ben-manes/caffeine) library for an in-memory application cache. ### Dependency { #dependency } ===! ":fontawesome-brands-java: `Java`" - Dependency ``build.gradle``: + [Dependency](general.md#dependencies) `build.gradle`: ```groovy implementation "ru.tinkoff.kora:cache-caffeine" ``` @@ -30,22 +32,22 @@ Library-based implementation of [Caffeine](https://github.com/ben-manes/caffeine === ":simple-kotlin: `Kotlin`" - Dependency ``build.gradle.kts``: + [Dependency](general.md#dependencies) `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:cache-caffeine") ``` Module: - ````kotlin + ```kotlin @KoraApp interface Application : CaffeineCacheModule ``` ### Configuration { #configuration } -Example of complete configuration for `mycache.config` cache, parameters are described in the `CaffeineCacheConfig` class (default or example values are specified): +Example of a complete configuration for a cache at `mycache.config`; parameters are described in the `CaffeineCacheConfig` class (example values or default values are shown): -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript mycache { @@ -53,17 +55,17 @@ Example of complete configuration for `mycache.config` cache, parameters are des expireAfterWrite = "10s" //(1)! expireAfterAccess = "10s" //(2)! initialSize = 10 //(3)! - maximumSize = 10 //(4)! + maximumSize = 100000 //(4)! } } ``` - 1. Time after which the value for the key will be deleted is reported after the value is added (optional) - 2. Time after which the value for the key will be deleted, counted after a read operation (optional) - 3. Initial cache size (helps to avoid cache expansion in case of active swelling) (optional) - 4. Maximum cache size (When the boundary is reached **or slightly earlier** will exclude the least relevant values from the cache) (default is `100000`) + 1. Time after which the value is removed from the cache; counted after the value is written (default not specified, optional) + 2. Time after which the value is removed from the cache; counted after the value is read (default not specified, optional) + 3. Initial cache size, helps avoid resizing when the number of values grows quickly (default not specified, optional) + 4. Maximum cache size; when the boundary is reached **or slightly earlier**, [least relevant values](https://blog.skillfactory.ru/glossary/lru/) are evicted (default: `100000`) -=== ":simple-yaml: ``YAML``" +=== ":simple-yaml: `YAML`" ```yaml mycache: @@ -71,13 +73,51 @@ Example of complete configuration for `mycache.config` cache, parameters are des expireAfterWrite: "10s" #(1)! expireAfterAccess: "10s" #(2)! initialSize: 10 #(3)! - maximumSize: 10 #(4)! + maximumSize: 100000 #(4)! + ``` + + 1. Time after which the value is removed from the cache; counted after the value is written (default not specified, optional) + 2. Time after which the value is removed from the cache; counted after the value is read (default not specified, optional) + 3. Initial cache size, helps avoid resizing when the number of values grows quickly (default not specified, optional) + 4. Maximum cache size; when the boundary is reached **or slightly earlier**, [least relevant values](https://blog.skillfactory.ru/glossary/lru/) are evicted (default: `100000`) + +The underlying `Caffeine` cache is built by a `CaffeineCacheFactory` supplied as a `@DefaultComponent`. +If tuning beyond the configuration options above is required (for example custom eviction, weak keys, or a custom weigher), +register your own `CaffeineCacheFactory` component to override the default and customize the `Caffeine` builder directly. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MyCaffeineCacheFactory implements CaffeineCacheFactory { + + @Nonnull + @Override + public Cache build(@Nonnull String name, @Nonnull CaffeineCacheConfig config) { + var builder = Caffeine.newBuilder().weakKeys(); + if (config.expireAfterWrite() != null) { + builder.expireAfterWrite(config.expireAfterWrite()); + } + builder.maximumSize(config.maximumSize()); + return builder.build(); + } + } ``` - 1. Time after which the value for the key will be expired is reported after the value is added (optional) - 2. Time after which the value for the key will be deleted is counted after a read operation (optional) - 3. Initial cache size (helps to avoid cache expansion in case of active swelling) (optional) - 4. Maximum cache size (When the boundary is reached **or slightly earlier** will exclude the least relevant values from the cache) (default is `100000`) +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyCaffeineCacheFactory : CaffeineCacheFactory { + + override fun build(name: String, config: CaffeineCacheConfig): Cache { + val builder = Caffeine.newBuilder().weakKeys() + config.expireAfterWrite()?.let { builder.expireAfterWrite(it) } + builder.maximumSize(config.maximumSize()) + return builder.build() + } + } + ``` ## Redis { #redis } @@ -100,96 +140,169 @@ Implementation based on in-memory database [Redis](https://redis.io/docs/about/) === ":simple-kotlin: `Kotlin`" - Dependency ``build.gradle.kts``: + [Dependency](general.md#dependencies) `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:cache-redis") ``` Module: - ````kotlin + ```kotlin @KoraApp interface Application : RedisCacheModule ``` ### Configuration { #configuration-2 } -It is required to separately configure the Lettuce driver to connect to Redis. -A single connection is used for all caches. +The `Lettuce` driver must be configured separately to connect to `Redis`. +A single connection is used for all `Redis` caches. -Example of a complete configuration for *lettuce* driver, parameters are described in the `LettuceConfig` class (default or example values are specified): +Basic Lettuce configuration parameters: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript lettuce { - uri = "redis://locahost:6379" //(1)! - user = "admin" //(2)! - password = "12345" //(3)! - database = 1 //(4)! - protocol = "RESP3" //(5)! - socketTimeout = "15s" //(6)! - commandTimeout = "15s" //(7)! - forceClusterClient = "false" //(8)! - ssl { - ciphers = [ "TLS_CHACHA20_POLY1305_SHA256" ] //(9)! - handshakeTimeout = "10s" //(10)! - } - } - ``` - - 1. URI to connect to Redis (**required**) - Connection for 1 server: `redis://locahost:6379`, - Connection for N servers: `redis://locahost:6379,locahost:6380`, - Connection with SSL: `rediss://locahost:6380` - Connection with TLS: `redis+tls://locahost:6380` - 2. Username for connection (optional) - 3. Password for connection (optional) - 4. Database number for connection (optional) - 5. Protocol for connection - 6. Connection timeout - 7. Command execution timeout - 8. Force cluster connection even if 1 URI is specified (optional) - 9. Ciphers algorithms to use for secure connections between client and server (optional) - 10. Timeout for establishing a secure connection between client and server (optional) - + uri = "redis://localhost:6379" //(1)! + commandTimeout = "60s" //(2)! + } + ``` + + 1. `URI` for connecting to `Redis` (`required`, no default) + 2. Command execution timeout (default: `60s`) + === ":simple-yaml: `YAML`" ```yaml lettuce: - uri: "redis://locahost:6379" #(1)! - user: "admin" #(2)! - password: "12345" #(3)! - database: 1 #(4)! - protocol: "RESP3" #(5)! - socketTimeout: "15s" #(6)! - commandTimeout: "15s" #(7)! - forceClusterClient: false #(8)! - ssl: - ciphers: - - "TLS_CHACHA20_POLY1305_SHA256" #(9)! - handshakeTimeout: "10s" #(10)! - ``` - - 1. URI to connect to Redis (**required**) - Connection for 1 server: `redis://locahost:6379`, - Connection for N servers: `redis://locahost:6379,locahost:6380`, - Connection with SSL: `rediss://locahost:6380` - Connection with TLS: `redis+tls://locahost:6380` - 2. Username for connection (optional) - 3. Password for connection (optional) - 4. Database number for connection (optional) - 5. Protocol for connection - 6. Connection timeout - 7. Command execution timeout - 8. Force cluster connection even if 1 URI is specified (optional) - 9. Ciphers algorithms to use for secure connections between client and server (optional) - 10. Timeout for establishing a secure connection between client and server (optional) - -Redis cache configurations configure the behavior of a particular cache. - -Example of a complete configuration for `mycache.config` cache, parameters are described in the `RedisCacheConfig` class (example values are specified): - -===! ":material-code-json: `Hocon`" + uri: "redis://localhost:6379" #(1)! + commandTimeout: "60s" #(2)! + ``` + + 1. `URI` for connecting to `Redis` (`required`, no default) + 2. Command execution timeout (default: `60s`) + +??? note "Full Configuration" + + Example of a complete configuration for the `Lettuce` driver; parameters are described in the `LettuceClientConfig` class (example values or default values are shown): + + ===! ":material-code-json: `HOCON`" + + ```javascript + lettuce { + uri = "redis://localhost:6379" //(1)! + user = "admin" //(2)! + password = "12345" //(3)! + database = 0 //(4)! + protocol = "RESP3" //(5)! + socketTimeout = "10s" //(6)! + commandTimeout = "60s" //(7)! + forceClusterClient = false //(8)! + ssl { + ciphers = [ "TLS_CHACHA20_POLY1305_SHA256" ] //(9)! + handshakeTimeout = "10s" //(10)! + } + telemetry { + logging { + enabled = false //(11)! + } + metrics { + enabled = true //(12)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(13)! + tags = { // (14)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(15)! + attributes = { // (16)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. `URI` for connecting to `Redis` (`required`, default not specified). + Single-server connection: `redis://localhost:6379`. + Multi-server connection: `redis://localhost:6379,localhost:6380`. + Connection with `SSL`: `rediss://localhost:6380`. + Connection with `TLS`: `redis+tls://localhost:6380`. + 2. Username for the connection (default not specified, optional) + 3. User password for the connection (default not specified, optional) + 4. Database number for the connection (default not specified, optional) + 5. Connection protocol, can be `RESP2` or `RESP3` (default: `RESP3`) + 6. Socket connection timeout (default: `10s`) + 7. Command execution timeout (default: `60s`) + 8. Create a cluster client even with a single connection `URI` (default: `false`) + 9. Cipher algorithms for a secure connection between client and server (default: `[]`) + 10. Timeout for establishing a secure connection with the server (default: `10s`) + 11. Enables module logging (default: `false`) + 12. Enables module metrics (default: `true`) + 13. [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) configuration for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Tags configuration for metrics (default: `{}`) + 15. Enables module tracing (default: `true`) + 16. Attributes configuration for tracing (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + lettuce: + uri: "redis://localhost:6379" #(1)! + user: "admin" #(2)! + password: "12345" #(3)! + database: 0 #(4)! + protocol: "RESP3" #(5)! + socketTimeout: "10s" #(6)! + commandTimeout: "60s" #(7)! + forceClusterClient: false #(8)! + ssl: + ciphers: + - "TLS_CHACHA20_POLY1305_SHA256" #(9)! + handshakeTimeout: "10s" #(10)! + telemetry: + logging: + enabled: false #(11)! + metrics: + enabled: true #(12)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! + tags: #(14)! + key1: value1 + key2: value2 + tracing: + enabled: true #(15)! + attributes: #(16)! + key1: value1 + key2: value2 + ``` + + 1. `URI` for connecting to `Redis` (`required`, default not specified). + Single-server connection: `redis://localhost:6379`. + Multi-server connection: `redis://localhost:6379,localhost:6380`. + Connection with `SSL`: `rediss://localhost:6380`. + Connection with `TLS`: `redis+tls://localhost:6380`. + 2. Username for the connection (default not specified, optional) + 3. User password for the connection (default not specified, optional) + 4. Database number for the connection (default not specified, optional) + 5. Connection protocol, can be `RESP2` or `RESP3` (default: `RESP3`) + 6. Socket connection timeout (default: `10s`) + 7. Command execution timeout (default: `60s`) + 8. Create a cluster client even with a single connection `URI` (default: `false`) + 9. Cipher algorithms for a secure connection between client and server (default: `[]`) + 10. Timeout for establishing a secure connection with the server (default: `10s`) + 11. Enables module logging (default: `false`) + 12. Enables module metrics (default: `true`) + 13. [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) configuration for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Tags configuration for metrics (default: `{}`) + 15. Enables module tracing (default: `true`) + 16. Attributes configuration for tracing (default: `{}`) + +The `Redis` cache configuration defines behavior for a specific cache. + +Example of a complete configuration for a cache at `mycache.config`; parameters are described in the `RedisCacheConfig` class (example values are shown): + +===! ":material-code-json: `HOCON`" ```javascript mycache { @@ -201,29 +314,109 @@ Example of a complete configuration for `mycache.config` cache, parameters are d } ``` - 1. When writing, sets the [expiration](https://redis.io/commands/psetex/) time (optional) - 2. When reading, sets the time [expiration](https://redis.io/commands/getex/) (optional) - 3. Prefix a key in a particular cache to avoid key collisions within a Redis database, can be an empty string then keys will be without prefixes (**required**) + 1. Sets the value [expiration](https://redis.io/commands/psetex/) time on write (default not specified, optional) + 2. Sets the value [expiration](https://redis.io/commands/getex/) time on read (default not specified, optional) + 3. Key prefix for the specific cache, used to avoid key collisions in one `Redis` database; can be an empty string, then keys will have no prefix (`required`, default not specified) -=== ":simple-yaml: ``YAML`" +=== ":simple-yaml: `YAML`" ```yaml mycache: config: expireAfterWrite: "10s" #(1)! expireAfterAccess: "10s" #(2)! - keyPrefix: "mykey" //(3)! + keyPrefix: "mykey" #(3)! ``` - 1. Sets the [expiration](https://redis.io/commands/psetex/) time when writing (optional) - 2. When reading, sets the time [expiration](https://redis.io/commands/getex/) (optional) - 3. Prefix a key in a specific cache to avoid key collisions within a Redis database, can be an empty string then keys will be without prefixes (**required**) + 1. Sets the value [expiration](https://redis.io/commands/psetex/) time on write (default not specified, optional) + 2. Sets the value [expiration](https://redis.io/commands/getex/) time on read (default not specified, optional) + 3. Key prefix for the specific cache, used to avoid key collisions in one `Redis` database; can be an empty string, then keys will have no prefix (`required`, default not specified) Module metrics are described in the [Metrics Reference](metrics.md#cache) section. +Custom cache telemetry for both backends can be plugged by registering the nullable `CacheMetrics` and `CacheTracer` components, +which receive a `CacheTelemetryOperation` describing the operation name, cache name, and origin. + +### Key and Value Mappers { #redis-mappers } + +`Redis` stores keys and values as byte arrays, so `RedisCache` uses two kinds of mappers: + +- `RedisCacheKeyMapper` turns a cache key into `byte[]`. +- `RedisCacheValueMapper` writes a cache value to `byte[]` and reads it back. + +Regular keys are built through `RedisCacheKeyMapper` for the key type. Built-in mappers are available for `String`, `byte[]`, +numbers, `BigInteger`, `BigDecimal`, `UUID`, `Boolean`, `Character`, `Instant`, `LocalDateTime`, `LocalDate`, `ZonedDateTime`, +`Duration`, `Period`, `Enum`, and `Collection` when a mapper for `T` is also available. +For `Enum`, `toString()` is used, so it can be overridden when another key format is needed. + +For values, built-in `RedisCacheValueMapper` implementations are available for the same simple types, date/time types, `Enum`, and `byte[]`. +For other types, a mapper based on `JsonWriter` and `JsonReader` is used when JSON serialization is available for the type. +If another representation is needed, register your own `RedisCacheValueMapper` or `RedisCacheKeyMapper` component. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class UserIdRedisKeyMapper implements RedisCacheKeyMapper { + + @Nonnull + @Override + public byte[] apply(@Nullable UserId key) { + return key == null + ? "NUL".getBytes(StandardCharsets.UTF_8) + : key.value().getBytes(StandardCharsets.UTF_8); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class UserIdRedisKeyMapper : RedisCacheKeyMapper { + + override fun apply(key: UserId?): ByteArray { + return key?.value?.toByteArray(Charsets.UTF_8) + ?: "NUL".toByteArray(Charsets.UTF_8) + } + } + ``` + +The common case is storing an object value as `JSON`. Annotate the value type with `@Json`: `Kora` generates a `JsonWriter` and `JsonReader` for it, +and `RedisCacheModule` provides a matching `RedisCacheValueMapper` (`jsonRedisValueMapper`) automatically, so no manual mapper is needed for `JSON`-serializable types. +To use a different representation for such a type, register your own `RedisCacheValueMapper` component, which overrides the default `JSON` mapper. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record UserData(String id, String name) { } + + @Cache("mycache.config") + public interface MyCache extends RedisCache { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class UserData(val id: String, val name: String) + + @Cache("mycache.config") + interface MyCache : RedisCache + ``` + +For a composite key based on a `record` or `data class`, Kora generates a separate `RedisCacheKeyMapper` for the whole key. +It receives a mapper for each field, converts every field to `byte[]`, and joins the parts with `RedisCacheKeyMapper.DELIMITER`. +The part order matches the order of `record` components or `data class` properties. + +For a single key, built-in `RedisCacheKeyMapper` implementations can encode `null` as a special byte value. +In a composite key, each field mapping result must be non-`null`: if a custom `RedisCacheKeyMapper` for a field returns `null`, +key creation fails. For optional fields in a composite key, a custom mapper must explicitly encode `null` +as a stable byte value. #### Configurator { #configurator } -Можно зарегистрировать `LettuceConfigurator` который позволит до настроить `Lettuce` клиент перед созданием. +You can register `LettuceConfigurator` to customize the `Lettuce` client before it is created. ===! ":fontawesome-brands-java: `Java`" @@ -231,8 +424,8 @@ Module metrics are described in the [Metrics Reference](metrics.md#cache) sectio @Component public final class MyLettuceConfigurator implements LettuceConfigurator { @Override - public DefaultClientResources.Builder configure(DefaultClientResources.Builder resouceBuilder) { - return resouceBuilder; + public DefaultClientResources.Builder configure(DefaultClientResources.Builder resourceBuilder) { + return resourceBuilder; } @Override @@ -252,8 +445,8 @@ Module metrics are described in the [Metrics Reference](metrics.md#cache) sectio ```kotlin class MyLettuceConfigurator : LettuceConfigurator { - override fun configure(resouceBuilder: DefaultClientResources.Builder): DefaultClientResources.Builder { - return resouceBuilder + override fun configure(resourceBuilder: DefaultClientResources.Builder): DefaultClientResources.Builder { + return resourceBuilder } override fun configure(clusterBuilder: ClusterClientOptions.Builder): ClusterClientOptions.Builder { @@ -266,13 +459,18 @@ Module metrics are described in the [Metrics Reference](metrics.md#cache) sectio } ``` +For advanced scenarios beyond the typed cache, `RedisCacheClient` is available for injection as a low-level client that operates on raw `byte[]` +(`scan`/`get`/`mget`/`getex`/`set`/`mset`/`psetex`/`del`/`flushAll`) over the shared `Lettuce` connection; it is the client that `RedisCache` is built on top of. + ## Usage { #usage } Creating a cache will require registering a typed `@Cache` contract. -The contract interface should only be inherited from Kora's provided implementations: `CaffeineCache` / `RedisCache`. -For such `@Cache` an implementation will be created and added to the graph, it can be used to enforce dependencies. +The contract interface must extend one of the `Kora` implementations: `CaffeineCache` or `RedisCache`. +For such an `@Cache`, an implementation is generated and added to the graph, so it can be injected as a dependency. -To register `@Cache` and specify the config, it is required to annotate with the `@Cache` annotation where the `value` argument means the full path to the config. +The `value` argument in `@Cache` defines the full path to the configuration of the specific cache. +It points at the configuration object of that cache, so the config keys can live under a nested path such as `mycache.config { ... }`, +or flat directly under the path such as `my-cache { ... }` as used in the example projects. Both forms are valid; pick one and keep the config keys under it. ===! ":fontawesome-brands-java: `Java`" @@ -283,30 +481,169 @@ To register `@Cache` and specify the config, it is required to annotate with the === ":simple-kotlin: `Kotlin`" - ````kotlin + ```kotlin @Cache("mycache.config") interface MyCache : CaffeineCache ``` +### Optional Values { #optional-values } + +If a `Java` method returns `Optional`, the caching aspect can work with that signature directly. +The same rule applies to asynchronous wrappers, for example `CompletionStage>` and `Mono>`. +The cache value type itself can be either `T` or `Optional`: + +- `CaffeineCache` and method `Optional get(String key)`; +- `CaffeineCache>` and method `String get(String key)`; +- `CaffeineCache>` and method `Optional get(String key)`. + +For `@Cacheable`, this makes it possible to distinguish a missing cache entry from a method result that also means no data. +For `@CachePut`, the `Optional` result is handled according to the cache value type: if the cache stores `Optional`, the `Optional` itself is stored, +and if the cache stores `T`, only a present value is stored. + ### Imperative { #imperative } Caches are available for injection as dependencies on the interface and can be used in conjunction with declarative operations. -The `CaffeineCache` implementation provides basic `Cache` interface contracts for synchronous operations, -and `RedisCache` provides both `Cache` and `AsyncCache` for asynchronous operations with `CompletionStage` signatures. +`CaffeineCache` provides the `Cache` contract for synchronous operations and the additional `getAll()` method. +`RedisCache` provides `Cache` and `AsyncCache`: it can be used synchronously and asynchronously through `CompletionStage`. + +`Cache` provides `get(...)`, `put(...)`, `computeIfAbsent(...)`, `invalidate(...)`, `invalidateAll(...)`, +as well as batch variants for a collection of keys or a map of values. `AsyncCache` provides the same operations with the `Async` suffix. +`computeIfAbsent(...)` methods first try to get a value from the cache; on a miss, they call the provided loader function and store the result. + +#### Composite Cache With `Cache.Builder` { #builder-composite-cache } -The interfaces provide get, delete, update, batch, etc. operations. -Cache implementations can also provide self-specific contracts. +If a composite cache is needed in imperative code, it can be built as a facade through `Cache.Builder`. +Layer order is defined by the add order: usually a fast local cache, such as `Caffeine`, is added first, +and a more shared cache, such as `Redis`, is added after it. + +- `get(key)` checks caches in order and returns the first found value. +- `put(...)`, `invalidate(...)`, and `invalidateAll()` are executed in all caches. +- `computeIfAbsent(...)` checks caches in order; if a value is found in a lower layer, it is written into previous layers. +- If the value is missing in every layer, the loader function is called and the result is written into all caches. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Cache("mycache.caffeine.config") + public interface MyCaffeineCache extends CaffeineCache { } + + @Cache("mycache.redis.config") + public interface MyRedisCache extends RedisCache { } + + @KoraApp + public interface Application extends CaffeineCacheModule, RedisCacheModule { + + default Cache compositeCache(MyCaffeineCache caffeineCache, MyRedisCache redisCache) { + return Cache.builder(caffeineCache) + .addCache(redisCache) + .build(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Cache("mycache.caffeine.config") + interface MyCaffeineCache : CaffeineCache + + @Cache("mycache.redis.config") + interface MyRedisCache : RedisCache + + @KoraApp + interface Application : CaffeineCacheModule, RedisCacheModule { + + fun compositeCache( + caffeineCache: MyCaffeineCache, + redisCache: MyRedisCache, + ): Cache { + return Cache.builder(caffeineCache) + .addCache(redisCache) + .build() + } + } + ``` + +For an asynchronous facade, use `AsyncCache.builder(...)`; only `AsyncCache` instances can be added to it. +This is suitable, for example, for several `RedisCache` instances or other asynchronous implementations with the same key and value types. + +===! ":fontawesome-brands-java: `Java`" + + ```java + default AsyncCache compositeAsyncCache(MyRedisCache redisCache1, MyRedisCache redisCache2) { + return AsyncCache.builder(redisCache1) + .addCache(redisCache2) + .build(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun compositeAsyncCache(redisCache1: MyRedisCache, redisCache2: MyRedisCache): AsyncCache { + return AsyncCache.builder(redisCache1) + .addCache(redisCache2) + .build() + } + ``` + +The facade built through `Cache.Builder` does not support direct `get(Collection)`, and the facade built through `AsyncCache.Builder` does not support direct `getAsync(Collection)`. +For batch loading, use `computeIfAbsent(Collection, ...)` or `computeIfAbsentAsync(Collection, ...)`. + +#### Manual Redis expiration { #redis-expiration-override } + +Beyond the shared `Cache`/`AsyncCache` surface, `RedisCache` adds methods to override the configured `expireAfterWrite` for a single write. +`putExpireAfterWrite(key, value, Duration)` and its `Map` batch overload write synchronously, while `putAsyncExpireAfterWrite(...)` +(single and `Map` batch) return `CompletionStage`. The provided `Duration` is applied to that specific write instead of the value from configuration. +These methods are `Redis`-only, since `RedisCache` extends `AsyncCache`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Cache("mycache.config") + public interface MyCache extends RedisCache { } + + @Component + public class SomeService { + + private final MyCache cache; + + public SomeService(MyCache cache) { + this.cache = cache; + } + + public void cacheFor(String key, String value) { + cache.putExpireAfterWrite(key, value, Duration.ofMinutes(5)); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Cache("mycache.config") + interface MyCache : RedisCache + + @Component + class SomeService(private val cache: MyCache) { + + fun cacheFor(key: String, value: String) { + cache.putExpireAfterWrite(key, value, Duration.ofMinutes(5)) + } + } + ``` ### Declarative { #declarative } -All aspect use cases will assume the cache implementation above. +All aspect examples below assume the cache implementation above. #### Get { #get } -To cache and retrieve a value from the cache for the *get()* method, annotate it with the `@Cacheable` annotation. +To cache and retrieve a value from the cache for the `get()` method, annotate it with `@Cacheable`. +If the value is found in the cache, the original method is not called; if there is no value, the method is executed and the result is stored in the cache. -The key for the cache is compiled from the method arguments, the order of the arguments matters, in this case it will be compiled from the value `arg1`. +The cache key is built from method arguments, and argument order matters. In this case it is built from `arg1`. ===! ":fontawesome-brands-java: `Java`" @@ -336,10 +673,10 @@ The key for the cache is compiled from the method arguments, the order of the ar #### Put { #put } -To add values to the cache via the *put()* method, annotate it with the `@CachePut` annotation. -The method annotated with `@CachePut` will be called and its value put into the cache defined in *value*. +To add values to the cache via the `put()` method, annotate it with `@CachePut`. +The method with `@CachePut` is always called, and its result is put into the cache defined in `value`. -The key for the cache is compiled from the method arguments, the order of the arguments matters, in this case it will be compiled from the value `arg1`. +The cache key is built from method arguments, and argument order matters. In this case it is built from `arg1`. ===! ":fontawesome-brands-java: `Java`" @@ -369,10 +706,10 @@ The key for the cache is compiled from the method arguments, the order of the ar #### Invalidate { #invalidate } -To remove a keyed value from the cache via the *evict()* method, annotate it with the `@CacheInvalidate` annotation. -The method annotated with `@CacheInvalidate` will be called and then the keyed values for the cache defined in *value* will be deleted by key. +To remove a value from the cache by key via the `evict()` method, annotate it with `@CacheInvalidate`. +The method with `@CacheInvalidate` is called, and then the value is removed by key from the cache defined in `value`. -The key for the cache is compiled from the method arguments, the order of the arguments matters, in this case it will be compiled from the value `arg1`. +The cache key is built from method arguments, and argument order matters. In this case it is built from `arg1`. ===! ":fontawesome-brands-java: `Java`" @@ -402,9 +739,10 @@ The key for the cache is compiled from the method arguments, the order of the ar #### Invalidate all { #invalidate-all } -To remove all values from the cache via the *evictAll()* method, annotate it with the `@CacheInvalidate` annotation and specify the *invalidateAll = true* parameter. +To remove all values from the cache via the `evictAll()` method, annotate it with `@CacheInvalidate` +and specify the `invalidateAll = true` parameter. -The method annotated with `@CacheInvalidate` will be called and then all of the cache values defined in *value* will be removed. +The method with `@CacheInvalidate` is called, and then all values are removed from the cache defined in `value`. ===! ":fontawesome-brands-java: `Java`" @@ -434,7 +772,8 @@ The method annotated with `@CacheInvalidate` will be called and then all of the #### Composite cache { #composite-cache } -In case you have multiple caches, you need to connect both modules and specify the appropriate number of annotations over the method. +If several caches need to be used, connect the required modules and specify several annotations on the method. +For example, this can combine a fast local layer on `Caffeine` and a shared layer on `Redis`. ===! ":fontawesome-brands-java: `Java`" @@ -464,7 +803,7 @@ In case you have multiple caches, you need to connect both modules and specify t } ``` -And the annotated class itself is like this: +And the annotated class itself: ===! ":fontawesome-brands-java: `Java`" @@ -494,11 +833,16 @@ And the annotated class itself is like this: } ``` -The order of aspect calls corresponds to the order of annotations above the method, top to bottom. +The call order follows the order of annotations on the method from top to bottom. +For `@Cacheable`, this means the upper cache is checked first; on a miss, the next cache is checked, +and after the value is loaded, the result is stored back into the checked caches. +The same composition model works for repeatable `@CachePut` and `@CacheInvalidate`: the method is called once, +and then the result is written to all listed caches or invalidation is executed in all listed caches. +The container annotations `@Cacheables`, `@CachePuts`, and `@CacheInvalidates` can also be used when that form is more convenient. ## Key { #key } -In case the cache key represents 1 argument, it is required to register `Cache` with a signature corresponding to the key and value types. +If the cache key consists of one argument, register `Cache` with a signature that matches the key and value types. ===! ":fontawesome-brands-java: `Java`" @@ -516,11 +860,11 @@ In case the cache key represents 1 argument, it is required to register `Cache` ### Conversion { #conversion } -In case an argument cannot be converted to a cache key, the cache implementation will require an appropriate converter -with the `CacheKeyMapper` interface, in case there are 2 arguments for the key then `CacheKeyMapper2` will be required, and so on. +If an argument cannot be used directly as a cache key, the implementation requires a mapper +with the `CacheKeyMapper` interface. If there are two arguments for the key, `CacheKeyMapper2` is required; if there are three, `CacheKeyMapper3` is required, and so on up to `CacheKeyMapper9`. -Such a converter can also be provided manually using the `@Mapping` annotation, -example of converting a complex object into a simple cache key: +Such a mapper can be provided manually with `@Mapping`. +Example of converting a complex object into a simple cache key: ===! ":fontawesome-brands-java: `Java`" @@ -553,8 +897,17 @@ example of converting a complex object into a simple cache key: @Component class SomeService { + data class UserContext(val userId: String, val traceId: String) + + class UserContextMapping : CacheKeyMapper { + override fun map(arg: UserContext): String { + return arg.userId + } + } + + @Mapping(UserContextMapping::class) @Cacheable(MyCache::class) - fun get(arg1: String, arg2: BigDecimal): String { + fun get(context: UserContext): String { // do something } } @@ -562,14 +915,14 @@ example of converting a complex object into a simple cache key: ### Composite key { #composite-key } -In case the cache key represents N arguments, it is required to register `Cache` using an -class to describe such a key. +If the cache key consists of several arguments, register `Cache` with a custom class +that describes that key. -Example for `Cache` where the composite key consists of 2 elements: +Example for `Cache` where the composite key consists of two elements: ===! ":fontawesome-brands-java: `Java`" - It is supposed to create its own `record` class that would describe the composite key. + Create a custom `record` that describes the composite key. ```java @Cache("mycache.config") @@ -581,7 +934,7 @@ Example for `Cache` where the composite key consists of 2 elements: === ":simple-kotlin: `Kotlin`" - It is supposed to create its own `data` class that would describe the composite key. + Create a custom `data class` that describes the composite key. ```kotlin @Cache("mycache.config") @@ -591,14 +944,20 @@ Example for `Cache` where the composite key consists of 2 elements: } ``` -If `RedisCache` is used, it is assumed that all composite key arguments will default to non `null`, -or a custom key resolver will need to be used. +If `RedisCache` is used, a `RedisCacheKeyMapper` is generated for the composite key. +It uses a mapper for each key field and expects the mapping result for every field to be non-`null`. +Built-in mappers can encode `null` with a special value, while custom mappers must do this explicitly. ### Argument ordering { #argument-ordering } -If the method accepts arguments that you want to exclude from the composite key, -or the order of the arguments does not match the order of the arguments of the composite key constructor, -you should use the `parameters` annotation attribute and define which method arguments to use and in what order. +If the method accepts arguments that should be excluded from the composite key, or the argument order does not match +the order of the composite-key constructor arguments, use the `parameters` attribute and specify +which method arguments to use and in what order. + +`parameters` defines the full set of method arguments used to build the key. Each name must match a method argument name, +and the order must match the key type: for a single argument, the `Cache` key type; for a composite key, +the constructor argument order of the `record` or `data class`. +If a name is missing, a type does not match, or the order does not fit the key, application generation fails. ===! ":fontawesome-brands-java: `Java`" @@ -628,7 +987,9 @@ you should use the `parameters` annotation attribute and define which method arg ## Loadable Cache { #loadable-cache } -The library provides a component for building an entity that combines GET and PUT operations without using aspects - `LoadableCache` +The library provides the `LoadableCache` component, which combines `get` and `put` operations without using aspects. +It is useful when value loading must be controlled manually while keeping the standard logic: first check the cache, +and on a miss load the data and store it. ===! ":fontawesome-brands-java: `Java`" @@ -637,7 +998,7 @@ The library provides a component for building an entity that combines GET and PU public interface MyCache extends CaffeineCache { } @KoraApp - public interface Application : CaffeineCacheModule { + public interface Application extends CaffeineCacheModule { default LoadableCache loadableCache(MyCache cache, SomeService someService) { return cache.asLoadable(someService::loadEntity); @@ -663,26 +1024,69 @@ The library provides a component for building an entity that combines GET and PU } ``` +For an asynchronous cache, use `AsyncLoadableCache`. It is created through `asLoadableAsyncSimple(...)` +for loading one key or through `asLoadableAsync(...)` for batch loading several keys. +Both variants return `CompletionStage` and are suitable for `RedisCache`, because it implements `AsyncCache`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Cache("mycache.config") + public interface MyCache extends RedisCache { } + + @KoraApp + public interface Application extends RedisCacheModule { + + default AsyncLoadableCache loadableCache(MyCache cache, SomeService someService) { + return cache.asLoadableAsync(someService::loadEntities); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Cache("mycache.config") + interface MyCache : RedisCache + + @KoraApp + interface Application : RedisCacheModule { + + fun loadableCache( + cache: MyCache, + someService: SomeService, + ): AsyncLoadableCache { + return cache.asLoadableAsync(someService::loadEntities) + } + } + ``` + ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Available signatures for methods supported by annotations: ===! ":fontawesome-brands-java: `Java`" - Class must be non `final` in order for aspects to work. + The class must not be `final` for aspects to work. The `T` refers to the type of the return value. - `T myMethod()` - - `@Nullable T myMethod()` - `Optional myMethod()` - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + + `@Cacheable` and `@CachePut` require a return value and cannot be applied to `void`, `Mono`, `CompletionStage`, `Flux`, or `Publisher`. + `@CacheInvalidate` can be applied to methods without a result, but cannot be applied to `Flux` or `Publisher`. === ":simple-kotlin: `Kotlin`" - Class must be `open` in order for aspects to work. + The class must be `open` for aspects to work. By `T` we mean the type of the return value, either `T?`, or `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + + `@Cacheable` and `@CachePut` require a return value and cannot be applied to `Unit`. + `@CacheInvalidate` can be applied to methods without a result. diff --git a/mkdocs/docs/en/documentation/camunda7-bpmn.md b/mkdocs/docs/en/documentation/camunda7-bpmn.md index a73d2e5..5fec94c 100644 --- a/mkdocs/docs/en/documentation/camunda7-bpmn.md +++ b/mkdocs/docs/en/documentation/camunda7-bpmn.md @@ -1,15 +1,18 @@ --- -description: "Explains Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry. Use when working with CamundaEngineBpmnModule, CamundaEngineConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." +description: "Explains Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry. Use when working with CamundaEngineBpmnModule, CamundaEngineBpmnConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry; key triggers include CamundaEngineBpmnModule, CamundaEngineConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." + use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry; key triggers include CamundaEngineBpmnModule, CamundaEngineBpmnConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." --- ??? warning "Experimental module" - **Experimental** module is fully working and tested, but requires additional approbation and usage analytics, - for this reason, API may potentially undergo minor changes before fully stable. + The **experimental** module is fully working and tested, but requires additional usage validation and analysis. + Therefore, the `API` may receive minor changes before full readiness. -Module for connecting a BPMN process workflow engine based on [Camunda 7](https://docs.camunda.org/manual/7.21/) +The module connects an embedded [Camunda 7](https://docs.camunda.org/manual/7.21/) engine for executing `BPMN` processes inside a Kora application. +It creates and configures `ProcessEngine`, connects it to a `JDBC` data source, registers delegates from the application graph, deploys `BPMN` / `FORM` / `DMN` resources from `classpath`, and adds execution telemetry. + +To expose the `Camunda 7 REST API` and the `Cockpit` / `Admin` / `Tasklist` web applications over HTTP, use the separate [Camunda 7 REST module](camunda7-rest.md) alongside this one. ## Dependency { #dependency } @@ -39,11 +42,12 @@ Module for connecting a BPMN process workflow engine based on [Camunda 7](https: interface Application : CamundaEngineBpmnModule ``` -Requires [JDBC module](database-jdbc.md) connection. +The module requires the [JDBC module](database-jdbc.md). +By default, the main application `DataSource` is used, but you can provide a separate `DataSource` with the `@Tag(CamundaBpmn.class)` tag when needed. ## Configuration { #configuration } -Example of the complete configuration described in the `CamundaEngineBpmnConfig` class (example values or default values are specified): +Example of the complete configuration described by the `CamundaEngineBpmnConfig` class: ===! ":material-code-json: `Hocon`" @@ -56,13 +60,13 @@ Example of the complete configuration described in the `CamundaEngineBpmnConfig` maxPoolSize = 25 //(2)! queueSize = 25 //(3)! maxJobsPerAcquisition = 2 //(4)! - virtualThreadsEnabled = true //(5)! + virtualThreadsEnabled = false //(5)! } deployment { tenantId = "Camunda" //(6)! name = "KoraEngineAutoDeployment" //(7)! deployChangedOnly = true //(8)! - resources = "classpath:bpm" //(9)! + resources = ["classpath:bpm"] //(9)! delay = "1m" //(10)! } parallelInitialization { @@ -83,8 +87,8 @@ Example of the complete configuration described in the `CamundaEngineBpmnConfig` } metrics { enabled = true //(20)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(21)! - tags = { // (22)! + slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(21)! + tags = { //(22)! "key1" = "value1" "key2" = "value2" } @@ -92,7 +96,7 @@ Example of the complete configuration described in the `CamundaEngineBpmnConfig` engineTelemetryEnabled = false //(23)! tracing { enabled = true //(24)! - attributes = { // (25)! + attributes = { //(25)! "key1" = "value1" "key2" = "value2" } @@ -103,31 +107,31 @@ Example of the complete configuration described in the `CamundaEngineBpmnConfig` } ``` - 1. Minimum number of live threads in [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 2. Maximum number of threads in [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 3. Size of the task queue before tasks are thrown out of the [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) execution queue - 4. Maximum number of tasks in the [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) execution (default is the number of CPU cores multiplied by 2) - 5. Use [virtual threads](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) as the basis for JobExecutor. All previous options are irrelevant when virtual threads are enabled - 6. Indeterminator tenant [load](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) resources (default is none) - 7. Name of [load](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) resources - 8. Flag that says that only modified resources should be loaded - 9. Paths to find BPMN/FORM/DMN resources that will be loaded into the engine after startup - 10. Delay before deploying new resources to engine - 11. Whether to enable parallel loading, which slightly improves the engine startup speed - 12. Whether to check for incomplete engine configuration requests - 13. Camunda administrator identifier (optional) - 14. Camunda Administrator Password (optional) - 15. Camunda Administrator Name (optional) - 16. Last name of Camunda administrator (optional) - 17. Email of the Camunda administrator (optional) - 18. Enables module logging (default is `false`) - 19. Enables error stack logging (default is `true`) - 20. Enables module metrics (default `true`) - 21. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 22. Configures tags for metrics (optional) - 23. Enables collection of engine metrics/telemetry (default is `false`) - 24. Enables module tracing (default `true`) - 25. Configures attributes for tracing (optional) + 1. Minimum number of permanently alive threads in [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (default: `5`). + 2. Maximum number of threads in [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (default: `25`). + 3. `JobExecutor` task queue size before new tasks are rejected (default: `25`). + 4. Maximum number of jobs acquired by `JobExecutor` in one request (default: `Runtime.getRuntime().availableProcessors() * 2`). + 5. Use [virtual threads](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) as the `JobExecutor` base (default: `false`). When this option is enabled, pool and queue size settings are not used. + 6. `tenant` identifier for resource [deployment](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) (default not specified, optional). + 7. Resource [deployment](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) name (default: `KoraEngineAutoDeployment`). + 8. Deploy only changed resources through `Camunda` duplicate filtering (default: `true`). + 9. List of paths for finding `BPMN` / `FORM` / `DMN` resources (`required`, default not specified). Only paths with the `classpath:` prefix are supported. + 10. Delay before deploying resources to the engine (default not specified, optional). + 11. Enable parallel engine initialization (default: `true`). + 12. Validate incomplete engine statements during parallel initialization (default: `true`). + 13. `Camunda` administrator identifier (`required`, default not specified). The whole `admin` section is optional. + 14. `Camunda` administrator password (`required`, default not specified). The whole `admin` section is optional. + 15. `Camunda` administrator first name (default not specified, optional). If not specified, uppercase `id` is used. + 16. `Camunda` administrator last name (default not specified, optional). If not specified, uppercase `id` is used. + 17. `Camunda` administrator email address (default not specified, optional). If not specified, `@localhost` is used. + 18. Enables module logging (default: `false`). + 19. Enables error stack trace logging (default: `true`). + 20. Enables module metrics (default: `true`). + 21. [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) configuration for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 22. Metric tags (default: `{}`). + 23. Enables built-in `Camunda` engine telemetry collection (default: `false`). + 24. Enables module tracing (default: `true`). + 25. Tracing attributes (default: `{}`). === ":simple-yaml: `YAML`" @@ -140,13 +144,14 @@ Example of the complete configuration described in the `CamundaEngineBpmnConfig` maxPoolSize: 25 #(2)! queueSize: 25 #(3)! maxJobsPerAcquisition: 2 #(4)! - virtualThreadsEnabled: true #(5)! + virtualThreadsEnabled: false #(5)! deployment: tenantId: "Camunda" #(6)! name: "KoraEngineAutoDeployment" #(7)! deployChangedOnly: true #(8)! - resources: "classpath:bpm" #(9)! - delay: "2m" #(9)! + resources: #(9)! + - "classpath:bpm" + delay: "1m" #(10)! parallelInitialization: enabled: true #(11)! validateIncompleteStatements: true #(12)! @@ -162,7 +167,7 @@ Example of the complete configuration described in the `CamundaEngineBpmnConfig` stacktrace: true #(19)! metrics: enabled: true #(20)! - slo: [ 0, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(21)! + slo: [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] #(21)! tags: #(22)! key1: value1 key2: value2 @@ -174,48 +179,106 @@ Example of the complete configuration described in the `CamundaEngineBpmnConfig` key2: value2 ``` - 1. Minimum number of live threads in [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 2. Maximum number of threads in [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 3. Size of the task queue before tasks are thrown out of the [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) execution queue - 4. Maximum number of tasks in the [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) execution (default is the number of CPU cores multiplied by 2) - 5. Use [virtual threads](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) as the basis for JobExecutor. All previous options are irrelevant when virtual threads are enabled - 6. Indeterminator tenant [load](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) resources (default is none) - 7. Name of [load](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) resources - 8. Flag that says that only modified resources should be loaded - 9. Paths to find BPMN/FORM/DMN resources that will be loaded into the engine after startup - 10. Delay before deploying new resources to engine - 11. Whether to enable parallel loading, which slightly improves the engine startup speed - 12. Whether to check for incomplete engine configuration requests - 13. Camunda administrator identifier (optional) - 14. Camunda Administrator Password (optional) - 15. Camunda Administrator Name (optional) - 16. Last name of Camunda administrator (optional) - 17. Email of the Camunda administrator (optional) - 18. Enables module logging (default is `false`) - 19. Enables error stack logging (default is `true`) - 20. Enables module metrics (default `true`) - 21. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 22. Configures tags for metrics (optional) - 23. Enables collection of engine metrics/telemetry (default is `false`) - 24. Enables module tracing (default `true`) - 25. Configures attributes for tracing (optional) + 1. Minimum number of permanently alive threads in [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (default: `5`). + 2. Maximum number of threads in [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (default: `25`). + 3. `JobExecutor` task queue size before new tasks are rejected (default: `25`). + 4. Maximum number of jobs acquired by `JobExecutor` in one request (default: `Runtime.getRuntime().availableProcessors() * 2`). + 5. Use [virtual threads](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) as the `JobExecutor` base (default: `false`). When this option is enabled, pool and queue size settings are not used. + 6. `tenant` identifier for resource [deployment](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) (default not specified, optional). + 7. Resource [deployment](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) name (default: `KoraEngineAutoDeployment`). + 8. Deploy only changed resources through `Camunda` duplicate filtering (default: `true`). + 9. List of paths for finding `BPMN` / `FORM` / `DMN` resources (`required`, default not specified). Only paths with the `classpath:` prefix are supported. + 10. Delay before deploying resources to the engine (default not specified, optional). + 11. Enable parallel engine initialization (default: `true`). + 12. Validate incomplete engine statements during parallel initialization (default: `true`). + 13. `Camunda` administrator identifier (`required`, default not specified). The whole `admin` section is optional. + 14. `Camunda` administrator password (`required`, default not specified). The whole `admin` section is optional. + 15. `Camunda` administrator first name (default not specified, optional). If not specified, uppercase `id` is used. + 16. `Camunda` administrator last name (default not specified, optional). If not specified, uppercase `id` is used. + 17. `Camunda` administrator email address (default not specified, optional). If not specified, `@localhost` is used. + 18. Enables module logging (default: `false`). + 19. Enables error stack trace logging (default: `true`). + 20. Enables module metrics (default: `true`). + 21. [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) configuration for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 22. Metric tags (default: `{}`). + 23. Enables built-in `Camunda` engine telemetry collection (default: `false`). + 24. Enables module tracing (default: `true`). + 25. Tracing attributes (default: `{}`). + +The `deployment` section is optional: if it is not specified, the module does not automatically deploy resources. +If the section is specified, `resources` must contain at least one path. +Resources are searched recursively in `classpath`; unsupported paths without the `classpath:` prefix are skipped. Module metrics are described in the [Metrics Reference](metrics.md#camunda-7-bpmn) section. -## Applications { #applications } +## Deployment { #deployment } + +When the `deployment` section is present, the module automatically deploys process resources into the engine after it is created. +Resources are placed on the `classpath` (usually under `src/main/resources`) and referenced by the `resources` list: + +===! ":material-code-json: `Hocon`" + + ```javascript + camunda.engine.bpmn { + deployment { + resources = ["classpath:bpm"] //(1)! + } + } + ``` + + 1. At least one path is required when the `deployment` section is present. Only paths with the `classpath:` prefix are supported. + +=== ":simple-yaml: `YAML`" + + ```yaml + camunda: + engine: + bpmn: + deployment: + resources: #(1)! + - "classpath:bpm" + ``` + + 1. At least one path is required when the `deployment` section is present. Only paths with the `classpath:` prefix are supported. + +Given the following layout, the `classpath:bpm` path is scanned recursively and every supported resource under it is deployed: + +``` +src/main/resources/bpm/ +├── approve.form +├── helloworld.bpmn +└── onboarding.bpmn +``` + +Deployment rules to keep in mind: + +- Supported resource types are `BPMN` process models, `FORM` forms, and `DMN` decision tables. +- Only paths with the `classpath:` prefix are deployed. Any other path is skipped with a warning in the log. +- Paths are scanned **recursively**, so nested directories under the listed path are included. +- With `deployChangedOnly = true` (default) `Camunda` duplicate filtering is enabled, so only resources that changed since the previous deployment are redeployed. +- The optional `tenantId` binds the deployment to a specific `tenant`, and `delay` postpones the deployment for the configured duration after startup. +- The deployment is registered under the `name` (default `KoraEngineAutoDeployment`). +- If the whole `deployment` section is omitted, the module does not deploy any resources — you are expected to deploy them yourself through `RepositoryService`. + +## Delegates { #applications } -You can register in Camunda user [JavaDelegate](https://docs.camunda.org/manual/7.21/user-guide/process-engine/delegation-code/) -which will be registered in the context by their full class name (`canonicalName`) and by their simplified class name (`simpleName`): +`Camunda` can call application components as process delegates. +Regular [`JavaDelegate`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/delegation-code/) instances are registered in the context by the full class name (`canonicalName`) and by the short class name (`simpleName`). +Inside `execute(...)` you read and write process variables through `DelegateExecution`: ===! ":fontawesome-brands-java: `Java`" ```java @Component - public final class SimpleDelegate implements JavaDelegate { + public final class ScoreCustomerDelegate implements JavaDelegate { - @Override - public void execute(DelegateExecution delegateExecution) throws Exception { + private static final Logger logger = LoggerFactory.getLogger(ScoreCustomerDelegate.class); + @Override + public void execute(DelegateExecution execution) { + int scoring = ThreadLocalRandom.current().nextInt(1, 100); + logger.info("Scored {} with result {}.", execution.getBusinessKey(), scoring); + execution.setVariable("result", scoring); } } ``` @@ -224,15 +287,30 @@ which will be registered in the context by their full class name (`canonicalName ```kotlin @Component - class SimpleKoraDelegate : JavaDelegate { + class ScoreCustomerDelegate : JavaDelegate { - fun execute(delegateExecution: DelegateExecution) { + private val logger = LoggerFactory.getLogger(ScoreCustomerDelegate::class.java) + override fun execute(execution: DelegateExecution) { + val scoring = ThreadLocalRandom.current().nextInt(1, 100) + logger.info("Scored {} with result {}.", execution.businessKey, scoring) + execution.setVariable("result", scoring) } } ``` -You can also register specialized `KoraDelegate`, which allow, in addition to standard naming, to register an executor with an arbitrary name in context via the `key()` method: +Because a `JavaDelegate` is registered by its short class name, a `serviceTask` in the `BPMN` model references it by `simpleName` through `camunda:delegateExpression`: + +```xml + + Flow_score_in + Flow_score_out + +``` + +Use `KoraDelegate` for an arbitrary delegate name. +The `key()` method returns `canonicalName` by default, but it can be overridden to specify the name used in `BPMN` expressions: ===! ":fontawesome-brands-java: `Java`" @@ -240,6 +318,7 @@ You can also register specialized `KoraDelegate`, which allow, in addition to st @Component public final class SimpleDelegate implements KoraDelegate { + @Override public String key() { return "myKey"; } @@ -257,17 +336,227 @@ You can also register specialized `KoraDelegate`, which allow, in addition to st @Component class SimpleKoraDelegate : KoraDelegate { - fun key() = "myKey" + override fun key(): String = "myKey" - fun execute(delegateExecution: DelegateExecution) { + override fun execute(delegateExecution: DelegateExecution) { } } ``` -## Engine configuration { #engine-configuration } +A delegate declared this way is referenced as `${myKey}` in `camunda:delegateExpression`, so the name used in the process model no longer depends on the class name. + +Every delegate is wrapped by `KoraDelegateWrapperFactory` before it is called: it forks the current Kora `Context` for the delegate execution and applies module telemetry around `execute(...)`. +You can provide your own `KoraDelegateWrapperFactory` as a `@Component` to change this behavior. -It is possible to register user `ProcessEngineConfigurator` that allow configuring [ProcessEngine](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-bootstrapping/): +## Engine Services { #engine-services } + +The module provides standard `Camunda` services as dependency graph components: + +- `RuntimeService` +- `RepositoryService` +- `ManagementService` +- `AuthorizationService` +- `DecisionService` +- `ExternalTaskService` +- `FilterService` +- `FormService` +- `TaskService` +- `HistoryService` +- `IdentityService` + +These services can be injected into your components in the usual way. + +## Starting and interacting with processes { #usage } + +Inject `ProcessEngine` (or any of the engine services above) into your components to start and drive process instances. +A process is started by its `BPMN` process `id` through `RuntimeService`, and process definitions can be queried through `RepositoryService`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController("/camunda") + public final class CamundaController { + + private final ProcessEngine processEngine; + + public CamundaController(ProcessEngine processEngine) { + this.processEngine = processEngine; + } + + @HttpRoute(method = HttpMethod.GET, path = "/start/onboarding") + public String startOnboarding() { + String businessKey = UUID.randomUUID().toString(); + ProcessInstance instance = processEngine.getRuntimeService() + .startProcessInstanceByKey("Onboarding", businessKey); + return instance.getId(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController("/camunda") + class CamundaController(private val processEngine: ProcessEngine) { + + @HttpRoute(method = HttpMethod.GET, path = "/start/onboarding") + fun startOnboarding(): String { + val businessKey = UUID.randomUUID().toString() + val instance = processEngine.runtimeService + .startProcessInstanceByKey("Onboarding", businessKey) + return instance.id + } + } + ``` + +A running process can be advanced from outside the engine as well: `RuntimeService.correlateMessage(...)` delivers a `BPMN` message event, and `TaskService` / `FormService` complete user tasks and submit forms: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController("/camunda/process/onboarding") + public final class OnboardingController { + + private final FormService formService; + private final TaskService taskService; + private final RuntimeService runtimeService; + + public OnboardingController(FormService formService, TaskService taskService, RuntimeService runtimeService) { + this.formService = formService; + this.taskService = taskService; + this.runtimeService = runtimeService; + } + + @HttpRoute(path = "/cancel/{businessKey}", method = HttpMethod.GET) + public String customerCancellation(@Path String businessKey) { + runtimeService.correlateMessage("MessageCustomerCancellation", businessKey); + return "Cancelled: " + businessKey; + } + + @HttpRoute(path = "/order/{businessKey}", method = HttpMethod.GET) + public String customerOrder(@Path String businessKey) { + Task task = taskService.createTaskQuery().processInstanceBusinessKey(businessKey).active().singleResult(); + formService.submitTaskForm(task.getId(), Map.of("approved", true)); + return "Approved: " + businessKey; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController("/camunda/process/onboarding") + class OnboardingController( + private val formService: FormService, + private val taskService: TaskService, + private val runtimeService: RuntimeService + ) { + + @HttpRoute(path = "/cancel/{businessKey}", method = HttpMethod.GET) + fun customerCancellation(@Path businessKey: String): String { + runtimeService.correlateMessage("MessageCustomerCancellation", businessKey) + return "Cancelled: $businessKey" + } + + @HttpRoute(path = "/order/{businessKey}", method = HttpMethod.GET) + fun customerOrder(@Path businessKey: String): String { + val task = taskService.createTaskQuery().processInstanceBusinessKey(businessKey).active().singleResult() + formService.submitTaskForm(task.id, mapOf("approved" to true)) + return "Approved: $businessKey" + } + } + ``` + +## DataSource and transactions { #datasource } + +The engine persists its state through a `JDBC` `DataSource`, so the [JDBC module](database-jdbc.md) is required. +By default the module reuses the main application `DataSource`, exposed to the engine under the `@Tag(CamundaBpmn.class)` tag. +To give the engine a dedicated data source, provide your own `DataSource` with that tag: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(CamundaBpmn.class) + @Component + public DataSource camundaDataSource(/* ... */) { + return dataSource; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(CamundaBpmn::class) + @Component + fun camundaDataSource(/* ... */): DataSource { + return dataSource + } + ``` + +The `CamundaEngineDataSource` component abstracts the engine's `DataSource` together with its `CamundaTransactionManager`. +The default implementation runs `JDBC` over the `@Tag(CamundaBpmn.class)` `DataSource`; you can override `CamundaEngineDataSource` as a `@Component` to fully control how the engine obtains connections and manages transactions. + +A delegate that performs its own `JDBC` work can run it inside the engine transaction through `CamundaTransactionManager`. +`inContinueTx(...)` reuses the connection of the current engine transaction (opening a new one only if none is active), while `inNewTx(...)` always opens a new transaction; `currentConnection()` returns a handle to `commit()` / `rollback()` the current transaction: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class AuditDelegate implements JavaDelegate { + + private final CamundaTransactionManager transactionManager; + + public AuditDelegate(CamundaTransactionManager transactionManager) { + this.transactionManager = transactionManager; + } + + @Override + public void execute(DelegateExecution execution) { + transactionManager.inContinueTx(() -> { + // JDBC work sharing the engine transaction + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class AuditDelegate(private val transactionManager: CamundaTransactionManager) : JavaDelegate { + + override fun execute(execution: DelegateExecution) { + transactionManager.inContinueTx(Runnable { + // JDBC work sharing the engine transaction + }) + } + } + ``` + +## Job executor and readiness { #job-executor } + +The engine runs asynchronous continuations and timers through a [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/). +The implementation is selected by the `jobExecutor.virtualThreadsEnabled` option: when `false` (default) a thread-pool executor is used and sized by `corePoolSize` / `maxPoolSize` / `queueSize` / `maxJobsPerAcquisition`; when `true` a [virtual-thread](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) executor is used and the pool/queue sizes are ignored (see the [Configuration](#configuration) callouts). + +The module automatically registers a [readiness probe](probes.md) that reports the application as `UP` only once the `JobExecutor` is active. +Until the job executor is activated the probe fails with `Camunda BPMN Engine JobExecutor is not active`, which keeps the application out of rotation while the engine is still starting. + +## Admin user and Cockpit { #admin } + +When the `admin` section is present, the module provisions a `Camunda` administrator user, ensures the `camunda-admin` group with full authorizations exists, and adds the user to it (see the [Configuration](#configuration) `admin` callouts). +This account is what you use to log into the `Cockpit` / `Admin` / `Tasklist` web applications served by the [Camunda 7 REST module](camunda7-rest.md). +If the `admin` section is omitted, no user is created. + +## Engine Configuration { #engine-configuration } + +For additional configuration, register a `ProcessEngineConfigurator` component. +The `prepare(...)` method is called before [ProcessEngine](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-bootstrapping/) is created and receives `ProcessEngineConfiguration`; `setup(...)` is called after the engine is created: ===! ":fontawesome-brands-java: `Java`" @@ -276,7 +565,12 @@ It is possible to register user `ProcessEngineConfigurator` that allow configuri public final class SimpleProcessEngineConfigurator implements ProcessEngineConfigurator { @Override - public void setup(ProcessEngine engine) { + public void prepare(ProcessEngineConfiguration configuration) { + + } + + @Override + public void setup(ProcessEngine engine) throws Exception { } } @@ -288,12 +582,100 @@ It is possible to register user `ProcessEngineConfigurator` that allow configuri @Component class SimpleProcessEngineConfigurator : ProcessEngineConfigurator { - fun setup(engine: ProcessEngine) { - + override fun prepare(configuration: ProcessEngineConfiguration) { + + } + + override fun setup(engine: ProcessEngine) { + } } ``` ## Plugins { #plugins } -You can register arbitrary [Plugin](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-plugins/) by providing them as components in a dependency container. +You can register arbitrary [`ProcessEnginePlugin`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-plugins/) by providing them as components in the Kora dependency container. +The module collects all such components and passes them to the engine configuration when creating `ProcessEngine`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SimpleProcessEnginePlugin implements ProcessEnginePlugin { + + @Override + public void preInit(ProcessEngineConfigurationImpl configuration) { + + } + + @Override + public void postInit(ProcessEngineConfigurationImpl configuration) { + + } + + @Override + public void postProcessEngineBuild(ProcessEngine engine) { + + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SimpleProcessEnginePlugin : ProcessEnginePlugin { + + override fun preInit(configuration: ProcessEngineConfigurationImpl) { + + } + + override fun postInit(configuration: ProcessEngineConfigurationImpl) { + + } + + override fun postProcessEngineBuild(engine: ProcessEngine) { + + } + } + ``` + +## Camunda version { #version } + +The detected `Camunda` version is available as an injectable `CamundaVersion` component. +Its `version()` returns the version string reported by the `Camunda` package, and `isEnterprise()` returns `true` when an enterprise (`-ee`) distribution is on the classpath: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class VersionPrinter { + + public VersionPrinter(CamundaVersion version) { + if (version.isEnterprise()) { + // enterprise-only behavior + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class VersionPrinter(version: CamundaVersion) { + + init { + if (version.isEnterprise()) { + // enterprise-only behavior + } + } + } + ``` + +## Telemetry { #telemetry } + +The module reports its own logging, metrics, and tracing for delegate executions through the `telemetry` configuration section. +Metrics are described in the [Metrics Reference](metrics.md#camunda-7-bpmn) section, and the `Context` fork performed by `KoraDelegateWrapperFactory` keeps this telemetry scoped to each delegate call. + +Independently of the module telemetry, `telemetry.engineTelemetryEnabled` toggles `Camunda`'s own built-in telemetry collection (disabled by default). diff --git a/mkdocs/docs/en/documentation/camunda7-rest.md b/mkdocs/docs/en/documentation/camunda7-rest.md index 6de816e..bed5610 100644 --- a/mkdocs/docs/en/documentation/camunda7-rest.md +++ b/mkdocs/docs/en/documentation/camunda7-rest.md @@ -6,10 +6,14 @@ agent: ??? warning "Experimental module" - **Experimental** module is fully working and tested, but requires additional approbation and usage analytics, - for this reason, API may potentially undergo minor changes before fully stable. + The **experimental** module is fully working and tested, but it requires additional validation and usage analysis. + For this reason, the `API` may undergo minor changes before it is considered fully stable. -Module to add [REST API](https://docs.camunda.org/manual/7.21/reference/rest/overview/) for [Camunda 7 BPMN module](camunda7-bpmn.md) +The module connects [`Camunda 7 REST API`](https://docs.camunda.org/manual/7.21/reference/rest/overview/) to a Kora application and exposes the standard `CamundaRestResources` through a separate `Undertow` HTTP server. +It is used together with the [`Camunda 7 BPMN` module](camunda7-bpmn.md): the `BPMN` engine executes processes, while the REST module provides HTTP access to `Camunda 7` operations. + +The module can also serve the `OpenAPI` description of the `REST API`, as well as `Swagger UI` and `RapiDoc` pages. +Requests to the `REST API` have separate settings for `CORS`, logging, metrics, tracing, and graceful server shutdown. ## Dependency { #dependency } @@ -39,11 +43,24 @@ Module to add [REST API](https://docs.camunda.org/manual/7.21/reference/rest/ove interface Application : CamundaRestUndertowModule ``` -Requires [Camunda BPMN module](camunda7-bpmn.md) to be added. +Requires the [`Camunda 7 BPMN` module](camunda7-bpmn.md). + +## HTTP server { #http-server } + +The module starts a **separate**, independent `Undertow` HTTP server dedicated to the `Camunda 7 REST API`. +It listens on its own `port` (default: `8081`) and is completely isolated from the main [HTTP server](http-server.md) module: +it has its own [CORS](#cors) filter, its own [telemetry](#telemetry), and its own [graceful shutdown](container.md#component-lifecycle). +The `Camunda REST API` and the application's own controllers therefore run on different ports and do not share request handling or configuration. + +The `ProcessEngine` that serves these requests is provided by the [`Camunda 7 BPMN` module](camunda7-bpmn.md); +this module only exposes it over HTTP under the configured `path` (default: `/engine-rest`). + +On shutdown, the server stops accepting new requests and waits up to `shutdownWait` (default: `30s`) +for in-flight requests to complete before it terminates. ## Configuration { #configuration } -Example of the complete configuration described in the `CamundaRestConfig` class (example values or default values are specified): +Example of the complete configuration described by the `CamundaRestConfig` class: ===! ":material-code-json: `Hocon`" @@ -82,7 +99,7 @@ Example of the complete configuration described in the `CamundaRestConfig` class stacktrace = true //(20)! mask = "***" //(21)! maskQueries = [ ] //(22)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(23)! + maskHeaders = [ "authorization" ] //(23)! pathTemplate = true //(24)! } metrics { @@ -105,37 +122,35 @@ Example of the complete configuration described in the `CamundaRestConfig` class } ``` - 1. Enable/disable REST API - 2. Prefix path to REST API - 3. Port on which the REST API server will be started - 4. Maximum time to wait for the server to complete after receiving a floating termination signal - 5. Relative path to OpenAPI files in the `resources` directory, default is the `openapi.json` OpenAPI file from [Camunda dependencies](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi) - 6. The on/off switch of the controller that gives the OpenAPI - 7. Path where OpenAPI will be available - 5. If a single OpenAPI file is specified, then represent entire path where file is available - 6. If multiple OpenAPI files are specified, is a path prefix to the file name `/openapi/{fileName}`, taking the specified path and appending the file name to it without the directories and its extension, example of the file `someDirectory/my-openapi-1.yaml` the file path will be `/openapi/my-openapi-1`. - 8. On/Off of the controller that gives SwaggerUI - 9. Path where the SwaggerUI will be accessed - 10. On/Off of the controller that gives Rapidoc - 11. Path where Rapidoc will be available - 12. Enables CORS filter (default `false`) - 13. Allowed origins for CORS (default `null`) - 14. Allowed headers for CORS requests (default `["*"]`) - 15. Allowed HTTP methods for CORS requests (default `["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD"]`) - 16. Allows transmission of credentials in CORS requests (default `true`) - 17. Headers that can be exposed to the client in CORS responses (default `["*"]`) - 18. Maximum caching time for CORS preflight requests (default `1 hour`) - 19. Enables module logging (default `false`) - 20. Enables call stack logging in case of exception - 21. Mask that is used to hide specified headers and request/response parameters - 22. List of request parameters to be hidden - 23. List of request/response headers that should be hidden - 24. Whether to always use the request path template when logging. The default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 25. Enables module metrics (default `true`) - 26. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 27. Configures tags for metrics (optional) - 28. Enables module tracing (default is `true`) - 29. Configures attributes for tracing (optional) + 1. Enables `Camunda 7 REST API` (default: `false`). + 2. Path prefix for `Camunda 7 REST API` (default: `/engine-rest`). + 3. Port of the separate `Undertow` HTTP server for the `REST API` (default: `8081`). + 4. Maximum time to wait for HTTP server [graceful shutdown](container.md#component-lifecycle) (default: `30s`). + 5. Path to the `OpenAPI` file in `resources` (default: `[ "openapi.json" ]`). By default, the file from the [`camunda-engine-rest-openapi` dependency](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi) is used. + 6. Enables the controller that serves the `OpenAPI` file (default: `false`). + 7. Path where the `OpenAPI` file will be available (default: `/openapi`). + 8. Enables the controller that serves `Swagger UI` (default: `false`). + 9. Path where `Swagger UI` will be available (default: `/swagger-ui`). + 10. Enables the controller that serves `RapiDoc` (default: `false`). + 11. Path where `RapiDoc` will be available (default: `/rapidoc`). + 12. Enables the `CORS` filter (default: `false`). + 13. Allowed origin for `CORS` (default: not specified, optional). If the value is not specified, the filter uses the request `Origin` header, and if it is absent, returns `*`. + 14. Allowed headers for `CORS` requests (default: `[ "*" ]`). + 15. Allowed HTTP methods for `CORS` requests (default: `[ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ]`). + 16. Allows credentials in `CORS` requests (default: `true`). + 17. Headers exposed to the client in a `CORS` response (default: `[ "*" ]`). + 18. Maximum caching time for `CORS` preflight requests (default: `1h`). + 19. Enables module logging (default: `false`). + 20. Enables stack trace logging when an exception occurs (default: `true`). + 21. Mask used to hide specified request or response headers and parameters (default: `***`). + 22. List of request parameters to hide in logs (default: `[ ]`). + 23. List of request or response headers to hide in logs (default: `[ "authorization" ]`). + 24. Defines whether the path template is used for logging (default: not specified, optional). If not specified, the full path is used only at the `TRACE` logging level; if `true`, the path template is used; if `false`, the full path is used. + 25. Enables module metrics (default: `true`). + 26. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 27. Additional tags for metrics (default: `{}`). + 28. Enables module tracing (default: `true`). + 29. Additional attributes for tracing (default: `{}`). === ":simple-yaml: `YAML`" @@ -156,21 +171,21 @@ Example of the complete configuration described in the `CamundaRestConfig` class rapidoc: enabled: false #(10)! endpoint: "/rapidoc" #(11)! - cors: - enabled: false #(12)! - allowOrigin: "*" #(13)! - allowHeaders: [ "*" ] #(14)! - allowMethods: [ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ] #(15)! - allowCredentials: true #(16)! - exposeHeaders: [ "*" ] #(17)! - maxAge: "1h" #(18)! + cors: + enabled: false #(12)! + allowOrigin: "*" #(13)! + allowHeaders: [ "*" ] #(14)! + allowMethods: [ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ] #(15)! + allowCredentials: true #(16)! + exposeHeaders: [ "*" ] #(17)! + maxAge: "1h" #(18)! telemetry: logging: enabled: false #(19)! stacktrace: true #(20)! mask: "***" #(21)! maskQueries: [ ] #(22)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(23)! + maskHeaders: [ "authorization" ] #(23)! pathTemplate: true #(24)! metrics: enabled: true #(25)! @@ -178,45 +193,176 @@ Example of the complete configuration described in the `CamundaRestConfig` class tags: #(27)! key1: value1 key2: value2 - tracing: - enabled: true #(28)! - attributes: #(29)! - key1: value1 - key2: value2 + tracing: + enabled: true #(28)! + attributes: #(29)! + key1: value1 + key2: value2 + ``` + + 1. Enables `Camunda 7 REST API` (default: `false`). + 2. Path prefix for `Camunda 7 REST API` (default: `/engine-rest`). + 3. Port of the separate `Undertow` HTTP server for the `REST API` (default: `8081`). + 4. Maximum time to wait for HTTP server [graceful shutdown](container.md#component-lifecycle) (default: `30s`). + 5. Path to the `OpenAPI` file in `resources` (default: `[ "openapi.json" ]`). By default, the file from the [`camunda-engine-rest-openapi` dependency](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi) is used. + 6. Enables the controller that serves the `OpenAPI` file (default: `false`). + 7. Path where the `OpenAPI` file will be available (default: `/openapi`). + 8. Enables the controller that serves `Swagger UI` (default: `false`). + 9. Path where `Swagger UI` will be available (default: `/swagger-ui`). + 10. Enables the controller that serves `RapiDoc` (default: `false`). + 11. Path where `RapiDoc` will be available (default: `/rapidoc`). + 12. Enables the `CORS` filter (default: `false`). + 13. Allowed origin for `CORS` (default: not specified, optional). If the value is not specified, the filter uses the request `Origin` header, and if it is absent, returns `*`. + 14. Allowed headers for `CORS` requests (default: `[ "*" ]`). + 15. Allowed HTTP methods for `CORS` requests (default: `[ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ]`). + 16. Allows credentials in `CORS` requests (default: `true`). + 17. Headers exposed to the client in a `CORS` response (default: `[ "*" ]`). + 18. Maximum caching time for `CORS` preflight requests (default: `1h`). + 19. Enables module logging (default: `false`). + 20. Enables stack trace logging when an exception occurs (default: `true`). + 21. Mask used to hide specified request or response headers and parameters (default: `***`). + 22. List of request parameters to hide in logs (default: `[ ]`). + 23. List of request or response headers to hide in logs (default: `[ "authorization" ]`). + 24. Defines whether the path template is used for logging (default: not specified, optional). If not specified, the full path is used only at the `TRACE` logging level; if `true`, the path template is used; if `false`, the full path is used. + 25. Enables module metrics (default: `true`). + 26. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 27. Additional tags for metrics (default: `{}`). + 28. Enables module tracing (default: `true`). + 29. Additional attributes for tracing (default: `{}`). + +The listing above shows every available option; in practice you enable only what you need. +A typical setup exposes the `REST API` on a custom `port` together with the `OpenAPI` description, `Swagger UI`, and request logging: + +===! ":material-code-json: `Hocon`" + + ```javascript + camunda { + rest { + enabled = true + port = 8090 + openapi { + enabled = true + swaggerui.enabled = true + } + telemetry.logging.enabled = true + } + } ``` - 1. Enable/disable REST API - 2. Prefix path to REST API - 3. Port on which the REST API server will be started - 4. Maximum time to wait for the server to complete after receiving a floating termination signal - 5. Relative path to OpenAPI files in the `resources` directory, default is the `openapi.json` OpenAPI file from [Camunda dependencies](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi) - 6. The on/off switch of the controller that gives the OpenAPI - 7. Path where OpenAPI will be available - 5. If a single OpenAPI file is specified, then represent entire path where file is available - 6. If multiple OpenAPI files are specified, is a path prefix to the file name `/openapi/{fileName}`, taking the specified path and appending the file name to it without the directories and its extension, example of the file `someDirectory/my-openapi-1.yaml` the file path will be `/openapi/my-openapi-1`. - 8. On/Off of the controller that gives SwaggerUI - 9. Path where the SwaggerUI will be accessed - 10. On/Off of the controller that gives Rapidoc - 11. Path where Rapidoc will be available - 12. Enables CORS filter (default `false`) - 13. Allowed origins for CORS (default `null`) - 14. Allowed headers for CORS requests (default `["*"]`) - 15. Allowed HTTP methods for CORS requests (default `["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD"]`) - 16. Allows transmission of credentials in CORS requests (default `true`) - 17. Headers that can be exposed to the client in CORS responses (default `["*"]`) - 18. Maximum caching time for CORS preflight requests (default `1 hour`) - 19. Enables module logging (default `false`) - 20. Enables call stack logging in case of exception - 21. Mask that is used to hide specified headers and request/response parameters - 22. List of request parameters to be hidden - 23. List of request/response headers that should be hidden - 24. Whether to always use the request path template when logging. The default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 25. Enables module metrics (default `true`) - 26. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 27. Configures tags for metrics (optional) - 28. Enables module tracing (default is `true`) - 29. Configures attributes for tracing (optional) +=== ":simple-yaml: `YAML`" + + ```yaml + camunda: + rest: + enabled: true + port: 8090 + openapi: + enabled: true + swaggerui: + enabled: true + telemetry: + logging: + enabled: true + ``` + +## OpenAPI { #openapi } + +Besides the `REST API` itself, the separate server can serve the API's `OpenAPI` description together with +[Swagger UI](https://swagger.io/tools/swagger-ui/) and [RapiDoc](https://rapidocweb.com/) pages. +All three are disabled by default and are enabled independently through the `openapi` configuration section. + +When enabled, the pages are available on the `REST` server `port` at the configured endpoints: + +| Page | Configuration flag | Default endpoint | +|--------------|-----------------------------|------------------| +| OpenAPI spec | `openapi.enabled` | `/openapi` | +| Swagger UI | `openapi.swaggerui.enabled` | `/swagger-ui` | +| RapiDoc | `openapi.rapidoc.enabled` | `/rapidoc` | + +For example, with `port = 8090` and `openapi.enabled = true` the specification is served at `http://localhost:8090/openapi`, +and `Swagger UI` (when enabled) at `http://localhost:8090/swagger-ui`. + +By default the module serves the `OpenAPI` specification bundled with the +[`camunda-engine-rest-openapi`](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi) dependency. +When this bundled specification is used, the module substitutes the configured `port` and `path` into it, +so the served `OpenAPI` always matches the live `REST API` address even when values other than `8081` or `/engine-rest` are configured. + +To serve a custom specification instead, point `openapi.file` at one or more files in `resources`: + +===! ":material-code-json: `Hocon`" + + ```javascript + camunda.rest.openapi { + enabled = true + file = [ "my-openapi.json" ] + } + ``` + +=== ":simple-yaml: `YAML`" + + ```yaml + camunda: + rest: + openapi: + enabled: true + file: [ "my-openapi.json" ] + ``` + +## CORS { #cors } + +The `REST` server has its own [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) filter, disabled by default and enabled through `cors.enabled`. +When `cors.allowOrigin` is not set, the filter reflects the request `Origin` header back in the response, +falling back to `*` when the request carries no `Origin` header. +The remaining `cors.*` options control the allowed headers and methods, whether credentials are allowed, +the headers exposed to the client, and the preflight cache duration. + +## Telemetry { #telemetry } + +Requests handled by the `REST` server are covered by the standard Kora telemetry signals — [logging](logging-slf4j.md), +[metrics](metrics.md), and [tracing](tracing.md) — configured under the `telemetry` section. +Logging is disabled by default (`telemetry.logging.enabled`), while metrics and tracing are enabled by default. + +The `telemetry.logging.pathTemplate` option controls how the request path appears in logs: when it is not set, +the path template is used except at the `TRACE` level, where the full path is logged; +`true` always uses the path template, and `false` always uses the full path. + +Module metrics are described in the [Metrics Reference](metrics.md#camunda-rest) section. + +The default telemetry can be overridden by registering your own `CamundaRestLoggerFactory`, `CamundaRestMetricsFactory`, +or `CamundaRestTracerFactory` component, which replaces the corresponding default provided via `@DefaultComponent`. ## Applications { #applications } -You can register custom `jakarta.ws.rs.core.Application` with resources for APIs (e.g. for other [webapp](https://docs.camunda.org/manual/7.21/webapps/)) by providing them as components in a dependency container. +The module already registers a default `@Tag(CamundaRest.class)` `jakarta.ws.rs.core.Application` that exposes the standard +`Camunda 7 REST API` resources (`CamundaRestResources`) together with a `ResteasyJackson2Provider` for `JSON` serialization. + +To add custom `JAX-RS` resources, register your own `jakarta.ws.rs.core.Application` component marked with the `@Tag(CamundaRest.class)` tag. +All such applications are collected and merged with the default one — their `getClasses()` and `getSingletons()` are combined — +so custom resources are served on the same `REST` server alongside the standard Camunda endpoints. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(CamundaRest.class) + @Component + public final class CustomCamundaApplication extends Application { + + @Override + public Set> getClasses() { + return Set.of(CustomResource.class); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(CamundaRest::class) + @Component + class CustomCamundaApplication : Application() { + + override fun getClasses(): Set> { + return setOf(CustomResource::class.java) + } + } + ``` diff --git a/mkdocs/docs/en/documentation/camunda8-worker.md b/mkdocs/docs/en/documentation/camunda8-worker.md index ad39845..201def9 100644 --- a/mkdocs/docs/en/documentation/camunda8-worker.md +++ b/mkdocs/docs/en/documentation/camunda8-worker.md @@ -1,15 +1,18 @@ --- -description: "Explains Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures. Use when working with @JobWorker, ZeebeClient, ActivatedJob, JobClient, Camunda8WorkerModule, Camunda8WorkerConfig." +description: "Explains Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures. Use when working with @JobWorker, @JobVariable, @JobVariables, ZeebeClient, JobContext, KoraJobWorker, JobWorkerException, ZeebeWorkerModule, ZeebeClientConfig, ZeebeWorkerConfig." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures; key triggers include @JobWorker, ZeebeClient, ActivatedJob, JobClient, Camunda8WorkerModule, Camunda8WorkerConfig." + use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures; key triggers include @JobWorker, @JobVariable, @JobVariables, ZeebeClient, JobContext, KoraJobWorker, JobWorkerException, ZeebeWorkerModule, ZeebeClientConfig, ZeebeWorkerConfig." --- ??? warning "Experimental module" - **Experimental** module is fully working and tested, but requires additional approbation and usage analytics, - for this reason, API may potentially undergo minor changes before fully stable. + The **experimental** module is fully working and tested, but requires additional validation and usage analytics. + For this reason, its `API` may potentially undergo minor changes before becoming fully stable. -Module for connecting a client and creating workers for an external process orchestrator [Camunda 8 (Zeebe)](https://docs.camunda.io/docs/components/concepts/job-workers/) +The module connects a [Camunda 8 (Zeebe)](https://docs.camunda.io/docs/components/concepts/job-workers/) client and +creates job workers for an external process orchestrator. In `Kora`, such a worker is declared as a regular component: +a method annotated with `@JobWorker` receives process variables, performs work, and returns a result that is sent back +to `Zeebe`. ## Dependency { #dependency } @@ -43,7 +46,7 @@ Module for connecting a client and creating workers for an external process orch Example of a complete client configuration described in the `ZeebeClientConfig` class (example values or default values are specified): -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript zeebe { @@ -62,10 +65,10 @@ Example of a complete client configuration described in the `ZeebeClientConfig` attempts = 5 //(10)! delay = "100ms" //(11)! delayMax = "5s" //(12)! - stepFactor = 3.0 //(13)! + step = 3.0 //(13)! } } - http { + rest { url = "http://localhost:8080" //(14)! } deployment { @@ -96,28 +99,28 @@ Example of a complete client configuration described in the `ZeebeClientConfig` } ``` - 1. Maximum number of threads for task workers, by default equal to the number of CPU cores or minimum `2`. - 2. Connection time without reading activity before sending `KeepAlive` check - 3. Whether to use TLS when connecting on a connection - 4. [File path](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) to the certificate file to use when connecting, or use the default system certificate - 5. Maximum time to wait for initialization of workers to start when the service starts (default is none) - 6. URL for connection via gRPC - 7. Time for how long the message should be buffered at the broker over gRPC connection - 8. Maximum message size over gRPC connection - 9. Whether the policy of execution repeat in case of connection error is enabled - 10. Number of attempts - 11. Delay between attempts - 12. maximum duration of retries - 13. Step coefficient for increasing the delay time between attempts - 14. URL for HTTP connection - 15. Paths to find resources that will be loaded into the orchestrator after startup - 16. Maximum time to wait for resources to be loaded - 17. Enables module logging (default is `false`) - 18. Enables module metrics (default `true`) - 19. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 20. Configures tags for metrics (optional) - 21. Enables module tracing (default `true`) - 22. Configures attributes for tracing (optional) + 1. Maximum number of threads for job workers (default: number of CPU cores, but not less than `2`) + 2. Time without read activity before sending a `KeepAlive` check (default: `45s`) + 3. Whether to use `TLS` for the connection (default: `true`) + 4. [File path](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) to the certificate for the connection; if not specified, the system certificate is used (default unspecified, optional) + 5. Maximum time to wait for topology availability check on client startup (default unspecified, optional) + 6. `URL` for connecting through `gRPC` (`required`, default unspecified) + 7. How long the message should be kept on the broker when sent through `gRPC` (default: `1h`) + 8. Maximum inbound message size for `gRPC` (default: `4Mib`) + 9. Whether the retry policy for the `gRPC` connection is enabled (default: `true`) + 10. Number of attempts (default: `5`) + 11. Initial delay between attempts (default: `100ms`) + 12. Maximum delay between attempts (default: `5s`) + 13. Delay multiplier between attempts (default: `3.0`) + 14. `URL` for connecting to the `Zeebe` `REST` address; if specified, the client prefers `REST` over `gRPC` for supported operations (`required` inside the optional `rest` section, default unspecified) + 15. Paths for searching resources that will be uploaded to the orchestrator after startup (default: `[]`) + 16. Maximum time to wait for resource upload (default: `45s`) + 17. Enables module logging (default: `false`) + 18. Enables module metrics (default: `true`) + 19. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 20. Configures tags for metrics (default: `{}`) + 21. Enables module tracing (default: `true`) + 22. Configures attributes for tracing (default: `{}`) === ":simple-yaml: `YAML`" @@ -130,7 +133,7 @@ Example of a complete client configuration described in the `ZeebeClientConfig` certificatePath: "/file/path/to/cert.crt" #(4)! initializationFailTimeout: "15s" #(5)! grpc: - url: "grpc:#localhost:8090" //(6)! + url: "grpc://localhost:8090" #(6)! ttl: "1h" #(7)! maxMessageSize: "4Mib" #(8)! retryPolicy: @@ -138,9 +141,9 @@ Example of a complete client configuration described in the `ZeebeClientConfig` attempts: 5 #(10)! delay: "100ms" #(11)! delayMax: "5s" #(12)! - stepFactor: 3.0 #(13)! - http: - url: "http:#localhost:8080" //(14)! + step: 3.0 #(13)! + rest: + url: "http://localhost:8080" #(14)! deployment: resources: "classpath:bpm" #(15)! timeout: "45s" #(16)! @@ -160,45 +163,190 @@ Example of a complete client configuration described in the `ZeebeClientConfig` key2: value2 ``` - 1. Maximum number of threads for task workers, by default equal to the number of CPU cores or minimum `2`. - 2. Connection time without reading activity before sending `KeepAlive` check - 3. Whether to use TLS when connecting on a connection - 4. [File path](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) to the certificate file to use when connecting, or use the default system certificate - 5. Maximum time to wait for initialization of workers to start when the service starts (default is none) - 6. URL for connection via gRPC - 7. Time for how long the message should be buffered at the broker over gRPC connection - 8. Maximum message size over gRPC connection - 9. Whether the policy of execution repeat in case of connection error is enabled - 10. Number of attempts - 11. Delay between attempts - 12. maximum duration of retries - 13. Step coefficient for increasing the delay time between attempts - 14. URL for HTTP connection - 15. Paths to find resources that will be loaded into the orchestrator after startup - 16. Maximum time to wait for resources to be loaded - 17. Enables module logging (default is `false`) - 18. Enables module metrics (default `true`) - 19. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 20. Configures tags for metrics (optional) - 21. Enables module tracing (default `true`) - 22. Configures attributes for tracing (optional) + 1. Maximum number of threads for job workers (default: number of CPU cores, but not less than `2`) + 2. Time without read activity before sending a `KeepAlive` check (default: `45s`) + 3. Whether to use `TLS` for the connection (default: `true`) + 4. [File path](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) to the certificate for the connection; if not specified, the system certificate is used (default unspecified, optional) + 5. Maximum time to wait for topology availability check on client startup (default unspecified, optional) + 6. `URL` for connecting through `gRPC` (`required`, default unspecified) + 7. How long the message should be kept on the broker when sent through `gRPC` (default: `1h`) + 8. Maximum inbound message size for `gRPC` (default: `4Mib`) + 9. Whether the retry policy for the `gRPC` connection is enabled (default: `true`) + 10. Number of attempts (default: `5`) + 11. Initial delay between attempts (default: `100ms`) + 12. Maximum delay between attempts (default: `5s`) + 13. Delay multiplier between attempts (default: `3.0`) + 14. `URL` for connecting to the `Zeebe` `REST` address; if specified, the client prefers `REST` over `gRPC` for supported operations (`required` inside the optional `rest` section, default unspecified) + 15. Paths for searching resources that will be uploaded to the orchestrator after startup (default: `[]`) + 16. Maximum time to wait for resource upload (default: `45s`) + 17. Enables module logging (default: `false`) + 18. Enables module metrics (default: `true`) + 19. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 20. Configures tags for metrics (default: `{}`) + 21. Enables module tracing (default: `true`) + 22. Configures attributes for tracing (default: `{}`) Module metrics are described in the [Metrics Reference](metrics.md#camunda-8-worker) section. +### Resource deployment { #resource-deployment } + +If `deployment.resources` contains paths, the module finds resources on the classpath during startup and deploys them to +`Zeebe` through the `ZeebeResourceDeployment` component. Both `BPMN` processes and `DMN` decisions found under the +configured locations are deployed. Only paths with the `classpath:` prefix are supported, for example `classpath:bpm`; +other locations are logged and skipped. + +Put the deployable resources under the corresponding classpath directory: + +```text +src/main/resources/ +└── bpm/ + └── demo.bpmn +``` + +===! ":material-code-json: `HOCON`" + + ```javascript + zeebe { + client { + deployment { + resources = "classpath:bpm" //(1)! + } + } + } + ``` + + 1. One or more classpath locations to scan for `BPMN` / `DMN` resources (a single value or a list) + +=== ":simple-yaml: `YAML`" + + ```yaml + zeebe: + client: + deployment: + resources: "classpath:bpm" #(1)! + ``` + + 1. One or more classpath locations to scan for `BPMN` / `DMN` resources (a single value or a list) + +### Client { #client } + +The module creates a `ZeebeClient` component that can be injected into your own services when you need to manually start +processes, publish messages, or execute other `Zeebe` commands. + +For example, to start a new process instance: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ProcessStarter { + + private final ZeebeClient client; + + public ProcessStarter(ZeebeClient client) { + this.client = client; + } + + public void start() { + ProcessInstanceEvent event = client.newCreateInstanceCommand() + .bpmnProcessId("demo") //(1)! + .latestVersion() //(2)! + .variables("{\"startId\":\"42\"}") //(3)! + .send() + .join(); //(4)! + } + } + ``` + + 1. `BPMN` process identifier of the process to start + 2. Start the latest deployed version of the process + 3. Initial process variables as a `JSON` string (a `Map` or a `@Json` object are also accepted) + 4. Send the command and block until `Zeebe` acknowledges it (use the returned `CompletionStage` for a non-blocking call) + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ProcessStarter(private val client: ZeebeClient) { + + fun start() { + val event = client.newCreateInstanceCommand() + .bpmnProcessId("demo") //(1)! + .latestVersion() //(2)! + .variables("""{"startId":"42"}""") //(3)! + .send() + .join() //(4)! + } + } + ``` + + 1. `BPMN` process identifier of the process to start + 2. Start the latest deployed version of the process + 3. Initial process variables as a `JSON` string (a `Map` or a `@Json` object are also accepted) + 4. Send the command and block until `Zeebe` acknowledges it (use the returned `CompletionStage` for a non-blocking call) + +The same client publishes messages (`client.newPublishMessageCommand()`) and executes any other `Zeebe` command. + +#### Client customization { #client-customization } + +The `ZeebeClient` can be tuned with optional graph components that the module picks up automatically: + +* `CredentialsProvider` — authorization for `Zeebe` (`Camunda 8 SaaS` or self-managed with `OAuth`); +* `JsonMapper` — custom `JSON` mapper used by `ZeebeClient` for variable (de)serialization; +* `ScheduledExecutorService` — executor used by job workers; +* `ClientInterceptor` — a `gRPC` interceptor applied to the `Zeebe` channel (all registered interceptors are collected). + +For example, to authenticate against `Camunda 8` with `OAuth`, provide a `CredentialsProvider` bean: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface ZeebeAuthModule { + + default CredentialsProvider zeebeCredentialsProvider() { + return CredentialsProvider.newCredentialsProviderBuilder() + .clientId("client-id") + .clientSecret("client-secret") + .audience("zeebe.camunda.io") + .authorizationServerUrl("https://login.cloud.camunda.io/oauth/token") + .build(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface ZeebeAuthModule { + + fun zeebeCredentialsProvider(): CredentialsProvider = + CredentialsProvider.newCredentialsProviderBuilder() + .clientId("client-id") + .clientSecret("client-secret") + .audience("zeebe.camunda.io") + .authorizationServerUrl("https://login.cloud.camunda.io/oauth/token") + .build() + } + ``` + ## Worker { #worker } Worker is a handler that can perform a specific job in a process. -Each time such a job needs to be performed, it is polled by worker. +When a process contains a job of the required type, `Zeebe` activates it and passes it to one of the workers. ### Configuration { #configuration-2 } -There is a default configuration that is applied to all workers at creation -and then the named worker-specific settings ([by `Type`](https://docs.camunda.io/docs/components/concepts/job-workers/)) are then applied overriding the default settings. -You can change the default settings for all interrupters at the same time by changing the default configuration (`default`). +There is a default configuration that is applied to all workers on creation, and then named settings for a concrete +worker are applied on top of it by [worker type (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/). +To change settings for all workers at once, override the `default` section. +To change settings only for one worker, add a section with the type name specified in `@JobWorker`. +If the `zeebe.worker.job` section is not specified, the built-in default configuration is used. -Example of a complete worker configuration is described in the `ZeebebeWorkerConfig` class (example values or default values are specified): +Example of a complete worker configuration described in the `ZeebeWorkerConfig` class (example values or default values are specified): -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript zeebe { @@ -217,7 +365,7 @@ Example of a complete worker configuration is described in the `ZeebebeWorkerCon minDelay = "100ms" //(11)! maxDelay = "500ms" //(12)! factor = 1.0 //(10)! - jitter = 1.3 //(13)! + jitter = 1.1 //(13)! } } } @@ -225,20 +373,19 @@ Example of a complete worker configuration is described in the `ZeebebeWorkerCon } ``` - 1. [Worker (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) or the name of the default settings (`default`) - 2. Whether to include an worker - 3. Maximum time for an worker to complete a single task - 4. The maximum number of tasks that will be activated simultaneously for this worker only. This is used to control the speed of the data producer to match the speed of the worker (`backpressure`) - 5. Limitation on the query time used to poll a new task by the worker - 6. Maximum interval between polling of new tasks. The worker automatically tries to always activate new tasks after the job is finished. If no task can be activated after completion, the performer will poll new tasks periodically - 7. Specifies the tenant identifiers that can own any entities (e.g., process definition, process instances, etc.) resulting from the execution of this command - 8. If set to enabled, the worker will use a combination of streaming and polling to activate jobs - 9. If streaming is enabled, sets the maximum lifetime for this thread - 10. Sets the minimum repetition delay. Note that due to `jitter`, the repeat delay may be lower than this minimum - 11. Sets the maximum repeat delay. Note that `jitter` may exceed this maximum delay - 12. Sets the delay multiplication factor. The previous delay is multiplied by this factor - 13. Sets the jitter coefficient. The next delay is varied randomly within the range +/- of this coefficient. - For example, if the next delay is calculated as 1s and `jitter` is 0.1, the actual next delay may be somewhere between 0.9 and 1.1s + 1. [Worker type (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) or the default settings name `default` + 2. Whether the worker is enabled (default: `true`) + 3. Maximum time for one job execution by the worker (default: `15m`) + 4. Maximum number of jobs that will be activated simultaneously for this worker; used to align job fetching speed with processing speed (`backpressure`) (default: `32`) + 5. Request timeout used for polling a new job by the worker (default: `15s`) + 6. Maximum interval between polling new jobs; if no jobs are activated after work is completed, the worker periodically polls the broker (default: `100ms`) + 7. `tenant` identifiers for which the worker can receive jobs (default: `[]`) + 8. Whether to use streaming together with polling for job activation (default: `false`) + 9. Maximum stream lifetime when streaming is enabled (default: `15s`) + 10. Minimum retry delay; due to `jitter`, the actual delay can be lower than this minimum (default: `100ms`) + 11. Maximum retry delay; due to `jitter`, the actual delay can exceed this value (default: `500ms`) + 12. Delay multiplication factor: the previous delay is multiplied by this value (default: `1.0`) + 13. `jitter` factor: the next delay is randomly changed within the `+/-` range of this factor (default: `1.1`) === ":simple-yaml: `YAML`" @@ -259,29 +406,75 @@ Example of a complete worker configuration is described in the `ZeebebeWorkerCon minDelay: "100ms" #(11)! maxDelay: "500ms" #(12)! factor: 1.0 #(10)! - jitter: 1.3 #(13)! - ``` - - 1. [Worker (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) or the name of the default settings (`default`) - 2. Whether to include an worker - 3. Maximum time for an worker to complete a single task - 4. The maximum number of tasks that will be activated simultaneously for this worker only. This is used to control the speed of the data producer to match the speed of the worker (`backpressure`) - 5. Limitation on the query time used to poll a new task by the worker - 6. Maximum interval between polling of new tasks. The worker automatically tries to always activate new tasks after the job is finished. If no task can be activated after completion, the performer will poll new tasks periodically - 7. Specifies the tenant identifiers that can own any entities (e.g., process definition, process instances, etc.) resulting from the execution of this command - 8. If set to enabled, the worker will use a combination of streaming and polling to activate jobs - 9. If streaming is enabled, sets the maximum lifetime for this thread - 10. Sets the minimum repetition delay. Note that due to `jitter`, the repeat delay may be lower than this minimum - 11. Sets the maximum repeat delay. Note that `jitter` may exceed this maximum delay - 12. Sets the delay multiplication factor. The previous delay is multiplied by this factor - 13. Sets the jitter coefficient. The next delay is varied randomly within the range +/- of this coefficient. - For example, if the next delay is calculated as 1s and `jitter` is 0.1, the actual next delay may be somewhere between 0.9 and 1.1s + jitter: 1.1 #(13)! + ``` + + 1. [Worker type (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) or the default settings name `default` + 2. Whether the worker is enabled (default: `true`) + 3. Maximum time for one job execution by the worker (default: `15m`) + 4. Maximum number of jobs that will be activated simultaneously for this worker; used to align job fetching speed with processing speed (`backpressure`) (default: `32`) + 5. Request timeout used for polling a new job by the worker (default: `15s`) + 6. Maximum interval between polling new jobs; if no jobs are activated after work is completed, the worker periodically polls the broker (default: `100ms`) + 7. `tenant` identifiers for which the worker can receive jobs (default: `[]`) + 8. Whether to use streaming together with polling for job activation (default: `false`) + 9. Maximum stream lifetime when streaming is enabled (default: `15s`) + 10. Minimum retry delay; due to `jitter`, the actual delay can be lower than this minimum (default: `100ms`) + 11. Maximum retry delay; due to `jitter`, the actual delay can exceed this value (default: `500ms`) + 12. Delay multiplication factor: the previous delay is multiplied by this value (default: `1.0`) + 13. `jitter` factor: the next delay is randomly changed within the `+/-` range of this factor (default: `1.1`) + +To override settings for a single worker, add a section keyed by the [worker type (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) +declared in `@JobWorker`. A named section is merged over `default`, which in turn is merged over the built-in defaults, +so a named section only needs to list the keys it changes. Setting `enabled = false` on a named type disables just that +one worker. + +===! ":material-code-json: `HOCON`" + + ```javascript + zeebe { + worker { + job { + foo { //(1)! + timeout = "30s" + maxJobsActive = 8 + } + bar { //(2)! + enabled = false + } + } + } + } + ``` + + 1. Overrides only `timeout` and `maxJobsActive` for the `@JobWorker("foo")` worker; all other settings come from `default` + 2. Disables the `@JobWorker("bar")` worker while leaving the rest of the configuration untouched + +=== ":simple-yaml: `YAML`" + + ```yaml + zeebe: + worker: + job: + foo: #(1)! + timeout: "30s" + maxJobsActive: 8 + bar: #(2)! + enabled: false + ``` + + 1. Overrides only `timeout` and `maxJobsActive` for the `@JobWorker("foo")` worker; all other settings come from `default` + 2. Disables the `@JobWorker("bar")` worker while leaving the rest of the configuration untouched ### Declarative { #declarative } -You can create declaratively [JobWorkers](https://docs.camunda.io/docs/components/concepts/job-workers/) that will perform work within the Zeebe orchestrator. +You can declaratively create [workers](https://docs.camunda.io/docs/components/concepts/job-workers/) that perform work +within the `Zeebe` orchestrator. -`JobWorker` annotation specifies the value of the [worker type (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) within the process. +The `@JobWorker` annotation specifies the [worker type (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) +from the process. `Zeebe` uses this value to connect a job from a `BPMN` process with a handler in the application. + +A worker method may only declare `@JobVariable`, `@JobVariables`, and `JobContext` parameters — any other parameter type +is rejected at compile time. The raw `JobClient` and `ActivatedJob` are available only in the [imperative](#imperative) worker. ===! ":fontawesome-brands-java: `Java`" @@ -311,8 +504,8 @@ You can create declaratively [JobWorkers](https://docs.camunda.io/docs/component #### Parameter context { #parameter-context } -You can embed the job context as a method argument. -Job Context has task, worker and process metadata available for the current task being executed. +You can inject the job context as a method argument. +`JobContext` contains metadata of the current job, worker, and process. ===! ":fontawesome-brands-java: `Java`" @@ -340,12 +533,62 @@ Job Context has task, worker and process metadata available for the current task } ``` +`JobContext` exposes the following read-only accessors: + +| Method | Description | +|------------------------------|--------------------------------------------------------------------------------------| +| `jobKey()` | Unique key of the activated job | +| `jobName()` | Worker name/type this handler is registered under (the `@JobWorker` value) | +| `jobType()` | Job type of the activated job as defined in the `BPMN` process | +| `jobWorker()` | Name of the worker that activated the job on the broker side | +| `tenantId()` | Tenant identifier the job belongs to | +| `processId()` | `BPMN` process identifier | +| `processInstanceKey()` | Key of the process instance the job belongs to | +| `processDefinitionVersion()` | Version of the deployed process definition | +| `processDefinitionKey()` | Key of the deployed process definition | +| `elementId()` | Identifier of the `BPMN` element the job was created for | +| `elementInstanceKey()` | Key of the `BPMN` element instance | +| `headers()` | Custom headers defined on the job in the `BPMN` model | +| `retryCount()` | Number of remaining retries for the job | +| `deadline()` | Moment (`Instant`) until which the job is exclusively assigned to the worker | +| `deadlineAsMillis()` | Same deadline expressed as epoch milliseconds | +| `variablesAsString()` | Raw job variables as a `JSON` string | + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeJob { + + @JobWorker("someJobType") + public void process(JobContext context) { + logger.info("Job {} of process {} at element {} with deadline {}", + context.jobType(), context.processInstanceKey(), context.elementId(), context.deadline()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeJob { + + @JobWorker("someJobType") + fun process(context: JobContext) { + logger.info("Job {} of process {} at element {} with deadline {}", + context.jobType(), context.processInstanceKey(), context.elementId(), context.deadline()) + } + } + ``` + #### Parameter variable { #parameter-variable } -You can embed [process variables](https://docs.camunda.io/docs/components/concepts/variables/) as method arguments, -a process variable is part of the process state and can be set on start or as part of the worker result. +You can inject [process variables](https://docs.camunda.io/docs/components/concepts/variables/) as method arguments. +A process variable is part of the process state and can be set on process start or as part of the worker result. -Importantly, if any named variables are specified, only those variables will be passed to receive from the orchestrator. +If at least one variable is specified through `@JobVariable`, the generated worker asks `Zeebe` only for those variables. +If `@JobVariable` is not used, the worker asks for all job variables. ===! ":fontawesome-brands-java: `Java`" @@ -373,10 +616,10 @@ Importantly, if any named variables are specified, only those variables will be } ``` -You can specify a variable name from context, or the default method argument name will be used. +You can specify the variable name explicitly in `@JobVariable`, or the method argument name will be used by default. -Since all process variables are required to be JSON objects, -the method argument can also be any mapping of a JSON object. +Since process variables are passed as `JSON`, the method argument can be a user type that has `JsonReader` and `JsonWriter` +available. ===! ":fontawesome-brands-java: `Java`" @@ -411,8 +654,8 @@ the method argument can also be any mapping of a JSON object. #### Parameter variables { #parameter-variables } -You can embed multiple [process variables](https://docs.camunda.io/docs/components/concepts/variables/) at once as a method argument, -as a single object that represents JSON objects in the process state. +You can inject multiple [process variables](https://docs.camunda.io/docs/components/concepts/variables/) as one method +argument through `@JobVariables`. This argument represents all job variables as one `JSON` object. ===! ":fontawesome-brands-java: `Java`" @@ -452,9 +695,9 @@ as a single object that represents JSON objects in the process state. #### Result { #result } -You can also execute a job with some result of the job execution and pass it as a variable to process context. +You can not only execute work, but also return the result as variables to the process context. -The result can be returned as a `Map` describing the JSON structure of the response. +The result can be returned as a `Map` that describes the `JSON` response structure. ===! ":fontawesome-brands-java: `Java`" @@ -482,8 +725,8 @@ The result can be returned as a `Map` describing the JSON struct } ``` -Or return the named result as a single variable at once, -which will be analogous to a single key and value in a `Map` object. +You can also return a named result as a single variable. This is equivalent to one key and value in a +`Map` object. In this case, it is obligatory to specify the name of the variable in the `@JobVariable` annotation: @@ -522,8 +765,20 @@ In this case, it is obligatory to specify the name of the variable in the `@JobV #### Errors { #errors } -In case you need to terminate execution with an error, you can throw a `JobWorkerException` exception where you can specify, -both the error code and the message and process variables if required. +If you need to complete execution with a process error, throw `JobWorkerException`. +The exception can contain an error code, message, and process variables if they are required. +This exception is converted to a `throwError` command for `Zeebe`: the `getCode()`, message, and `getVariables()` +of the exception are sent as the error code, error message, and variables of the command. + +If the handler throws any other exception, the module wraps it into a `JobWorkerException` with one of the following +built-in codes: + +| Code | When it is used | +|-------------------|-----------------------------------------------------------------------------| +| `DESERIALIZATION` | A job variable could not be read/deserialized into a method argument | +| `SERIALIZATION` | The worker result could not be written/serialized into variables | +| `UNEXPECTED` | An unexpected error was thrown from a synchronous handler | +| `INTERNAL` | Fallback code for any other error not covered above | ===! ":fontawesome-brands-java: `Java`" @@ -533,11 +788,13 @@ both the error code and the message and process variables if required. @JobWorker("someJobType") public User process() { - throw new JobWorkerException("DOESNT_WORK"); + throw new JobWorkerException("DOESNT_WORK"); //(1)! } } ``` + 1. Additional overloads accept a message/cause and a `Map` of variables to attach to the `throwError` command + === ":simple-kotlin: `Kotlin`" ```kotlin @@ -546,14 +803,17 @@ both the error code and the message and process variables if required. @JobWorker("someJobType") fun process(): User { - throw JobWorkerException("DOESNT_WORK") + throw JobWorkerException("DOESNT_WORK") //(1)! } } ``` -### Imperative. { #imperative } + 1. Additional overloads accept a message/cause and a `Map` of variables to attach to the `throwError` command + +### Imperative { #imperative } -You can also create more low-level workers and work directly with `ZeebeClient` contracts and its interface. +You can also create lower-level workers and work directly with `ZeebeClient` contracts. +To do that, the component must implement the `KoraJobWorker` interface. ===! ":fontawesome-brands-java: `Java`" @@ -566,34 +826,51 @@ You can also create more low-level workers and work directly with `ZeebeClient` return "someJobType"; } + @Override + public List fetchVariables() { + return List.of("startId"); //(1)! + } + @Override public CompletionStage> handle(JobClient client, ActivatedJob job) { - return client.newCompleteCommand(job); + return CompletableFuture.completedFuture(client.newCompleteCommand(job)); } } ``` + 1. Only these variables are fetched from `Zeebe`; return an empty list (the default) to fetch **all** variables + === ":simple-kotlin: `Kotlin`" ```kotlin @Component class SomeJob : KoraJobWorker { - fun type(): String = "someJobType" + override fun type(): String = "someJobType" - fun handle(client: JobClient, job: ActivatedJob): CompletionStage> { - return client.newCompleteCommand(job) + override fun fetchVariables(): List = listOf("startId") //(1)! + + override fun handle(client: JobClient, job: ActivatedJob): CompletionStage> { + return CompletableFuture.completedFuture(client.newCompleteCommand(job)) } } ``` + 1. Only these variables are fetched from `Zeebe`; return an empty list (the default) to fetch **all** variables + +The `fetchVariables()` method is the imperative analogue of `@JobVariable`: it controls which process variables `Zeebe` +sends with the job. By default it returns an empty list, which fetches all variables; returning a non-empty list limits +the payload to just those variables. Unlike declarative workers, `handle` receives the raw `JobClient` and `ActivatedJob` +and is responsible for completing the job (for example with `client.newCompleteCommand(job)`). + ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Available signatures for worker methods out of the box: ===! ":fontawesome-brands-java: `Java`" The `T` refers to the type of the return value or `Void`. + If the result is `null` or `Optional.empty()`, the job is completed without adding variables. - `T myMethod()` - `Optional myMethod()` @@ -603,6 +880,7 @@ Available signatures for repository methods out of the box: === ":simple-kotlin: `Kotlin`" By `T` we mean the type of the return value, either `T?` or `Unit`. + If the result is `null`, the job is completed without adding variables. - `myMethod(): T` - - `suspend myMethod(): Deferred` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (add [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `myMethod(): Deferred` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (add [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) diff --git a/mkdocs/docs/en/documentation/config.md b/mkdocs/docs/en/documentation/config.md index 5510c02..8e30d94 100644 --- a/mkdocs/docs/en/documentation/config.md +++ b/mkdocs/docs/en/documentation/config.md @@ -4,14 +4,22 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora configuration system for HOCON and YAML, typed config extraction, config injection, config sources, watchers, and supported value types; key triggers include @ConfigSource, @ConfigValueExtractor, @Environment, @SystemProperties, Config, HoconConfigModule, YamlConfigModule." --- -Module is responsible for mapping the values of configuration files to classes in Kora and then using them for application settings. +The configuration module reads application settings from `HOCON` or `YAML` files, environment variables, `Java` system +properties, and maps them to typed classes in `Kora`. The resulting configuration objects become regular dependency +graph components and can be injected into services, clients, servers, and other integrations. + +In `Kora`, application configuration is usually described by an interface annotated with `@ConfigSource`: the path in +the file points to the section to read, and the interface methods describe required values, optional values, and defaults. +Libraries and reusable configuration shapes use `@ConfigValueExtractor`, which creates only the extraction rule, while +the concrete path is selected in the library module. For a step-by-step walkthrough before the reference details, see [HOCON Configuration](../guides/config-hocon.md) and [YAML Configuration](../guides/config-yaml.md). ## HOCON { #hocon } Support for [HOCON](https://github.com/lightbend/config/blob/master/HOCON.md) is implemented with [Typesafe Config](https://github.com/lightbend/config). -HOCON is a JSON-based config file format. The format is less strict than JSON and has a slightly different syntax. +`HOCON` is a `JSON`-based configuration file format. It is less strict than `JSON` and supports substitutions, defaults, +and a convenient syntax for nested objects. ```javascript services { @@ -49,16 +57,20 @@ services { 1. String configuration value 2. Numeric configuration value -3. Mandatory configuration value that is substituted from the `REQUIRED_ENV_VALUE` environment variable. -4. Optional configuration value which is substituted from the `OPTIONAL_ENV_VALUE` environment variable, if no such variable is found, the configuration value will be omitted. 5. -5. Configuration value with default value, the default value is specified in `propDefault = 10` and if `NON_DEFAULT_ENV_VALUE` environment variable is found, its value will replace the default value. -6. Configuration value assembled from substitutions of other parts of the configuration and the `Other` value between the -7. String list configuration value, the value is set as an array of strings or can also be set as a string with values separated by commas -8. String list configuration value, the value is set as a string with values separated by commas or can also be set as an array of strings -9. Configuration value as a dictionary key and value +3. Required configuration value substituted from the `REQUIRED_ENV_VALUE` environment variable +4. Optional configuration value substituted from the `OPTIONAL_ENV_VALUE` environment variable; if the variable is not found, the configuration value is omitted +5. Configuration value with a default: the default is specified as `propDefault = 10`, and `NON_DEFAULT_ENV_VALUE`, if found, replaces it +6. Configuration value assembled from substitutions of other configuration parts with the `Other` value between them +7. String list configuration value; the value can be set as an array of strings or as a comma-separated string +8. String list configuration value; the value can be set as a comma-separated string or as an array of strings +9. Configuration value as a key-value dictionary 10. Configuration value as a mapped class 11. Configuration value as a list of mapped classes +Values can also reference other configuration keys (self-reference / cross-reference) via `${path}`, and environment +variables via `${VAR}` (required), `${?VAR}` (optional), or a default fallback. All substitutions are resolved after +every layer is merged, so a reference can point at a key defined in another file or in another configuration layer. + Configuration representation in code: ===! ":fontawesome-brands-java: `Java`" @@ -168,19 +180,44 @@ Configuration representation in code: ### File { #file } -By default, the configuration files [reference.conf and application.conf](https://github.com/lightbend/config#note-about-resolving-substitutions-in-referenceconf-and-applicationconf) are expected. +By default, the [`reference.conf` and `application.conf`](https://github.com/lightbend/config#note-about-resolving-substitutions-in-referenceconf-and-applicationconf) configuration files are expected. -First, all `reference.conf` files are merged, second, the `application.conf` file is overlaid on the unresolved -`reference.conf` file, the result is calculated and checked that all variable values are available. +First, all `reference.conf` files from the classpath are merged, then `application.conf` is overlaid on top of the +unresolved `reference.conf`, and after that the result is resolved and required substitutions are checked. -It is assumed that the application configuration is in `application.conf` and the library configurations are in `reference.conf`. +The application configuration is expected to be in `application.conf`, while library configuration is expected to be in `reference.conf`. -Prioritize reading the `application.conf` configuration file: +`HOCON` also supports the [`include`](https://github.com/lightbend/config/blob/master/HOCON.md#includes) directive: +files pulled in through `include` participate in the same merge and substitution resolution as the main file, +and are tracked by the [Config Watcher](#config-watcher) so that changes in an included file also refresh the graph. -- Use the file from `config.resource` if specified (file from `resources` directory) +Application file selection priority for `HOCON`: + +- Use the file from `config.resource` if specified (file from the `resources` directory) - Use the file from `config.file` if specified (file from the file system) -- Use the `application.conf` file if available (file from `resources` directory) -- Use an empty configuration file if none of the above is present +- Use `application.conf` if present (file from the `resources` directory) +- Use an empty configuration if none of the above is present + +Only one property can be specified at the same time: `config.resource` or `config.file`. If both properties are specified, +the application will fail on startup. + +===! ":fontawesome-brands-java: `java`" + + Example of specifying configuration on startup through `java`: + ```shell + java -Dconfig.file=path/to/configFile application + ``` + +=== ":simple-kotlin: `gradle`" + + Example of specifying configuration in `build.gradle`: + ```groovy + run { + jvmArgs += [ + "-Dconfig.file=path/to/configFile" + ] + } + ``` ## YAML { #yaml } @@ -212,13 +249,13 @@ services: 1. String configuration value 2. Numeric configuration value -3. Mandatory configuration value that is substituted from the `REQUIRED_ENV_VALUE` environment variable. -4. Optional configuration value which is substituted from the `OPTIONAL_ENV_VALUE` environment variable, if no such variable is found, the configuration value will be omitted. 5. -5. Configuration value with default value, the default value is specified as `10` and if `NON_DEFAULT_ENV_VALUE` environment variable is found, its value will replace the default value. -6. Configuration value assembled from substitutions of other parts of the configuration and the `Other` value between the -7. String list configuration value, the value is set as an array of strings or can also be set as a string with values separated by commas -8. String list configuration value, the value is set as a string with values separated by commas or can also be set as an array of strings -9. Configuration value as a dictionary key and value +3. Required configuration value substituted from the `REQUIRED_ENV_VALUE` environment variable +4. Optional configuration value substituted from the `OPTIONAL_ENV_VALUE` environment variable; if the variable is not found, the configuration value is omitted +5. Configuration value with a default: the default is `10`, and `NON_DEFAULT_ENV_VALUE`, if found, replaces it +6. Configuration value assembled from substitutions of other configuration parts with the `Other` value between them +7. String list configuration value; the value can be set as an array of strings or as a comma-separated string +8. String list configuration value; the value can be set as a comma-separated string or as an array of strings +9. Configuration value as a key-value dictionary 10. Configuration value as a mapped class 11. Configuration value as a list of mapped classes @@ -329,30 +366,53 @@ Configuration representation in code: interface Application : YamlConfigModule ``` -#### File { #file-2 } +### File { #file-2 } By default, the `reference.yaml` and `application.yaml` configuration files are expected. -First, all `reference.yaml` files are merged, second, the `application.yaml` file is overlaid on an unresolved -`reference.yaml` file, the result is calculated and it is checked that all variable values are available. +First, all `reference.yaml` files from the classpath are merged, then `application.yaml` is overlaid on top of +`reference.yaml`, and after that the result is resolved and required substitutions are checked. -It is assumed that the application configuration is in the `application.yaml` file and the library configurations are in `reference.yaml`. +The application configuration is expected to be in `application.yaml`, while library configuration is expected to be in `reference.yaml`. -Prioritize reading the `application.yaml` configuration file: +Application file selection priority for `YAML`: -- Use the file from `config.resource` if specified (file from `resources` directory) +- Use the file from `config.resource` if specified (file from the `resources` directory) - Use the file from `config.file` if specified (file from the file system) -- Use the `application.yaml` file if available (file from `resources` directory) -- Use an empty configuration file if none of the above is present +- Use `application.yaml` if present (file from the `resources` directory) +- Use an empty configuration if none of the above is present + +Only one property can be specified at the same time: `config.resource` or `config.file`. If both properties are specified, +the application will fail on startup. + +===! ":fontawesome-brands-java: `java`" + + Example of specifying configuration on startup through `java`: + ```shell + java -Dconfig.file=path/to/configFile application + ``` + +=== ":simple-kotlin: `gradle`" + + Example of specifying configuration in `build.gradle`: + ```groovy + run { + jvmArgs += [ + "-Dconfig.file=path/to/configFile" + ] + } + ``` ## Custom configuration { #custom-configuration } -A custom configuration provides a mapping of the configuration file to a user interface. -Such a user interface can later be injected as a dependency along with other components. +A custom configuration maps a configuration file section to a user type. +That type can then be injected as a dependency just like any other component. ### Application config { #application-config } -In order to simplify the creation of custom configurations, the `@ConfigSource` annotation should be used: +Use the `@ConfigSource` annotation to create custom configurations in an application. +It generates a `ConfigValueExtractor` for the interface and a module that adds the ready configuration object to the +dependency graph. The annotation value points to the section path inside the resulting configuration: ===! ":fontawesome-brands-java: `Java`" @@ -425,10 +485,16 @@ After that, the `FooServiceConfig` class can already be used as a dependency in ### Library config { #library-config } -In order to create custom configurations within custom libraries, use the `@ConfigValueExtractor` annotation -which will create rules for processing a configuration file into an instance of a configuration class. +Use the `@ConfigValueExtractor` annotation to create custom configurations in libraries. +It creates a rule for extracting a value from `ConfigValue`, but does not bind it to a concrete configuration path. +The path is selected in a library module factory method, so the same configuration shape can be reused for different sections. +`@ConfigValueExtractor` can be used on a `Java` interface, `record`, or class, and on a `Kotlin` interface or `data class`. -Let's consider an example when there is such a configuration class: +The annotation has the `mapNullAsEmptyObject` parameter (default: `true`). When enabled, a missing section is treated +as an empty object: required fields still fail, while optional fields and defaults behave as if an empty section was present. +If `mapNullAsEmptyObject = false`, a missing section is treated as `null` for the whole configuration object. + +Consider this configuration class: ===! ":fontawesome-brands-java: `Java`" @@ -454,7 +520,7 @@ Let's consider an example when there is such a configuration class: } ``` -In order for the library to provide configuration, you need to implement the factory in a module: +For the library to provide configuration, implement a factory in a module: ===! ":fontawesome-brands-java: `Java`" @@ -500,11 +566,13 @@ The factory will expect a configuration of the following kind: baz: 10 ``` -Then by connecting the `FooLibraryModule` module in the application, the `FooServiceConfig` config can be used as a dependency in other classes. +Then, after connecting `FooLibraryModule` in the application, `FooLibraryConfig` can be used as a dependency in other classes. ### Required values { #required-values } -By default, all values declared in the config are considered **required** (*NotNull*) and must be present in the configuration file. +By default, all values declared in the configuration are considered **required** (`NotNull`) and must be present in the +resulting configuration. If a required value is missing or has the `null` value, the application will fail while creating +the configuration object. ### Optional values { #optional-values } @@ -525,11 +593,11 @@ If you need to specify a value from the configuration file as optional, you can } ``` - 1. Any `@Nullable` annotation will do, such as `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / etc. + 1. Any `@Nullable` annotation will do, for example `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - It is expected to use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark such a parameter as Nullable: + Use [`Kotlin` null-safety](https://kotlinlang.org/docs/null-safety.html) syntax and mark the parameter as nullable: ```kotlin @ConfigSource("services.foo") @@ -541,9 +609,12 @@ If you need to specify a value from the configuration file as optional, you can } ``` +An `Optional` return type is also supported (an absent value maps to `Optional.empty()`), but a `@Nullable` value +(or a `Kotlin` nullable type) is the recommended style. + ### Default values { #default-values } -If there is a need to use default values in a class, you can use this format: +If you need to set a default value in configuration mapping, use a `default` method: ===! ":fontawesome-brands-java: `Java`" @@ -573,19 +644,157 @@ If there is a need to use default values in a class, you can use this format: } ``` +### Relaxed key names { #relaxed-key-names } + +Configuration keys are matched with relaxed naming. A method name is compared against the key in the file not only in +its exact form, but also in its `kebab-case` and `snake_case` variants. This means a method `someBarString()` resolves +equally from `someBarString`, `some-bar-string`, or `some_bar_string` in the configuration file, so teams that prefer +kebab-case or snake_case keys can keep their style without renaming methods. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @ConfigValueExtractor + public interface BarConfig { + + String someBarString(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @ConfigValueExtractor + interface BarConfig { + + fun someBarString(): String + } + ``` + +All three key spellings below are read into `someBarString()`: + +===! ":material-code-json: `Hocon`" + + ```javascript + bar { + someBarString = "value" //(1)! + # some-bar-string = "value" //(2)! + # some_bar_string = "value" //(3)! + } + ``` + + 1. Exact `camelCase` spelling of the method name + 2. Relaxed `kebab-case` spelling + 3. Relaxed `snake_case` spelling + +=== ":simple-yaml: `YAML`" + + ```yaml + bar: + someBarString: "value" #(1)! + # some-bar-string: "value" #(2)! + # some_bar_string: "value" #(3)! + ``` + + 1. Exact `camelCase` spelling of the method name + 2. Relaxed `kebab-case` spelling + 3. Relaxed `snake_case` spelling + +### Recommended style { #recommended-configuration-style } + +It is usually more convenient to describe configuration as a separate type for a concrete integration or subsystem: +an HTTP client, an external service connection, a queue handler, and so on. Such a type should clearly separate required +values, optional values, and values that come from environment variables. + +In the example below: + +1. `baseUrl` is a required value from the configuration file +2. `clientName` is an optional value from the `ORDERS_CLIENT_NAME` environment variable +3. `token` is a required value from the `ORDERS_API_TOKEN` environment variable +4. `requestTimeout` has the `2s` default value and can be overridden by the optional `ORDERS_REQUEST_TIMEOUT` environment variable + +===! ":fontawesome-brands-java: `Java`" + + ```java + import java.time.Duration; + import javax.annotation.Nullable; + + @ConfigSource("clients.orders") + public interface OrdersClientConfig { + + String baseUrl(); + + @Nullable + String clientName(); + + String token(); + + Duration requestTimeout(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import java.time.Duration + + @ConfigSource("clients.orders") + interface OrdersClientConfig { + + fun baseUrl(): String + + fun clientName(): String? + + fun token(): String + + fun requestTimeout(): Duration + } + ``` + +===! "`HOCON`" + + ```javascript + clients { + orders { + baseUrl = "https://orders.example.com" + clientName = ${?ORDERS_CLIENT_NAME} + token = ${ORDERS_API_TOKEN} + requestTimeout = 2s + requestTimeout = ${?ORDERS_REQUEST_TIMEOUT} + } + } + ``` + +=== "`YAML`" + + ```yaml + clients: + orders: + baseUrl: "https://orders.example.com" + clientName: ${?ORDERS_CLIENT_NAME} + token: ${ORDERS_API_TOKEN} + requestTimeout: ${?ORDERS_REQUEST_TIMEOUT:2s} + ``` + +This keeps the configuration structure readable: required settings are visible in the configuration type, secrets can be +passed through environment variables, and safe defaults stay directly in the configuration file. + ## Injecting configuration { #injecting-configuration } -You can inject the base class `ru.tinkoff.kora.config.common.Config` which provides a common abstraction over the -configuration file mapping. The resulting configuration mapping consists of several layers that represent: +You can inject the base class `ru.tinkoff.kora.config.common.Config`, which represents the configuration tree and gives +access to values through the `get(...)` method. The resulting configuration consists of several layers: - Environment variables -- System variables +- `Java` system properties - Configuration file -#### Environment variables { #environment-variables } +Layers are merged in this order: environment variables, then system properties, then the application configuration file. +Each next layer overlays the previous one. -In case you want to embed the configuration **only** [environment variables](https://ru.hexlet.io/courses/cli-basics/lessons/environment-variables/theory_unit), -you can use the `@Environment` annotation as a tag for the configuration class: +### Environment variables { #environment-variables } + +If you need to inject configuration that contains **only** [environment variables](https://en.wikipedia.org/wiki/Environment_variable), +use the `@Environment` annotation as a tag for the configuration class: ===! ":fontawesome-brands-java: `Java`" @@ -608,10 +817,10 @@ you can use the `@Environment` annotation as a tag for the configuration class: class FooService(@Environment val config: Config) ``` -### System variables { #system-variables } +### System properties { #system-variables } -In case you want to inject a configuration of **only** [system variables](https://www.baeldung.com/java-system-get-property-vs-system-getenv), -then you can use the `@SystemProperties` annotation as a tag for the configuration class: +If you need to inject configuration that contains **only** [`Java` system properties](https://www.baeldung.com/java-system-get-property-vs-system-getenv), +use the `@SystemProperties` annotation as a tag for the configuration class: ===! ":fontawesome-brands-java: `Java`" @@ -636,8 +845,8 @@ then you can use the `@SystemProperties` annotation as a tag for the configurati ### Configuration file { #configuration-file } -In case you want to inject a complete application configuration that consists **only** of a configuration file, -you can use the `@ApplicationConfig` annotation as a tag for the configuration class: +If you need to inject application configuration that consists **only** of the configuration file, +use the `@ApplicationConfig` annotation as a tag for the configuration class: ===! ":fontawesome-brands-java: `Java`" @@ -662,8 +871,8 @@ you can use the `@ApplicationConfig` annotation as a tag for the configuration c ### Resulting configuration { #resulting-configuration } -If you want to inject a complete application configuration that consists of a configuration file, -environment variables and system variables, you simply inject the configuration class without the tag: +If you need to inject the complete resulting application configuration, which consists of the configuration file, +environment variables and system properties, simply inject the configuration class without a tag: ===! ":fontawesome-brands-java: `Java`" @@ -686,28 +895,74 @@ environment variables and system variables, you simply inject the configuration class FooService(val config: Config) ``` -### Recommendations { #recommendations } +### Reading raw Config values { #reading-raw-config-values } + +When a raw `Config` is injected, values are read through the `get(...)` method, which returns a `ConfigValue` node +for the requested path. `ConfigValue` is a sealed type with typed accessors: `asString()`, `asNumber()`, +`asBoolean()`, `asObject()`, `asArray()`, and `isNull()`. If the value has an unexpected type, the accessor throws +`ConfigValueExtractionException`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class FooService { + + public FooService(Config config) { + ConfigValue value = config.get("services.foo.bar"); + if (!value.isNull()) { + String bar = value.asString(); + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class FooService(config: Config) { + + init { + val value = config["services.foo.bar"] + if (!value.isNull) { + val bar = value.asString() + } + } + } + ``` + +As noted in [Recommendations](#recommendations), prefer typed [custom configurations](#custom-configuration) over +reading a raw `Config`. +Use the raw read API only for dynamic or generic access when no other choice and use `ValueOf` to avoid component refresh. -???+ warning "Recommendation" +???+ warning "Attention" **We do not recommend** using `ru.tinkoff.kora.config.common.Config` directly as a dependency in components, - because when you update the configuration it will cause all graph components that use it to be updated, - it is recommended to always create custom user configuration interfaces. + because when configuration is updated, all graph components that use it will be updated as well. + We recommend always creating [custom configurations](#custom-configuration). ## Config Watcher { #config-watcher } -By default, Kora has a configuration file watcher that updates the contents of the configuration file, -which causes the dependency graph for the affected components to be updated if the configuration file is changed. +By default, `Kora` has a configuration file watcher that checks the application file for changes and starts dependency +graph refresh if the file changes. The check runs every `1000` milliseconds. + +For `HOCON`, the watcher also tracks files included through `include` inside the main configuration file. +If such an included file changes, the configuration is reread and the dependency graph is refreshed as well. + +The watcher works only for file-based configuration that has a trackable source. If configuration came from a resource +inside an archive or was built without an application file, there is nothing on disk to update. You can disable the watcher by using: -1. Environment variable `KORA_CONFIG_WATCHER_ENABLED`. -2. System property `kora.config.watcher.enabled`. +1. Environment variable `KORA_CONFIG_WATCHER_ENABLED` (default: `true`) +2. System property `kora.config.watcher.enabled` (default: `true`) ## Supported types { #supported-types } -Configuration Extractors provide an extensive list of supported types that covers most of what -you might need to specify in custom configurations, or you can extend the behavior with your custom `ConfigValueExtractor` component. +Configuration extractors provide an extensive list of supported types that covers most values you may need in custom +configurations. If the standard conversion is not enough, the behavior can be extended with a custom +`ConfigValueExtractor` component. ??? abstract "List of supported types" @@ -727,27 +982,188 @@ you might need to specify in custom configurations, or you can extend the behavi * Properties * Pattern * UUID - * Properties * LocalDate * LocalTime * LocalDateTime * OffsetTime * OffsetDateTime - * Enum (any custom ENUM type) (Change mapping change `toString()` contract) - * `List` (where `T` is any of the above listed types) - * `Set` (where `T` is any of the above types) - * `Map` (where `K` or `V` is any of the above types) - * `Either` (where `A` and `B` are any of the above types) + * ConfigValue.ObjectValue + * Enum (any custom `enum`; mapping can be overridden through `toString()`) + * `Optional` (where `T` is any supported type) + * `List` (where `T` is any supported type) + * `Set` (where `T` is any supported type) + * `Map` or `Map` (where `K` and `V` are supported by corresponding extractors) + * `Either` (where `A` and `B` are any supported types) + +### Custom extractor { #custom-extractor } + +If there is no standard conversion for a type or special parsing logic is required, add a custom +`ConfigValueExtractor` component. The `extract(...)` method receives the configuration value as `ConfigValue` +and must return the ready value of the required type. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public final class TokenConfigValueExtractor implements ConfigValueExtractor { + + @Override + public Token extract(ConfigValue value) { + if (value instanceof ConfigValue.NullValue) { + return null; + } + return new Token(value.asString()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + class TokenConfigValueExtractor : ConfigValueExtractor { + + override fun extract(value: ConfigValue<*>): Token? { + if (value is ConfigValue.NullValue) { + return null + } + return Token(value.asString()) + } + } + ``` + +If a specific extractor should be used only for one field, specify it through `@Mapping`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @ConfigValueExtractor + public interface ApiConfig { + + @Mapping(TokenConfigValueExtractor.class) + Token token(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @ConfigValueExtractor + interface ApiConfig { + + @Mapping(TokenConfigValueExtractor::class) + fun token(): Token + } + ``` + +### Duration { #duration } + +`Duration` can be set as a number or a string. +If a number is specified, it is treated as milliseconds. +If a string is specified, the `java.time.Duration` format is supported, for example `PT10S`, as well as `HOCON` style: + +- `500ms` +- `10 seconds` +- `2 minutes` +- `1h` +- `1d` + +### Period { #period } + +`Period` can be set as a number or a string. +If a number is specified, it is treated as days. +If a string is specified, these units are supported: + +- `d` / `days` +- `w` / `weeks` +- `m` / `mo` / `months` +- `y` / `years` + +For example, `7d`, `2 weeks`, `3mo`, or `1 year`. ### Size { #size } -`Size` is a special type that allows you to specify the size of bytes in a human-friendly system of calculations according to both the [IEEE 1541-2002](https://en.wikipedia.org/wiki/IEEE_1541-2002) (binary) standard and the [SI](https://en.wikipedia.org/wiki/Binary_prefix) (decimal) standard. +`Size` is a special type that allows specifying byte sizes in a human-friendly notation: according to the +[IEEE 1541-2002](https://en.wikipedia.org/wiki/IEEE_1541-2002) standard (binary) or the +[SI](https://en.wikipedia.org/wiki/Binary_prefix) standard (decimal). Example values: -- `1Mb` - 1 megabytes (`1.000.000` bytes) -- `1Mib` - 1 megabit (`1.048.576` bytes) +- `1Mb` - 1 megabyte (`1.000.000` bytes) +- `1Mib` - 1 mebibyte (`1.048.576` bytes) - `1024b` - 1024 bytes - `1024` - 1024 bytes If just a number without a suffix is specified, it is considered that bytes are specified. + +### Either { #either } + +`Either` lets a single field accept two alternative shapes. The extractor tries the left type `A` first, and if +extraction fails with any exception, it falls back to the right type `B`. This is useful when a value may be either a +plain scalar or a structured object. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.common.util.Either; + + @ConfigValueExtractor + public interface EndpointConfig { + + String host(); + + int port(); + } + + @ConfigSource("services.foo") + public interface FooServiceConfig { + + Either endpoint(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.util.Either + + @ConfigValueExtractor + interface EndpointConfig { + + fun host(): String + + fun port(): Int + } + + @ConfigSource("services.foo") + interface FooServiceConfig { + + fun endpoint(): Either + } + ``` + +Both of these forms are valid for the `endpoint` field: + +===! ":material-code-json: `Hocon`" + + ```javascript + services { + foo { + endpoint = "https://example.com" //(1)! + } + } + ``` + + 1. Resolved as the left type (`String`) + +=== ":simple-yaml: `YAML`" + + ```yaml + services: + foo: + endpoint: #(1)! + host: "example.com" + port: 8080 + ``` + + 1. Resolved as the right type (`EndpointConfig`) + +Use `isLeft()` / `isRight()` to check which side was resolved, and `left()` / `right()` to read the value. diff --git a/mkdocs/docs/en/documentation/container.md b/mkdocs/docs/en/documentation/container.md index 70c3a83..6366900 100644 --- a/mkdocs/docs/en/documentation/container.md +++ b/mkdocs/docs/en/documentation/container.md @@ -4,24 +4,33 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora compile-time dependency injection container, components, modules, factories, tags, lifecycle, graph resolution, and dependency wrappers; key triggers include @KoraApp, @Component, @Module, @KoraSubmodule, @Root, @Tag, @DefaultComponent, ValueOf, All, PromiseOf." --- -The dependency container is the core of the Kora framework and is responsible for building the dependency container, validating them, -injecting and then parallel initialization. +The dependency container is the core of the `Kora` framework. It builds the dependency graph, validates it, +injects components, initializes them, and releases them later. +Unlike containers that assemble an application at startup by scanning the classpath, `Kora` builds most of the graph +at compile time and generates regular `Java` code for application startup. -The work of the container in Kora is divided into two parts: what is done at runtime and what is done at compile time. +Container work in `Kora` is split into two parts: compile time and runtime. +At compile time, `Kora` checks that all dependencies can be found and connected. At runtime, the container creates +components, manages their lifecycle, and updates affected graph parts when changes happen. For a step-by-step walkthrough before the reference details, see [Dependency Injection Introduction](../guides/dependency-injection-introduction.md) and [Dependency Injection](../guides/dependency-injection.md). ## Compile Time { #compile-time } -At compile time, components are searched building the dependency container of the entire application. -This allows validation of the dependency container at compile time, before the application actually starts. +At compile time, components are discovered to build the dependency container for the whole application. +This allows the dependency container to be validated before the application actually starts. ### Container { #container } -The core of the dependency container is the interface labeled with the `@KoraApp` annotation. -This annotation should be used to label the interface within which the factory methods for creating components and [modules](#module-factory) dependencies are attached. +The core of the dependency container is the interface marked with the `@KoraApp` annotation. +This annotation should be used on the interface that contains factory methods for creating components +and connects [external modules](#external-module-factory). There can be only one such interface within an application. +`Kora` annotation processors analyze source code in the compilation module where `@KoraApp` is declared, +and in modules where [`@KoraSubmodule`](#submodule-factory) is declared. Regular project modules without +`@KoraApp` or `@KoraSubmodule` do not become component discovery scopes automatically. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -39,8 +48,8 @@ There can be only one such interface within an application. ### Components { #components } A component is a dependency in a dependency container. -All components in Kora are Singletons. A Singleton is a class that has an instance created only once. -Components are injected only if they are [root component](#root-component), or if they are required in other components as dependencies. +All components in `Kora` are created as a single instance (`Singleton`). +Components are injected only if they are [root components](#root-component) or if other components need them as dependencies. Components that do not meet these requirements are not included in the dependency container. @@ -77,10 +86,10 @@ The `@Component` annotation marks the class as accessible via the container. The class SomeService(val otherService: OtherService) { } ``` -#### Basic factory { #basic-factory } +#### Method factory { #method-factory } -A factory method is a method with the `default` modifier that returns a component, the method can take -arguments to other components as dependencies. +A factory method is a method with the `default` modifier that returns a component. +The method can take other dependency components as arguments. The dependency container below describes two factories, where the `otherService` factory requires a component created by the `someService` factory. This is the most basic way in which components can be registered in a container: @@ -119,12 +128,12 @@ The factory method **should not provide** a `null` value as a component. #### Module factory { #module-factory } -Components for a dependency container can also be searched for in modules in an application project. +Components for a dependency container can also be located in modules within an application project. A module refers to an interface that contains the factory methods. -The `@Module` annotation marks the interface as a module to be injected in our container at compile time. -Module must be within the same source code directory as the class labeled `@KoraApp`. +The `@Module` annotation marks the interface as a module to be injected into the application container at compile time. +The module must be within the same source code directory as the class marked with `@KoraApp`. -All factory methods within module become available to the dependency container: +All factory methods within the module become available to the dependency container: ===! ":fontawesome-brands-java: `Java`" @@ -152,12 +161,15 @@ All factory methods within module become available to the dependency container: Components for a dependency container can also be looked up in external modules from third-party dependencies. A module refers to the interface that contains the factory methods. -Kora does not automatically search for modules from external dependencies as some other DI solutions do. -This allows the developer to precisely control and realize which dependencies are used in their application and to avoid -from initializing a lot of unnecessary dependencies and thus degrading the application's performance. +`Kora` does not automatically search for modules from external dependencies as some other DI solutions do. +This lets the developer precisely control which dependencies are used in the application and avoid +initializing unnecessary components. All required external modules from dependencies must be connected explicitly in the interface marked with the `@KoraApp` annotation through inheritance: +Such a module can be declared in any interface: in a third-party library, in a separate project module, or next to `@KoraApp` itself. +The important part is that the `@KoraApp` interface explicitly connects it through inheritance. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -174,15 +186,18 @@ All required external modules from dependencies must be connected explicitly in #### Submodule factory { #submodule-factory } -The `@KoraSubmodule` annotation marks the interface for which to build a module for the current compilation module, -it will contain all components marked with the `@Module` and `@Component` annotations. -Annotation is useful if you are breaking your project into [multi-modules application](https://docs.gradle.org/current/userguide/multi_project_builds.html) -in terms of `Gradle` build tool, where -each is responsible for some piece of functionality, and the `@KoraApp` application itself is built in a separate module from the logic. +The `@KoraSubmodule` annotation marks the interface for which a module should be built for the current compilation module. +It will contain all components marked with the `@Module` and `@Component` annotations. +This annotation is useful when you split a project into a [multi-project application](https://docs.gradle.org/current/userguide/multi_project_builds.html) +from the `Gradle` build tool point of view, where each module is responsible for its own part of functionality, +and the application with `@KoraApp` is assembled in a separate compilation module. +This approach helps structure a large project by domain areas and improve build time: +changes in one project module do not force the annotation processor to analyze the entire application code again. -An inheritor interface will be created for the interface, where all interfaces labeled `@Module` will be inherited and default-methods for classes labeled as `@Component` will be created. +An inheritor interface will be generated for the interface. It will inherit all interfaces marked with `@Module` +and create factory methods for classes marked as `@Component`. -For example, you have application-module that contains submodule with module: +For example, you have a separate application module that contains this `@KoraSubmodule`: ===! ":fontawesome-brands-java: `Java`" @@ -222,7 +237,7 @@ For example, you have application-module that contains submodule with module: } ``` -And there's core application build module with application entrypoint: +And there is the main application module with the assembly point for the whole application: ===! ":fontawesome-brands-java: `Java`" @@ -238,12 +253,41 @@ And there's core application build module with application entrypoint: interface Application : SomeSubModule ``` -This will plug both `SomeSubModule` and `SomeModule` modules. +This will connect the `SomeModule` module found through `SomeSubModule` to the final application container. + +A common real-world use of `@KoraSubmodule` is a separate `Gradle` module that owns one domain area and simply +aggregates the [external modules](#external-module-factory) it needs (databases, caches, and so on) by extending them, +together with its own `@Component` classes and `@Module` interfaces. The application module then connects the +generated submodules the same way it connects any other module: + +===! ":fontawesome-brands-java: `Java`" + + ```java + // in the "pet" Gradle module + @KoraSubmodule + public interface PetModule extends JdbcDatabaseModule, CaffeineCacheModule { } + + // in the application Gradle module + @KoraApp + public interface Application extends PetModule, VetModule { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // in the "pet" Gradle module + @KoraSubmodule + interface PetModule : JdbcDatabaseModule, CaffeineCacheModule + + // in the application Gradle module + @KoraApp + interface Application : PetModule, VetModule + ``` #### Generic factory { #generic-factory } -If the dependency container could not find a generic factory for a particular type, the Kora container at compile time can try looking for -methods with [Generic](https://docs.oracle.com/javase/tutorial/java/generics/types.html) parameters, and use that method to create an instance of the desired class. +If the dependency container could not find a factory for a particular type, the `Kora` container can try to find +methods with generic parameters at compile time and use such a method to create an instance of the required class. ===! ":fontawesome-brands-java: `Java`" @@ -269,25 +313,57 @@ methods with [Generic](https://docs.oracle.com/javase/tutorial/java/generics/typ } ``` -Now if some component needs GenericValidator as a dependency, this factory will be used to create it. +Now, if some component needs `GenericValidator` as a dependency, this factory will be used to create it. + +##### Generic Type Information { #type-ref } + +If a factory method needs to know the exact generic type currently requested by the container, it can inject `TypeRef`. +This is useful for infrastructure components that create a dependency by the shape of the type, not only by the raw class. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface SomeModule { + + default Validator> listValidator(Validator validator, TypeRef typeRef) { + return new ListValidator<>(validator, typeRef); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface SomeModule { + + fun listValidator(validator: Validator, typeRef: TypeRef): Validator> { + return ListValidator(validator, typeRef) + } + } + ``` + +`TypeRef` carries generic type information through `Java` type erasure. Most application components do not need it, +but it is useful for universal factories, mappers, and container extensions. #### Extension mechanism { #extension-mechanism } -In case none of the factories were able to provide a component, Kora can try to create that dependency at compile time itself. +In case none of the factories were able to provide a component, `Kora` can try to create that dependency at compile time itself. The extensions mechanism is provided for this purpose. Each extension is able to tell if it can create a component of the desired type. -If the extension can do this, it does the necessary codogeneration and tells you how to get that component. +If the extension can do this, it performs the required code generation and reports how to obtain that component. -For example, there are extensions that know how to create optimal Json readers and writers, JDBC repositories, and other components. -The available extensions are searched thanks to the `ServiceLocator` mechanism from all dependencies provided in the Annotation Processor scope. +For example, there are extensions that know how to create optimal `JsonReader` and `JsonWriter` components, repositories, and other components. +Available extensions are discovered through the `ServiceLocator` mechanism from all dependencies provided in the annotation processor scope. -The mechanism is rather system specific and is often used by internal Kora modules. +This mechanism is system-level and is most often used by internal `Kora` modules. -#### Standard factory { #standard-factory } +#### Standard factory { #default-factory } In order to provide default components by factory methods, which it is assumed that the user can override, it is required to use the `@DefaultComponent` annotation. -If any component that does not use this annotation is found in the dependency container at compile time, -it will be given preference during injection. +If the dependency container finds any component of the same type and with the same tags at compile time, but without `@DefaultComponent`, +the user component will be preferred during injection. ===! ":fontawesome-brands-java: `Java`" @@ -316,7 +392,7 @@ it will be given preference during injection. #### Auto creation { #auto-creation } If none of the methods above were able to provide a component, -then Kora can try to create a component on its own if it meets the requirements similar to [auto factory](#auto-factory): +then `Kora` can try to create a component on its own if it meets the requirements similar to [auto factory](#auto-factory): ===! ":fontawesome-brands-java: `Java`" @@ -372,7 +448,7 @@ then Kora can try to create a component on its own if it meets the requirements In case a component is provided by the library as a default dependency, it is possible to create a factory in an application without the `@DefaultComponent` annotation and such a dependency will override it. -Since all external modules are plugged as interfaces into the core `@KoraApp` container and their factories are available, +Since all external modules are connected as interfaces to the `@KoraApp` container core and their factories are available, you can simply override them as a method and provide your custom implementation. ### Root component { #root-component } @@ -380,7 +456,7 @@ you can simply override them as a method and provide your custom implementation. When a component is required to always be initialized with application startup, even if it is not a dependency of other components, it is expected to use the `@Root` annotation over a factory method or class annotated with `@Component`. -An example of such a component might be HTTP server, Kafka consumer, cache warming component. +An example of such a component might be an `HTTP` server, a `Kafka` consumer, a cache warming component, or a runnable background task handler. ===! ":fontawesome-brands-java: `Java`" @@ -426,12 +502,12 @@ An example of such a component might be HTTP server, Kafka consumer, cache warmi } ``` - 1. Any `@Nullable` annotation will do, such as `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / etc. + 1. Any `@Nullable` annotation will do, for example `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - If you want to introduce an optional dependency that may not exist, you should use [Kotlin Nullability]() syntax and mark such a component as Nullable. - you should use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark such a component as Nullable, + If you want to inject an optional dependency that may be absent, use the [`Kotlin` null-safety syntax](https://kotlinlang.org/docs/null-safety.html) + and mark that component as allowing `null`, then the dependency container will not crash at compile time due to the absence of the component: ```kotlin @@ -439,6 +515,10 @@ An example of such a component might be HTTP server, Kafka consumer, cache warmi class SomeService(val otherService: OtherService?) { } ``` +Optionality can be combined with container wrappers: `ValueOf>`, `Optional>`, +`PromiseOf>`, and `Optional>`. This is useful when a dependency may be absent, +but the component still needs deferred access or the ability to refresh it through the container. + ### List of components { #list-of-components } There can be many instances of the same type in a container, and if you want to collect them all in one place, you should use the special type `All`. @@ -479,14 +559,17 @@ There can be many instances of the same type in a container, and if you want to For example, we have some entity `Handler` and it is injected by N different types in a container. `SomeProcessor` while consuming all possible implementations of that type. +**Important**: the example above takes all `Handler` instances without tags. The `All` type itself has the following contract: ```java -public interface All extends List {} +public sealed interface All extends List permits AllImpl {} ``` This is a token type that extends `List` and can be given to constructors that expect `List`. +If you need to collect references to components instead of the components themselves, the container also supports +`All>` and `All>`. ### Tags { #tags } @@ -532,7 +615,7 @@ This is how you can inject different instances of a class with a common interfac fun someService1(): SomeService = SomeService1() @Tag(MyTag2::class) - fun someService1(): SomeService = SomeService2() + fun someService2(): SomeService = SomeService2() fun serviceA(@Tag(MyTag1::class) service: SomeService): ServiceA { return ServiceA(service) @@ -597,7 +680,7 @@ Tags also work on constructor parameters, in conjunction with `@Component` or fi #### Tag custom { #tag-custom } -You can also create your own tag annotations and work with them, such an example is [@Json annotation](json.md) +You can also create your own tag annotations and work with them. One example is the [`@Json` annotation](json.md). ===! ":fontawesome-brands-java: `Java`" @@ -625,7 +708,7 @@ You can also create your own tag annotations and work with them, such an example annotation class MyTag interface SomeModule { - + @MyTag fun someService(): SomeService = SomeService() @@ -717,26 +800,109 @@ To get a list of all components with and without a tag, you need to use a specia } ``` +### Circular dependencies { #circular-dependencies } + +Because `Kora` builds and validates the whole dependency graph at compile time, a dependency cycle +(component `A` needs `B`, `B` needs `A`, possibly through more components in between) is detected during compilation +rather than blowing up at runtime. How such a cycle is handled depends on how the dependency inside the cycle is declared. + +**Direct dependency on a `final` class (or any non-interface type).** +Such a cycle cannot be resolved and compilation fails. The error points at the type that closes the cycle and lists the +cycle candidates: + +``` +Encountered circular dependency in graph for source type: ru.tinkoff.kora.example.ServiceA (no tags) + Cycle dependency candidates: + - ru.tinkoff.kora.example.ServiceA + - ru.tinkoff.kora.example.ServiceB +Please check that you are not using cycle dependency in ru.tinkoff.kora.application.graph.Lifecycle, this is forbidden. +``` + +**Dependency declared through an interface (or a non-`final` class).** +`Kora` breaks the cycle automatically: for the interface-typed dependency it generates a lazy proxy that implements +`ru.tinkoff.kora.common.PromisedProxy` and injects the proxy instead of the real component. The proxy resolves the +actual component from the graph on first access (and re-resolves it after a graph refresh), so both components can be +constructed. No action is required from the developer, but keep in mind that the proxied side becomes usable only after +the graph is fully bound, so it must not be called from a constructor. + +In the example below `ServiceAImpl` and `ServiceBImpl` reference each other through interfaces, so the cycle is broken +by an auto-generated `PromisedProxy` and the graph resolves successfully: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public interface ServiceA { } + + public interface ServiceB { } + + @Component + public final class ServiceAImpl implements ServiceA { + + public ServiceAImpl(ServiceB serviceB) { } + } + + @Component + public final class ServiceBImpl implements ServiceB { + + public ServiceBImpl(ServiceA serviceA) { } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + interface ServiceA + + interface ServiceB + + @Component + class ServiceAImpl(serviceB: ServiceB) : ServiceA + + @Component + class ServiceBImpl(serviceA: ServiceA) : ServiceB + ``` + +The reliable way to break a cycle deliberately is to inject one side through [`ValueOf`](#indirect-dependency) +or [`PromiseOf`](#updating-components) instead of a direct dependency. This decouples the consumer from the other +component's lifecycle, so the container no longer treats the two as a hard cycle: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ServiceAImpl implements ServiceA { + + public ServiceAImpl(ValueOf serviceB) { } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ServiceAImpl(serviceB: ValueOf) : ServiceA + ``` + ## Runtime { #runtime } -The dependency container is initialized as parallel as possible within the dependency container that has been constructed. +The dependency container uses as much parallelism as possible within the graph that has been built. -During the execution phase of the application, the following things are done: +During application execution, the container does the following: * Initializes all components in the dependency container -* Track changes in the dependency container -* Atomically updates the dependency container when changes are made. -* Performs a [Graceful Shutdown](#graceful-shutdown) when a SIGTERM signal is received. +* Tracks changes in the dependency container +* Atomically updates the dependency container when changes are made +* Performs a [graceful shutdown](#graceful-shutdown) when a `SIGTERM` signal is received All components use eager initialization, which means they are initialized immediately upon application startup. ### Entrypoint { #entrypoint } -The application entry point should cause `KoraApplication.run` to run using the dependency container created at compile time. +The application entry point should call `KoraApplication.run` using the dependency container created at compile time. -In case the interface labeled `@KoraApp` is called `Application`, the same package will create an `Application` class at compile time -class `ApplicationGraph` will be created in the same package to represent the dependency container implementation, and then the entry point -within the same package will look like this: +If the interface marked with `@KoraApp` is named `Application`, then during compilation a class named `ApplicationGraph` +will be generated in the same package. It represents the dependency container implementation, and the entry point +in the same package will look like this: ===! ":fontawesome-brands-java: `Java`" @@ -761,6 +927,10 @@ within the same package will look like this: } ``` +`KoraApplication.run` boots the container and returns a `RefreshableGraph` (a `Graph` combined with [`Lifecycle`](#component-lifecycle)). +For a running application you usually do not interact with it directly, but it is useful in tests and advanced flows where +you need to look up a component from the graph or trigger a refresh manually. + ### Container lifecycle { #container-lifecycle } The dependency container knows how to initialize all components in the correct order, and it does so in as much parallel as possible, in order to achieve the fastest possible startup time. @@ -773,7 +943,7 @@ which closes only if all components are successfully initialized and rolls back ### Component lifecycle { #component-lifecycle } -By default, all components are singletons through the constructor. +By default, all components are created as singletons through a constructor or a factory method during initialization. If you need to do some actions when the component is initialized, or before it is released, you must implement `Lifecycle` interface: ```java @@ -787,7 +957,11 @@ public interface Lifecycle { In a dependency container, all components are initialized asynchronously and in parallel as much as possible. -If you need to provide a component in a factory method with a lifecycle, you can use the `LifecycleWrapper` class: +If you need to provide a component with a lifecycle from a factory method, you can use the `LifecycleWrapper` class. +It implements two contracts at once: + +* `Lifecycle` — the container will call `init()` on startup and `release()` when releasing the component +* `Wrapped` — the container will inject the `T` value returned by the `value()` method ===! ":fontawesome-brands-java: `Java`" @@ -818,19 +992,30 @@ If you need to provide a component in a factory method with a lifecycle, you can // initialize logic }, { component -> - // initialize logic + // release logic } ) } } ``` +If you need to return a custom wrapper, it must implement `Wrapped`: + +```java +public interface Wrapped { + + T value(); +} +``` + ### Graceful shutdown { #graceful-shutdown } -All integrations that Kora provides such as [HTTP server](http-server.md), [Kafka-consumer](kafka.md), -etc., support [graceful shutdown](https://www.techtarget.com/whatis/definition/graceful-shutdown-and-hard-shutdown) out of the box using +All integrations that `Kora` provides, such as [HTTP server](http-server.md) and [Kafka consumer](kafka.md), +support [graceful shutdown](https://www.techtarget.com/whatis/definition/graceful-shutdown-and-hard-shutdown) out of the box using [component lifecycle](#component-lifecycle). +All components that implement `AutoCloseable` will also be automatically closed by the dependency container before release. + ### Indirect dependency { #indirect-dependency } Consider the following example: @@ -870,12 +1055,12 @@ Consider the following example: We have two services, and a third service that depends on them. But there is a difference in the lifecycle. If we take the type as a dependency directly, then we tell the container that when we update the `ServiceA` component, we need to update the `ServiceC` component in the same way. -But when we use the type wrapper `ValueOf`, we tell the container, -that `ServiceC` is in no way related to the life cycle of `ServiceB` and if `ServiceB` changes, we don't need to update `ServiceC`. +But when we use the `ValueOf` type wrapper, we tell the container +that `ServiceC` is not connected to the lifecycle of `ServiceB`, and if `ServiceB` changes, `ServiceC` does not need to be updated. #### Updating components { #updating-components } -Updating of components is possible if the `ValueOf` wrapper is used to inject dependencies: +Component refresh is possible if the `ValueOf` wrapper is used for dependency injection: ```java public interface ValueOf { @@ -886,18 +1071,71 @@ public interface ValueOf { } ``` -We can get the actual state of the component in the container using the `get` method. -This mechanism is used in such components that cannot be reloaded during the execution of the application. -For example, this is the case for various servers that listen to sockets (http, grpc) - for them, request handlers are supplied via `ValueOf`, which may be subject to changes. +The `get()` method returns the current component state in the container. +This mechanism is used in components that cannot be reloaded while the application is running. +For example, this applies to various servers that listen on sockets (`HTTP`, `gRPC`): request handlers that may change +are supplied to them through `ValueOf`. + +With the `refresh()` method, you can initiate a component refresh. This mechanism is used, for example, by a component +that tracks configuration file changes on disk. +When the file content changes, it initiates a refresh of the configuration component, and then all changes propagate +through the chain of components connected by direct dependencies. + +`ValueOf` also has additional methods for convenient work with the wrapped value: + +* `map(...)` — transforms the value inside `ValueOf` without changing the connection to the source component +* `optional()` — converts `ValueOf` to `ValueOf>` + +If a component needs a deferred reference, it can use `PromiseOf`. +The `get()` method returns `Optional`: before graph binding it is empty, and after binding it receives the current component from the container. + +```java +public interface PromiseOf { + + Optional get(); +} +``` + +Like `ValueOf`, `PromiseOf` supports `map(...)` and `optional()`. +Most business code only needs a direct dependency or `ValueOf`; `PromiseOf` is intended for lower-level scenarios +where a component needs deferred access to another graph part. -With the `refresh` function we can initiate a component refresh. This mechanism is for example used in a component that tracks changes to a configuration file on disk. -When the content of the file changes, it initiates a refresh of the configuration component, and further all changes are propagated through the chain of components linked by a direct link. +If a component received through `ValueOf>` needs to be passed further as a regular `ValueOf`, +you can use `Wrapped.UnwrappedValue.unwrap(...)`. This is useful for wrappers that add lifecycle or other behavior +but should expose a regular value outward. + +#### Refresh listeners { #refresh-listener } + +If a component needs to know that the graph was successfully refreshed, it can implement `RefreshListener`: + +```java +public interface RefreshListener { + + void graphRefreshed() throws Exception; +} +``` + +The container calls `graphRefreshed()` after a successful graph refresh. If a component is both a value wrapper and a refresh listener, +it can implement the combined `WrappedRefreshListener` interface. + +`RefreshListener` is only needed to receive a notification after refresh completion. It is not required for the container +to recreate a component. If a refresh affects a component or its dependencies, and other components injected it directly, +without `ValueOf` or `PromiseOf`, those dependent components will also be refreshed automatically. ### Component inspection { #component-inspection } -There are situations where there is some component in a dependency container that needs to be further modified or initialized, -but we need to make sure that nobody starts working with this component before we do these actions. -For this case there is a mechanism of component interception. You need to put an object implementing the `GraphInterceptor` interface into the container. +There are situations where a component in the container needs to be additionally modified or initialized, +but no one should start working with this component before those actions are complete. +For this case, there is a component interception mechanism. Put an object implementing the `GraphInterceptor` interface into the container. + +```java +public interface GraphInterceptor { + + T init(T value); + + T release(T value); +} +``` For example, this mechanism can be used to warm up the cache based on `JdbcDatabase`: @@ -936,5 +1174,6 @@ For example, this mechanism can be used to warm up the cache based on `JdbcDatab ``` The `GraphInterceptor` interface is almost the same as the `Lifecycle` contract, except for the return type. -Here we expect that the method may return a modified or a different instance of an object of the given type, -and this object will be used as a dependency by other components. +The `init(T value)` method receives an already fully initialized component. The method may return a modified or completely different +instance of the given type, and that object will be used as a dependency by other components. +The `release(T value)` method receives the component before release, meaning it is still a working and not yet cleaned-up instance. diff --git a/mkdocs/docs/en/documentation/database-cassandra.md b/mkdocs/docs/en/documentation/database-cassandra.md index 06cc1e3..52422ff 100644 --- a/mkdocs/docs/en/documentation/database-cassandra.md +++ b/mkdocs/docs/en/documentation/database-cassandra.md @@ -5,6 +5,11 @@ agent: --- Module provides a repository implementation for the [Cassandra](https://cassandra.apache.org/_/cassandra-basics.html) database using the [DataStax](https://docs.datastax.com/en/developer/java-driver/4.17/) driver. +`Cassandra` is a distributed column-oriented database where queries are written in `CQL`, and the data model is usually designed around specific read scenarios. +In Kora, the Cassandra module provides declarative repositories on top of `CqlSession`: the application writes `CQL` queries in `@Query`, and Kora generates query preparation, parameter binding, and result mapping code at compile time. + +Common rules for entities, `@Repository`, `@Query`, macros, batch queries, and the `@Table`, `@Column`, `@Id`, `@Embedded` annotations are described in the [common database section](database-common.md). +This document covers the Cassandra-specific parts: driver connection, `CqlSession` configuration, execution profiles, `UDT`, mappers, and supported method signatures. For a step-by-step walkthrough before the reference details, see [Cassandra Database](../guides/database-cassandra.md). @@ -38,7 +43,10 @@ For a step-by-step walkthrough before the reference details, see [Cassandra Data ## Configuration { #configuration } -Example of a simple configuration described in `CassandraConfig` class (example values are indicated): +Configuration is read from the `cassandra` section and described by the `CassandraConfig` interface. +At minimum, `basic.contactPoints` must be specified. Other parameters are optional or passed to the driver only when explicitly configured. + +Simple configuration example: ===! ":material-code-json: `Hocon`" @@ -59,14 +67,14 @@ Example of a simple configuration described in `CassandraConfig` class (example } ``` - 1. Cassandra node addresses for connection to the database (**required**) - 2. Cassandra datacenter name (optional) - 3. Name of keyspace for connection (optional) - 4. Query execution timeout (optional) - 5. Username for connection (optional) - 6. Password for connection (optional) + 1. `Cassandra` node addresses for connecting to the database (`required`, no default) + 2. `Cassandra` datacenter name (not specified by default, optional) + 3. `keyspace` name for the connection (not specified by default, optional) + 4. Query execution timeout for the connection (not specified by default, optional) + 5. Username for the connection (not specified by default, optional) + 6. Password for the connection (not specified by default, optional) -=== ":simple-yaml: ``YAML`" +=== ":simple-yaml: `YAML`" ```yaml cassandra: @@ -81,208 +89,218 @@ Example of a simple configuration described in `CassandraConfig` class (example password: "password" #(6)! ``` - 1. Cassandra node addresses for connection to the database (**required**) - 2. Cassandra datacenter name (optional) - 3. Name of keyspace for connection (optional) - 4. Query execution timeout (optional) - 5. Username for connection (optional) - 6. Password for connection (optional) + 1. `Cassandra` node addresses for connecting to the database (`required`, no default) + 2. `Cassandra` datacenter name (not specified by default, optional) + 3. `keyspace` name for the connection (not specified by default, optional) + 4. Query execution timeout for the connection (not specified by default, optional) + 5. Username for the connection (not specified by default, optional) + 6. Password for the connection (not specified by default, optional) ??? abstract "Full configuration example" - Full configuration with example values (configuration is in `CassandraConfig` class): + Full configuration with example values. Parameter descriptions are shared by the `HOCON` and `YAML` examples. ===! ":material-code-json: `Hocon`" ```javascript cassandra { auth { - login = "username" - password = "password" + login = "username" //(1)! + password = "password" //(2)! } basic { - contactPoints = [ "127.0.0.1:9042", "127.0.0.2:9042" ] // Nod cassandra hosts - sessionName = "some-session-name" // session name - dc = "datacenter1" // Datacenter Name - sessionKeyspace = "test-db" // The name of the keyspace for this session - - loadBalancingPolicy.slowReplicaAvoidance = true // Flag to enable the slow cue avoidance mechanism - cloud.secureConnectBundle = "/location/of/secure/connect/bundle" // Bandle locations to connect to Datastax Apache Cassandra. The path must be a valid URL. By default, if no protocol is specified, it will be assumed to be file:// - request { // Request settings - timeout = "5s" // request timeout - consistency = "LOCAL_ONE" // consistency level, permissible values: ANY, ONE, TWO, THREE, QUORUM, ALL, LOCAL_QUORUM, EACH_QUORUM, SERIAL, LOCAL_SERIAL, LOCAL_ONE - pageSize = 5000 // Page size limit (determines how many lines can be returned in a single request) - serialConsistency = "LOCAL_SERIAL" // Consistency level for lightweight transactions(LWT). Allowed values of SERIAL and LOCAL_SERIAL. - defaultIdempotence = false // Settings of idempotency value for queries + contactPoints = [ "127.0.0.1:9042", "127.0.0.2:9042" ] //(3)! + sessionName = "some-session-name" //(4)! + dc = "datacenter1" //(5)! + sessionKeyspace = "test-db" //(6)! + + loadBalancingPolicy.slowReplicaAvoidance = true //(7)! + cloud.secureConnectBundle = "/location/of/secure/connect/bundle" //(8)! + request { + timeout = "5s" //(9)! + consistency = "LOCAL_ONE" //(10)! + pageSize = 5000 //(11)! + serialConsistency = "LOCAL_SERIAL" //(12)! + defaultIdempotence = false //(13)! } } - advanced { // Advanced settings - sessionLeak.threshold = 4 // Maximum number of active sessions + + advanced { + sessionLeak.threshold = 4 //(14)! connection { - connectTimeout = "10s" // Connection timeout - initQueryTimeout = "10s" // Request initialization timeout - setKeyspaceTimeout = "10s" // Keyspace setting timeout - maxRequestsPerConnection = 1024 // Limiting requests per connection - maxOrphanRequests = 256 // The maximum number of "orphaned" requests, i.e., those for which a response has ceased to be expected for one reason or another. - warnOnInitError = true // Output initialization errors to the log - pool { // Pool Settings - localSize = 10 - remoteSize = 10 + connectTimeout = "10s" //(15)! + initQueryTimeout = "10s" //(16)! + setKeyspaceTimeout = "10s" //(17)! + maxRequestsPerConnection = 1024 //(18)! + maxOrphanRequests = 256 //(19)! + warnOnInitError = true //(20)! + pool { + localSize = 10 //(21)! + remoteSize = 10 //(22)! } } - reconnectOnInit = false // Retry initialization if all nodes specified in contactpoints did not respond at the first attempt - reconnectionPolicy { // Reconnect Policy - Base and Maximum Delay. By default, the first value is used when an attempt fails, then doubles with each subsequent attempt until it reaches the maximum value - baseDelay = "1s" - maxDelay = "60s" + reconnectOnInit = false //(23)! + reconnectionPolicy { + baseDelay = "1s" //(24)! + maxDelay = "60s" //(25)! + } + loadBalancingPolicy.dcFailover { + maxNodesPerRemoveDc = 1 //(26)! + allowForLocalConsistencyLevels = false //(27)! } - sslEngineFactory { - cipherSuites = [ "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_CBC_SHA" ] - hostnameValidation = true // Host name validation - keystorePath = "/path/to/client.keystore" // Path to the key vault - keystorePassword = "password" // The password to the key vault - truststorePath = "/path/to/client.truststore" // Path to trusted storage - truststorePassword = "password" // Trusted storage password + cipherSuites = [ "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_CBC_SHA" ] //(28)! + hostnameValidation = true //(29)! + keystorePath = "/path/to/client.keystore" //(30)! + keystorePassword = "password" //(31)! + truststorePath = "/path/to/client.truststore" //(32)! + truststorePassword = "password" //(33)! } - - timestampGenerator { // A generator that adds a timestamp to each request. AtomicTimestampGenerator is used by default - forceJavaClock = false // Forced to use Java system clock - driftWarning.threshold = "1s" // Indicates how far into the future timestamps can "run away" under high loads - driftWarning.interval = "10s" // Interval for logging warnings if timestamps continue to "run" forward. + timestampGenerator { + forceJavaClock = false //(34)! + driftWarning.threshold = "1s" //(35)! + driftWarning.interval = "10s" //(36)! } - protocol { - version = "V4" // Cassandra protocol version - compression = "lz4" // Compression - maxFrameLength = 268435456 // Maximum frame length in bytes + version = "V4" //(37)! + compression = "lz4" //(38)! + maxFrameLength = 268435456 //(39)! } request { - warnIfSetKeyspace = true // Log a warning that keyspace is being set in a query - trace { // Settings of the built-in query tracing mechanism - attempts = 5 // Number of attempts - interval = "1ms" // Interval between attempts - consistency = "ONE" // Consistency level + warnIfSetKeyspace = true //(40)! + trace { + attempts = 5 //(41)! + interval = "1ms" //(42)! + consistency = "ONE" //(43)! } - logWarnings = true + logWarnings = true //(44)! } - metrics { // session-level metrics, with all metrics turned off by default - node.enabled = [] // List of enabled metrics. Included: bytes-sent, connected-nodes, cql-requests, cql-client-timeouts, cql-prepared-cache-size, throttling.delay, throttling.errors, continuous-cql-requests - session.enabled = [] - publishPercentileHistogram = false // whether to publish persentils in metrics within min/max along with SLOs - node.cqlMessages { // Additional customizations for metrics if needed: - lowestLatency = "1ms" - highestLatency = "90s" - significantDigits = 1 - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] + metrics { + idGenerator { + name = "TaggingMetricIdGenerator" //(45)! + prefix = "my-app" //(46)! } - session.cqlRequests { - lowestLatency = "1ms" - highestLatency = "90s" - significantDigits = 1 - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] + node { + enabled = [ "bytes-sent", "bytes-received", "open-connections" ] //(47)! + cqlMessages { + lowestLatency = "1ms" //(48)! + highestLatency = "90s" //(49)! + significantDigits = 1 //(50)! + refreshInterval = "10s" //(51)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(52)! + } } - session.throttlingDelay { - lowestLatency = "1ms" - highestLatency = "90s" - significantDigits = 1 - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] + session { + enabled = [ "connected-nodes", "cql-requests", "cql-client-timeouts" ] //(53)! + cqlRequests { + lowestLatency = "1ms" //(54)! + highestLatency = "90s" //(55)! + significantDigits = 1 //(56)! + refreshInterval = "10s" //(57)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(58)! + } + throttlingDelay { + lowestLatency = "1ms" //(59)! + highestLatency = "90s" //(60)! + significantDigits = 1 //(61)! + refreshInterval = "10s" //(62)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(63)! + } } + publishPercentileHistogram = false //(64)! } socket { - tcpNoDelay = true // Flag to disable Nagle algorithm, by default true(off), because the driver has its own message coalescing algorithm. - keepAlive = false - reuseAddress = true // Allow the address to be reused - lingerInterval = 0 - receiveBufferSize = 65535 - sendBufferSize = 65535 + tcpNoDelay = true //(65)! + keepAlive = false //(66)! + reuseAddress = true //(67)! + lingerInterval = 0 //(68)! + receiveBufferSize = 65535 //(69)! + sendBufferSize = 65535 //(70)! } heartbeat { - interval = "30s" - timeout = "2m" + interval = "30s" //(71)! + timeout = "2m" //(72)! } - metadata { // Settings responsible for schema metadata + metadata { schema { - enabled = true - requestTimeout = "20s" - requestPageSize = 20 - refreshedKeyspaces = [ "ks1", "ks2" ] - debouncer.window = "1s" // The amount of time the driver waits before applying the update - debouncer.maxEvents = 20 // Maximum number of updates that can be accumulated + enabled = true //(73)! + requestTimeout = "20s" //(74)! + requestPageSize = 20 //(75)! + refreshedKeyspaces = [ "ks1", "ks2" ] //(76)! + debouncer.window = "1s" //(77)! + debouncer.maxEvents = 20 //(78)! } - topologyEventDebouncer.window = "1s" // A window for sending the event. - topologyEventDebouncer.maxEvents = 20 // Maximum number of events in a bundle - tokenMapEnabled = true + topologyEventDebouncer.window = "1s" //(79)! + topologyEventDebouncer.maxEvents = 20 //(80)! + tokenMapEnabled = true //(81)! } controlConnection { - timeout = "10s" + timeout = "10s" //(82)! schemaAgreement { - interval = 200ms - timeout = "10s" - warnOnFailure = true + interval = "200ms" //(83)! + timeout = "10s" //(84)! + warnOnFailure = true //(85)! } } preparedStatements { - prepareOnAllNodes = true // Execute query preparation on all nodes after its successful execution on one node. + prepareOnAllNodes = true //(86)! reprepareOnUp { - enabled = true // Prepare queries for new nodes - checkSystemTable = false // Check if there is a prepare statement in system.prepared_statements node before preparation - maxStatements = 0 // Maximum number of requests that can be retrained - maxParallelism = 100 // Maximum number of competitive requests - timeout = 20s + enabled = true //(87)! + checkSystemTable = false //(88)! + maxStatements = 0 //(89)! + maxParallelism = 100 //(90)! + timeout = "20s" //(91)! } - preparedCache.weakValues = false + preparedCache.weakValues = false //(92)! } - netty { // Netty event loop settings used in the driver - ioGroup.size = 0 // Number of tracks - ioGroup.shutdown { // Graceful shutdown settings - quietPeriod = 2 - timeout = 15 - unit = "SECONDS" + netty { + ioGroup.size = 0 //(93)! + ioGroup.shutdown { + quietPeriod = 2 //(94)! + timeout = 15 //(95)! + unit = "SECONDS" //(96)! } - adminGroup.size = 2 // Event loop group used only for admin tasks not related to IO + adminGroup.size = 2 //(97)! adminGroup.shutdown { - quietPeriod = 2 - timeout = 15 - unit = "SECONDS" + quietPeriod = 2 //(98)! + timeout = 15 //(99)! + unit = "SECONDS" //(100)! } - timer.tickDuration = "100ms" // Settings for how often the timer should wake up to check for overdue tasks - timer.ticksPerWheel = 2048 - daemon = false + timer.tickDuration = "100ms" //(101)! + timer.ticksPerWheel = 2048 //(102)! + daemon = false //(103)! + } + coalescer.rescheduleInterval = "10ms" //(104)! + resolveContactPoints = false //(105)! + throttler { + throttlerClass = "ConcurrencyLimitingRequestThrottler" //(106)! + maxConcurrentRequests = 1024 //(107)! + maxRequestsPerSecond = 10000 //(108)! + maxQueueSize = 10000 //(109)! + drainInterval = "1ms" //(110)! } - coalescer.rescheduleInterval = "10ms" - resolveContactPoints = false } - profiles { // Settings overridden in the profile + + profiles { someProfile { - basic { - // basic.request.timeout - // basic.request.consistency - } - advanced { - // advanced.request.trace.consistency - // advanced.request.trace.attempts - } + basic.request.timeout = "10s" //(111)! + basic.request.consistency = "LOCAL_QUORUM" //(112)! + advanced.request.trace.attempts = 3 //(113)! + advanced.request.trace.consistency = "ONE" //(114)! } - } + } + telemetry { - logging { - enabled = false - } + logging.enabled = false //(115)! metrics { - enabled = true - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - tags = { - "key1" = "value1" - "key2" = "value2" - } + enabled = true //(116)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(117)! + tags = { "key1" = "value1", "key2" = "value2" } //(118)! } tracing { - enabled = true - attributes = { - "key1" = "value1" - "key2" = "value2" - } + enabled = true //(119)! + attributes = { "key1" = "value1", "key2" = "value2" } //(120)! } } } @@ -292,177 +310,309 @@ Example of a simple configuration described in `CassandraConfig` class (example ```yaml cassandra: - advanced: # Advanced settings - coalescer: - rescheduleInterval: "10ms" + auth: + login: "username" #(1)! + password: "password" #(2)! + basic: + contactPoints: [ "127.0.0.1:9042", "127.0.0.2:9042" ] #(3)! + sessionName: "some-session-name" #(4)! + dc: "datacenter1" #(5)! + sessionKeyspace: "test-db" #(6)! + loadBalancingPolicy: + slowReplicaAvoidance: true #(7)! + cloud: + secureConnectBundle: "/location/of/secure/connect/bundle" #(8)! + request: + timeout: "5s" #(9)! + consistency: "LOCAL_ONE" #(10)! + pageSize: 5000 #(11)! + serialConsistency: "LOCAL_SERIAL" #(12)! + defaultIdempotence: false #(13)! + advanced: + sessionLeak: + threshold: 4 #(14)! connection: - connectTimeout: "10s" # Connection timeout - initQueryTimeout: "10s" # Request initialization timeout - setKeyspaceTimeout: "10s" # Keyspace setting timeout - maxOrphanRequests: 256 # The maximum number of "orphaned" requests, i.e., those for which a response has ceased to be expected for one reason or another. - maxRequestsPerConnection: 1024 # Limiting requests per connection - pool: # Pool Settings. - localSize: 10 - remoteSize: 10 - warnOnInitError: true # Output initialization errors to the log - controlConnection: - schemaAgreement: - interval: "200ms" - timeout: "10s" - warnOnFailure: true - timeout: "10s" + connectTimeout: "10s" #(15)! + initQueryTimeout: "10s" #(16)! + setKeyspaceTimeout: "10s" #(17)! + maxRequestsPerConnection: 1024 #(18)! + maxOrphanRequests: 256 #(19)! + warnOnInitError: true #(20)! + pool: + localSize: 10 #(21)! + remoteSize: 10 #(22)! + reconnectOnInit: false #(23)! + reconnectionPolicy: + baseDelay: "1s" #(24)! + maxDelay: "60s" #(25)! + loadBalancingPolicy: + dcFailover: + maxNodesPerRemoveDc: 1 #(26)! + allowForLocalConsistencyLevels: false #(27)! + sslEngineFactory: + cipherSuites: [ "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_CBC_SHA" ] #(28)! + hostnameValidation: true #(29)! + keystorePath: "/path/to/client.keystore" #(30)! + keystorePassword: "password" #(31)! + truststorePath: "/path/to/client.truststore" #(32)! + truststorePassword: "password" #(33)! + timestampGenerator: + forceJavaClock: false #(34)! + driftWarning: + threshold: "1s" #(35)! + interval: "10s" #(36)! + protocol: + version: "V4" #(37)! + compression: "lz4" #(38)! + maxFrameLength: 268435456 #(39)! + request: + warnIfSetKeyspace: true #(40)! + trace: + attempts: 5 #(41)! + interval: "1ms" #(42)! + consistency: "ONE" #(43)! + logWarnings: true #(44)! + metrics: + idGenerator: + name: "TaggingMetricIdGenerator" #(45)! + prefix: "my-app" #(46)! + node: + enabled: [ "bytes-sent", "bytes-received", "open-connections" ] #(47)! + cqlMessages: + lowestLatency: "1ms" #(48)! + highestLatency: "90s" #(49)! + significantDigits: 1 #(50)! + refreshInterval: "10s" #(51)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(52)! + session: + enabled: [ "connected-nodes", "cql-requests", "cql-client-timeouts" ] #(53)! + cqlRequests: + lowestLatency: "1ms" #(54)! + highestLatency: "90s" #(55)! + significantDigits: 1 #(56)! + refreshInterval: "10s" #(57)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(58)! + throttlingDelay: + lowestLatency: "1ms" #(59)! + highestLatency: "90s" #(60)! + significantDigits: 1 #(61)! + refreshInterval: "10s" #(62)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(63)! + publishPercentileHistogram: false #(64)! + socket: + tcpNoDelay: true #(65)! + keepAlive: false #(66)! + reuseAddress: true #(67)! + lingerInterval: 0 #(68)! + receiveBufferSize: 65535 #(69)! + sendBufferSize: 65535 #(70)! heartbeat: - interval: "30s" - timeout: "2m" - metadata: # Settings responsible for schema metadata + interval: "30s" #(71)! + timeout: "2m" #(72)! + metadata: schema: + enabled: true #(73)! + requestTimeout: "20s" #(74)! + requestPageSize: 20 #(75)! + refreshedKeyspaces: [ "ks1", "ks2" ] #(76)! debouncer: - maxEvents: 20 # Maximum number of updates that can be accumulated - window: "1s" # The amount of time the driver waits before applying the update - enabled: true - refreshedKeyspaces: - - ks1 - - ks2 - requestPageSize: 10 - requestTimeout: "20s" - tokenMapEnabled: true - topologyEventDebouncer: - maxEvents: 20 # Maximum number of events in a bundle - window: "1s" # A window for sending the event. - metrics: - publishPercentileHistogram: false # whether to publish persentils in metrics within min/max along with SLOs - node: - enabled: [] # List of enabled metrics. Included: bytes-sent, connected-nodes, cql-requests, cql-client-timeouts, cql-prepared-cache-size, throttling.delay, throttling.errors, continuous-cql-requests - cqlMessages: # Additional customizations for metrics if needed: - lowestLatency: "1ms" - highestLatency: "90s" - significantDigits: 1 - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - session: # session-level metrics, with all metrics turned off by default - enabled: [] # List of enabled metrics. Included: bytes-sent, connected-nodes, cql-requests, cql-client-timeouts, cql-prepared-cache-size, throttling.delay, throttling.errors, continuous-cql-requests - cqlRequests: - lowestLatency: "1ms" - highestLatency: "90s" - significantDigits: 1 - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - throttlingDelay: - lowestLatency: "1ms" - highestLatency: "90s" - significantDigits: 1 - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - netty: # Netty event loop settings used in the driver - adminGroup: # Event loop group used only for admin tasks not related to IO - shutdown: - quietPeriod: 2 - timeout: 15 - unit: SECONDS - size: 2 - daemon: false - ioGroup: - shutdown: # Graceful shutdown settings - quietPeriod: 2 - timeout: 15 - unit: SECONDS - size: 0 # Number of tracks - timer: - tickDuration: "100ms" # Settings for how often the timer should wake up to check for overdue tasks - ticksPerWheel: 2048 - preparedStatements: - prepareOnAllNodes: true # Execute query preparation on all nodes after its successful execution on one node. - preparedCache: - weakValues: false - reprepareOnUp: - enabled: true # Prepare queries for new nodes - checkSystemTable: false # Check if there is a prepare statement in system.prepared_statements node before preparation - maxParallelism: 100 # Maximum number of competitive requests - maxStatements: 0 # Maximum number of requests that can be retrained - timeout: "20s" - protocol: - compression: "lz4" # Compression - maxFrameLength: 268435456 # Maximum frame length in bytes - version: "V4" # Cassandra protocol version - reconnectOnInit: false # Retry initialization if all nodes specified in contactpoints did not respond at the first attempt - reconnectionPolicy: # Reconnect Policy - Base and Maximum Delay. By default, the first value is used when an attempt fails, then doubles with each subsequent attempt until it reaches the maximum value - baseDelay: "1s" - maxDelay: "60s" - request: - logWarnings: true - trace: - attempts: 5 # Number of attempts - consistency: ONE # Consistency level - interval: "1ms" # Interval between attempts - warnIfSetKeyspace: true # Log a warning that keyspace is being set in a query - resolveContactPoints: false - sessionLeak: - threshold: 4 - socket: - keepAlive: false - lingerInterval: 0 - receiveBufferSize: 65535 - reuseAddress: true # Allow the address to be reused - sendBufferSize: 65535 - tcpNoDelay: true # Flag to disable Nagle algorithm, by default true(off), because the driver has its own message coalescing algorithm. - sslEngineFactory: - cipherSuites: - - TLS_RSA_WITH_AES_128_CBC_SHA - - TLS_RSA_WITH_AES_256_CBC_SHA - hostnameValidation: true # Host name validation - keystorePassword: "password" # The password to the key vault - keystorePath: "/path/to/client.keystore" # Path to the key vault - truststorePassword: "password" # Trusted storage password - truststorePath: "/path/to/client.truststore" # Path to trusted storage - timestampGenerator: # A generator that adds a timestamp to each request. AtomicTimestampGenerator is used by default - driftWarning: - interval: "10s" # Interval for logging warnings if timestamps continue to "run" forward. - threshold: "1s" # Indicates how far into the future timestamps can "run away" under high loads - forceJavaClock: false # Forced to use Java system clock - auth: - login: "username" - password: "password" - basic: - cloud: - secureConnectBundle: "/location/of/secure/connect/bundle" - contactPoints: - - "127.0.0.1:9042" - - "127.0.0.2:9042" - dc: datacenter1 - loadBalancingPolicy: - slowReplicaAvoidance: true - request: - consistency: LOCAL_ONE - defaultIdempotence: false - pageSize: 5000 - serialConsistency: LOCAL_SERIAL - timeout: "5s" - sessionKeyspace: "test-db" - sessionName: "some-session-name" - profiles: # Settings overridden in the profile - someProfile: - advanced: - #advanced.request.trace.consistency - #advanced.request.trace.attempts - basic: - #basic.request.timeout - #basic.request.consistency + window: "1s" #(77)! + maxEvents: 20 #(78)! + topologyEventDebouncer: + window: "1s" #(79)! + maxEvents: 20 #(80)! + tokenMapEnabled: true #(81)! + controlConnection: + timeout: "10s" #(82)! + schemaAgreement: + interval: "200ms" #(83)! + timeout: "10s" #(84)! + warnOnFailure: true #(85)! + preparedStatements: + prepareOnAllNodes: true #(86)! + reprepareOnUp: + enabled: true #(87)! + checkSystemTable: false #(88)! + maxStatements: 0 #(89)! + maxParallelism: 100 #(90)! + timeout: "20s" #(91)! + preparedCache: + weakValues: false #(92)! + netty: + ioGroup: + size: 0 #(93)! + shutdown: + quietPeriod: 2 #(94)! + timeout: 15 #(95)! + unit: "SECONDS" #(96)! + adminGroup: + size: 2 #(97)! + shutdown: + quietPeriod: 2 #(98)! + timeout: 15 #(99)! + unit: "SECONDS" #(100)! + timer: + tickDuration: "100ms" #(101)! + ticksPerWheel: 2048 #(102)! + daemon: false #(103)! + coalescer: + rescheduleInterval: "10ms" #(104)! + resolveContactPoints: false #(105)! + throttler: + throttlerClass: "ConcurrencyLimitingRequestThrottler" #(106)! + maxConcurrentRequests: 1024 #(107)! + maxRequestsPerSecond: 10000 #(108)! + maxQueueSize: 10000 #(109)! + drainInterval: "1ms" #(110)! + profiles: + someProfile: + basic: + request: + timeout: "10s" #(111)! + consistency: "LOCAL_QUORUM" #(112)! + advanced: + request: + trace: + attempts: 3 #(113)! + consistency: "ONE" #(114)! telemetry: logging: - enabled: false + enabled: false #(115)! metrics: - enabled: true - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - tags: - key1: value1 - key2: value2 + enabled: true #(116)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(117)! + tags: { key1: "value1", key2: "value2" } #(118)! tracing: - enabled: true - attributes: - key1: value1 - key2: value2 + enabled: true #(119)! + attributes: { key1: "value1", key2: "value2" } #(120)! ``` + 1. Username for authentication in `Cassandra` (not specified by default, optional). + 2. Password for authentication in `Cassandra` (not specified by default, optional). + 3. `Cassandra` node addresses in `host:port` format (`required`, no default). + 4. Driver session name used in logs, metrics, and diagnostics (not specified by default, optional). + 5. Local datacenter for the load-balancing policy (not specified by default, optional). + 6. `keyspace` that will be set for the session after connection (not specified by default, optional). + 7. Enables slow replica avoidance in the default load-balancing policy (not specified by default, optional). + 8. Path or `URL` to the `Secure Connect Bundle` for connecting to `DataStax Astra` / cloud Cassandra (not specified by default, optional). + 9. Regular request timeout (not specified by default, optional). + 10. Regular request consistency level, for example `ONE`, `LOCAL_ONE`, `LOCAL_QUORUM`, `QUORUM`, `ALL` (not specified by default, optional). + 11. Result page size, meaning the maximum number of rows requested in one network round trip (not specified by default, optional). + 12. Serial consistency level for lightweight transactions `LWT`: `SERIAL` or `LOCAL_SERIAL` (not specified by default, optional). + 13. Default request idempotence value; affects whether retries and speculative execution can be applied safely (not specified by default, optional). + 14. Driver session leak warning threshold (not specified by default, optional). + 15. Timeout for opening a network connection to a node (not specified by default, optional). + 16. Timeout for requests that the driver executes while initializing a connection (not specified by default, optional). + 17. Timeout for setting the `keyspace` on a connection (not specified by default, optional). + 18. Maximum number of simultaneous requests per connection (not specified by default, optional). + 19. Maximum number of requests whose response is no longer awaited but may still complete inside the driver (not specified by default, optional). + 20. Logs a warning when connection initialization fails for an individual node (not specified by default, optional). + 21. Connection pool size for local datacenter nodes (not specified by default, optional). + 22. Connection pool size for remote nodes (not specified by default, optional). + 23. Allows initialization retry when all `contactPoints` do not answer during startup (not specified by default, optional). + 24. Initial delay of the reconnection policy (not specified by default, optional). + 25. Maximum delay of the reconnection policy (not specified by default, optional). + 26. Maximum number of remote datacenter nodes that can be used for failover (not specified by default, optional). + 27. Allows failover to a remote datacenter for local consistency levels (not specified by default, optional). + 28. Allowed cipher suites for `SSL/TLS` (not specified by default, optional). + 29. Checks that the node hostname matches the `SSL/TLS` certificate (not specified by default, optional). + 30. Path to the client keystore (not specified by default, optional). + 31. Client keystore password (not specified by default, optional). + 32. Path to the truststore (not specified by default, optional). + 33. Truststore password (not specified by default, optional). + 34. Forces Java system clock usage for query timestamp generation (not specified by default, optional). + 35. Warning threshold for timestamp drift into the future (not specified by default, optional). + 36. Minimum interval between timestamp drift warnings (not specified by default, optional). + 37. Cassandra binary protocol version, for example `V4` (not specified by default, optional). + 38. Protocol compression algorithm, for example `lz4` or `snappy` (not specified by default, optional). + 39. Maximum protocol frame size in bytes (not specified by default, optional). + 40. Logs a warning when a query explicitly changes the `keyspace` (not specified by default, optional). + 41. Number of attempts to fetch query tracing information from Cassandra (not specified by default, optional). + 42. Interval between attempts to fetch query tracing information (not specified by default, optional). + 43. Consistency level for queries to tracing tables (not specified by default, optional). + 44. Logs warnings returned by Cassandra with a query response (not specified by default, optional). + 45. Driver metric identifier generator name (default: `TaggingMetricIdGenerator`). + 46. Driver metric name prefix (not specified by default, optional). + 47. Enabled node-level metrics (default: `open-connections`, `in-flight`, `bytes-received`, `bytes-sent`, `write-timeouts`, `read-timeouts`, `aborted-requests`). + 48. Lowest expected latency for the `node.cqlMessages` metric histogram (default: `1ms`). + 49. Highest expected latency for the `node.cqlMessages` metric histogram (default: `90s`). + 50. Number of significant digits for the `node.cqlMessages` metric histogram (not specified by default, optional). + 51. Snapshot refresh interval for the `node.cqlMessages` metric histogram (not specified by default, optional). + 52. `SLO` boundaries for the `node.cqlMessages` metric (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 53. Enabled session-level metrics (default: `connected-nodes`, `cql-requests`, `cql-client-timeouts`, `cql-prepared-cache-size`, `throttling.delay`, `throttling.queue-size`). + 54. Lowest expected latency for the `session.cqlRequests` metric histogram (default: `1ms`). + 55. Highest expected latency for the `session.cqlRequests` metric histogram (default: `90s`). + 56. Number of significant digits for the `session.cqlRequests` metric histogram (not specified by default, optional). + 57. Snapshot refresh interval for the `session.cqlRequests` metric histogram (not specified by default, optional). + 58. `SLO` boundaries for the `session.cqlRequests` metric (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 59. Lowest expected latency for the `session.throttlingDelay` metric histogram (default: `1ms`). + 60. Highest expected latency for the `session.throttlingDelay` metric histogram (default: `90s`). + 61. Number of significant digits for the `session.throttlingDelay` metric histogram (not specified by default, optional). + 62. Snapshot refresh interval for the `session.throttlingDelay` metric histogram (not specified by default, optional). + 63. `SLO` boundaries for the `session.throttlingDelay` metric (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 64. Publishes percentile histograms for driver metrics (default: `false`). + 65. Enables `TCP_NODELAY`, which disables Nagle's algorithm (not specified by default, optional). + 66. Enables `SO_KEEPALIVE` for TCP sockets (not specified by default, optional). + 67. Enables `SO_REUSEADDR` for TCP sockets (not specified by default, optional). + 68. `SO_LINGER` value for TCP sockets (not specified by default, optional). + 69. TCP socket receive buffer size in bytes (not specified by default, optional). + 70. TCP socket send buffer size in bytes (not specified by default, optional). + 71. Interval for sending `heartbeat` on an idle connection (not specified by default, optional). + 72. Timeout for waiting for a `heartbeat` response (not specified by default, optional). + 73. Enables schema metadata loading and refresh (not specified by default, optional). + 74. Timeout for schema metadata queries (not specified by default, optional). + 75. Page size for schema metadata queries (not specified by default, optional). + 76. List of `keyspace` names whose schema metadata is refreshed by the driver (not specified by default, optional). + 77. Window for coalescing schema refresh events before processing (not specified by default, optional). + 78. Maximum number of schema refresh events that can be accumulated in the window (not specified by default, optional). + 79. Window for coalescing cluster topology change events (not specified by default, optional). + 80. Maximum number of topology change events that can be accumulated in the window (not specified by default, optional). + 81. Enables the token map for routing requests by data owners (not specified by default, optional). + 82. Service `control connection` timeout (not specified by default, optional). + 83. Interval for checking `schema agreement` between nodes (not specified by default, optional). + 84. Maximum time to wait for `schema agreement` (not specified by default, optional). + 85. Logs a warning if `schema agreement` is not reached in time (not specified by default, optional). + 86. Prepares a statement on all nodes after it has been prepared successfully on one node (not specified by default, optional). + 87. Re-prepares statements on a node that became available again (not specified by default, optional). + 88. Checks the `system.prepared_statements` system table before re-preparing a statement (not specified by default, optional). + 89. Maximum number of statements to re-prepare; `0` means no driver-side limit (not specified by default, optional). + 90. Maximum number of parallel re-prepare requests (not specified by default, optional). + 91. Timeout for re-preparing statements on one node (not specified by default, optional). + 92. Stores prepared statement cache values through weak references (not specified by default, optional). + 93. Number of `Netty` threads for network I/O; `0` lets the driver choose automatically (not specified by default, optional). + 94. Quiet period for graceful `ioGroup` shutdown (not specified by default, optional). + 95. Maximum wait time for `ioGroup` shutdown (not specified by default, optional). + 96. Unit for `ioGroup` shutdown parameters (not specified by default, optional). + 97. Number of `Netty` threads for driver administrative tasks (not specified by default, optional). + 98. Quiet period for graceful `adminGroup` shutdown (not specified by default, optional). + 99. Maximum wait time for `adminGroup` shutdown (not specified by default, optional). + 100. Unit for `adminGroup` shutdown parameters (not specified by default, optional). + 101. Duration of one `Netty` timer tick for delayed driver tasks (not specified by default, optional). + 102. Number of ticks in the `Netty` timer wheel (not specified by default, optional). + 103. Makes `Netty` threads daemon threads (not specified by default, optional). + 104. Rescheduling interval for message coalescing before sending (not specified by default, optional). + 105. Allows the driver to resolve `contactPoints` through DNS during startup (not specified by default, optional). + 106. Driver request throttler class (not specified by default, optional). + 107. Maximum number of concurrent requests for the throttler (not specified by default, optional). + 108. Maximum number of requests per second for the throttler (not specified by default, optional). + 109. Maximum throttler request queue size (not specified by default, optional). + 110. Interval at which the throttler releases requests from the queue (not specified by default, optional). + 111. `basic.request.timeout` override for the `someProfile` profile (not specified by default, optional). + 112. `basic.request.consistency` override for the `someProfile` profile (not specified by default, optional). + 113. `advanced.request.trace.attempts` override for the `someProfile` profile (not specified by default, optional). + 114. `advanced.request.trace.consistency` override for the `someProfile` profile (not specified by default, optional). + 115. Enables Kora query logging (default: `false`). + 116. Enables Kora query metrics (default: `true`). + 117. Kora metrics `SLO` boundaries (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 118. Additional Kora metric tags (default: `{}`). + 119. Enables Kora query tracing (default: `true`). + 120. Additional Kora tracing attributes (default: `{}`). + ### Code configuration { #code-configuration } -You can configure the driver manually in your code using `CassandraConfigurer` to modify the `CqlSession` builder: +You can configure the driver manually in your code by registering a `CassandraConfigurer` component. +The `configure` method receives the `CqlSessionBuilder` and the `ProgrammaticDriverConfigLoaderBuilder`, +so you can adjust the session builder and override raw driver options that are not exposed through the `cassandra` configuration section: ===! ":fontawesome-brands-java: `Java`" @@ -471,7 +621,7 @@ You can configure the driver manually in your code using `CassandraConfigurer` t public final class MyCassandraConfigurer implements CassandraConfigurer { @Override - public CqlSessionBuilder configure(CqlSessionBuilder builder) { + public CqlSessionBuilder configure(CqlSessionBuilder builder, ProgrammaticDriverConfigLoaderBuilder loaderBuilder) { return builder.withClientId(UUID.randomUUID()); } } @@ -482,7 +632,8 @@ You can configure the driver manually in your code using `CassandraConfigurer` t ```kotlin @Component class MyCassandraConfigurer : CassandraConfigurer { - override fun configure(builder: CqlSessionBuilder): CqlSessionBuilder { + + override fun configure(builder: CqlSessionBuilder, loaderBuilder: ProgrammaticDriverConfigLoaderBuilder): CqlSessionBuilder { return builder.withClientId(UUID.randomUUID()) } } @@ -490,20 +641,105 @@ You can configure the driver manually in your code using `CassandraConfigurer` t ## Usage { #usage } +To create a repository, declare an interface with `@Repository` and extend `CassandraRepository`. +Such a repository gets access to `CqlSession` through generated code and uses `@Query` to execute `CQL` queries. +Query parameters are bound by name: `:id`, `:entity.field`, `:filter.value`. + +Views are described with the [common database annotations](database-common.md) and marked with `@EntityCassandra` +so that `Kora` generates the view mapper at compile time (see [View](#view)): + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends CassandraRepository { } + public interface EntityRepository extends CassandraRepository { + + @EntityCassandra + @Table("entities") + record Entity(@Id String id, + @Column("value1") int field1, + String value2, + @Nullable String value3) {} + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + @Nullable + Entity findById(String id); + + @Query("SELECT id, value1, value2, value3 FROM entities") //(2)! + List findAll(); + + @Query("INSERT INTO %{entity#inserts}") //(3)! + void insert(Entity entity); + } ``` + 1. Uses macros `%{return#selects}` and `%{return#table}`. Expands to query: + ```sql + SELECT id, value1, value2, value3 + FROM entities + WHERE id = :id + ``` + Method uses macros for `SELECT`. Details: [Common Database Rules — Macros](database-common.md#macros) + 2. Fields listed manually without macros — this is valid but requires maintenance when the view changes. + 3. Uses macro `%{entity#inserts}`. Expands to query: + ```sql + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + ``` + Method uses macros for `INSERT`. Details: [Common Database Rules — Macros](database-common.md#macros) + === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : CassandraRepository + interface EntityRepository : CassandraRepository { + + @EntityCassandra + @Table("entities") + data class Entity( + @field:Id val id: String, + @field:Column("value1") val field1: Int, + val value2: String, + val value3: String? + ) + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + fun findById(id: String): Entity? + + @Query("INSERT INTO %{entity#inserts}") //(3)! + fun insert(entity: Entity) + } ``` + 1. Uses macros `%{return#selects}` and `%{return#table}`. Expands to query: + ```sql + SELECT id, value1, value2, value3 + FROM entities + WHERE id = :id + ``` + Method uses macros for `SELECT`. Details: [Common Database Rules — Macros](database-common.md#macros) + 2. Fields listed manually without macros — this is valid but requires maintenance when the view changes. + 3. Uses macro `%{entity#inserts}`. Expands to query: + ```sql + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + ``` + Method uses macros for `INSERT`. Details: [Common Database Rules — Macros](database-common.md#macros) + +`CQL` remains under the developer's control: you write the query text yourself, while `Kora` only handles parameter binding, +query execution, and result mapping. +Common rules for entities, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch`, and macros are described in +[Common database rules](database-common.md#macros). + +**Parameter binding:** Kora performs typed injection of arguments into the CQL query at compile time. +Query parameters (e.g., `:id`, `:entity.field1`) are replaced in the generated code with corresponding Cassandra driver calls. +For example, for a `String id` parameter, something like `statement.setString(1, id)` will be generated, where the index corresponds to the parameter order in the query. +This ensures security (protection against CQL injection) and performance (using driver prepared statements). + +Unlike relational databases, `Cassandra` has no transactions. +When you need several statements to be applied atomically, use a `@Batch` method (a `CQL` `BATCH`) as shown above; +its semantics and macros are documented in the [common database section](database-common.md). + ### Profile { #profile } It is possible to override common settings with private settings from a profile, suppose there is such a profile configuration `someProfile`: @@ -559,23 +795,54 @@ In order to apply the settings from the `someProfile` profile, just do the follo ``` The settings specified in the profile will be applied to each request, specifically in this case - a timeout of 10s will be set. +The profile applies only to the method annotated with `@CassandraProfile`; other repository methods continue to use the base configuration. ## Mapping { #mapping } -It is possible to override the mapping of different parts of [entity](database-common.md) and query parameters, Kora provides special interfaces for this. +It is possible to override the mapping of different parts of [view](database-common.md) and query parameters, Kora provides special interfaces for this. +Out of the box, `CassandraModule` provides mappers for common types: `String`, numeric types, `Boolean`, `BigDecimal`, `BigInteger`, `UUID`, `ByteBuffer`, `LocalDate`, `LocalTime`, `LocalDateTime`, `ZonedDateTime`, and `Instant`. +If a type is not covered by that set, or if it needs a custom representation in `CQL`, add a custom mapper through `@Mapping`. + +### View { #view } + +Use the `@EntityCassandra` annotation for optimal view mapping. +The annotation allows the annotation processor to generate all necessary mappers in **one round** of annotation processing. +Without this annotation, mappers are generated on-demand, which can require **multiple rounds** of processing and significantly increase compilation time. +This is the recommended way to map every view returned from or bound into a repository. + +All nested views and [UDT](#udt) types are also expected to use this annotation. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @EntityCassandra + public record Entity(String id, String name) {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @EntityCassandra + data class Entity(val id: String, val name: String) + ``` ### Result { #result } -If you need to convert the result manually, it is suggested to use `CassandraResultSetMapper`: +If you need to convert the whole synchronous query result manually, use `CassandraResultSetMapper`. +It receives `ResultSet` and returns the repository method value: a single object, list, `Optional`, or another supported type. ===! ":fontawesome-brands-java: `Java`" ```java - final class ResultMapper implements CassandraResultSetMapper { + final class ResultMapper implements CassandraResultSetMapper> { @Override - public UUID apply(ResultSet rows) { - // mapping code + public List apply(ResultSet rows) { + var result = new ArrayList(); + for (var row : rows) { + result.add(row.getUuid("id")); + } + return result; } } @@ -593,9 +860,9 @@ If you need to convert the result manually, it is suggested to use `CassandraRes In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. ```kotlin - class ResultMapper : CassandraResultSetMapper { - override fun apply(rows: ResultSet): UUID { - // mapping code + class ResultMapper : CassandraResultSetMapper> { + override fun apply(rows: ResultSet): List { + return rows.map { it.getUuid("id") } } } @@ -608,9 +875,17 @@ If you need to convert the result manually, it is suggested to use `CassandraRes } ``` +Each result-mapper interface also exposes static factory helpers that build a full result mapper from a `CassandraRowMapper`, +so you can reuse a single row mapper across signatures: + +- `CassandraResultSetMapper` — `singleResultSetMapper`, `optionalResultSetMapper`, `listResultSetMapper`; +- `CassandraAsyncResultSetMapper` — `one`, `list` (auto-paginates across result pages); +- `CassandraReactiveResultSetMapper` — `flux`, `mono`, `monoVoid`, `monoList`. + ### Row { #row } -If you need to convert the string manually, it is suggested to use `CassandraRowMapper`: +If you need to convert one result row manually, use `CassandraRowMapper`. +This mapper is applied to every row and suits return values like `T`, `Optional`, `List`, `Flux`, and `Flow`. ===! ":fontawesome-brands-java: `Java`" @@ -619,7 +894,7 @@ If you need to convert the string manually, it is suggested to use `CassandraRow @Override public UUID apply(Row row) { - return UUID.fromString(rs.getString(0)); + return UUID.fromString(row.getString(0)); } } @@ -640,7 +915,7 @@ If you need to convert the string manually, it is suggested to use `CassandraRow class RowMapper : CassandraRowMapper { override fun apply(row: Row): UUID { - return UUID.fromString(rs.getString(0)) + return UUID.fromString(row.getString(0)) } } @@ -707,7 +982,8 @@ If you need to convert the column value manually, it is suggested to use the `Ca ### Parameter { #parameter } -If you want to convert the value of a query parameter manually, it is suggested to use `CassandraParameterColumnMapper`: +If you want to convert the value of a query parameter manually, use `CassandraParameterColumnMapper`. +It receives `SettableByName`, the parameter index, and the value from the repository method. ===! ":fontawesome-brands-java: `Java`" @@ -715,7 +991,7 @@ If you want to convert the value of a query parameter manually, it is suggested public final class ParameterMapper implements CassandraParameterColumnMapper { @Override - public void set(SettableByName stmt, int index, @Nullable UUID value) { + public void apply(SettableByName stmt, int index, @Nullable UUID value) { if (value != null) { stmt.setString(index, value.toString()); } @@ -737,7 +1013,7 @@ If you want to convert the value of a query parameter manually, it is suggested ```kotlin class ParameterMapper : CassandraParameterColumnMapper { - override fun set(stmt: SettableByName<*>, index: Int, value: UUID?) { + override fun apply(stmt: SettableByName<*>, index: Int, value: UUID?) { if (value != null) { stmt.setString(index, value.toString()) } @@ -754,15 +1030,17 @@ If you want to convert the value of a query parameter manually, it is suggested ### Async { #async } -Due to the nature of the helper class for extracting data from `AsyncResultSet` for asynchronous queries (Mono or Suspend), only `CassandraReactiveResultSetMapper` can be used: +For `CompletionStage` and `CompletableFuture`, use `CassandraAsyncResultSetMapper`, which receives `AsyncResultSet` and returns `CompletionStage`. +Its `list` helper automatically requests subsequent result pages, so a `List` result gathers every page before completing. +For reactive types `Mono` / `Flux`, use `CassandraReactiveResultSetMapper`, which receives `ReactiveResultSet` and returns the required `Publisher`. ===! ":fontawesome-brands-java: `Java`" ```java - final class AsyncResultMapper implements CassandraReactiveResultSetMapper> { + final class ReactiveResultMapper implements CassandraReactiveResultSetMapper> { @Override - public UUID apply(ResultSet rows) { + public Flux apply(ReactiveResultSet rows) { return Flux.from(rows).map(r -> UUID.fromString(r.getString(0))); } } @@ -770,7 +1048,7 @@ Due to the nature of the helper class for extracting data from `AsyncResultSet` @Repository public interface EntityRepository extends CassandraRepository { - @Mapping(AsyncResultMapper.class) + @Mapping(ReactiveResultMapper.class) @Query("SELECT id FROM entities") Flux getIds(); } @@ -779,7 +1057,7 @@ Due to the nature of the helper class for extracting data from `AsyncResultSet` === ":simple-kotlin: `Kotlin`" ```kotlin - class AsyncResultMapper : CassandraReactiveResultSetMapper> { + class ReactiveResultMapper : CassandraReactiveResultSetMapper> { override fun apply(rows: ReactiveResultSet): Flux { return Flux.from(rows).map { r -> UUID.fromString(r.getString(0)) } } @@ -788,58 +1066,216 @@ Due to the nature of the helper class for extracting data from `AsyncResultSet` @Repository interface EntityRepository : CassandraRepository { - @Mapping(AsyncResultMapper::class) + @Mapping(ReactiveResultMapper::class) @Query("SELECT id FROM entities") fun getIds(): Flux } ``` +## Manual Query { #query } + +If a query is hard to express as a single static `@Query`, you can declare a regular method with an implementation and build `CQL` manually. +The repository exposes `getCassandraConnectionFactory()`, and `CassandraConnectionFactory#query` executes such a query: +it prepares the statement through the current `CqlSession`, wraps execution in `Kora` telemetry, and returns the value produced by the callback. +The `currentSession()` accessor returns the active `CqlSession`, and `telemetry()` returns the `DataBaseTelemetry` used for reporting. + +`QueryContext` carries the query identifier and the final `CQL`. +The identifier is reported to telemetry, so use a stable name such as `Repository.method`. +Bind values through a `BoundStatement` obtained from the prepared statement; never concatenate values directly into the query string. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends CassandraRepository { + + default List findByFilter(@Nullable String value2) { + var sql = new StringBuilder("SELECT id, value1, value2, value3 FROM entities"); + if (value2 != null) { + sql.append(" WHERE value2 = ? ALLOW FILTERING"); + } + + var connectionFactory = getCassandraConnectionFactory(); + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return connectionFactory.query(queryContext, statement -> { + var boundStatement = (value2 != null) + ? statement.bind(value2) + : statement.bind(); + var resultSet = connectionFactory.currentSession().execute(boundStatement); + + var result = new ArrayList(); + for (var row : resultSet) { + result.add(new Entity( + row.getString("id"), + row.getInt("value1"), + row.getString("value2"), + row.getString("value3"))); + } + return result; + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : CassandraRepository { + + fun findByFilter(value2: String?): List { + val sql = StringBuilder("SELECT id, value1, value2, value3 FROM entities") + if (value2 != null) { + sql.append(" WHERE value2 = ? ALLOW FILTERING") + } + + val connectionFactory = cassandraConnectionFactory + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return connectionFactory.query(queryContext) { statement -> + val boundStatement = if (value2 != null) statement.bind(value2) else statement.bind() + val resultSet = connectionFactory.currentSession().execute(boundStatement) + + resultSet.map { row -> + Entity( + row.getString("id"), + row.getInt("value1"), + row.getString("value2"), + row.getString("value3") + ) + } + } + } + } + ``` + +Because `Cassandra` has no transactions, `query` simply runs on the current session with telemetry; there is no commit or rollback to manage. + ## UDT { #udt } -There is support for [UDT](https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useCreateUDT.html) -types using the `@UDT` annotation: +There is support for [UDT](https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useCreateUDT.html) types through the `@UDT` annotation. +`UDT` describes a Cassandra user-defined type and can be used as a field of a regular entity. +The `@UDT` type is mapped like any other entity, so the enclosing entity is annotated with `@EntityCassandra`. + +Given the following schema, where `username` is a user-defined type stored as a `FROZEN` column: + +```cql +CREATE TYPE IF NOT EXISTS username(first text, last text); + +CREATE TABLE IF NOT EXISTS entities_udt +( + id VARCHAR, + name FROZEN, + PRIMARY KEY (id) +); +``` + +the view and repository look like this: ===! ":fontawesome-brands-java: `Java`" ```java - @Table("entities") - public record Entity(String id, Name name) { + @Repository + public interface EntityRepository extends CassandraRepository { + + @EntityCassandra + record Entity(String id, Name name) { + + @UDT + record Name(String first, String last) {} + } - @UDT - public record Name(String first, String middle, String last) { } + @Query("SELECT * FROM entities_udt WHERE id = :id") + @Nullable + Entity findById(String id); + + @Query(""" + INSERT INTO entities_udt(id, name) + VALUES (:entity.id, :entity.name) + """) + void insert(Entity entity); } ``` === ":simple-kotlin: `Kotlin`" ```kotlin - @Table("entities") - data class Entity(val id: String, val name: Name) { + @Repository + interface EntityRepository : CassandraRepository { + + @EntityCassandra + data class Entity(val id: String, val name: Name) { + + @UDT + data class Name(val first: String, val last: String) + } - @UDT - data class Name(val first: String, val middle: String, val last: String) + @Query("SELECT * FROM entities_udt WHERE id = :id") + fun findById(id: String): Entity? + + @Query(""" + INSERT INTO entities_udt(id, name) + VALUES (:entity.id, :entity.name) + """) + fun insert(entity: Entity) } ``` +If the `UDT` type is not used through an enclosing entity, but as a standalone Cassandra type, mapper generation can be enabled explicitly with `@EntityCassandra`. +This is useful when the mapper is needed as a separate graph component. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @EntityCassandra + public record Name(String first, String last) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @EntityCassandra + data class Name(val first: String, val last: String) + ``` + +### Macros { #macros } + +To simplify writing `CQL` queries, use macros — they expand into `CQL` constructs at compile time. +Usage examples are shown above in the [Usage](#usage) section (`findById` and `insert` methods). + +**Detailed documentation:** [Common Database Rules — Macros](database-common.md#macros) + ## Signatures { #signatures } Available signatures for repository methods out of the box: ===! ":fontawesome-brands-java: `Java`" - The `T` refers to the type of the return value, either `List` or `Void`. + `T` means the return value type, `List`, or `Void`. - `T myMethod()` - `@Nullable T myMethod()` - `Optional myMethod()` - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) + - `CompletableFuture myMethod()` [CompletableFuture](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html) - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + The `CompletionStage`, `CompletableFuture`, and `Mono` wrappers can also wrap `List`, + for example `CompletionStage>` or `Mono>`. + + Method parameters can include regular values, DTOs, `@Batch List` for batch execution, and `CqlSession` when the method needs access to the current driver session. + === ":simple-kotlin: `Kotlin`" - By `T` we mean the type of the return value, either `T?`, either `List`, or `Unit`. + `T` means the return value type, `T?`, `List`, or `Unit`. - `myMethod(): T` - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + + Method parameters can include regular values, DTOs, `@Batch List` for batch execution, and `CqlSession` when the method needs access to the current driver session. + +## Telemetry { #telemetry } + +Logging, metrics, and tracing are configured via the `telemetry` block in the [configuration](#configuration) and described in the [Metrics Reference](metrics.md#database) section. +To completely override telemetry, you can provide custom SPI factories; see the [Common Database Documentation](database-common.md#telemetry) for details. diff --git a/mkdocs/docs/en/documentation/database-common.md b/mkdocs/docs/en/documentation/database-common.md index c62769f..bfc8bfd 100644 --- a/mkdocs/docs/en/documentation/database-common.md +++ b/mkdocs/docs/en/documentation/database-common.md @@ -1,10 +1,17 @@ ---- +--- description: "Explains Common Kora database model and repository conventions: entities, identifiers, naming, embedded fields, query macros, batch queries, and repository inheritance. Use when working with @Table, @Column, @Id, @Embedded, @Repository, @Query, @Batch, @Mapping." agent: use_when: "Use this file for Kora docs or implementation questions about Common Kora database model and repository conventions: entities, identifiers, naming, embedded fields, query macros, batch queries, and repository inheritance; key triggers include @Table, @Column, @Id, @Embedded, @Repository, @Query, @Batch, @Mapping, Entity, Repository." --- Basic principles and mechanisms of database modules in Kora. +This section describes the common model for `JDBC`, `Cassandra`, `R2DBC`, and `Vertx`: entities, repositories, query parameters, batch queries, affected row counts, and macros. +Connection configuration, transactions, supported signatures, and driver-specific mappers are described in the documentation for each database implementation. + +This section intentionally does not describe driver-specific details. +For connection configuration, transactions, return value types, database-generated identifiers, service method parameters, +and exact mapper interfaces, see the documentation for the required implementation: +[`JDBC`](database-jdbc.md), [`Cassandra`](database-cassandra.md), [`R2DBC`](database-r2dbc.md), or [`Vertx`](database-vertx.md). We think that the best way to communicate with a SQL database is to communicate in its native SQL language. Other tools often have limitations on using specific functions of a particular database, @@ -13,13 +20,13 @@ carries a lot of non-obviousness and potential errors on the part of the develop For a step-by-step walkthrough before the reference details, see [JDBC Database](../guides/database-jdbc.md) and [Advanced JDBC Database](../guides/database-jdbc-advanced.md). -## Entity { #entity } +## View { #view } -An entity is a representation of data from a database in the form of a class with fields. +A view is a representation of data from a database in the form of a class with fields. -Entities used as a return value must contain a single public +Views used as a return value must contain a single public constructor. This can be either a default constructor or a constructor with parameters. -If Kora finds a constructor with parameters, the entity object will be created based on it. +If Kora finds a constructor with parameters, the view object will be created based on it. In the case of an empty constructor, the fields will be filled [via setters](https://docs.oracle.com/cd/E19316-01/819-3669/bnais/index.html). ===! ":fontawesome-brands-java: `Java`" @@ -27,7 +34,6 @@ In the case of an empty constructor, the fields will be filled [via setters](htt ```java public record Entity(String id, String name) {} ``` - === ":simple-kotlin: `Kotlin`" ```kotlin @@ -36,9 +42,9 @@ In the case of an empty constructor, the fields will be filled [via setters](htt ### Table { #table } -You can specify which table the entity belongs to, this will be needed if you use [macros](#macros) when building queries. +You can specify which table the view belongs to, this will be needed if you use [macros](#macros) when building queries. -If no table is specified, macros will use the class name in [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). +If no table is specified, macros will use the class name in [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). ===! ":fontawesome-brands-java: `Java`" @@ -56,8 +62,8 @@ If no table is specified, macros will use the class name in [snake_lower_case](h ### Identifier { #identifier } -Since all data manipulations are performed by converting the entity into a driver query, -there is no need to allocate a special primary key within an entity to work with the entity. +Since all data manipulations are performed by converting the view into a driver query, +there is no need to allocate a special primary key within a view to work with the view. Identifying what exactly is a primary key can be useful when using [macros](#macros), the `@Id` annotation can be used for this purpose. @@ -79,7 +85,7 @@ the `@Id` annotation can be used for this purpose. Let's look at creating an identity as a sequence of numbers using Postgres as an example, Kora suggests using the database mechanism [identity column](https://www.tutorialsteacher.com/postgresql/identity-column). -An example table for such an entity would look like this: +An example table for such a view would look like this: ```sql CREATE TABLE IF NOT EXISTS entities @@ -127,11 +133,58 @@ or use [special constructs](https://www.postgresql.org/docs/current/dml-returnin } ``` +Instead of a driver-specific `RETURNING`, the primary key generated by the database on insertion can be returned +by marking the repository **method** itself with `@Id` (the annotation targets both a view field and a method). +The exact generated-identifier behavior and the supported return signatures are driver-specific and are described +for [JDBC](database-jdbc.md#generated-identifier) and [R2DBC](database-r2dbc.md#generated-identifier): + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends JdbcRepository { + + @Table("entities") + public record Entity(@Id Long id, String name) {} + + @Id //(1)! + @Query("INSERT INTO %{entity#inserts -= id}") //(2)! + long insert(Entity entity); + } + ``` + + 1. Marks the method so that the identifier generated by the database is returned. + 2. Expands into a query: + ```sql + INSERT INTO entities(name) VALUES(:entity.name) + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + @Table("entities") + data class Entity(@field:Id val id: Long, val name: String) + + @Id //(1)! + @Query("INSERT INTO %{entity#inserts -= id}") //(2)! + fun insert(entity: Entity): Long + } + ``` + + 1. Marks the method so that the identifier generated by the database is returned. + 2. Expands into a query: + ```sql + INSERT INTO entities(name) VALUES(:entity.name) + ``` + #### Random { #random } It is suggested to use the standard `UUID` from Java to create a random identifier: -An example table for such an entity would look like this: +An example table for such a view would look like this: ```sql CREATE TABLE IF NOT EXISTS entities @@ -185,10 +238,10 @@ When a composite key is required, it is intended to use the `@Embedded` annotati ### Naming { #naming } -By default, entity field names are translated to [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) when retrieving a +By default, view field names are translated to [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) when retrieving a result. -If you want to customize the mapping of specific fields from the database to an entity, you can use the `@Column` annotation: +If you want to customize the mapping of specific fields from the database to a view, you can use the `@Column` annotation: ===! ":fontawesome-brands-java: `Java`" @@ -206,13 +259,13 @@ If you want to customize the mapping of specific fields from the database to an #### Naming Strategy { #naming-strategy } -If you want to use a naming strategy for the entire entity, it is suggested to create a `NameConverter` implementation and then use it in the `@NamingStrategy` annotation. +If you want to use a naming strategy for the entire view, it is suggested to create a `NameConverter` implementation and then use it in the `@NamingStrategy` annotation. It is required that the `NameConverter` implementation has a constructor without parameters. Either use the available strategies from Kora: - `NoopNameConverter` - the strategy uses the default field name. -- `SnakeCaseNameConverter` - strategy uses [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). +- `SnakeCaseNameConverter` - strategy uses [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). - `SnakeCaseUpperNameConverter` - strategy uses [SNAKE_UPPER_CASE](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). - `PascalCaseNameConverter` - the strategy uses [PascalCase](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). - `CamelCaseNameConverter` - the strategy uses [camelCase](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). @@ -237,7 +290,7 @@ Either use the available strategies from Kora: ===! ":fontawesome-brands-java: `Java`" - By default, all fields declared in an entity are considered **required** (*NotNull*). + By default, all fields declared in a view are considered **required** (*NotNull*). ```java public record Entity(String id, @@ -246,7 +299,7 @@ Either use the available strategies from Kora: === ":simple-kotlin: `Kotlin`" - By default, all fields declared in an entity that do not use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax are considered **required** (*NotNull*). + By default, all fields declared in a view that do not use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax are considered **required** (*NotNull). ```kotlin data class Entity(val id: String, @@ -257,8 +310,8 @@ Either use the available strategies from Kora: ===! ":fontawesome-brands-java: `Java`" - In case a field in an entity is optional, that is, it may not exist then, - you can use the `@Nullable` annotation to match the field in Json and DTO. + If a view field is optional, meaning it may be absent, + use the `@Nullable` annotation to mark it explicitly. ```java public record Entity(String id, @@ -294,7 +347,7 @@ Either use the available strategies from Kora: ### Embedded fields { #embedded-fields } -In case you want to use nested fields, i.e. convert entity fields into specific classes, you can use the `@Embedded` annotation. +In case you want to use nested fields, i.e. convert view fields into specific classes, you can use the `@Embedded` annotation. Suppose there is a SQL table where there is a composite key which we want to express as a separate class: @@ -308,7 +361,7 @@ CREATE TABLE IF NOT EXISTS entities ) ``` -Then the entity will look like this: +Then the view will look like this: ===! ":fontawesome-brands-java: `Java`" @@ -325,7 +378,7 @@ Then the entity will look like this: ```kotlin data class Entity( @field:Id @field:Embedded val id: UserID, - @field:Column("name") val info: String + @field:Column("info") val info: String ) { data class UserID( @@ -335,7 +388,7 @@ Then the entity will look like this: } ``` -Then the repository for such an entity would look like this: +Then the repository for such a view would look like this: ===! ":fontawesome-brands-java: `Java`" @@ -370,7 +423,7 @@ Then the repository for such an entity would look like this: WHERE name = :id.name AND surname = :id.surname; """ ) - fun findById(id: Entity.CompositeID): Entity? + fun findById(id: Entity.UserID): Entity? @Query( """ @@ -401,8 +454,8 @@ Repository interface must be annotated with `@Repository`. Queries for repository methods are described using the `@Query` annotation. Repository implementation is created at compile time, all `@Query` methods will execute described query and assemble the query arguments and process the result optimally. -SQL queries are supposed to be written by the developer because it increases the developer's understanding of the query plan, -gives more insight and context to the developer about what he is doing and how his query will work. +`SQL` queries are supposed to be written by the developer because it increases the developer's understanding of the query plan, +gives more insight and context about what the query does and how it will work. You can use [macros](#macros) to improve the user experience to avoid writing all model fields/columns. Repository must extend of one of the implementations, in the examples below the [JDBC](database-jdbc.md) implementation will be considered: @@ -423,7 +476,7 @@ Repository must extend of one of the implementations, in the examples below the ``` 1. Indicates that the interface is a repository. - 2. Indicates that it is necessary to create a method implementation that executes the SQL query specified in the annotation. + 2. Indicates that Kora should create a method implementation that executes the `SQL` query specified in the annotation. === ":simple-kotlin: `Kotlin`" @@ -440,14 +493,97 @@ Repository must extend of one of the implementations, in the examples below the ``` 1. Indicates that the interface is a repository. - 2. Indicates that it is necessary to create a method implementation that executes the SQL query specified in the annotation. + 2. Indicates that Kora should create a method implementation that executes the `SQL` query specified in the annotation. + +### Query parameters { #query-parameters } + +Repository method parameters are bound to named parameters in `@Query`. +A simple parameter is referenced by the method parameter name: `:id`, `:name`, `:status`. +If a parameter is an entity or a `DTO`, its fields can be referenced with dot notation: `:entity.id`, `:entity.name`, `:filter.status`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends JdbcRepository { + + @Query(""" + SELECT id, name FROM entities + WHERE id = :id AND name = :filter.name + """) + @Nullable + Entity findById(String id, Filter filter); + + record Filter(String name) {} + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + @Query( + """ + SELECT id, name FROM entities + WHERE id = :id AND name = :filter.name + """ + ) + fun findById(id: String, filter: Filter): Entity? + + data class Filter(val name: String) + } + ``` + +If a parameter appears in the query more than once, Kora binds it to every occurrence. +If a method parameter is not used in the query and is not a service parameter of a specific driver, compilation fails. + +### Mappers { #mappers } + +Use the `@Mapping` annotation when a value needs a non-standard database representation. +It can be placed on a view field, a method parameter, or a repository method: + +- on a view field, to customize reading or writing a specific column; +- on a method parameter, to customize writing a specific query parameter; +- on a repository method, to customize processing the whole query result or a result row. + +An arbitrary mapper cannot be used in every location: its type must match where it is applied. +A parameter mapper is applied to a query parameter, a column mapper to a view field, and a result or row mapper to a repository method. +The exact set of supported interfaces depends on the driver: for example, `JDBC` uses `JdbcRowMapper`, `JdbcResultSetMapper`, `JdbcResultColumnMapper`, and `JdbcParameterColumnMapper`. +Similar interfaces for `Cassandra`, `R2DBC`, and `Vertx`, as well as their usage details, are described in the documentation for each database implementation. +All driver row mappers share the common `RowMapper` (`ru.tinkoff.kora.database.common.RowMapper`) marker interface, which is the base type behind driver-specific mappers such as `JdbcRowMapper` and `CassandraRowMapper`. +The `@Mapping` annotation itself comes from the core `common` module (`ru.tinkoff.kora.common.Mapping`). +If a mapper is specified with `@Mapping`, Kora adds it as a dependency of the generated repository and uses it instead of the default mapper. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Table("entities") + public record Entity(@Id String id, + @Mapping(JsonParameterMapper.class) + @Column("payload") + String payload) {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Table("entities") + data class Entity( + @field:Id val id: String, + @field:Mapping(JsonParameterMapper::class) + @field:Column("payload") + val payload: String + ) + ``` ### Batch query { #batch-query } Kora supports batch queries with the `@Batch` annotation. Unlike executing SQL queries sequentially, batch processing allows you to send an entire set of queries in a single call, -reducing the number of network connections required and allowing some queries to be executed in parallel on the database side, +reducing the number of network round trips required and allowing some queries to be executed in parallel on the database side, which can increase the speed of execution. ===! ":fontawesome-brands-java: `Java`" @@ -478,10 +614,34 @@ which can increase the speed of execution. **Batch query** can't return arbitrary values, such a method can return `Unit`, or `UpdateCount`, or database-generated identifiers for [JDBC](database-jdbc.md#generated-identifier) or [R2DBC](database-r2dbc.md#generated-identifier) drivers. +`@Batch` is placed on a collection parameter, and each collection element is substituted into the same query one by one. +All other method parameters, if present, are shared by all batch elements. +For example, in `INSERT INTO logs(tenant_id, id, value) VALUES (:tenantId, :entity.id, :entity.value)`, +the `tenantId` parameter is the same for every element, while `entity` fields are taken from each collection element. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Query("INSERT INTO logs(tenant_id, id, value) VALUES (:tenantId, :entity.id, :entity.value)") + UpdateCount insert(String tenantId, @Batch List entity); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Query("INSERT INTO logs(tenant_id, id, value) VALUES (:tenantId, :entity.id, :entity.value)") + fun insert(tenantId: String, @Batch entity: List): UpdateCount + ``` + +A method must have no more than one parameter annotated with `@Batch`. +Support for database-generated identifiers in batch queries depends on the specific driver and is described in the corresponding section. + ### Affected rows { #affected-rows } Kora does not process the contents of the query, the result of the method is always derived from the rows returned by the database. -If you want to get the number of updated rows as a result, you should use a special type `UpdateCount`. +If you want to get the number of affected rows as a result, use the special `UpdateCount` type. +For a regular query, `UpdateCount#value()` contains the row count returned by the driver for the executed query. +For a batch query, the value is usually the sum of results for all batch elements; exact behavior depends on the database driver. ===! ":fontawesome-brands-java: `Java`" @@ -513,6 +673,10 @@ you can use the built-in connection factory method to create a method with fully You can also use other repository methods within the method and they will also be executed within a single transaction if required. For more details about transactions, see the documentation for the specific repository implementation. +Repositories can declare regular methods with implementations. +This is useful when a more complex operation should stay close to the queries: for example, executing several `@Query` methods in one transaction, +building a result from several queries, or keeping a database operation sequence inside the repository instead of moving it to a service layer. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -521,15 +685,17 @@ For more details about transactions, see the documentation for the specific repo public record Entity(Long id, String name) {} - default int insert(Entity entity) { - return getJdbcConnectionFactory().inTx(connection -> { - String sql = "INSERT INTO entities(name) VALUES (?) RETURNING id"; - try(PreparedStatement preparedStatement = connection.prepareStatement(sql)) { - preparedStatement.setString(1, entity.name()); - try(ResultSet resultSet = preparedStatement.executeQuery()) { - return resultSet.getInt(1); - } - } + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + UpdateCount insert(Entity entity); + + @Query("UPDATE entities SET name = :name WHERE id = :id") + UpdateCount updateName(Long id, String name); + + default Entity saveAndRename(Entity entity, String name) { + return getJdbcConnectionFactory().inTx(() -> { + insert(entity); + updateName(entity.id(), name); + return new Entity(entity.id(), name); }); } } @@ -543,18 +709,28 @@ For more details about transactions, see the documentation for the specific repo data class Entity(val id: Long, val name: String) - fun insert(entity: Entity): Int { - return jdbcConnectionFactory.inTx { connection -> - val sql = "INSERT INTO entities(name) VALUES (?) RETURNING id" - connection.prepareStatement(sql).use { preparedStatement -> - preparedStatement.setString(1, entity.name) - preparedStatement.executeQuery().use { resultSet -> resultSet.getInt(1) } - } + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + fun insert(entity: Entity): UpdateCount + + @Query("UPDATE entities SET name = :name WHERE id = :id") + fun updateName(id: Long, name: String): UpdateCount + + fun saveAndRename(entity: Entity, name: String): Entity { + return jdbcConnectionFactory.inTx { + insert(entity) + updateName(entity.id, name) + Entity(entity.id, name) } } } ``` +When you build `SQL` manually through the driver's connection factory rather than using a `@Query` method, +the query still flows through Kora telemetry. The executed query is described by a shared +`QueryContext(queryId, sql, operation)`: `queryId` is a stable query identifier reported to telemetry +(a name such as `Repository.method` is convenient), `sql` is the final query text, and `operation` defaults to `db_query`. +The exact connection-factory method and its signature are driver-specific — see [JDBC](database-jdbc.md#query) for a worked example. + ### Multiple databases { #multiple-databases } Sometimes you need to access different databases in different repositories within the same application, @@ -641,18 +817,18 @@ Repositories with a main database connection, doesn't require tag. ### Macros { #macros } -The most frustrating part of writing SQL queries can be listing and keeping the columns and fields of an entity up to date. +The most frustrating part of writing SQL queries can be listing and keeping the columns and fields of a view up to date. -In order to solve this problem you can use special macros constructions within the SQL query within the `@Query` annotation. -These constructions allow you to operate target [entity](#entity) and expand it into specific SQL constructions and easily augment into SQL queries. -Macros is an assistant when writing SQL queries, expands into constructions that the user could write with his own hands. +To solve this problem, use special macro constructions inside an `SQL` query in the `@Query` annotation. +These constructions operate on the target [view](#view), expand it into specific `SQL` constructions, and make it easier to extend `SQL` queries. +A macro is a helper for writing `SQL` queries and expands into constructions that the user could write manually. The syntax of the macros looks as follows: `%{return#selects}`. 1. The macros is limited by the syntactic construction `%{` and `}` 2. The target of the macros is specified first, it can be either the name of any method argument or the return value using the `return` keyword 3. Then the `#` character is used to separate the macros target and the macros command -4. The macros command is then specified, which tells which SQL construction to expand the entity into +4. The macros command is then specified, which tells which SQL construction to expand the view into ===! ":fontawesome-brands-java: `Java`" @@ -700,11 +876,11 @@ The syntax of the macros looks as follows: `%{return#selects}`. Available macros commands: -- `table` - construction exposes the entity value in [annotation](#table) `@Table` or if none is available, translates the entity name to [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) -- `selects` - creates an entity column enumeration construction for a `SELECT` query -- `inserts` - creates a table, column enumeration construction and corresponding entity fields for an `INSERT` query -- `updates` - creates a column enumeration construction and corresponding entity fields for `UPDATE` query -- `where` - creates a column enumeration construction with a value from the entity for the `WHERE` part of the query +- `table` - expands the view value from the `@Table` [annotation](#table), or, if it is absent, translates the view name to [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) +- `selects` - creates a view column enumeration construction for a `SELECT` query +- `inserts` - creates a table, column enumeration construction and corresponding view fields for an `INSERT` query +- `updates` - creates a column enumeration construction and corresponding view fields for `UPDATE` query +- `where` - creates a column enumeration construction with a value from the view for the `WHERE` part of the query #### Field enumeration { #field-enumeration } @@ -716,8 +892,8 @@ Spaces can be placed **only** between fields in the enumeration or special enume Special enumeration symbols are available: -1. `=` - only the entity fields name specified after the symbol will participate in the command expansion -2. `-=` - all entity fields except those specified after the symbol will participate in command expansion +1. `=` - only the view fields name specified after the symbol will participate in the command expansion +2. `-=` - all view fields except those specified after the symbol will participate in command expansion ===! ":fontawesome-brands-java: `Java`" @@ -766,7 +942,7 @@ Special enumeration symbols are available: ##### Identifier { #identifier-2 } When listing fields in a macro, it is possible to use the special keyword `@id` -to refer immediately to the entity identifier annotated with [annotation](#identifier) `@Id`. +to refer immediately to the view identifier annotated with [annotation](#identifier) `@Id`. This can be especially useful when the identifier is a [compound key](#embedded-fields), to list all columns at once. @@ -816,7 +992,7 @@ This can be especially useful when the identifier is a [compound key](#embedded- #### Repository example { #repository-example } -Example of a complete repository with all the basic methods for operating an entity for [Postgres SQL](https://postgrespro.com/docs/postgresql): +Example of a complete repository with all the basic methods for operating a view for [Postgres SQL](https://postgrespro.com/docs/postgresql): ===! ":fontawesome-brands-java: `Java`" @@ -879,7 +1055,7 @@ Example of a complete repository with all the basic methods for operating an ent 5. Expands into a query: ```sql INSERT INTO entities(id, value1, value2, value3) - VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + VALUES(:entity.id, :entity.field1, :entity.value2, :entity.value3) ON CONFLICT (id) DO UPDATE SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` @@ -920,7 +1096,6 @@ Example of a complete repository with all the basic methods for operating an ent fun deleteAll(): UpdateCount } ``` - 1. Expands into a query: ```sql SELECT id, value1, value2, value3 @@ -935,7 +1110,7 @@ Example of a complete repository with all the basic methods for operating an ent 3. Expands into a query: ```sql INSERT INTO entities(id, value1, value2, value3) - VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + VALUES(:entity.id, :entity.field1, :entity.value2, :entity.value3) ``` 4. Expands into a query: ```sql @@ -946,7 +1121,7 @@ Example of a complete repository with all the basic methods for operating an ent 5. Expands into a query: ```sql INSERT INTO entities(id, value1, value2, value3) - VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + VALUES(:entity.id, :entity.field1, :entity.value2, :entity.value3) ON CONFLICT (id) DO UPDATE SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` @@ -995,32 +1170,32 @@ it is almost identical to the previous one except for the `WHERE` conditions for } ``` - 1. Раскрывается в запрос: + 1. Expands into a query: ```sql SELECT code, type, value1, value2, value3 FROM entities WHERE code = :code AND type = :type ``` - 2. Раскрывается в запрос: + 2. Expands into a query: ```sql SELECT code, type, value1, value2, value3 FROM entities ``` - 3. Раскрывается в запрос: + 3. Expands into a query: ```sql INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) ``` - 4. Раскрывается в запрос: + 4. Expands into a query: ```sql UPDATE entities SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 WHERE code = :entity.id.code AND type = :entity.id.type ``` - 5. Раскрывается в запрос: + 5. Expands into a query: ```sql INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) ON CONFLICT (code, type) DO UPDATE SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` @@ -1064,33 +1239,32 @@ it is almost identical to the previous one except for the `WHERE` conditions for fun deleteAll(): UpdateCount } ``` - - 1. Раскрывается в запрос: + 1. Expands into a query: ```sql SELECT code, type, value1, value2, value3 FROM entities WHERE code = :code AND type = :type ``` - 2. Раскрывается в запрос: + 2. Expands into a query: ```sql SELECT code, type, value1, value2, value3 FROM entities ``` - 3. Раскрывается в запрос: + 3. Expands into a query: ```sql INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) ``` - 4. Раскрывается в запрос: + 4. Expands into a query: ```sql UPDATE entities SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 WHERE code = :entity.id.code AND type = :entity.id.type ``` - 5. Раскрывается в запрос: + 5. Expands into a query: ```sql INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) ON CONFLICT (code, type) DO UPDATE SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` @@ -1201,3 +1375,61 @@ You can also create an abstract CRUD repository and then use it in inheritance f fun deleteAll(): UpdateCount } ``` + +## Telemetry { #telemetry } + +All database drivers share a common telemetry contract for logging, metrics, and tracing of queries. +The concrete configuration knobs (the `telemetry { logging / metrics / tracing }` section) are described in the documentation +for each driver, for example [JDBC](database-jdbc.md#configuration); this section documents only the shared extension points +that live in `ru.tinkoff.kora.database.common.telemetry`. + +For every executed query a `DataBaseTelemetry.DataBaseTelemetryContext` is created and closed when the query finishes +(receiving the thrown exception, if any). +The query being executed is described by `QueryContext(queryId, sql, operation)`, where `queryId` is a stable query +identifier reported to telemetry, `sql` is the final query text, and `operation` defaults to `db_query`. + +The default factory `DefaultDataBaseTelemetryFactory` combines three optional sub-factories: + +- `DataBaseLoggerFactory` builds a `DataBaseLogger` that logs query begin/end (`logQueryBegin` / `logQueryEnd`); +- `DataBaseMetricWriterFactory` builds a `DataBaseMetricWriter` that records per-query metrics (`recordQuery`); +- `DataBaseTracerFactory` builds a `DataBaseTracer` that creates query and call spans for distributed tracing. + +If none of the sub-factories produces an implementation (for example, when logging, [metrics](metrics.md), and [tracing](tracing.md) +are all disabled in configuration), `DataBaseTelemetryFactory.EMPTY` is used and telemetry becomes a no-op. + +In case you want to provide fully customize telemetry, provide your own `DataBaseTelemetryFactory` in the [application graph](container.md), +which [overrides](container.md#component-override) the default one: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends JdbcDatabaseModule { + + default DataBaseTelemetryFactory dataBaseTelemetryFactory() { //(1)! + return (config, name, driverType, dbType, username) -> { + // build and return a custom DataBaseTelemetry + return DataBaseTelemetryFactory.EMPTY; + }; + } + } + ``` + + 1. Overrides the default `DataBaseTelemetryFactory` provided by `DataBaseModule`. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : JdbcDatabaseModule { + + fun dataBaseTelemetryFactory(): DataBaseTelemetryFactory { //(1)! + return DataBaseTelemetryFactory { config, name, driverType, dbType, username -> + // build and return a custom DataBaseTelemetry + DataBaseTelemetryFactory.EMPTY + } + } + } + ``` + + 1. Overrides the default `DataBaseTelemetryFactory` provided by `DataBaseModule`. diff --git a/mkdocs/docs/en/documentation/database-jdbc.md b/mkdocs/docs/en/documentation/database-jdbc.md index 0a6dce9..e4b5e31 100644 --- a/mkdocs/docs/en/documentation/database-jdbc.md +++ b/mkdocs/docs/en/documentation/database-jdbc.md @@ -4,8 +4,14 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora JDBC repositories, JDBC configuration, result and parameter mapping, generated identifiers, transactions, and repository method signatures; key triggers include @Repository, @Query, @EntityJdbc, @Table, @Id, @Column, @Batch, JdbcDatabaseModule, JdbcConnectionFactory, JdbcRepository." --- -Module provides a repository implementation based on the [JDBC](https://proselyte.net/tutorials/jdbc/introduction/) database protocol -and using [Hikari](https://github.com/brettwooldridge/HikariCP) for connection set management. +The module provides a repository implementation based on [JDBC](https://proselyte.net/tutorials/jdbc/introduction/) for +working with relational databases and uses [Hikari](https://github.com/brettwooldridge/HikariCP) to manage the connection +pool. +You describe a repository interface and `SQL` queries with `@Repository` and `@Query`, and `Kora` generates an implementation +that obtains a connection from the pool, binds parameters, reads the result, and participates in transactions. + +Common rules for entities, `@Repository`, `@Query`, `@Batch`, `UpdateCount`, macros, manual queries, and other repository +mechanisms are described in [Common database rules](database-common.md). For a step-by-step walkthrough before the reference details, see [JDBC Database](../guides/database-jdbc.md) and [Advanced JDBC Database](../guides/database-jdbc-advanced.md). @@ -37,77 +43,29 @@ For a step-by-step walkthrough before the reference details, see [JDBC Database] interface Application : JdbcDatabaseModule ``` -Also **required to provide** the database driver implementation as a dependency. +You also **must provide** the database driver implementation as a dependency. ## Configuration { #configuration } -Example of the complete configuration described in the `JdbcDatabaseConfig` class (default or example values are specified): +Basic JDBC configuration parameters: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript db { jdbcUrl = "jdbc:postgresql://localhost:5432/postgres" //(1)! username = "postgres" //(2)! password = "postgres" //(3)! - schema = "public" //(4)! - poolName = "kora" //(5)! - maxPoolSize = 10 //(6)! - minIdle = 0 //(7)! - connectionTimeout = "10s" //(8)! - validationTimeout = "5s" //(9)! - idleTimeout = "10m" //(10)! - maxLifetime = "15m" //(11)! - leakDetectionThreshold = "0s" //(12)! - initializationFailTimeout = "0s" //(13)! - readinessProbe = false //(14)! - dsProperties { //(15)! - "hostRecheckSeconds": "2" - } - telemetry { - logging { - enabled = false //(16)! - } - metrics { - enabled = true //(17)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(18)! - tags = { // (19)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(20)! - attributes = { // (21)! - "key1" = "value1" - "key2" = "value2" - } - } - } + poolName = "kora" //(4)! + maxPoolSize = 10 //(5)! } ``` - 1. JDBC database connection URL (**required**) - 2. Username to connect (**required**) - 3. Password of the user to connect (**required**) - 4. Database schema for the connection - 5. Name of the database connection set in Hikari (**required**) - 6. Maximum size of the database connection set in Hikari - 7. Minimum size of the set of ready connections to the database in Hikari in standby mode - 8. Maximum time to establish a connection in Hikari - 9. Maximum time for connection verification in Hikari - 10. Maximum time for connection downtime in Hikari - 11. Maximum lifetime of a connection in Hikari - 12. Maximum time a connection can be idle in Hikari before it is considered a leak (optional) - 13. Maximum time to wait for connection initialization at service startup (optional) - 14. Whether to enable [readiness probe](probes.md#readiness) for database connection - 15. Additional JDBC connection attributes `dataSourceProperties` (below example `hostRecheckSeconds` parameters) (optional) - 16. Enables module logging (default `false`) - 17. Enables module metrics (default `true`) - 18. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 19. Configures tags for metrics (optional) - 20. Enables module tracing (default `true`) - 21. Configures attributes for tracing (optional) + 1. `JDBC URL` for database connection (`required`, no default) + 2. Username for connection (`required`, no default) + 3. Password for connection (`required`, no default) + 4. `Hikari` connection pool name (`required`, no default) + 5. Maximum `Hikari` connection pool size (default: `10`) === ":simple-yaml: `YAML`" @@ -116,81 +74,247 @@ Example of the complete configuration described in the `JdbcDatabaseConfig` clas jdbcUrl: "jdbc:postgresql://localhost:5432/postgres" #(1)! username: "postgres" #(2)! password: "postgres" #(3)! - schema: "public" #(4)! - poolName: "kora" #(5)! - maxPoolSize: 10 #(6)! - minIdle: 0 #(7)! - connectionTimeout: "10s" #(8)! - validationTimeout: "5s" #(9)! - idleTimeout: "10m" #(10)! - maxLifetime: "15m" #(11)! - leakDetectionThreshold: "0s" #(12)! - initializationFailTimeout: "0s" //(13)! - readinessProbe: false //(14)! - dsProperties: #(15)! - hostRecheckSeconds: "1" - telemetry: - logging: - enabled: false #(16)! - metrics: - enabled: true #(17)! - slo: [ 2, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(18)! - tags: #(19)! - key1: value1 - key2: value2 - tracing: - enabled: true #(20)! - attributes: #(21)! - key1: value1 - key2: value2 - } - ``` - - 1. JDBC database connection URL (**required**) - 2. Username to connect (**required**) - 3. Password of the user to connect (**required**) - 4. Database schema for the connection - 5. Name of the database connection set in Hikari (**required**) - 6. Maximum size of the database connection set in Hikari - 7. Minimum size of the set of ready connections to the database in Hikari in standby mode - 8. Maximum time to establish a connection in Hikari - 9. Maximum time for connection verification in Hikari - 10. Maximum time for connection downtime in Hikari - 11. Maximum lifetime of a connection in Hikari - 12. Maximum time a connection can be idle in Hikari before it is considered a leak (optional) - 13. Maximum time to wait for connection initialization at service startup (optional) - 14. Whether to enable [readiness probe](probes.md#readiness) for database connection - 15. Additional JDBC connection attributes `dataSourceProperties` (below example `hostRecheckSeconds` parameters) (optional) - 16. Enables module logging (default `false`) - 17. Enables module metrics (default `true`) - 18. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 19. Configures tags for metrics (optional) - 20. Enables module tracing (default `true`) - 21. Configures attributes for tracing (optional) + poolName: "kora" #(4)! + maxPoolSize: 10 #(5)! + ``` + + 1. `JDBC URL` for database connection (`required`, no default) + 2. Username for connection (`required`, no default) + 3. Password for connection (`required`, no default) + 4. `Hikari` connection pool name (`required`, no default) + 5. Maximum `Hikari` connection pool size (default: `10`) + +??? note "Full Configuration" + + Example of the complete configuration described by `JdbcDatabaseConfig` (example values or default values are shown): + + ===! ":material-code-json: `HOCON`" + + ```javascript + db { + jdbcUrl = "jdbc:postgresql://localhost:5432/postgres" //(1)! + username = "postgres" //(2)! + password = "postgres" //(3)! + schema = "public" //(4)! + poolName = "kora" //(5)! + maxPoolSize = 10 //(6)! + minIdle = 0 //(7)! + connectionTimeout = "10s" //(8)! + validationTimeout = "5s" //(9)! + idleTimeout = "10m" //(10)! + maxLifetime = "15m" //(11)! + leakDetectionThreshold = "0s" //(12)! + initializationFailTimeout = "0s" //(13)! + readinessProbe = false //(14)! + dsProperties { //(15)! + "hostRecheckSeconds": "2" + } + telemetry { + logging { + enabled = false //(16)! + } + metrics { + enabled = true //(17)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(18)! + tags = { // (19)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(20)! + attributes = { // (21)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. `JDBC URL` for connecting to the database (`required`, default: not specified) + 2. Username for the connection (`required`, default: not specified) + 3. User password for the connection (`required`, default: not specified) + 4. Database schema for the connection (default: not specified, optional) + 5. `Hikari` connection pool name (`required`, default: not specified) + 6. Maximum `Hikari` connection pool size (default: `10`) + 7. Minimum number of idle ready connections in the `Hikari` pool (default: `0`) + 8. Maximum time to wait for a connection from the `Hikari` pool (default: `10s`) + 9. Maximum time for `Hikari` connection validation (default: `5s`) + 10. Maximum idle time for a `Hikari` connection (default: `10m`) + 11. Maximum lifetime of a `Hikari` connection (default: `15m`) + 12. Time after which a busy connection is considered a possible leak (default: `0s`) + 13. Maximum time to wait for connection initialization at service startup (default: not specified, optional) + 14. Whether to enable the [readiness probe](probes.md#readiness) for the database connection (default: `false`) + 15. Additional `JDBC` connection properties passed to `Hikari` `dataSourceProperties` (default: `{}`) + 16. Enables module logging (default: `false`) + 17. Enables module metrics (default: `true`) + 18. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Configures metric tags (default: `{}`) + 20. Enables module tracing (default: `true`) + 21. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + db: + jdbcUrl: "jdbc:postgresql://localhost:5432/postgres" #(1)! + username: "postgres" #(2)! + password: "postgres" #(3)! + schema: "public" #(4)! + poolName: "kora" #(5)! + maxPoolSize: 10 #(6)! + minIdle: 0 #(7)! + connectionTimeout: "10s" #(8)! + validationTimeout: "5s" #(9)! + idleTimeout: "10m" #(10)! + maxLifetime: "15m" #(11)! + leakDetectionThreshold: "0s" #(12)! + initializationFailTimeout: "0s" #(13)! + readinessProbe: false #(14)! + dsProperties: #(15)! + hostRecheckSeconds: "1" + telemetry: + logging: + enabled: false #(16)! + metrics: + enabled: true #(17)! + slo: [ 2, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(18)! + tags: #(19)! + key1: value1 + key2: value2 + tracing: + enabled: true #(20)! + attributes: #(21)! + key1: value1 + key2: value2 + ``` + + 1. `JDBC URL` for connecting to the database (`required`, default: not specified) + 2. Username for the connection (`required`, default: not specified) + 3. User password for the connection (`required`, default: not specified) + 4. Database schema for the connection (default: not specified, optional) + 5. `Hikari` connection pool name (`required`, default: not specified) + 6. Maximum `Hikari` connection pool size (default: `10`) + 7. Minimum number of idle ready connections in the `Hikari` pool (default: `0`) + 8. Maximum time to wait for a connection from the `Hikari` pool (default: `10s`) + 9. Maximum time for `Hikari` connection validation (default: `5s`) + 10. Maximum idle time for a `Hikari` connection (default: `10m`) + 11. Maximum lifetime of a `Hikari` connection (default: `15m`) + 12. Time after which a busy connection is considered a possible leak (default: `0s`) + 13. Maximum time to wait for connection initialization at service startup (default: not specified, optional) + 14. Whether to enable the [readiness probe](probes.md#readiness) for the database connection (default: `false`) + 15. Additional `JDBC` connection properties passed to `Hikari` `dataSourceProperties` (default: `{}`) + 16. Enables module logging (default: `false`) + 17. Enables module metrics (default: `true`) + 18. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Configures metric tags (default: `{}`) + 20. Enables module tracing (default: `true`) + 21. Configures tracing attributes (default: `{}`) ## Usage { #usage } +A `JDBC` repository is declared as an interface annotated with `@Repository` and must extend `JdbcRepository`. +Each method annotated with `@Query` contains a regular `SQL` query. Method parameters are bound by name with the +`:parameter` syntax, and object fields can be referenced as `:entity.field`. + +Entities are described with the [common database annotations](database-common.md) and marked with `@EntityJdbc` +so that `Kora` generates the view mapper at compile time (see [View](database-common.md#view)): + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends JdbcRepository { } + public interface EntityRepository extends JdbcRepository { + + @EntityJdbc + @Table("entities") + record Entity(@Id long id, + String name, + @Nullable String description) {} + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + @Nullable + Entity findById(long id); + + @Query("SELECT id, name, description FROM entities") //(2)! + List findAll(); + + @Query("INSERT INTO %{entity#inserts}") //(3)! + UpdateCount insert(Entity entity); + } ``` + 1. Uses macros `%{return#selects}` and `%{return#table}`. Expands to query: + ```sql + SELECT id, name, description + FROM entities + WHERE id = :id + ``` + Method uses macros for `SELECT`. Details: [Common Database Rules — Macros](database-common.md#macros) + 2. Fields listed manually without macros — this is valid but requires maintenance when the view changes. + 3. Uses macro `%{entity#inserts}`. Expands to query: + ```sql + INSERT INTO entities(id, name, description) + VALUES(:entity.id, :entity.name, :entity.description) + ``` + Method uses macros for `INSERT`. Details: [Common Database Rules — Macros](database-common.md#macros) + === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : JdbcRepository + interface EntityRepository : JdbcRepository { + + @EntityJdbc + @Table("entities") + data class Entity( + @field:Id val id: Long, + val name: String, + val description: String? + ) + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + fun findById(id: Long): Entity? + + @Query("INSERT INTO %{entity#inserts}") //(3)! + fun insert(entity: Entity): UpdateCount + } ``` + 1. Uses macros `%{return#selects}` and `%{return#table}`. Expands to query: + ```sql + SELECT id, name, description + FROM entities + WHERE id = :id + ``` + Method uses macros for `SELECT`. Details: [Common Database Rules — Macros](database-common.md#macros) + 3. Uses macro `%{entity#inserts}`. Expands to query: + ```sql + INSERT INTO entities(id, name, description) + VALUES(:entity.id, :entity.name, :entity.description) + ``` + Method uses macros for `INSERT`. Details: [Common Database Rules — Macros](database-common.md#macros) + +`SQL` remains under the developer's control: you can use database-specific features, while `Kora` only handles safe +parameter binding, query execution, and result mapping. +Common rules for entities, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch`, and macros are described in +[Common database rules](database-common.md#macros). + +**Parameter binding:** Kora performs typed injection of arguments into the SQL query at compile time. +Query parameters (e.g., `:id`, `:entity.name`) are replaced in the generated code with corresponding `PreparedStatement` calls. +For example, for a `String name` parameter, something like `statement.setString(1, name)` will be generated, where the index corresponds to the parameter order in the query. +This ensures security (protection against SQL injection) and performance (using prepared statements). + ## Mapping { #mapping } -It is possible to override the conversion of different parts of [entity](database-common.md) and query parameters, Kora provides special interfaces for this. +You can override the mapping of different parts of an [entity](database-common.md), a query result, and query parameters. +For this, `Kora` provides several mapper interfaces. ### Result { #result } -If you need to convert the result manually, it is suggested to use `JdbcResultSetMapper`: +Use `JdbcResultSetMapper` when you need to manually map the whole `ResultSet`. +This mapper receives the whole query result and decides how many rows to read and what to return. ===! ":fontawesome-brands-java: `Java`" @@ -214,10 +338,8 @@ If you need to convert the result manually, it is suggested to use `JdbcResultSe === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin - class ResultMapper : JdbcResultSetMapper { + class ResultMapper : JdbcResultSetMapper { @Throws(SQLException::class) override fun apply(rs: ResultSet): UUID { @@ -234,30 +356,35 @@ If you need to convert the result manually, it is suggested to use `JdbcResultSe } ``` -#### Entity { #entity } +`JdbcResultSetMapper` also exposes static helpers `singleResultSetMapper`, `listResultSetMapper`, +and `optionalResultSetMapper` that build a full-`ResultSet` mapper from a `JdbcRowMapper`. + +#### View { #view } -Optimal entity mapping intend to use with `@EntityJdbc` annotation for result converter generation. +Use the `@EntityJdbc` annotation for optimal view mapping. +The annotation allows the annotation processor to generate all necessary mappers in **one round** of annotation processing. +Without this annotation, mappers are generated on-demand, which can require **multiple rounds** of processing and significantly increase compilation time. -All embedded entities also should use this annotation: +All nested views are also expected to use this annotation. ===! ":fontawesome-brands-java: `Java`" ```java @EntityJdbc - Public record Entity(String id, String name) {} + public record Entity(String id, String name) {} ``` === ":simple-kotlin: `Kotlin`" ```kotlin @EntityJdbc - Data class Entity(val id: String, val name: String) + data class Entity(val id: String, val name: String) ``` ### Row { #row } -If you need to convert the string manually, it is suggested to use `JdbcRowMapper`, -keep in mind that the order of the columns starts from `1`: +Use `JdbcRowMapper` when you need to manually map one row. +Keep in mind that in `JDBC`, column indexes in `ResultSet` start from `1`: ===! ":fontawesome-brands-java: `Java`" @@ -281,8 +408,6 @@ keep in mind that the order of the columns starts from `1`: === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin class RowMapper : JdbcRowMapper { @@ -303,7 +428,7 @@ keep in mind that the order of the columns starts from `1`: ### Column { #column } -If you need to convert the column value manually, it is suggested to use the `JdbcResultColumnMapper`: +Use `JdbcResultColumnMapper` when you need to manually map a single column value: ===! ":fontawesome-brands-java: `Java`" @@ -330,8 +455,6 @@ If you need to convert the column value manually, it is suggested to use the `Jd === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin class ColumnMapper : JdbcResultColumnMapper { @@ -358,7 +481,7 @@ If you need to convert the column value manually, it is suggested to use the `Jd ### Parameter { #parameter } -If you want to convert the value of a query parameter manually, it is suggested to use `JdbcParameterColumnMapper`: +Use `JdbcParameterColumnMapper` when you need to manually map a query parameter value: ===! ":fontawesome-brands-java: `Java`" @@ -383,8 +506,6 @@ If you want to convert the value of a query parameter manually, it is suggested === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin class ParameterMapper : JdbcParameterColumnMapper { @@ -404,11 +525,12 @@ If you want to convert the value of a query parameter manually, it is suggested } ``` -### Supported types { #supported-types } +### Supported Types { #supported-types } ??? abstract "List of supported types for arguments/return values out of the box" - These types are chosen because they are supported by most popular databases. + These types are selected because they are supported by most popular databases. + `Kora` provides built-in row, column, and parameter mappers for them. * void * boolean / Boolean @@ -427,14 +549,19 @@ If you want to convert the value of a query parameter manually, it is suggested * OffsetTime * OffsetDateTime -## Select by list { #select-by-list } + View fields without an explicit `@Mapping` natively support `boolean` / `Boolean`, `short` / `Short`, + `int` / `Integer`, `long` / `Long`, `double` / `Double`, `float` / `Float`, `byte[]`, `String`, + `BigDecimal`, `LocalDate`, and `LocalDateTime`. + For other types, use built-in `JdbcResultColumnMapper` / `JdbcParameterColumnMapper` mappers or declare custom mappers. + +## Select by List { #select-by-list } -Sometimes a list of values from the database needs to be fetched, all these parameters must be set separately at the driver level, as the length of the list is not known -this is not the most obvious task as Kora tries to do all conversions at compile time and remove any string conversions especially in SQL at runtime, -such functionality would require adding a separate parameter converter. +Sometimes you need to select rows by a list of values. +At the `JDBC` level, such parameters must be prepared separately by the driver because the list length is not known in advance. +`Kora` tries to perform mappings at compile time and does not rewrite `SQL` at runtime, so such parameters require a custom mapper. -What is certain at this point is that it is easy to add support for such parameters without manual connection factory for popular databases like Postgres/Oracle. -Out of the box Kora does not provide conversion of such parameters, but it is easy to add it yourself, an example for `Postgres` is shown below: +`Kora` does not provide this parameter mapping out of the box, but it is easy to add yourself. +The example below shows `Postgres` through a `JDBC Array`: ===! ":fontawesome-brands-java: `Java`" @@ -480,11 +607,137 @@ Out of the box Kora does not provide conversion of such parameters, but it is ea } ``` -### Generated identifier { #generated-identifier } +## JSON / JSONB { #json } + +A `JSON` / `JSONB` column can be mapped to a view field by registering generic +`JdbcParameterColumnMapper` and `JdbcResultColumnMapper` as default `@Module` components tagged with `@Json`. +These mappers bridge the [JSON](json.md) module `JsonWriter` / `JsonReader` to a driver-specific value. +The `Postgres` example below serializes the value into a `PGobject` of type `jsonb` when binding a parameter, +handles `null` via `setNull(index, Types.NULL)`, and reads the column back as a `String`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface JdbcJsonbMapperModule { + + @Json + default JdbcParameterColumnMapper jdbcJsonParameterColumnMapper(JsonWriter writer) { + return (stmt, index, value) -> { + if (value != null) { + PGobject jsonb = new PGobject(); + jsonb.setType("jsonb"); + jsonb.setValue(writer.toStringUnchecked(value)); + stmt.setObject(index, jsonb); + } else { + stmt.setNull(index, Types.NULL); + } + }; + } + + @Json + default JdbcResultColumnMapper jdbcJsonResultColumnMapper(JsonReader reader) { + return (row, index) -> { + var value = row.getString(index); + if (value == null) { + return null; + } else { + return reader.readUnchecked(value); + } + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface JdbcJsonbMapperModule { + + @Json + fun jdbcJsonParameterColumnMapper(writer: JsonWriter): JdbcParameterColumnMapper { + return JdbcParameterColumnMapper { stmt, index, value -> + if (value == null) { + stmt.setNull(index, Types.NULL) + } else { + val jsonb = PGobject() + jsonb.type = "jsonb" + jsonb.value = writer.toStringUnchecked(value) + stmt.setObject(index, jsonb) + } + } + } + + @Json + fun jdbcJsonResultColumnMapper(reader: JsonReader): JdbcResultColumnMapper { + return JdbcResultColumnMapper { row, index -> + val value = row.getString(index) + if (value == null) null else reader.readUnchecked(value) + } + } + } + ``` + +Annotate the view field with `@Json` (and `@Column` if the column name differs), where the field type is itself a `@Json` type. +The `INSERT` uses the `::jsonb` cast so `Postgres` accepts the serialized string as `JSONB`; +`findById` reads it back through the same `@Json`-tagged column mapper: -If you want to get the primary keys of an entity created by the database as the result, -it is suggested to use the `@Id` annotation over a method where the return value type is identifiers. -This approach works for `@Batch` queries as well. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface JdbcJsonbRepository extends JdbcRepository { + + @EntityJdbc + record Entity(UUID id, + @Column("value") @Json JsonbValue value) { + + @Json + record JsonbValue(String name, String surname) {} + } + + @Query("SELECT * FROM entities_jsonb WHERE id = :id") + @Nullable + Entity findById(UUID id); + + @Query("INSERT INTO entities_jsonb(id, value) VALUES (:entity.id, :entity.value::jsonb)") + void insert(Entity entity); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface JdbcJsonbRepository : JdbcRepository { + + @EntityJdbc + data class Entity( + val id: UUID, + @field:Column("value") @Json val value: JsonbValue + ) { + + @Json + data class JsonbValue(val name: String, val surname: String) + } + + @Query("SELECT * FROM entities_jsonb WHERE id = :id") + fun findById(id: UUID): Entity? + + @Query("INSERT INTO entities_jsonb(id, value) VALUES (:entity.id, :entity.value::jsonb)") + fun insert(entity: Entity) + } + ``` + +The [JSON](json.md) module dependency is required so `Kora` can generate `JsonWriter` / `JsonReader` for the field type, +and the mapper `@Module` must be added to the [application graph](container.md). + +## Generated Identifier { #generated-identifier } + +If you need to return primary keys generated by the database, +use the `@Id` annotation on the method. +This approach also works for `@Batch` queries. ===! ":fontawesome-brands-java: `Java`" @@ -508,7 +761,7 @@ This approach works for `@Batch` queries as well. interface EntityRepository : JdbcRepository { @EntityJdbc - public record Entity(Long id, String name) {} + data class Entity(val id: Long, val name: String) @Query("INSERT INTO entities(name) VALUES (:entity.name)") @Id @@ -516,69 +769,236 @@ This approach works for `@Batch` queries as well. } ``` -## Transaction { #transaction } +The generated key can also be returned as the view key type rather than a scalar. +When the identifier is a composite key described by an [`@Embedded`](database-common.md#embedded-fields) record, +the `@Id` method returns that record, and a `@Batch` insert returns a `List` of keys, one per inserted row: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends JdbcRepository { -In order to execute blocking queries, Kora has a `JdbcConnectionFactory` interface, which is provided in a method within the `JdbcRepository` contract. -All repository methods called within a transaction lambda will be executed in that transaction. + @EntityJdbc + record Entity(@Id @Embedded EntityId id, @Column("name") String name) { -In order to execute queries transactionally, the `inTx` contract can be used: + @EntityJdbc + record EntityId(Long a, Long b) {} + } + + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + @Id + Entity.EntityId insertGenerated(Entity entity); + + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + @Id + List insertGenerated(@Batch List entities); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + @EntityJdbc + data class Entity( + @field:Id @field:Embedded val id: EntityId?, + @field:Column("name") val name: String + ) { + + @EntityJdbc + data class EntityId(val a: Long?, val b: Long?) + } + + @Id + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + fun insertGenerated(entity: Entity): Entity.EntityId + + @Id + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + fun insertGenerated(@Batch entities: List): List + } + ``` + +## Manual Query With Telemetry { #query } + +If a query is hard to express as a single static `@Query`, you can create a regular method with an implementation and build `SQL` manually. +Use `JdbcConnectionFactory#query` to execute such a query. +This method creates a `PreparedStatement`, runs the query through Kora telemetry, and uses the same connection as other repository methods. +If `query` is called inside an active `inTx` transaction, the query is executed on the current transactional connection. + +`QueryContext` contains the query identifier and the final `SQL`. +The query identifier is reported to telemetry, so it is convenient to use a stable name such as `Repository.method`. +Values must be passed through `PreparedStatement` parameters, not concatenated directly into the query string. ===! ":fontawesome-brands-java: `Java`" ```java - @Component - public final class SomeService { + @Repository + public interface EntityRepository extends JdbcRepository { - private final EntityRepository repository; + default List findByFilter(@Nullable String name, boolean onlyActive) { + var sql = new StringBuilder("SELECT id, name FROM entities WHERE 1 = 1"); + var params = new ArrayList(); - public SomeService(EntityRepository repository) { - this.repository = repository; + if (name != null) { + sql.append(" AND name = ?"); + params.add(name); + } + if (onlyActive) { + sql.append(" AND active = true"); + } + + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return getJdbcConnectionFactory().query(queryContext, statement -> { + for (int i = 0; i < params.size(); i++) { + statement.setString(i + 1, params.get(i)); + } + try (var resultSet = statement.executeQuery()) { + var result = new ArrayList(); + while (resultSet.next()) { + result.add(new Entity(resultSet.getLong("id"), resultSet.getString("name"))); + } + return result; + } + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + fun findByFilter(name: String?, onlyActive: Boolean): List { + val sql = StringBuilder("SELECT id, name FROM entities WHERE 1 = 1") + val params = mutableListOf() + + if (name != null) { + sql.append(" AND name = ?") + params += name + } + if (onlyActive) { + sql.append(" AND active = true") + } + + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return jdbcConnectionFactory.query(queryContext) { statement -> + params.forEachIndexed { index, value -> + statement.setString(index + 1, value) + } + statement.executeQuery().use { resultSet -> + val result = mutableListOf() + while (resultSet.next()) { + result += Entity(resultSet.getLong("id"), resultSet.getString("name")) + } + result + } + } } + } + ``` + +## Transactions { #transaction } + +For executing blocking queries, `Kora` provides the `JdbcConnectionFactory` interface through the `JdbcRepository` contract. +All repository methods called inside the transaction lambda are executed in that same transaction. + +Use `inTx` to execute queries transactionally. +If there is already an active transaction on the current thread, a nested `inTx` call uses the same connection and does not open +a new transaction. + +A transactional sequence of operations can stay inside the repository itself as a regular method with an implementation. +This is useful when several `@Query` methods or a complex manual `SQL` query should stay next to the rest of the repository queries, +without moving technical database work to a service layer. +Inside such a method, you can use both repository `@Query` methods and `JdbcConnectionFactory#query` for a manual query with telemetry. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends JdbcRepository { + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + UpdateCount insert(Entity entity); + + @Query("UPDATE entities SET name = :name WHERE id = :id") + UpdateCount updateName(long id, String name); public List saveAll(Entity one, Entity two) { - return repository.getJdbcConnectionFactory().inTx(() -> { - repository.insert(one); //(1)! - // do some work - repository.insert(two); //(2)! + return getJdbcConnectionFactory().inTx(() -> { + insert(one); //(1)! + updateName(two.id(), two.name()); //(2)! return List.of(one, two); }); } } ``` - 1. will be executed within the transaction or rolled back if the entire lambda throws an exception - 2. will be executed within the transaction or rolled back if the entire lambda throws an exception + 1. Executed within the transaction, or rolled back if the whole lambda throws an exception + 2. Executed within the transaction, or rolled back if the whole lambda throws an exception === ":simple-kotlin: `Kotlin`" ```kotlin - @Component - class SomeService(private val repository: EntityRepository) { + @Repository + interface EntityRepository : JdbcRepository { - fun saveAll(one: List, two: List): List { - return repository.jdbcConnectionFactory.inTx(SqlFunction1 { - repository.insert(one) //(1)! - // do some work - repository.insert(two) //(2)! - one + two - }) + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insert(entity: Entity): UpdateCount + + @Query("UPDATE entities SET name = :name WHERE id = :id") + fun updateName(id: Long, name: String): UpdateCount + + fun saveAll(one: Entity, two: Entity): List { + return jdbcConnectionFactory.inTx> { + insert(one) //(1)! + updateName(two.id, two.name) //(2)! + listOf(one, two) + } } } ``` - 1. will be executed within the transaction or rolled back if the entire lambda throws an exception - 2. will be executed within the transaction or rolled back if the entire lambda throws an exception + 1. Executed within the transaction, or rolled back if the whole lambda throws an exception + 2. Executed within the transaction, or rolled back if the whole lambda throws an exception + +The transaction is considered successfully committed after the method completes if it did not throw an exception. +If the method throws an exception, all database changes made within the transaction are not applied. -The isolation level is taken from the `dsProperties` configuration of the Hikari pool, -or you can change it yourself via `java.sql.Connection` before executing queries. +The transaction isolation level is taken from the `Hikari` pool `dsProperties` configuration, +or you can change it manually through `java.sql.Connection` before executing queries. ```java connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); ``` -### Connection { #connection } +### Manual Connection Management { #connection } + +If a query needs more complex logic or queries outside a repository, you can use `java.sql.Connection`. +The `withConnection` method executes code with a connection, but does not open a transaction by itself. + +`withConnection` works as follows: + +- if the current `Context` already contains a `ConnectionContext`, the method passes the current connection to the lambda; +- if the current `Context` does not contain a connection, the method takes a new connection from the `DataSource`, stores it in `ConnectionContext` for the duration of the lambda, and closes it after completion; +- nested calls to `withConnection`, `JdbcConnectionFactory#query`, and repository methods inside this lambda use the same current connection; +- if a `JDBC` exception is a `SQLException`, it is wrapped in `RuntimeSqlException`. + +!!! note -If you need some more complex logic for a query and `@Query` is not enough, you can use `java.sql.Connection`: + Manual `query`, `withConnection`, and `inTx` calls surface a `JDBC` failure as an unchecked `RuntimeSqlException` + that wraps the original `java.sql.SQLException`. Catch `RuntimeSqlException` (not `SQLException`) at the call site, + and use `getCause()` to reach the underlying `SQLException`. + +The `inTx` method opens a transaction and is built on top of `withConnection`. +If the current connection is already in an active transaction, meaning `autoCommit = false`, nested `inTx` uses the same transaction. +If there is no active transaction, `inTx` disables `autoCommit`, executes the lambda, and then calls `commit` on success or `rollback` on exception. +After the transaction completes, registered `addPostCommitAction` or `addPostRollbackAction` callbacks are executed. ===! ":fontawesome-brands-java: `Java`" @@ -616,10 +1036,11 @@ If you need some more complex logic for a query and `@Query` is not enough, you } ``` -### Post-commit actions { #post-commit-actions } +### Post-Commit Actions { #post-commit-actions } -If you need to perform any actions after committing a transaction, -you can add the appropriate actions using `addPostCommitAction`. +If you need to perform actions after a transaction is successfully committed, add them with `addPostCommitAction`. +The action is executed after `commit` and only if the transaction completed successfully. +Such actions can be added only inside an active transaction. ===! ":fontawesome-brands-java: `Java`" @@ -636,7 +1057,7 @@ you can add the appropriate actions using `addPostCommitAction`. public List saveAll(Entity one, Entity two) { return repository.getJdbcConnectionFactory().inTx(connection -> { var ccc = repository.getJdbcConnectionFactory().currentConnectionContext(); - ccc.addPostCommitAction(conn) -> { + ccc.addPostCommitAction(conn -> { // do some work }); @@ -655,9 +1076,9 @@ you can add the appropriate actions using `addPostCommitAction`. fun saveAll(one: Entity, two: Entity): List { return repository.jdbcConnectionFactory.inTx(SqlFunction1 { connection: Connection -> - val ccc = repository.jdbcConnectionFactory.currentConnectionContext() - ccc.addPostCommitAction { conn -> { - // do some work + val ccc = repository.jdbcConnectionFactory.currentConnectionContext()!! + ccc.addPostCommitAction { conn -> + // do some work } // do some work @@ -667,10 +1088,11 @@ you can add the appropriate actions using `addPostCommitAction`. } ``` -### Post-rollback actions { #post-rollback-actions } +### Post-Rollback Actions { #post-rollback-actions } -If you need to perform any actions after rolling back a transaction, -you can add the appropriate actions using `addPostRollbackAction`. +If you need to perform actions after a transaction is rolled back, add them with `addPostRollbackAction`. +The action receives the connection and the exception that caused the transaction to roll back. +Such actions can be added only inside an active transaction. ===! ":fontawesome-brands-java: `Java`" @@ -706,9 +1128,9 @@ you can add the appropriate actions using `addPostRollbackAction`. fun saveAll(one: Entity, two: Entity): List { return repository.jdbcConnectionFactory.inTx(SqlFunction1 { connection: Connection -> - val ccc = repository.jdbcConnectionFactory.currentConnectionContext() + val ccc = repository.jdbcConnectionFactory.currentConnectionContext()!! ccc.addPostRollbackAction { conn, e -> - // do some work + // do some work } // do some work @@ -720,21 +1142,57 @@ you can add the appropriate actions using `addPostRollbackAction`. ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Available repository method signatures out of the box: ===! ":fontawesome-brands-java: `Java`" - The `T` refers to the type of the return value, either `List`, either `Void` or `UpdateCount`. + `T` means the return value type, or `List`, or `Void`, or `UpdateCount`. + `CompletionStage`, `CompletableFuture`, and `Mono` require an `Executor` component. - `T myMethod()` - `@Nullable T myMethod()` - `Optional myMethod()` - - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) (provide `Executor`) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (provide `Executor` and add [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) (requires `Executor`) + - `CompletableFuture myMethod()` [CompletableFuture](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html) (requires `Executor`) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires `Executor` and the [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - By `T` we mean the type of the return value, either `T?`, either `List`, either `Unit` or `UpdateCount`. + `T` means the return value type, or `T?`, or `List`, or `Unit`, or `UpdateCount`. + `suspend` methods require an `Executor` component. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (provide `Executor` and add [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires `Executor` and the [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + +For asynchronous methods, you can specify a separate `Executor` tag through the `executorTag` parameter in `@Repository`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public final class BlockingJdbcExecutorTag {} + + @Repository(executorTag = @Tag(BlockingJdbcExecutorTag.class)) + public interface EntityRepository extends JdbcRepository { + + @Query("SELECT id, name FROM entities") + CompletionStage> findAll(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + class BlockingJdbcExecutorTag + + @Repository(executorTag = Tag(BlockingJdbcExecutorTag::class)) + interface EntityRepository : JdbcRepository { + + @Query("SELECT id, name FROM entities") + suspend fun findAll(): List + } + ``` + +## Telemetry { #telemetry } + +Logging, metrics, and tracing are configured via the `telemetry` block in the [configuration](#configuration) and described in the [Metrics Reference](metrics.md#database) section. +To completely override telemetry, you can provide custom SPI factories; see the [Common Database Documentation](database-common.md#telemetry) for details. diff --git a/mkdocs/docs/en/documentation/database-migration.md b/mkdocs/docs/en/documentation/database-migration.md index 59aa69e..89a82f3 100644 --- a/mkdocs/docs/en/documentation/database-migration.md +++ b/mkdocs/docs/en/documentation/database-migration.md @@ -4,11 +4,21 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora database migration modules for Flyway and Liquibase, migration configuration, startup behavior, and database integration; key triggers include FlywayJdbcDatabaseInterceptor, LiquibaseJdbcDatabaseInterceptor, FlywayConfig, LiquibaseConfig, JdbcDatabaseModule." --- -Modules to migrate the database along with the service launch. +Database migrations apply schema and reference data changes in a controlled order: they create tables, indexes, constraints, and perform other `SQL` operations required by a new application version. +In Kora, migration modules are bound to `JdbcDatabase` initialization through a `GraphInterceptor`: when the application starts, `JdbcDatabase` is created as a graph component, and the interceptor's `init()` runs migrations before the component is published to the rest of the graph. +If a migration fails, `init()` throws, so `JdbcDatabase` component initialization and the whole graph build (application startup) fail as well. +The interceptor's `release()` is a no-op: migrations are never rolled back or re-run when the application stops. + +This approach is convenient for local development, tests, and small installations where the application runs as a single instance. +For environments with multiple replicas, choose a separate migration execution method in advance so migrations are not run simultaneously from every application instance. +Repositories do not create the database schema themselves: tables, indexes, constraints, and reference data must be created by migrations or by an external database preparation process. ## Flyway { #flyway } Module for database migration using the [Flyway](https://documentation.red-gate.com/fd) tool. +During `JdbcDatabase` initialization, the module calls `Flyway.migrate()` with settings from the `flyway` section. +Migrations are run by `FlywayJdbcDatabaseInterceptor`, which is provided by `FlywayJdbcDatabaseModule`. +`Flyway` is wired to `SLF4J` (`loggers("slf4j")`), so migration output and the `FlyWay migration applied in ...` timing line (logged at `INFO`) appear in the application's normal logs. ### Dependency { #dependency } @@ -38,13 +48,14 @@ Module for database migration using the [Flyway](https://documentation.red-gate. interface Application : FlywayJdbcDatabaseModule ``` -Requires [JDBC module](database-jdbc.md) dependency. +Requires the [`JDBC` module](database-jdbc.md) because migrations are executed through `DataSource`. +Applications usually include both modules: `JdbcDatabaseModule` creates `JdbcDatabase`, and `FlywayJdbcDatabaseModule` adds the migration interceptor. ### Configuration { #configuration } -Example of the complete configuration described in the `FlywayConfig` class (default values are specified): +Example of the complete configuration described by the `FlywayConfig` class: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript flyway { @@ -57,13 +68,15 @@ Example of the complete configuration described in the `FlywayConfig` class (def } ``` - 1. Whether database migration is enabled when the application starts. If `false`, migrations will not be executed. - 2. Directory paths where migration scripts are located. - 3. Whether to execute migrations within a transaction. - 4. Whether to verify checksums of existing migrations before execution. An error will occur if they do not match. - 5. Whether to allow mixing transactional and non-transactional SQL operations in a single migration. If enabled, the entire migration will be executed **without a transaction** to avoid errors in databases where certain operations cannot be run inside a transaction. - This setting is only relevant for databases that do not support executing certain operations within a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite. - 6. Additional key-value configuration properties for `Flyway#configurationProperties`. + 1. Enables migration execution during `JdbcDatabase` initialization (default: `true`). If set to `false`, the module skips the `Flyway.migrate()` call. + 2. Paths to directories with migration scripts (default: `["db/migration"]`). + 3. Executes migrations inside a transaction when supported by the database and the `SQL` operations themselves (default: `true`). + 4. Validates checksums of already applied migrations before executing new ones (default: `true`). If checksums do not match, startup fails with an error. + 5. Allows mixing transactional and non-transactional `SQL` operations in one migration (default: `false`). + If enabled, the whole migration is executed **without a transaction** to avoid errors in databases where some operations cannot run inside a transaction. + This setting is relevant for databases that do not support executing certain operations inside a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite. + 6. Additional `Flyway` key-value properties (default: `{}`). + Use them to pass settings that do not have a separate Kora configuration option, such as `schemas`, `baselineOnMigrate`, `placeholderReplacement`, or `placeholders.*`. === ":simple-yaml: `YAML`" @@ -77,17 +90,44 @@ Example of the complete configuration described in the `FlywayConfig` class (def configurationProperties: {} #(6)! ``` - 1. Whether database migration is enabled when the application starts. If `false`, migrations will not be executed. - 2. Directory paths where migration scripts are located. - 3. Whether to execute migrations within a transaction. - 4. Whether to verify checksums of existing migrations before execution. An error will occur if they do not match. - 5. Whether to allow mixing transactional and non-transactional SQL operations in a single migration. If enabled, the entire migration will be executed **without a transaction** to avoid errors in databases where certain operations cannot be run inside a transaction. - This setting is only relevant for databases that do not support executing certain operations within a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite. - 6. Additional key-value configuration properties for `Flyway#configurationProperties`. + 1. Enables migration execution during `JdbcDatabase` initialization (default: `true`). If set to `false`, the module skips the `Flyway.migrate()` call. + 2. Paths to directories with migration scripts (default: `["db/migration"]`). + 3. Executes migrations inside a transaction when supported by the database and the `SQL` operations themselves (default: `true`). + 4. Validates checksums of already applied migrations before executing new ones (default: `true`). If checksums do not match, startup fails with an error. + 5. Allows mixing transactional and non-transactional `SQL` operations in one migration (default: `false`). + If enabled, the whole migration is executed **without a transaction** to avoid errors in databases where some operations cannot run inside a transaction. + This setting is relevant for databases that do not support executing certain operations inside a transaction: PostgreSQL, Aurora PostgreSQL, SQL Server, and SQLite. + 6. Additional `Flyway` key-value properties (default: `{}`). + Use them to pass settings that do not have a separate Kora configuration option, such as `schemas`, `baselineOnMigrate`, `placeholderReplacement`, or `placeholders.*`. + +### Migration Files { #flyway-files } + +By default, `Flyway` looks for migrations in `src/main/resources/db/migration`. +A regular migration file has a name like `V1__init_schema.sql`, where `V1` is the version and the part after the double underscore is the description. + +```text +src/main/resources/db/migration/ + V1__init_users.sql + V2__add_user_status.sql +``` + +Example of a simple migration: + +```sql +CREATE TABLE users ( + id BIGSERIAL PRIMARY KEY, + name TEXT NOT NULL +); +``` + +When `Flyway` starts, it creates a service migration history table and applies only new versions. +If `validateOnMigrate` is enabled, already applied files must not be changed without a separate migration history repair process. ## Liquibase { #liquibase } Module for database migration using the [Liquibase](https://www.liquibase.com/supported-databases) tool. +During `JdbcDatabase` initialization, the module obtains a connection from `DataSource`, creates a `Liquibase` instance, and calls `update()`. +Migrations are run by `LiquibaseJdbcDatabaseInterceptor`, which is provided by `LiquibaseJdbcDatabaseModule`. ### Dependency { #dependency-2 } @@ -117,13 +157,14 @@ Module for database migration using the [Liquibase](https://www.liquibase.com/su interface Application : LiquibaseJdbcDatabaseModule ``` -Requires [JDBC module](database-jdbc.md) dependency. +Requires the [`JDBC` module](database-jdbc.md) because migrations are executed through `DataSource`. +Applications usually include both modules: `JdbcDatabaseModule` creates `JdbcDatabase`, and `LiquibaseJdbcDatabaseModule` adds the migration interceptor. ### Configuration { #configuration-2 } -Example of the complete configuration described in the `LiquibaseConfig` class (default values are specified): +Example of the complete configuration described by the `LiquibaseConfig` class: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript liquibase { @@ -131,7 +172,7 @@ Example of the complete configuration described in the `LiquibaseConfig` class ( } ``` - 1. Path to [master file](https://docs.liquibase.com/concepts/changelogs/home.html) migration configuration + 1. Path to the main [`changelog`](https://docs.liquibase.com/concepts/changelogs/home.html) file with migration definitions (default: `db/changelog/db.changelog-master.xml`). === ":simple-yaml: `YAML`" @@ -140,16 +181,60 @@ Example of the complete configuration described in the `LiquibaseConfig` class ( changelog: "db/changelog/db.changelog-master.xml" #(1)! ``` - 1. Path to [master file](https://docs.liquibase.com/concepts/changelogs/home.html) migration configuration + 1. Path to the main [`changelog`](https://docs.liquibase.com/concepts/changelogs/home.html) file with migration definitions (default: `db/changelog/db.changelog-master.xml`). + +Unlike `Flyway`, the `Liquibase` module does not have an `enabled` setting: if the module is connected to the application graph, migrations run during `JdbcDatabase` initialization. +If a `Liquibase` migration fails, the module wraps the error in `IllegalStateException`, and application startup is interrupted. + +### Migration Files { #liquibase-files } + +By default, `Liquibase` looks for the main `changelog` file at `src/main/resources/db/changelog/db.changelog-master.xml`. +`Liquibase` supports different `changelog` formats, but an `SQL`-oriented project often benefits from keeping migrations as formatted `SQL`. +The main file can include such migrations with `include`. + +```text +src/main/resources/db/changelog/ + db.changelog-master.xml + changes/ + 001-init-users.sql +``` + +Minimal main `changelog`: + +```xml + + + + +``` + +Example of an included migration in formatted `SQL`: + +```sql +--liquibase formatted sql + +--changeset app:001-init-users +CREATE TABLE users ( + id BIGSERIAL PRIMARY KEY, + name TEXT NOT NULL +); +``` ## Recommendations { #recommendations } ???+ warning "Recommendation" - **We do not recommend** using migration modules to run applications in an environment where with horizontal scaling - by increasing the number of working application replicas. Since this will lead to a migration call on each replica setup. - Also keep in mind that every restart of the application will also trigger migrations. + **Migration modules are not recommended** for running migrations on application startup in horizontally scaled environments + where the application runs with multiple replicas. Each replica will try to execute migrations during startup. + Also keep in mind that every application restart triggers the migration mechanism again. + + In such cases, use the [Flyway Gradle Plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway) for local development, + run `Flyway` from code after database startup in tests, + use a [Kubernetes Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) for production Kubernetes environments, + or run migrations separately from `CI`. + - In such cases we recommend using something like [Flyway Gradle plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway) for local development, - for tests use Flyway startup from code after database startup, for Kubernetes combat environment use [K8S Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) - or migration from CI via [Flyway Gradle plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway). diff --git a/mkdocs/docs/en/documentation/database-r2dbc.md b/mkdocs/docs/en/documentation/database-r2dbc.md index a8e1895..699ae81 100644 --- a/mkdocs/docs/en/documentation/database-r2dbc.md +++ b/mkdocs/docs/en/documentation/database-r2dbc.md @@ -1,11 +1,17 @@ --- -description: "Explains Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and signatures. Use when working with @Repository, @Query, @EntityR2dbc, @Table, @Id, @Column, R2dbcDatabaseModule, R2dbcConnectionFactory." +description: "Explains Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and repository method signatures. Use when working with @Repository, @Query, @Table, @Id, @Column, @Batch, R2dbcDatabaseModule, R2dbcConnectionFactory." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and signatures; key triggers include @Repository, @Query, @EntityR2dbc, @Table, @Id, @Column, R2dbcDatabaseModule, R2dbcConnectionFactory, R2dbcRepository." + use_when: "Use this file for Kora docs or implementation questions about Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and repository method signatures; key triggers include @Repository, @Query, @Table, @Id, @Column, @Batch, R2dbcDatabaseModule, R2dbcConnectionFactory, R2dbcRepository." --- -Module provides a repository implementation based on [R2DBC](https://r2dbc.io/) reactive database protocol, -the implementation as an example is [Postgres R2DBC](https://github.com/pgjdbc/r2dbc-postgresql). +The module provides a repository implementation based on the [R2DBC](https://r2dbc.io/) reactive database protocol; +the driver implementation, for example, is [Postgres R2DBC](https://github.com/pgjdbc/r2dbc-postgresql). +It uses the [io.r2dbc.pool](https://github.com/r2dbc/r2dbc-pool) connection pool to manage connections. +You describe a repository interface and `SQL` queries with `@Repository` and `@Query`, and `Kora` generates an implementation +that obtains a reactive connection from the pool, binds parameters, maps the `Flux`, and participates in transactions. + +Common rules for entities, `@Repository`, `@Query`, `@Batch`, `UpdateCount`, macros, manual queries, and other repository +mechanisms are described in [Common database rules](database-common.md). ## Dependency { #dependency } @@ -35,13 +41,13 @@ the implementation as an example is [Postgres R2DBC](https://github.com/pgjdbc/r interface Application : R2dbcDatabaseModule ``` -Also **required to provide** the database driver implementation as a dependency. +You also **must provide** the database driver implementation as a dependency. ## Configuration { #configuration } -Example of the complete configuration described in the `R2dbcDatabaseConfig` class (default or example values are specified): +Basic R2DBC configuration parameters: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript db { @@ -50,60 +56,14 @@ Example of the complete configuration described in the `R2dbcDatabaseConfig` cla password = "postgres" //(3)! poolName = "kora" //(4)! maxPoolSize = 10 //(5)! - minIdle = 0 //(6)! - acquireRetry = 3 //(7)! - connectionTimeout = "10s" //(8)! - connectionCreateTimeout = "30s" //(9)! - idleTimeout = "1m" //(10)! - maxLifetime = "0s" //(11)! - statementTimeout = "0s" //(12)! - readinessProbe = false //(13)! - options { //(14)! - "backgroundEvictionInterval": "PT120S" - } - telemetry { - logging { - enabled = false //(15)! - } - metrics { - enabled = true //(16)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(17)! - tags = { // (18)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(19)! - attributes = { // (20)! - "key1" = "value1" - "key2" = "value2" - } - } - } } ``` - 1. R2DBC database connection URL (**required**) - 2. User name for connection (**required**) - 3. Password of the user to connect (**required**) - 4. Database Connection Set Name (**required**) - 5. Maximum size of the database connection set - 6. Minimum idle size of the ready database connection set - 7. Maximum number of attempts to obtain a connection - 8. Maximum time to establish a connection - 9. Maximum time to establish a connection - 10. Maximum time for connection downtime - 11. Maximum connection lifetime (optional) - 12. Maximum time to execute a query to the database (optional) - 13. Whether to enable [readiness probe](probes.md#readiness) for database connection - 14. Additional attributes of R2DBC connection (optional) - 15. Enables module logging (default `false`) - 16. Enables module metrics (default `true`) - 17. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 18. Configures tags for metrics (optional) - 19. Enables module tracing (default `true`) - 20. Configures attributes for tracing (optional) + 1. `R2DBC URL` for database connection (`required`, no default) + 2. Username for connection (`required`, no default) + 3. Password for connection (`required`, no default) + 4. Connection pool name (`required`, no default) + 5. Maximum connection pool size (default: `10`) === ":simple-yaml: `YAML`" @@ -114,76 +74,202 @@ Example of the complete configuration described in the `R2dbcDatabaseConfig` cla password: "postgres" #(3)! poolName: "kora" #(4)! maxPoolSize: 10 #(5)! - minIdle: 0 #(6)! - acquireRetry: 3 #(7)! - connectionTimeout: "10s" #(8)! - connectionCreateTimeout: "30s" #(9)! - idleTimeout: "1m" #(10)! - maxLifetime: "0s" #(11)! - statementTimeout: "0ms" #(12)! - readinessProbe: false #(13)! - options: #(14)! - backgroundEvictionInterval: "PT120S" - telemetry: - logging: - enabled: false #(15)! - metrics: - enabled: true #(16)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(17)! - tags: #(18)! - key1: value1 - key2: value2 - tracing: - enabled: true #(19)! - attributes: #(20)! - key1: value1 - key2: value2 ``` - 1. R2DBC database connection URL (**required**) - 2. User name for connection (**required**) - 3. Password of the user to connect (**required**) - 4. Database Connection Set Name (**required**) - 5. Maximum size of the database connection set - 6. Minimum idle size of the ready database connection set - 7. Maximum number of attempts to obtain a connection - 8. Maximum time to establish a connection - 9. Maximum time to establish a connection - 10. Maximum time for connection downtime - 11. Maximum connection lifetime (optional) - 12. Maximum time to execute a query to the database (optional) - 13. Whether to enable [readiness probe](probes.md#readiness) for database connection - 14. Additional attributes of R2DBC connection (optional) - 15. Enables module logging (default `false`) - 16. Enables module metrics (default `true`) - 17. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 18. Configures tags for metrics (optional) - 19. Enables module tracing (default `true`) - 20. Configures attributes for tracing (optional) + 1. `R2DBC URL` for database connection (`required`, no default) + 2. Username for connection (`required`, no default) + 3. Password for connection (`required`, no default) + 4. Connection pool name (`required`, no default) + 5. Maximum connection pool size (default: `10`) + +??? note "Full Configuration" + + Example of the complete configuration described by `R2dbcDatabaseConfig` (example values or default values are shown): + + ===! ":material-code-json: `HOCON`" + + ```javascript + db { + r2dbcUrl = "r2dbc:postgresql://localhost:5432/postgres" //(1)! + username = "postgres" //(2)! + password = "postgres" //(3)! + poolName = "kora" //(4)! + maxPoolSize = 10 //(5)! + minIdle = 0 //(6)! + acquireRetry = 3 //(7)! + connectionTimeout = "10s" //(8)! + connectionCreateTimeout = "30s" //(9)! + idleTimeout = "10m" //(10)! + maxLifetime = "0s" //(11)! + statementTimeout = "0s" //(12)! + readinessProbe = false //(13)! + options { //(14)! + "backgroundEvictionInterval": "PT120S" + } + telemetry { + logging { + enabled = false //(15)! + } + metrics { + enabled = true //(16)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(17)! + tags = { // (18)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(19)! + attributes = { // (20)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. `R2DBC URL` for connecting to the database (`required`, default: not specified) + 2. Username for the connection (`required`, default: not specified) + 3. User password for the connection (`required`, default: not specified) + 4. Connection pool name (`required`, default: not specified) + 5. Maximum connection pool size (default: `10`) + 6. Minimum number of idle ready connections in the pool (default: `0`) + 7. Maximum number of attempts to acquire a connection (default: `3`) + 8. Maximum time to acquire a connection from the pool (default: `10s`) + 9. Maximum time to create a new physical connection (default: `30s`) + 10. Maximum idle time for a connection (default: `10m`) + 11. Maximum lifetime of a connection, `0s` means no limit (default: `0s`) + 12. Maximum time to execute a query on the database (default: not specified, optional) + 13. Whether to enable the [readiness probe](probes.md#readiness) for the database connection (default: `false`) + 14. Additional `R2DBC` connection options passed to the driver (default: `{}`) + 15. Enables module logging (default: `false`) + 16. Enables module metrics (default: `true`) + 17. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Configures metric tags (default: `{}`) + 19. Enables module tracing (default: `true`) + 20. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + db: + r2dbcUrl: "r2dbc:postgresql://localhost:5432/postgres" #(1)! + username: "postgres" #(2)! + password: "postgres" #(3)! + poolName: "kora" #(4)! + maxPoolSize: 10 #(5)! + minIdle: 0 #(6)! + acquireRetry: 3 #(7)! + connectionTimeout: "10s" #(8)! + connectionCreateTimeout: "30s" #(9)! + idleTimeout: "10m" #(10)! + maxLifetime: "0s" #(11)! + statementTimeout: "0s" #(12)! + readinessProbe: false #(13)! + options: #(14)! + backgroundEvictionInterval: "PT120S" + telemetry: + logging: + enabled: false #(15)! + metrics: + enabled: true #(16)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(17)! + tags: #(18)! + key1: value1 + key2: value2 + tracing: + enabled: true #(19)! + attributes: #(20)! + key1: value1 + key2: value2 + ``` + + 1. `R2DBC URL` for connecting to the database (`required`, default: not specified) + 2. Username for the connection (`required`, default: not specified) + 3. User password for the connection (`required`, default: not specified) + 4. Connection pool name (`required`, default: not specified) + 5. Maximum connection pool size (default: `10`) + 6. Minimum number of idle ready connections in the pool (default: `0`) + 7. Maximum number of attempts to acquire a connection (default: `3`) + 8. Maximum time to acquire a connection from the pool (default: `10s`) + 9. Maximum time to create a new physical connection (default: `30s`) + 10. Maximum idle time for a connection (default: `10m`) + 11. Maximum lifetime of a connection, `0s` means no limit (default: `0s`) + 12. Maximum time to execute a query on the database (default: not specified, optional) + 13. Whether to enable the [readiness probe](probes.md#readiness) for the database connection (default: `false`) + 14. Additional `R2DBC` connection options passed to the driver (default: `{}`) + 15. Enables module logging (default: `false`) + 16. Enables module metrics (default: `true`) + 17. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Configures metric tags (default: `{}`) + 19. Enables module tracing (default: `true`) + 20. Configures tracing attributes (default: `{}`) ## Usage { #usage } +An `R2DBC` repository is declared as an interface annotated with `@Repository` and must extend `R2dbcRepository`. +Each method annotated with `@Query` contains a regular `SQL` query. Method parameters are bound by name with the +`:parameter` syntax, and object fields can be referenced as `:entity.field`. + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends R2dbcRepository { } + public interface EntityRepository extends R2dbcRepository { + + @Query("SELECT id, name FROM entities WHERE id = :id") + Mono findById(String id); + + @Query("SELECT id, name FROM entities") + Flux findAll(); + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + Mono insert(Entity entity); + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + Mono insertBatch(@Batch List entities); + } ``` === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : R2dbcRepository + interface EntityRepository : R2dbcRepository { + + @Query("SELECT id, name FROM entities WHERE id = :id") + fun findById(id: String): Mono + + @Query("SELECT id, name FROM entities") + fun findAll(): Flux + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insert(entity: Entity): Mono + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insertBatch(@Batch entities: List): Mono + } ``` +`SQL` remains under the developer's control: you can use database-specific features, while `Kora` only handles safe +parameter binding, query execution, and result mapping. +Common rules for entities, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch`, and macros are described in +[Common database rules](database-common.md). + +Reactive `Mono` and `Flux` returns are the native signatures for this module. +Blocking returns such as `Entity`, `List`, `void`, and `UpdateCount` are also supported, but they block the calling +thread until the reactive result completes, so prefer the reactive signatures when running in a reactive context. + ## Mapping { #mapping } -It is possible to override the conversion of different parts of [entity](database-common.md) and query parameters, Kora provides special interfaces for this. +You can override the mapping of different parts of an [entity](database-common.md), a query result, and query parameters. +For this, `Kora` provides several mapper interfaces. ### Result { #result } -If you need to convert the result manually, it is suggested to use `R2dbcResultFluxMapper`: +Use `R2dbcResultFluxMapper` when you need to control the whole `Flux`. +This mapper receives the entire reactive result stream and decides how to consume it and what to return. ===! ":fontawesome-brands-java: `Java`" @@ -192,7 +278,8 @@ If you need to convert the result manually, it is suggested to use `R2dbcResultF @Override public Flux apply(Flux resultFlux) { - // mapping code + return resultFlux.flatMap(result -> result.map((row, meta) -> + UUID.fromString(row.get(0, String.class)))); } } @@ -207,12 +294,12 @@ If you need to convert the result manually, it is suggested to use `R2dbcResultF === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin class ResultMapper : R2dbcResultFluxMapper> { override fun apply(resultFlux: Flux): Flux { - // mapping code + return resultFlux.flatMap { result -> + result.map { row, _ -> UUID.fromString(row.get(0, String::class.java)) } + } } } @@ -225,18 +312,24 @@ If you need to convert the result manually, it is suggested to use `R2dbcResultF } ``` +In most cases you do not need to control the whole `Flux`. +It is enough to provide a [R2dbcRowMapper](#row), and `Kora` automatically adapts it to the method return type: +the module provides ready `mono` (`Mono`), `monoList` (`Mono>`), and `flux` (`Flux`) result-flux mappers built from a +single row mapper. There is also a `R2dbcResultFluxMapper.monoOptional` helper that adapts a row mapper to `Mono>`. + ### Row { #row } -If you need to convert the string manually, it is suggested to use `R2dbcRowMapper`: +Use `R2dbcRowMapper` when you need to manually map one row of the result. +Columns are read from `io.r2dbc.spi.Row` by index (starting from `0`) or by label: ===! ":fontawesome-brands-java: `Java`" ```java - final class RowMapper implements R2dbcRowMapper { + final class RowMapper implements R2dbcRowMapper { @Override - public UUID apply(Row row) { - return UUID.fromString(rs.get(0, String.class)); + public EntityPart apply(Row row) { + return new EntityPart(row.get(0, String.class), row.get(1, Integer.class)); } } @@ -244,20 +337,18 @@ If you need to convert the string manually, it is suggested to use `R2dbcRowMapp public interface EntityRepository extends R2dbcRepository { @Mapping(RowMapper.class) - @Query("SELECT id FROM entities") - Flux findAll(); + @Query("SELECT id, value1 FROM entities") + Flux findAllParts(); } ``` === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin - class RowMapper : R2dbcRowMapper { + class RowMapper : R2dbcRowMapper { - override fun apply(row: Row): UUID { - return UUID.fromString(rs.get(0, String.class)) + override fun apply(row: Row): EntityPart { + return EntityPart(row.get(0, String::class.java), row.get(1, Integer::class.java)) } } @@ -265,14 +356,14 @@ If you need to convert the string manually, it is suggested to use `R2dbcRowMapp interface EntityRepository : R2dbcRepository { @Mapping(RowMapper::class) - @Query("SELECT id FROM entities") - fun findAll(): Flux + @Query("SELECT id, value1 FROM entities") + fun findAllParts(): Flux } ``` ### Column { #column } -If you need to convert the column value manually, it is suggested to use the `R2dbcResultColumnMapper`: +Use `R2dbcResultColumnMapper` when you need to manually map a single column value by its label: ===! ":fontawesome-brands-java: `Java`" @@ -298,13 +389,11 @@ If you need to convert the column value manually, it is suggested to use the `R2 === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin class ColumnMapper : R2dbcResultColumnMapper { override fun apply(row: Row, label: String): UUID { - return UUID.fromString(row.get(label, String.class)) + return UUID.fromString(row.get(label, String::class.java)) } } @@ -324,7 +413,8 @@ If you need to convert the column value manually, it is suggested to use the `R2 ### Parameter { #parameter } -If you want to convert the value of a query parameter manually, it is suggested to use `R2dbcParameterColumnMapper`: +Use `R2dbcParameterColumnMapper` when you need to manually bind a query parameter value onto `io.r2dbc.spi.Statement`. +The value is bound with `stmt.bind(index, value)`, and `stmt.bindNull(index, type)` is used for `null`: ===! ":fontawesome-brands-java: `Java`" @@ -332,7 +422,7 @@ If you want to convert the value of a query parameter manually, it is suggested public final class ParameterMapper implements R2dbcParameterColumnMapper { @Override - public void set(Statement stmt, int index, @Nullable UUID value) { + public void apply(Statement stmt, int index, @Nullable UUID value) { if (value != null) { stmt.bind(index, value.toString()); } @@ -349,12 +439,10 @@ If you want to convert the value of a query parameter manually, it is suggested === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin class ParameterMapper : R2dbcParameterColumnMapper { - override fun set(stmt: Statement, index: Int, value: UUID?) { + override fun apply(stmt: Statement, index: Int, value: UUID?) { if (value != null) { stmt.bind(index, value.toString()) } @@ -369,11 +457,35 @@ If you want to convert the value of a query parameter manually, it is suggested } ``` +The result column mapper and the parameter mapper can be stacked on the same view field. +This is convenient for mapping, for example, an enum both when reading a row and when binding a parameter: + +===! ":fontawesome-brands-java: `Java`" + + ```java + record Entity(String id, + @Mapping(FieldTypeResultMapper.class) + @Mapping(FieldTypeParameterMapper.class) + @Column("value1") FieldType field1) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class Entity( + val id: String, + @Mapping(FieldTypeResultMapper::class) + @Mapping(FieldTypeParameterMapper::class) + @Column("value1") val field1: FieldType + ) + ``` + ### Supported types { #supported-types } ??? abstract "List of supported types for arguments/return values out of the box" - These types are chosen because they are supported by most popular databases. + These types are selected because they are supported by most popular databases. + `Kora` provides built-in row, column, and parameter mappers for them. * void * boolean / Boolean @@ -393,11 +505,14 @@ If you want to convert the value of a query parameter manually, it is suggested * OffsetTime * OffsetDateTime + For other types, use custom `R2dbcResultColumnMapper` / `R2dbcParameterColumnMapper` mappers, + or a `R2dbcRowMapper` / `R2dbcResultFluxMapper`. + ## Generated identifier { #generated-identifier } -If you want to get the primary keys of an entity created by the database as the result, -it is suggested to use the `@Id` annotation over a method where the return value type is identifiers. -This approach works for `@Batch` queries as well. +If you need to return primary keys generated by the database, +use the `@Id` annotation on the method where the return value type is the identifier. +This approach also works for `@Batch` queries, in which case the method returns the list of generated identifiers. ===! ":fontawesome-brands-java: `Java`" @@ -405,11 +520,15 @@ This approach works for `@Batch` queries as well. @Repository public interface EntityRepository extends R2dbcRepository { - public record Entity(Long id, String name) {} + record Entity(Long id, String name) {} @Query("INSERT INTO entities(name) VALUES (:entity.name)") @Id Mono insert(Entity entity); + + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + @Id + Mono> insertBatch(@Batch List entities); } ``` @@ -419,71 +538,203 @@ This approach works for `@Batch` queries as well. @Repository interface EntityRepository : R2dbcRepository { - public record Entity(Long id, String name) {} + data class Entity(val id: Long?, val name: String) @Query("INSERT INTO entities(name) VALUES (:entity.name)") @Id fun insert(entity: Entity): Mono + + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + @Id + fun insertBatch(@Batch entities: List): Mono> } ``` -## Transactions { #transactions } +Alternatively, you can return generated columns explicitly with a `RETURNING` clause and map them as a regular result, +without the `@Id` annotation: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Query("INSERT INTO entities(name) VALUES (:entity.name) RETURNING id") + Mono insert(Entity entity); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Query("INSERT INTO entities(name) VALUES (:entity.name) RETURNING id") + fun insert(entity: Entity): Mono + ``` -In order to perform manual queries in Kora, there is an interface `ru.tinkoff.kora.database.r2dbc.R2dbcConnectionFactory`, -which is provided in a method within the `R2dbcRepository` contract. -All repository methods called within a transaction lambda will be executed in that transaction. +## Manual Query With Telemetry { #query } -In order to perform queries transactionally, the `inTx` contract can be used: +If a query is hard to express as a single static `@Query`, you can create a regular method with an implementation and build `SQL` manually. +Use `R2dbcConnectionFactory#query` to execute such a query. +This method creates an `io.r2dbc.spi.Statement`, runs the query through Kora telemetry, and uses the same connection as other repository methods. +If `query` is called inside an active `inTx` transaction, the query is executed on the current transactional connection. + +`query` takes three arguments: + +- a `QueryContext` with the query identifier and the final `SQL`. The query identifier is reported to telemetry, so it is convenient to use a stable name such as `Repository.method`; +- a `Consumer` that binds parameter values. Values must be bound through `Statement` (`stmt.bind(...)`), not concatenated directly into the query string; +- a `Function, Mono>` that consumes the reactive result and produces the return value. ===! ":fontawesome-brands-java: `Java`" ```java - @Component - public final class SomeService { + @Repository + public interface EntityRepository extends R2dbcRepository { - private final EntityRepository repository; + default Mono> findByFilter(@Nullable String name, boolean onlyActive) { + var sql = new StringBuilder("SELECT id, name FROM entities WHERE 1 = 1"); + var params = new ArrayList(); - public SomeService(EntityRepository repository) { - this.repository = repository; + if (name != null) { + params.add(name); + sql.append(" AND name = $").append(params.size()); + } + if (onlyActive) { + sql.append(" AND active = true"); + } + + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return getR2dbcConnectionFactory().query( + queryContext, + statement -> { + for (int i = 0; i < params.size(); i++) { + statement.bind(i, params.get(i)); + } + }, + resultFlux -> resultFlux + .flatMap(result -> result.map((row, meta) -> + new Entity(row.get("id", String.class), row.get("name", String.class)))) + .collectList() + ); } + } + ``` - public Mono> saveAll(Entity one, Entity two) { - return repository.getR2dbcConnectionFactory().inTx(connection -> { - // do some work - return repository.insert(one) //(1)! - .zipWith(repository.insert(two), //(2)! - (r1, r2) -> List.of(one, two)); - }); +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : R2dbcRepository { + + fun findByFilter(name: String?, onlyActive: Boolean): Mono> { + val sql = StringBuilder("SELECT id, name FROM entities WHERE 1 = 1") + val params = mutableListOf() + + if (name != null) { + params += name + sql.append(" AND name = $").append(params.size) + } + if (onlyActive) { + sql.append(" AND active = true") + } + + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return r2dbcConnectionFactory.query( + queryContext, + { statement -> + params.forEachIndexed { index, value -> + statement.bind(index, value) + } + }, + { resultFlux -> + resultFlux + .flatMap { result -> result.map { row, _ -> + Entity(row.get("id", String::class.java), row.get("name", String::class.java)) + } } + .collectList() + } + ) + } + } + ``` + +## Transactions { #transactions } + +For executing manual queries and grouping queries into a transaction, `Kora` provides the `R2dbcConnectionFactory` interface +through the `R2dbcRepository` contract, obtained via `getR2dbcConnectionFactory()`. +All repository methods called inside the transaction lambda are executed in that same transaction. + +Use `inTx` to execute queries transactionally. +If there is already an active transaction on the current reactive `Context`, a nested `inTx` call reuses the same connection and does not open +a new transaction. + +A transactional sequence of operations can stay inside the repository itself as a regular method with an implementation. +This is useful when several `@Query` methods or a complex manual `SQL` query should stay next to the rest of the repository queries, +without moving technical database work to a service layer. +Inside such a method, you can use both repository `@Query` methods and `R2dbcConnectionFactory#query` for a manual query with telemetry. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends R2dbcRepository { + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + Mono insert(Entity entity); + + @Query("UPDATE entities SET name = :name WHERE id = :id") + Mono updateName(String id, String name); + + default Mono> saveAll(Entity one, Entity two) { + return getR2dbcConnectionFactory().inTx(connection -> + insert(one) //(1)! + .then(updateName(two.id(), two.name())) //(2)! + .thenReturn(List.of(one, two))); } } ``` - 1. will be executed within the transaction or rolled back if the entire lambda throws an exception - 2. will be executed within the transaction or rolled back if the entire lambda throws an exception + 1. Executed within the transaction, or rolled back if the whole chain signals an error + 2. Executed within the transaction, or rolled back if the whole chain signals an error === ":simple-kotlin: `Kotlin`" ```kotlin - @Component - class SomeService(private val repository: EntityRepository) { + @Repository + interface EntityRepository : R2dbcRepository { - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.r2dbcConnectionFactory.inTx { - repository.insert(one).zipWith(repository.insert(two)) //(1)! - { r1: UpdateCount, r2: UpdateCount -> listOf(one, two) } + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insert(entity: Entity): Mono + + @Query("UPDATE entities SET name = :name WHERE id = :id") + fun updateName(id: String, name: String): Mono + + fun saveAll(one: Entity, two: Entity): Mono> { + return r2dbcConnectionFactory.inTx { _ -> + insert(one) //(1)! + .then(updateName(two.id, two.name)) //(2)! + .thenReturn(listOf(one, two)) } } } ``` - 1. will be executed within the transaction or will be rolled back if the entire lambda throws an exception + 1. Executed within the transaction, or rolled back if the whole chain signals an error + 2. Executed within the transaction, or rolled back if the whole chain signals an error + +The transaction is committed when the returned `Mono` completes successfully. +If the `Mono` signals an error, the transaction is rolled back and the error is propagated, so all database changes made within +the transaction are not applied. + +### Manual Connection Management { #connection } + +If a query needs more complex logic or queries outside a repository, you can work with `io.r2dbc.spi.Connection` directly. +The `withConnection` method executes code with a connection, but does not open a transaction by itself. -### Connection { #connection } +`withConnection` works as follows: -If you need some more complex logic for the query and `@Query` is not enough, you can use `io.r2dbc.spi.Connection`: +- if the current reactive `Context` already contains a connection, the method passes that current connection to the lambda; +- if the current `Context` does not contain a connection, the method takes a new connection from the pool, stores it in the `Context` for the duration of the lambda, and closes it after completion; +- nested calls to `withConnection`, `R2dbcConnectionFactory#query`, and repository methods inside this lambda use the same current connection. + +`withConnection` returns a `Mono`. For results that are naturally a stream of rows, use `withConnectionFlux`, which is the +`Flux`-returning variant with the same connection semantics. +The `inTx` method opens a transaction and is built on top of `withConnection`. ===! ":fontawesome-brands-java: `Java`" @@ -497,9 +748,15 @@ If you need some more complex logic for the query and `@Query` is not enough, yo this.repository = repository; } - public Mono> saveAll(Entity one, Entity two) { - return repository.getR2dbcConnectionFactory().inTx(connection -> { - // do some work + public Mono> loadAll() { + return repository.getR2dbcConnectionFactory().withConnection(connection -> { + // do some work, returns Mono + }); + } + + public Flux streamAll() { + return repository.getR2dbcConnectionFactory().withConnectionFlux(connection -> { + // do some work, returns Flux }); } } @@ -511,35 +768,92 @@ If you need some more complex logic for the query and `@Query` is not enough, yo @Component class SomeService(private val repository: EntityRepository) { - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.r2dbcConnectionFactory.inTx { connection -> - // do some work + fun loadAll(): Mono> { + return repository.r2dbcConnectionFactory.withConnection { connection -> + // do some work, returns Mono + } + } + + fun streamAll(): Flux { + return repository.r2dbcConnectionFactory.withConnectionFlux { connection -> + // do some work, returns Flux } } } ``` +## Select by List { #select-by-list } + +Sometimes you need to select rows by a list of values. +`Kora` tries to perform mappings at compile time and does not rewrite `SQL` at runtime, so a list parameter requires a custom mapper +that binds the whole collection as a single value. + +The example below shows `Postgres` through an array bound with `ANY(:ids)`. +Note that many `R2DBC` drivers (for example `Postgres`) can bind a Java array directly, so such a mapper is often short: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + class ListOfStringR2dbcParameterMapper implements R2dbcParameterColumnMapper> { + + @Override + public void apply(Statement stmt, int index, @Nullable List value) { + stmt.bind(index, value.toArray(String[]::new)); + } + } + + @Repository + public interface EntityRepository extends R2dbcRepository { + + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + Flux findAllByIds(@Mapping(ListOfStringR2dbcParameterMapper.class) List ids); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ListOfStringR2dbcParameterMapper : R2dbcParameterColumnMapper> { + + override fun apply(stmt: Statement, index: Int, value: List?) { + stmt.bind(index, value!!.toTypedArray()) + } + } + + @Repository + interface EntityRepository : R2dbcRepository { + + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + fun findAllByIds(@Mapping(ListOfStringR2dbcParameterMapper::class) ids: List): Flux + } + ``` + ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Available repository method signatures out of the box. +Because `R2DBC` is natively reactive, no `Executor` component is required for the asynchronous signatures. ===! ":fontawesome-brands-java: `Java`" - The `T` refers to the type of the return value, either `List`, either `Void` or `UpdateCount`. + `T` means the return value type, or `List`, or `Void`, or `UpdateCount`. - `T myMethod()` - `@Nullable T myMethod()` - `Optional myMethod()` - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires the [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires the [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - By `T` we mean the type of the return value, either `T?`, either `List`, either `Unit` or `UpdateCount`. + `T` means the return value type, or `T?`, or `List`, or `Unit`, or `UpdateCount`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires the [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires the [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + +## Telemetry { #telemetry } + +Logging, metrics, and tracing are configured via the `telemetry` block in the [configuration](#configuration) and described in the [Metrics Reference](metrics.md#database) section. +To completely override telemetry, you can provide custom SPI factories; see the [Common Database Documentation](database-common.md#telemetry) for details. diff --git a/mkdocs/docs/en/documentation/database-vertx.md b/mkdocs/docs/en/documentation/database-vertx.md index 2ace3a1..acda621 100644 --- a/mkdocs/docs/en/documentation/database-vertx.md +++ b/mkdocs/docs/en/documentation/database-vertx.md @@ -1,10 +1,17 @@ --- -description: "Explains Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures. Use when working with @Repository, @Query, @EntityVertx, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory." +description: "Explains Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures. Use when working with @Repository, @Query, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures; key triggers include @Repository, @Query, @EntityVertx, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory, VertxRepository." + use_when: "Use this file for Kora docs or implementation questions about Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures; key triggers include @Repository, @Query, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory, VertxRepository." --- -Module provides a repository implementation based on the [Vertx](https://vertx.io/docs/#databases) reactive protocol. +The module provides a repository implementation based on the [Vert.x](https://vertx.io/docs/#databases) reactive SQL client. +The Vert.x connection [pool](https://vertx.io/docs/vertx-pg-client/java/#_using_connection_pool) runs on top of the +[Netty](netty.md) transport. You describe a repository interface and `SQL` queries with `@Repository` and `@Query`, and +`Kora` generates an implementation that binds a Vert.x `Tuple`, runs the prepared query through telemetry, maps the +`RowSet`, and participates in transactions. + +Common rules for entities, `@Repository`, `@Query`, `@Batch`, `UpdateCount`, macros, and other repository +mechanisms are described in [Common database rules](database-common.md). ## Dependency { #dependency } @@ -34,15 +41,33 @@ Module provides a repository implementation based on the [Vertx](https://vertx.i interface Application : VertxDatabaseModule ``` -It is also **required to provide** a driver implementation as a dependency of a version no higher than [4.3.8](https://mvnrepository.com/artifact/io.vertx/vertx-pg-client/4.3.8) +You also **must provide** a Vert.x driver implementation as a dependency, of a version no higher than +[4.3.8](https://mvnrepository.com/artifact/io.vertx/vertx-pg-client/4.3.8), for example +[vertx-pg-client](https://mvnrepository.com/artifact/io.vertx/vertx-pg-client/4.3.8) for `PostgreSQL`: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + implementation "io.vertx:vertx-pg-client:4.3.8" + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + implementation("io.vertx:vertx-pg-client:4.3.8") + ``` -In some cases, such as with a [PostgreSQL](https://postgrespro.ru/docs/postgresql) database, it is also required to add [dependency](https://mvnrepository.com/artifact/com.ongres.scram/client/2.1). +For a [PostgreSQL](https://postgrespro.ru/docs/postgresql) database using `SCRAM` authentication, you also need to add the +[com.ongres.scram:client](https://mvnrepository.com/artifact/com.ongres.scram/client/2.1) dependency. + +The [io.projectreactor:reactor-core](https://mvnrepository.com/artifact/io.projectreactor/reactor-core) dependency is +required only if you use `Mono`/`Flux` method signatures. ## Configuration { #configuration } -Example of the complete configuration described in the `VertxDatabaseConfig` class (default or example values are specified): +Basic Vert.x configuration parameters: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript db { @@ -51,136 +76,258 @@ Example of the complete configuration described in the `VertxDatabaseConfig` cla password = "postgres" //(3)! poolName = "kora" //(4)! maxPoolSize = 10 //(5)! - connectionTimeout = "10s" //(6)! - acquireTimeout = "0s" //(7)! - idleTimeout = "10m" //(8)! - cachePreparedStatements = true //(9)! - initializationFailTimeout = "0s" //(10)! - readinessProbe = false //(11)! - telemetry { - logging { - enabled = false //(12)! - } - metrics { - enabled = true //(13)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(14)! - tags = { // (15)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(16)! - attributes = { // (17)! - "key1" = "value1" - "key2" = "value2" - } - } - } } ``` - 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) connection URI (**required**) - 2. User name for connection (**required**) - 3. Password of the user to connect (**required**) - 4. Database connection set name (**required**) - 5. Maximum size of the database connection set - 6. Maximum time to establish a connection - 7. Maximum time to get a connection from a connection set (optional) - 8. Maximum time for connection downtime - 9. Whether to cache prepared requests - 10. Maximum time to wait for connection initialization at service startup (optional) - 11. Whether to enable [readiness probe](probes.md#readiness) for database connection - 12. Enables module logging (default `false`) - 13. Enables module metrics (default `true`) - 14. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 15. Configures tags for metrics (optional) - 16. Enables module tracing (default `true`) - 17. Configures attributes for tracing (optional) + 1. Database connection `URL` (`required`, no default) + 2. Username for connection (`required`, no default) + 3. Password for connection (`required`, no default) + 4. Connection pool name (`required`, no default) + 5. Maximum connection pool size (default: `10`) === ":simple-yaml: `YAML`" ```yaml db: - connectionUri = "postgresql://localhost:5432/postgres" #(1)! + connectionUri: "postgresql://localhost:5432/postgres" #(1)! username: "postgres" #(2)! password: "postgres" #(3)! poolName: "kora" #(4)! maxPoolSize: 10 #(5)! - connectionTimeout: "10s" #(6)! - acquireTimeout: "10s" #(7)! - idleTimeout: "10m" #(8)! - cachePreparedStatements: true #(9)! - initializationFailTimeout: "0s" #(10)! - readinessProbe: false #(11)! - telemetry: - logging: - enabled: false #(12)! - metrics: - enabled: true #(13)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(14)! - tags: #(15)! - key1: value1 - key2: value2 - tracing: - enabled: true #(16)! - attributes: #(17)! - key1: value1 - key2: value2 ``` - 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) connection URI (**required**) - 2. User name for connection (**required**) - 3. Password of the user to connect (**required**) - 4. Database connection set name (**required**) - 5. Maximum size of the database connection set - 6. Maximum time to establish a connection - 7. Maximum time to get a connection from a connection set (optional) - 8. Maximum time for connection downtime - 9. Whether to cache prepared requests - 10. Maximum time to wait for connection initialization at service startup (optional) - 11. Whether to enable [readiness probe](probes.md#readiness) for database connection - 12. Enables module logging (default `false`) - 13. Enables module metrics (default `true`) - 14. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 15. Configures tags for metrics (optional) - 16. Enables module tracing (default `true`) - 17. Configures attributes for tracing (optional) - -You can also configure [Netty transport](netty.md). + 1. Database connection `URL` (`required`, no default) + 2. Username for connection (`required`, no default) + 3. Password for connection (`required`, no default) + 4. Connection pool name (`required`, no default) + 5. Maximum connection pool size (default: `10`) + +??? note "Full Configuration" + + Example of the complete configuration described by `VertxDatabaseConfig` (example values or default values are shown): + + ===! ":material-code-json: `HOCON`" + + ```javascript + db { + connectionUri = "postgresql://localhost:5432/postgres" //(1)! + username = "postgres" //(2)! + password = "postgres" //(3)! + poolName = "kora" //(4)! + maxPoolSize = 10 //(5)! + connectionTimeout = "10s" //(6)! + acquireTimeout = "10s" //(7)! + idleTimeout = "10m" //(8)! + cachePreparedStatements = true //(9)! + initializationFailTimeout = "10s" //(10)! + readinessProbe = false //(11)! + telemetry { + logging { + enabled = false //(12)! + } + metrics { + enabled = true //(13)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(14)! + tags = { // (15)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(16)! + attributes = { // (17)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Connection [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) for the database (`required`, default: not specified) + 2. Username for the connection (`required`, default: not specified) + 3. User password for the connection (`required`, default: not specified) + 4. Connection pool name (`required`, default: not specified) + 5. Maximum connection pool size (default: `10`) + 6. Maximum time to establish a physical connection (default: `10s`) + 7. Maximum time to acquire a connection from the pool; if not set, `connectionTimeout` is used instead (default: not specified, optional) + 8. Maximum idle time for a connection (default: `10m`) + 9. Whether to cache prepared statements (default: `true`) + 10. Maximum time to wait for a `SELECT 1` connection check at service startup; if not set, no startup check is performed (default: not specified, optional) + 11. Whether to enable the [readiness probe](probes.md#readiness) for the database connection (default: `false`) + 12. Enables module logging (default: `false`) + 13. Enables module metrics (default: `true`) + 14. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Configures metric tags (default: `{}`) + 16. Enables module tracing (default: `true`) + 17. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + db: + connectionUri: "postgresql://localhost:5432/postgres" #(1)! + username: "postgres" #(2)! + password: "postgres" #(3)! + poolName: "kora" #(4)! + maxPoolSize: 10 #(5)! + connectionTimeout: "10s" #(6)! + acquireTimeout: "10s" #(7)! + idleTimeout: "10m" #(8)! + cachePreparedStatements: true #(9)! + initializationFailTimeout: "10s" #(10)! + readinessProbe: false #(11)! + telemetry: + logging: + enabled: false #(12)! + metrics: + enabled: true #(13)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(14)! + tags: #(15)! + key1: value1 + key2: value2 + tracing: + enabled: true #(16)! + attributes: #(17)! + key1: value1 + key2: value2 + ``` + + 1. Connection [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) for the database (`required`, default: not specified) + 2. Username for the connection (`required`, default: not specified) + 3. User password for the connection (`required`, default: not specified) + 4. Connection pool name (`required`, default: not specified) + 5. Maximum connection pool size (default: `10`) + 6. Maximum time to establish a physical connection (default: `10s`) + 7. Maximum time to acquire a connection from the pool; if not set, `connectionTimeout` is used instead (default: not specified, optional) + 8. Maximum idle time for a connection (default: `10m`) + 9. Whether to cache prepared statements (default: `true`) + 10. Maximum time to wait for a `SELECT 1` connection check at service startup; if not set, no startup check is performed (default: not specified, optional) + 11. Whether to enable the [readiness probe](probes.md#readiness) for the database connection (default: `false`) + 12. Enables module logging (default: `false`) + 13. Enables module metrics (default: `true`) + 14. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Configures metric tags (default: `{}`) + 16. Enables module tracing (default: `true`) + 17. Configures tracing attributes (default: `{}`) + +Because the pool runs on the [Netty](netty.md) transport, you can also configure the [Netty transport](netty.md) separately. ## Usage { #usage } +A Vert.x repository is declared as an interface annotated with `@Repository` that must extend `VertxRepository`. +Each method annotated with `@Query` contains a regular `SQL` query. Method parameters are bound by name with the +`:parameter` syntax, and object fields can be referenced as `:entity.field`. + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends VertxRepository { } + public interface EntityRepository extends VertxRepository { + + record Entity(String id, @Column("value1") int field1, String value2, @Nullable String value3) {} + + @Query("SELECT * FROM entities WHERE id = :id") + Mono findById(String id); + + @Query("SELECT * FROM entities") + Flux findAll(); + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + Mono insert(Entity entity); + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + Mono insertBatch(@Batch List entities); + + @Query(""" + UPDATE entities + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + WHERE id = :entity.id + """) + Mono update(Entity entity); + + @Query("DELETE FROM entities WHERE id = :id") + Mono deleteById(String id); + } ``` === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : VertxRepository + interface EntityRepository : VertxRepository { + + data class Entity(val id: String, @Column("value1") val field1: Int, val value2: String, val value3: String?) + + @Query("SELECT * FROM entities WHERE id = :id") + fun findById(id: String): Mono + + @Query("SELECT * FROM entities") + fun findAll(): Flux + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + fun insert(entity: Entity): Mono + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + fun insertBatch(@Batch entities: List): Mono + + @Query(""" + UPDATE entities + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + WHERE id = :entity.id + """) + fun update(entity: Entity): Mono + + @Query("DELETE FROM entities WHERE id = :id") + fun deleteById(id: String): Mono + } ``` +`SQL` remains under the developer's control: you can use database-specific features, while `Kora` only handles safe +parameter binding, query execution, and result mapping. +Common rules for entities, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch`, and macros are described in +[Common database rules](database-common.md). + +Reactive `Mono` and `Flux` returns are the native signatures for this module, since the Vert.x client is asynchronous. +Blocking returns such as `Entity`, `List`, `void`, and `UpdateCount` are also supported, but they block the calling +thread until the asynchronous result completes, so prefer the reactive signatures when running in a reactive context. + ## Mapping { #mapping } -It is possible to override the conversion of different parts of [entity](database-common.md) and query parameters, Kora provides special interfaces for this. +You can override the mapping of different parts of an [entity](database-common.md), a query result, and query parameters. +For this, `Kora` provides several mapper interfaces. ### Result { #result } -If you need to convert the result manually, it is suggested to use `VertxRowSetMapper`: +Use `VertxRowSetMapper` when you need to control the whole `io.vertx.sqlclient.RowSet`. +This mapper receives the entire result set and decides how to consume it and what to return: ===! ":fontawesome-brands-java: `Java`" ```java - final class ResultMapper implements VertxRowSetMapper> { + final class ResultMapper implements VertxRowSetMapper>> { @Override - public List apply(RowSet rows) { - // mapping code + public Map> apply(RowSet rows) { + var result = new LinkedHashMap>(rows.size()); + for (Row row : rows) { + var entityPart = new EntityPart(row.getString(0), row.getInteger(1)); + var entityParts = result.computeIfAbsent(entityPart.field1(), k -> new ArrayList<>()); + entityParts.add(entityPart); + } + return result; } } @@ -188,19 +335,22 @@ If you need to convert the result manually, it is suggested to use `VertxRowSetM public interface EntityRepository extends VertxRepository { @Mapping(ResultMapper.class) - @Query("SELECT id FROM entities") - Mono> getIds(); + @Query("SELECT id, value1 FROM entities") + Mono>> findAllParts(); } ``` === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin - class ResultMapper : VertxRowSetMapper> { - override fun apply(rows: RowSet): List { - // mapping code + class ResultMapper : VertxRowSetMapper>> { + override fun apply(rows: RowSet): Map> { + val result = LinkedHashMap>(rows.size()) + for (row in rows) { + val entityPart = EntityPart(row.getString(0), row.getInteger(1)) + result.computeIfAbsent(entityPart.field1) { ArrayList() }.add(entityPart) + } + return result } } @@ -208,23 +358,29 @@ If you need to convert the result manually, it is suggested to use `VertxRowSetM interface EntityRepository : VertxRepository { @Mapping(ResultMapper::class) - @Query("SELECT id FROM entities") - fun getIds(): Mono> + @Query("SELECT id, value1 FROM entities") + fun findAllParts(): Mono>> } ``` +In most cases you do not need to control the whole `RowSet`. +It is enough to provide a [VertxRowMapper](#row), and `Kora` automatically adapts it to the method return type using the +built-in `VertxRowSetMapper` helpers: `singleRowSetMapper` (single `T`), `listRowSetMapper` (`List`), and +`optionalRowSetMapper` (`Optional`). There is also `VertxRowSetMapper.extractUpdateCount` used to adapt a result to `UpdateCount`. + ### Row { #row } -If you need to convert the string manually, it is suggested to use `VertxRowMapper`: +Use `VertxRowMapper` when you need to manually map one row of the result. +Columns are read from `io.vertx.sqlclient.Row` by type and index (starting from `0`): ===! ":fontawesome-brands-java: `Java`" ```java - final class RowMapper implements VertxRowMapper { + final class RowMapper implements VertxRowMapper { @Override - public UUID apply(Row row) { - return UUID.fromString(rs.get(0, String.class)); + public EntityPart apply(Row row) { + return new EntityPart(row.get(String.class, 0), row.get(Integer.class, 1)); } } @@ -232,20 +388,18 @@ If you need to convert the string manually, it is suggested to use `VertxRowMapp public interface EntityRepository extends VertxRepository { @Mapping(RowMapper.class) - @Query("SELECT id FROM entities") - Flux findAll(); + @Query("SELECT id, value1 FROM entities") + Flux findAllParts(); } ``` === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin - class RowMapper : VertxRowMapper { + class RowMapper : VertxRowMapper { - override fun apply(row: Row): UUID { - return UUID.fromString(rs.get(0, String.class)) + override fun apply(row: Row): EntityPart { + return EntityPart(row.get(String::class.java, 0), row.get(Integer::class.java, 1)) } } @@ -253,110 +407,161 @@ If you need to convert the string manually, it is suggested to use `VertxRowMapp interface EntityRepository : VertxRepository { @Mapping(RowMapper::class) - @Query("SELECT id FROM entities") - fun findAll(): Flux + @Query("SELECT id, value1 FROM entities") + fun findAllParts(): Flux } ``` ### Column { #column } -If you need to convert the column value manually, it is suggested to use the `VertxResultColumnMapper`: +Use `VertxResultColumnMapper` when you need to manually map a single column value by its index: ===! ":fontawesome-brands-java: `Java`" ```java - public final class ColumnMapper implements VertxResultColumnMapper { + public final class ColumnMapper implements VertxResultColumnMapper { + + private static final Entity.FieldType[] ALL = Entity.FieldType.values(); + @Nullable @Override - public UUID apply(Row row, int index) { - return UUID.fromString(row.get(String.class, index)); + public Entity.FieldType apply(Row row, int index) { + var fieldAsInt = row.get(Integer.class, index); + if (fieldAsInt == null) { + return null; + } + for (var type : ALL) { + if (type.code() == fieldAsInt) { + return type; + } + } + return Entity.FieldType.UNKNOWN; } } @Table("entities") - public record Entity(@Mapping(ColumnMapper.class) @Id UUID id, String name) { } + public record Entity(String id, @Mapping(ColumnMapper.class) @Column("value1") FieldType field1) { + + enum FieldType { + UNKNOWN(-10), ONE(1), TWO(2); + + private final int code; + + FieldType(int code) { this.code = code; } + + public int code() { return code; } + } + } @Repository public interface EntityRepository extends VertxRepository { - @Query("SELECT id, name FROM entities") + @Query("SELECT id, value1 FROM entities") Flux findAll(); } ``` === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin - class ColumnMapper : VertxResultColumnMapper { + class ColumnMapper : VertxResultColumnMapper { - override fun apply(row: Row, index: Int): UUID { - return UUID.fromString(row.get(String.class, index)) + override fun apply(row: Row, index: Int): Entity.FieldType? { + val fieldAsInt = row.get(Integer::class.java, index) ?: return null + return Entity.FieldType.entries.firstOrNull { it.code == fieldAsInt } ?: Entity.FieldType.UNKNOWN } } @Table("entities") data class Entity( - @Id @Mapping(ColumnMapper::class) val id: UUID, - val name: String - ) + val id: String, + @Mapping(ColumnMapper::class) @Column("value1") val field1: FieldType + ) { + enum class FieldType(val code: Int) { UNKNOWN(-10), ONE(1), TWO(2) } + } @Repository interface EntityRepository : VertxRepository { - @Query("SELECT id, name FROM entities") + @Query("SELECT id, value1 FROM entities") fun findAll(): Flux } ``` +Unlike some other database modules, the column mapper here reads by numeric `index`, not by column label. + ### Parameter { #parameter } -If you want to convert the value of a query parameter manually, it is suggested to use `VertxParameterColumnMapper`: +Use `VertxParameterColumnMapper` when you need to manually convert a query parameter value. +The mapper returns the raw value that `Kora` binds into the Vert.x `Tuple`; return `null` for a `null` value: ===! ":fontawesome-brands-java: `Java`" ```java - public final class ParameterMapper implements VertxParameterColumnMapper { + public final class ParameterMapper implements VertxParameterColumnMapper { + @Nullable @Override - public Object apply(@Nullable UUID value) { - return value.toString(); + public Object apply(@Nullable Entity.FieldType fieldType) { + return (fieldType == null) ? null : fieldType.code(); } } @Repository public interface EntityRepository extends VertxRepository { - @Query("SELECT id, name FROM entities WHERE id = :id") - Flux findById(@Mapping(ParameterMapper.class) UUID id); + @Query("UPDATE entities SET value1 = :fieldType WHERE id = :id") + UpdateCount updateFieldType(String id, @Mapping(ParameterMapper.class) Entity.FieldType fieldType); } ``` === ":simple-kotlin: `Kotlin`" - In Kotlin, you only need to write mappers for `T?` types, so the type is specified as `@Nullable` in the interfaces. - ```kotlin - class ParameterMapper : VertxParameterColumnMapper { - override fun apply(value: UUID?): Any { - return value.toString() + class ParameterMapper : VertxParameterColumnMapper { + + override fun apply(fieldType: Entity.FieldType?): Any? { + return fieldType?.code } } @Repository interface EntityRepository : VertxRepository { - @Query("SELECT id, name FROM entities WHERE id = :id") - fun findById(@Mapping(ParameterMapper::class) id: UUID): Flux + @Query("UPDATE entities SET value1 = :fieldType WHERE id = :id") + fun updateFieldType(id: String, @Mapping(ParameterMapper::class) fieldType: Entity.FieldType): UpdateCount } ``` +The result column mapper and the parameter mapper can be stacked on the same view field. +This is convenient for mapping, for example, an enum both when reading a row and when binding a parameter: + +===! ":fontawesome-brands-java: `Java`" + + ```java + record Entity(String id, + @Mapping(FieldTypeColumnMapper.class) + @Mapping(FieldTypeParameterMapper.class) + @Column("value1") FieldType field1) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class Entity( + val id: String, + @Mapping(FieldTypeColumnMapper::class) + @Mapping(FieldTypeParameterMapper::class) + @Column("value1") val field1: FieldType + ) + ``` + ### Supported types { #supported-types } ??? abstract "List of supported types for arguments/return values out of the box" - These types are chosen because they are supported by most popular databases. + These types are selected because they are supported by most popular databases. + `Kora` provides built-in row, column, and parameter mappers for them. * void * boolean / Boolean @@ -365,71 +570,97 @@ If you want to convert the value of a query parameter manually, it is suggested * long / Long * double / Double * float / Float - * Buffer + * Buffer (`io.vertx.core.buffer.Buffer`) * String * BigInteger * BigDecimal * UUID - * LocalTime + * LocalDate * LocalDateTime + For other types, use custom `VertxResultColumnMapper` / `VertxParameterColumnMapper` mappers, + or a `VertxRowMapper` / `VertxRowSetMapper`. + ## Transactions { #transactions } -In order to perform manual queries, Kora has an interface `ru.tinkoff.kora.database.vertx.VertxConnectionFactory`, -which is provided in a method within the `VertxRepository` contract. -All repository methods called within a transaction lambda will be executed in that transaction. +For grouping queries into a transaction, `Kora` provides the `VertxConnectionFactory` interface +through the `VertxRepository` contract, obtained via `getVertxConnectionFactory()`. +All repository methods called inside the transaction lambda are executed in that same transaction. -In order to perform queries transactionally, the `inTx` contract can be used: +Use `inTx` to execute queries transactionally. The lambda receives an `io.vertx.sqlclient.SqlConnection` and must return +a `java.util.concurrent.CompletionStage`, so reactive `Mono` results from repository methods must be converted with +`.toFuture()`. If there is already an active transaction on the current `Context`, a nested `inTx` call reuses the same +connection and does not open a new transaction. + +A transactional sequence of operations can stay inside the repository itself as a regular method with an implementation. +This is useful when several `@Query` methods should stay next to the rest of the repository queries, +without moving technical database work to a service layer. ===! ":fontawesome-brands-java: `Java`" ```java - @Component - public final class SomeService { + @Repository + public interface EntityRepository extends VertxRepository { - private final EntityRepository repository; + @Query("INSERT INTO entities(id, value2) VALUES (:entity.id, :entity.value2)") + Mono insert(Entity entity); - public SomeService(EntityRepository repository) { - this.repository = repository; - } + @Query("UPDATE entities SET value2 = :value2 WHERE id = :id") + Mono updateValue(String id, String value2); - public Mono> saveAll(Entity one, Entity two) { - return repository.getVertxConnectionFactory().inTx(connection -> { - // do some work - return repository.insert(one) //(1)! - .zipWith(repository.insert(two), //(2)! - (r1, r2) -> List.of(one, two)); - }); + default CompletionStage> saveAll(Entity one, Entity two) { + return getVertxConnectionFactory().inTx(connection -> + insert(one) //(1)! + .then(updateValue(two.id(), two.value2())) //(2)! + .thenReturn(List.of(one, two)) + .toFuture()); } } ``` - 1. will be executed within the transaction or rolled back if the entire lambda throws an exception - 2. will be executed within the transaction or rolled back if the entire lambda throws an exception + 1. Executed within the transaction, or rolled back if the whole chain signals an error + 2. Executed within the transaction, or rolled back if the whole chain signals an error === ":simple-kotlin: `Kotlin`" ```kotlin - @Component - class SomeService(private val repository: EntityRepository) { + @Repository + interface EntityRepository : VertxRepository { + + @Query("INSERT INTO entities(id, value2) VALUES (:entity.id, :entity.value2)") + fun insert(entity: Entity): Mono + + @Query("UPDATE entities SET value2 = :value2 WHERE id = :id") + fun updateValue(id: String, value2: String): Mono - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.getVertxConnectionFactory.inTx { - repository.insert(one).zipWith(repository.insert(two)) //(1)! - { r1: UpdateCount, r2: UpdateCount -> listOf(one, two) } + fun saveAll(one: Entity, two: Entity): CompletionStage> { + return vertxConnectionFactory.inTx { _ -> + insert(one) //(1)! + .then(updateValue(two.id, two.value2)) //(2)! + .thenReturn(listOf(one, two)) + .toFuture() } } } ``` - 1. will be executed within the transaction or will be rolled back if the entire lambda throws an exception + 1. Executed within the transaction, or rolled back if the whole chain signals an error + 2. Executed within the transaction, or rolled back if the whole chain signals an error + +The transaction is committed when the returned `CompletionStage` completes successfully. +If the `CompletionStage` completes exceptionally, the transaction is rolled back and the error is propagated, so all database +changes made within the transaction are not applied. + +### Manual Connection Management { #connection } -### Connection { #connection } +If a query needs more complex logic or queries outside a repository, you can work with `io.vertx.sqlclient.SqlConnection` +directly. The `withConnection` method executes the lambda with a connection, but does not open a transaction by itself: -If some more complex logic is needed for the query, and `@Query` is not enough, you can use `io.r2dbc.spi.Connection`: +- if the current `Context` already contains a connection, the method passes that current connection to the lambda; +- if the current `Context` does not contain a connection, the method takes a new connection from the pool, stores it in the `Context` for the duration of the lambda, and closes it afterwards; +- nested calls to `withConnection` and repository methods inside this lambda use the same current connection. + +Both `withConnection` and `inTx` return a `CompletionStage`, and the `inTx` method is built on top of `withConnection`. ===! ":fontawesome-brands-java: `Java`" @@ -443,9 +674,9 @@ If some more complex logic is needed for the query, and `@Query` is not enough, this.repository = repository; } - public Mono> saveAll(Entity one, Entity two) { - return repository.getVertxConnectionFactory().inTx(connection -> { - // do some work + public CompletionStage> loadAll() { + return repository.getVertxConnectionFactory().withConnection(connection -> { + // do some work, returns CompletionStage }); } } @@ -457,35 +688,173 @@ If some more complex logic is needed for the query, and `@Query` is not enough, @Component class SomeService(private val repository: EntityRepository) { - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.getVertxConnectionFactory.inTx { connection -> - // do some work + fun loadAll(): CompletionStage> { + return repository.vertxConnectionFactory.withConnection { connection -> + // do some work, returns CompletionStage } } } ``` +The `VertxConnectionFactory` also exposes lower-level accessors: `currentConnection()` returns the connection bound to the +current `Context` (or `null` if none), `newConnection()` acquires a new `CompletionStage` from the pool, +`pool()` returns the underlying `io.vertx.sqlclient.Pool`, and `telemetry()` returns the database telemetry. + +## Manual Query With Telemetry { #query } + +If a query is hard to express as a single static `@Query`, you can create a regular method with an implementation and build +`SQL` manually. There is no `query` method on the factory; instead, use the `VertxRepositoryHelper` helper to run a query +through Kora telemetry on the current or a new connection. + +`VertxRepositoryHelper.completionStage` takes four arguments: + +- the `VertxConnectionFactory` (from `getVertxConnectionFactory()`); +- a `QueryContext` with the query identifier and the final `SQL`. The identifier is reported to telemetry, so it is convenient to use a stable name such as `Repository.method`; +- a Vert.x `Tuple` with the bound parameter values; +- a `VertxRowSetMapper` that consumes the `RowSet` and produces the return value. + +If a connection is already bound to the current `Context` (for example inside `inTx` or `withConnection`), the query is +executed on that connection; otherwise a new connection is taken from the pool and closed afterwards. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends VertxRepository { + + default CompletionStage> findByFilter(@Nullable String name, boolean onlyActive) { + var sql = new StringBuilder("SELECT id, name FROM entities WHERE 1 = 1"); + var params = new ArrayList(); + + if (name != null) { + params.add(name); + sql.append(" AND name = $").append(params.size()); + } + if (onlyActive) { + sql.append(" AND active = true"); + } + + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return VertxRepositoryHelper.completionStage( + getVertxConnectionFactory(), + queryContext, + Tuple.from(params), + rows -> { + var result = new ArrayList(rows.size()); + for (var row : rows) { + result.add(new Entity(row.getString(0), row.getString(1))); + } + return result; + } + ); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : VertxRepository { + + fun findByFilter(name: String?, onlyActive: Boolean): CompletionStage> { + val sql = StringBuilder("SELECT id, name FROM entities WHERE 1 = 1") + val params = mutableListOf() + + if (name != null) { + params += name + sql.append(" AND name = $").append(params.size) + } + if (onlyActive) { + sql.append(" AND active = true") + } + + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return VertxRepositoryHelper.completionStage( + vertxConnectionFactory, + queryContext, + Tuple.from(params) + ) { rows -> + rows.map { row -> Entity(row.getString(0), row.getString(1)) } + } + } + } + ``` + +If you prefer reactive return types, use `VertxRepositoryHelper.Reactor.mono` (returns `Mono` with a `VertxRowSetMapper`) +or `VertxRepositoryHelper.Reactor.flux` (returns `Flux` with a `VertxRowMapper`) instead. This is an advanced path; for +static queries prefer plain `@Query` methods. + +## Select by List { #select-by-list } + +Sometimes you need to select rows by a list of values. +`Kora` performs mappings at compile time and does not rewrite `SQL` at runtime, so a list parameter requires a custom mapper +that binds the whole collection as a single value. + +The example below shows `Postgres` through an array bound with `ANY(:ids)`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public final class ListOfStringParameterMapper implements VertxParameterColumnMapper> { + + @Override + public Object apply(@Nullable List value) { + return value == null ? null : value.toArray(String[]::new); + } + } + + @Repository + public interface EntityRepository extends VertxRepository { + + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + Flux findAllByIds(@Mapping(ListOfStringParameterMapper.class) List ids); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + class ListOfStringParameterMapper : VertxParameterColumnMapper?> { + + override fun apply(value: List?): Any? { + return value?.toTypedArray() + } + } + + @Repository + interface EntityRepository : VertxRepository { + + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + fun findAllByIds(@Mapping(ListOfStringParameterMapper::class) ids: List): Flux + } + ``` + ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Available repository method signatures out of the box. +Because the Vert.x client is natively asynchronous, no `Executor` component is required for the asynchronous signatures. ===! ":fontawesome-brands-java: `Java`" - The `T` refers to the type of the return value, either `List`, either `Void` or `UpdateCount`. + `T` means the return value type, or `List`, or `Void`, or `UpdateCount`. - `T myMethod()` - `@Nullable T myMethod()` - `Optional myMethod()` - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires the [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires the [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - By `T` we mean the type of the return value, either `T?`, either `List`, either `Unit` or `UpdateCount`. + `T` means the return value type, or `T?`, or `List`, or `Unit`, or `UpdateCount`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires the [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires the [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + +## Telemetry { #telemetry } + +Logging, metrics, and tracing are configured via the `telemetry` block in the [configuration](#configuration) and described in the [Metrics Reference](metrics.md#database) section. +To completely override telemetry, you can provide custom SPI factories; see the [Common Database Documentation](database-common.md#telemetry) for details. diff --git a/mkdocs/docs/en/documentation/general.md b/mkdocs/docs/en/documentation/general.md index 64a62f1..0c0d130 100644 --- a/mkdocs/docs/en/documentation/general.md +++ b/mkdocs/docs/en/documentation/general.md @@ -1,100 +1,113 @@ --- description: "Explains Kora framework fundamentals, annotation processors, compatibility, Gradle build setup, dependencies, application runtime, and terminology. Use when working with @KoraApp, annotation processors, Gradle, BOM, kora-parent, application plugin." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora framework fundamentals, annotation processors, compatibility, Gradle build setup, dependencies, application runtime, and terminology; key triggers include @KoraApp, annotation processors, Gradle, BOM, kora-parent, application plugin." + use_when: "Use this file for Kora docs or implementation questions about Kora framework fundamentals, annotation processors, Gradle, BOM, kora-parent, application plugin." --- -Kora is a cloud-oriented server-side Java framework and offers -many different modules for quickly building applications such as HTTP server and client, Kafka consumers, -database abstraction in the form of repositories, S3 client, gRPC server and client, -Camunda integration, telemetry for all modules, resilient module and much more. +Kora is a cloud-oriented server framework written in `Java` for applications written in `Java` and `Kotlin`. +This page describes the basic principles of Kora, environment requirements, annotation processor setup, minimal `Gradle` configuration, dependency management, and application startup. -You can read about the core features of Kora [on home page](../index.md). +Kora provides a set of modules for quickly building server applications: `HTTP` server and `HTTP` client, `Kafka` consumers, repositories for working with databases, `S3` client, `gRPC` server and `gRPC` client, `Camunda` integrations, module telemetry, resilience, and other capabilities. +The main framework characteristics are described [on the home page](../index.md). -Kora provides all the tools needed for modern Java or Kotlin server-side development: +Kora provides the tools usually needed for modern server-side development: -- Dependency injection and inversion via annotations -- Sufficiently high-level simple abstractions and development tools -- Aspect-oriented programming via annotations -- Large set of preconfigured integrations -- Observability, tracing, metrics according to `OpenTelemetry` standard and logging for all modules -- Easy and rapid testing with [JUnit5](junit5.md) -- Simple and detailed documentation supported by [guides and examples of working services](../guides/home.md) +- dependency injection through annotations; +- inversion of control without a separate container at runtime; +- aspect-oriented programming through annotations; +- sufficiently high-level simple abstractions and development tools; +- a large set of preconfigured integrations; +- telemetry, tracing, metrics according to the `OpenTelemetry` standard, and module logging; +- fast testing with [JUnit5](junit5.md); +- working [examples and guides](../guides/home.md). -In order to achieve high-performance and efficient code, Kora stands on these principles: +For high-performance, efficient, and predictable code, Kora follows these principles: -- Avoiding Reflection API in runtime -- Avoiding dynamic proxies in runtime -- Avoiding bytecode generation at compile time and runtime -- Source code generation via compile-time annotation processors -- Fine-grained abstractions -- Free aspects -- Using the most efficient implementations for integrations -- Encouraging and using effective programming practices and natural language constructs +- does not use `Reflection` during application runtime; +- does not use `dynamic proxy` during application runtime; +- does not generate bytecode during compilation or application runtime; +- creates source code at compile time through annotation processors; +- keeps thin abstractions over integrations; +- provides free aspects: without additional cost during application runtime; +- uses only the most efficient implementations for integrations; +- encourages and uses the most effective development principles and natural language constructs. -For a step-by-step walkthrough before the reference details, see [Creating Your First Kora Application](../guides/getting-started.md) and [Dependency Injection Introduction](../guides/dependency-injection-introduction.md). +If you need a step-by-step walkthrough before the reference description, see [Creating Your First Kora Application](../guides/getting-started.md) and [Dependency Injection Introduction](../guides/dependency-injection-introduction.md). -## Annotation Handlers { #annotation-handlers } +## Annotation Processors { #annotation-processor } -The main pillar on which the Kora framework is built is annotation processors. +Kora builds the application at compile time: processors read annotations, validate the code, and generate source files that are then compiled together with the application code. +As a result, the dependency graph, aspects, `HTTP` handlers, repositories, and other components become ordinary compiled code without `Reflection` at runtime. ===! ":fontawesome-brands-java: `Java`" - An annotation is a construct associated with Java source code elements such as classes, methods, and variables. - Annotations provide the program with information at compile time based on which the program can take further action. - The [Annotation Processor](https://docs.oracle.com/en/java/javase/17/docs/api/java.compiler/javax/annotation/processing/Processor.html) processes these annotations at compile time to provide functions such as code generation, error checking, etc. + An annotation is a construct associated with `Java` source code elements: classes, methods, parameters, and fields. + An [annotation processor](https://docs.oracle.com/en/java/javase/17/docs/api/java.compiler/javax/annotation/processing/Processor.html) is started by the compiler, reads these annotations, and can generate additional source code or stop compilation with a clear error. - Kora provides within a single dependency all the [annotation processors](https://docs.oracle.com/en/java/javase/17/docs/api/java.compiler/javax/annotation/processing/Processor.html) that will be required for all modules, - processors do not pull in any unnecessary dependencies that would leak at compile time or application runtime. + Kora provides all annotation processors in a single dependency: + + ```groovy + annotationProcessor "ru.tinkoff.kora:annotation-processors" + ``` + + This dependency is needed only at compile time and does not add extra libraries to the application runtime classpath. === ":simple-kotlin: `Kotlin`" - [Kotlin Symbol Processing (KSP)](https://kotlinlang.org/docs/ksp-overview.html) is an API that can be used to develop lightweight plugins for compilers. - KSP is a simplified API for compiler plugins that allows you to utilize the features of Kotlin - while minimizing the learning curve. Compared to kapt, annotation processors using KSP can run twice as fast (still drastically slower than Java). + [`KSP`](https://kotlinlang.org/docs/ksp-overview.html) (`Kotlin Symbol Processing`) is used for `Kotlin`. + `KSP` reads `Kotlin` source code symbols, passes them to Kora processors, and allows code generation before the main compilation step. - Another way to think of [KSP](https://kotlinlang.org/docs/ksp-overview.html) is as a preprocessor framework for Kotlin programs. If we think of KSP-based plugins as symbolic processors, or simply processors, the data flow at compile time can be described by the following steps: + Kora provides `KSP` processors in a single dependency: - - Processors read and analyze source programs and resources. - - Processors generate code or other forms of output. - - The Kotlin compiler compiles the source programs along with the generated code. + ```kotlin + ksp("ru.tinkoff.kora:symbol-processors") + ``` + + At the same time, `Kotlin` processing is usually slower than annotation processing in `Java`. -This approach allows you to use the familiar paradigm of programming by means of creating HTTP handlers, -Kafka producers, database repositories and so on, but gives a huge performance and transparency advantage over the well-known JVM frameworks. +### `KSP` { #ksp } + +`KSP` is needed only for `Kotlin` projects. +If the application is written in `Java`, use the regular `annotationProcessor`; if the application is written in `Kotlin`, connect `com.google.devtools.ksp` and the `ru.tinkoff.kora:symbol-processors` dependency. ## Compatibility { #compatibility } ===! ":fontawesome-brands-java: `Java`" - Requires a minimum version of [JDK 17](https://openjdk.org/projects/jdk/17/). + Requires at least [JDK 17](https://openjdk.org/projects/jdk/17/), recommended [`JDK` `21`](https://openjdk.org/projects/jdk/21/) — the version used by the official examples and application templates. - Configuration in `build.gradle`: + Minimal configuration in `build.gradle`: ```groovy plugins { id "java" - } + } - sourceCompatibility = JavaVersion.VERSION_17 - targetCompatibility = JavaVersion.VERSION_17 + java { + toolchain { + languageVersion = JavaLanguageVersion.of(21) + vendor = JvmVendorSpec.ADOPTIUM + } + } ``` -=== ":simple-kotlin: `Kotlin`" + The `vendor` pin is optional and simply matches the `Adoptium` toolchain used by the example projects; you may omit it or select another vendor. - Requires a minimum version of [JDK 17](https://openjdk.org/projects/jdk/17/). +=== ":simple-kotlin: `Kotlin`" - Recommended version of [Kotlin `1.9+`](https://github.com/JetBrains/kotlin/releases), compatibility with `1.8+` or `2+` is not guaranteed. + Requires at least [JDK 17](https://openjdk.org/projects/jdk/17/), recommended [`JDK` `21`](https://openjdk.org/projects/jdk/21/) because of `Kotlin` compatibility. - Recommended version of [KSP `1.9+`](https://github.com/google/ksp/releases) corresponds to the Kotlin version. + Recommended [`Kotlin` `1.9+`](https://github.com/JetBrains/kotlin/releases), compatibility with versions `1.8+` and `2+` is not guaranteed. + Recommended [`KSP` `1.9+`](https://github.com/google/ksp/releases) should match the `Kotlin` version. - Configuration in `build.gradle.kts`: - ```groovy + Minimal configuration in `build.gradle.kts`: + ```kotlin plugins { - kotlin("jvm") version ("1.9.25") - id("com.google.devtools.ksp") version ("1.9.25-1.0.20") + kotlin("jvm") version "1.9.25" + id("com.google.devtools.ksp") version "1.9.25-1.0.20" } kotlin { - jvmToolchain { languageVersion.set(JavaLanguageVersion.of("17")) } + jvmToolchain { languageVersion.set(JavaLanguageVersion.of("21")) } sourceSets.main { kotlin.srcDir("build/generated/ksp/main/kotlin") } sourceSets.test { kotlin.srcDir("build/generated/ksp/test/kotlin") } } @@ -102,87 +115,86 @@ Kafka producers, database repositories and so on, but gives a huge performance a ## Build System { #build-system } -Since annotation processors are the main pillar, it is assumed that you will use the [Gradle](https://gradle.org/guides/) build system, -because it supports annotation processors, incremental builds and is the most advanced build system in the JVM ecosystem. -Gradle version `7+` is required. +Kora is designed to be built with [Gradle](https://gradle.org/guides/) because `Gradle` has good support for annotation processors, `KSP`, incremental builds, and dependency management. +Requires `Gradle` `7+`, recommended `Gradle` `9.5+`. -In order to avoid having to specify versions for each dependency, it is suggested to use [BOM](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import) -dependency `ru.tinkoff.kora:kora-parent` which requires to specify the version once for all Kora dependencies at once. +To avoid specifying versions for each Kora dependency separately, use the [`BOM`](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import) `ru.tinkoff.kora:kora-parent`. +The `BOM` version is specified once, and the rest of the Kora dependencies are declared without an explicit version. ===! ":fontawesome-brands-java: `Java`" - Kora supports incremental builds of multiple rounds at the annotation processing stage, - a more detailed description of working with Gradle and Java can be found in their [official documentation](https://docs.gradle.org/current/userguide/java_plugin.html). - - The minimum required configuration of the application will be presented below `build.gradle`: + Minimal application configuration in `build.gradle`: ```groovy plugins { id "java" id "application" - } - - sourceCompatibility = JavaVersion.VERSION_17 - targetCompatibility = JavaVersion.VERSION_17 + } - configurations { - koraBom - annotationProcessor.extendsFrom(koraBom) - compileOnly.extendsFrom(koraBom) - implementation.extendsFrom(koraBom) - api.extendsFrom(koraBom) + java { + toolchain { + languageVersion = JavaLanguageVersion.of(21) + vendor = JvmVendorSpec.ADOPTIUM + } } dependencies { - koraBom platform("ru.tinkoff.kora:kora-parent:1.2.18") - annotationProcessor "ru.tinkoff.kora:annotation-processors" + annotationProcessor "ru.tinkoff.kora:annotation-processors:1.2.18" + implementation platform("ru.tinkoff.kora:kora-parent:1.2.18") } ``` - You can also check out [Creating Your First Kora Application](../guides/getting-started.md) for a more detailed guided example. + A more detailed example is available in [Creating Your First Kora Application](../guides/getting-started.md). === ":simple-kotlin: `Kotlin`" - Kora supports incremental builds of multiple rounds at the annotation processing stage, - a more detailed description of working with Gradle and Kotlin can be found in their [official documentation](https://kotlinlang.org/docs/get-started-with-jvm-gradle-project.html), - and it will also be useful to familiarize yourself with [KSP in Gradle](https://kotlinlang.org/docs/ksp-quickstart.html). - - For Kotlin it is assumed that [Gradle Kotlin DSL](https://docs.gradle.org/current/userguide/kotlin_dsl.html) will be used, - so all examples for Kotlin will be given in this syntax, if you use Groovy syntax, use Java examples. + [Gradle Kotlin DSL](https://docs.gradle.org/current/userguide/kotlin_dsl.html) is assumed for `Kotlin`. + If the project uses `Groovy DSL`, follow the `Java` examples. - The minimum required configuration of the application will be presented below `build.gradle.kts`: - ```groovy + Minimal application configuration in `build.gradle.kts`: + ```kotlin plugins { - kotlin("jvm") version ("1.9.25") - id("com.google.devtools.ksp") version ("1.9.25-1.0.20") id("application") + kotlin("jvm") version "1.9.25" + id("com.google.devtools.ksp") version "1.9.25-1.0.20" } kotlin { - jvmToolchain { languageVersion.set(JavaLanguageVersion.of("17")) } + jvmToolchain { languageVersion.set(JavaLanguageVersion.of("21")) } sourceSets.main { kotlin.srcDir("build/generated/ksp/main/kotlin") } sourceSets.test { kotlin.srcDir("build/generated/ksp/test/kotlin") } } - val koraBom: Configuration by configurations.creating - configurations { - ksp.get().extendsFrom(koraBom) - compileOnly.get().extendsFrom(koraBom) - api.get().extendsFrom(koraBom) - implementation.get().extendsFrom(koraBom) - } - dependencies { - koraBom(platform("ru.tinkoff.kora:kora-parent:1.2.18")) - ksp("ru.tinkoff.kora:symbol-processors") + ksp("ru.tinkoff.kora:symbol-processors:1.2.18") + implementation(platform("ru.tinkoff.kora:kora-parent:1.2.18")) } ``` - You can also check out [Creating Your First Kora Application](../guides/getting-started.md) for a more detailed guided example. + A more detailed example is available in [Creating Your First Kora Application](../guides/getting-started.md). + +The `testAnnotationProcessor` (`Java`) and `kspTest` (`Kotlin`) entries connect the same Kora processors to the test source set. +They are required so that test-time generated sources — for example the ones produced for [`@KoraAppTest`](junit5.md) — are created; without them the test compilation will not generate the necessary Kora code. + +In real projects the `BOM` version is usually extracted into a `gradle.properties` property (for example `koraVersion`) and referenced as `platform("ru.tinkoff.kora:kora-parent:$koraVersion")`, so the version is declared in a single place instead of being hardcoded in every module. + +!!! note "Compiler internal access" + + Some `Java` annotation processors read `jdk.compiler` internals. On newer `JDK`s this may require exporting the corresponding packages to the compiler. + If compilation fails with `IllegalAccessError` or `module jdk.compiler does not export ...` errors, add the following `JVM` arguments to `gradle.properties`: + + ```properties + org.gradle.jvmargs=--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED + ``` ## Dependencies { #dependencies } -Annotation processors are the main pillar on which Kora is built, they are a mandatory dependency, -and the [BOM dependency](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import) should not be forgotten: +Kora module documentation usually shows only the dependency of a specific module. +But the application must also connect the [`BOM`](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import) and processors shown below. ===! ":fontawesome-brands-java: `Java`" @@ -195,11 +207,14 @@ and the [BOM dependency](https://docs.gradle.org/current/userguide/platforms.htm compileOnly.extendsFrom(koraBom) implementation.extendsFrom(koraBom) api.extendsFrom(koraBom) + testImplementation.extendsFrom(koraBom) + testAnnotationProcessor.extendsFrom(koraBom) } dependencies { koraBom platform("ru.tinkoff.kora:kora-parent:1.2.18") annotationProcessor "ru.tinkoff.kora:annotation-processors" + testAnnotationProcessor "ru.tinkoff.kora:annotation-processors" } ``` @@ -207,10 +222,11 @@ and the [BOM dependency](https://docs.gradle.org/current/userguide/platforms.htm `build.gradle.kts`: - ```groovy + ```kotlin val koraBom: Configuration by configurations.creating configurations { ksp.get().extendsFrom(koraBom) + kspTest.get().extendsFrom(koraBom) compileOnly.get().extendsFrom(koraBom) api.get().extendsFrom(koraBom) implementation.get().extendsFrom(koraBom) @@ -219,24 +235,42 @@ and the [BOM dependency](https://docs.gradle.org/current/userguide/platforms.htm dependencies { koraBom(platform("ru.tinkoff.kora:kora-parent:1.2.18")) ksp("ru.tinkoff.kora:symbol-processors") + kspTest("ru.tinkoff.kora:symbol-processors") } ``` + + +After that, module dependencies can be declared without a version, for example: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + implementation "ru.tinkoff.kora:http-server-undertow" + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + implementation("ru.tinkoff.kora:http-server-undertow") + ``` + + + ## Run { #run } -Running and working with the application through the build system is supposed to be done using the [application plugin](https://docs.gradle.org/current/userguide/application_plugin.html) -which is provided by Gradle. +The [application plugin](https://docs.gradle.org/current/userguide/application_plugin.html) is usually used for local startup and building an executable archive. ===! ":fontawesome-brands-java: `Java`" - Requires the plugin to be specified in `build.gradle`: + Connect the plugin in `build.gradle`: ```groovy plugins { id "application" } ``` - You can specify both system variables and environment variables when running the application locally in `build.gradle`: + System properties and environment variables for local startup can be set in the `run` task: ```groovy run { jvmArgs += [ @@ -249,17 +283,16 @@ which is provided by Gradle. } ``` - You can run it with command: + Run: ```shell ./gradlew run ``` - You can configure the artifact build this way in `build.gradle`: + Archive build configuration: ```groovy - applicationName = "application" - mainClassName = "ru.tinkoff.kora.java.Application" - application { + applicationName = "application" + mainClassName = "ru.tinkoff.kora.java.Application" applicationDefaultJvmArgs = ["-Dfile.encoding=UTF-8"] } @@ -268,26 +301,26 @@ which is provided by Gradle. } ``` - You can build an artifact with the command: + Build the archive: ```shell ./gradlew distTar ``` - Example of configured application can be seen [here](https://github.com/kora-projects/kora-java-template/blob/master/build.gradle) + A configured application example is available in the [Java application template](https://github.com/kora-projects/kora-java-template/blob/master/build.gradle). === ":simple-kotlin: `Kotlin`" - Requires the plugin to be specified in `build.gradle.kts`: - ```groovy + Connect the plugin in `build.gradle.kts`: + ```kotlin plugins { id("application") - kotlin("jvm") version ("1.9.25") - id("com.google.devtools.ksp") version ("1.9.25-1.0.20") + kotlin("jvm") version "1.9.25" + id("com.google.devtools.ksp") version "1.9.25-1.0.20" } ``` - You can specify both system variables and environment variables when running the application locally in `build.gradle.kts`: - ```groovy + System properties and environment variables for local startup can be set in `JavaExec` tasks: + ```kotlin tasks.withType { jvmArgs( "-Xmx256m", @@ -299,13 +332,13 @@ which is provided by Gradle. } ``` - You can run it with command: + Run: ```shell ./gradlew run ``` - You can configure the artifact build this way in `build.gradle.kts`: - ```groovy + Archive build configuration: + ```kotlin application { applicationName = "application" mainClass.set("ru.tinkoff.kora.kotlin.ApplicationKt") @@ -317,22 +350,24 @@ which is provided by Gradle. } ``` - You can build an artifact with the command: + Build the archive: ```shell ./gradlew distTar ``` - Example of configured application can be seen [here](https://github.com/kora-projects/kora-kotlin-template/blob/master/build.gradle.kts) + A configured application example is available in the [Kotlin application template](https://github.com/kora-projects/kora-kotlin-template/blob/master/build.gradle.kts). ## Terminology { #terminology } -This section describes the basic terms found throughout the documentation and within the Kora framework: +This section describes the basic terms used in the Kora documentation: -- Factory - is factory a method that, creates instances of a component/classes/dependencies. -- Module - [module](container.md#external-module-factory) is a pluggable dependency, often external, that provides some factory methods and new functionality to the application. -- Component - [component](container.md#components) is a singleton class that implements some logic and is a dependency in a dependency container. -- Aspect - is aspect logic that will extend the standard behavior of a method by via annotation before and/or after its execution. +- Factory - a method that creates and returns an instance of a component or dependency. +- [Module](container.md#external-module-factory) - a connected dependency or interface with factory methods that add new components to the application. +- [Component](container.md#components) - an object in the Kora dependency graph. Usually this is a single class instance that implements part of the application logic. +- Aspect - logic that extends method behavior before, after, or around its execution based on an annotation. +- Dependency graph - a set of application components and links between them, built by Kora at compile time. -## First guide +## First Guide -After reading the general overview, continue with [Creating Your First Kora Application](../guides/getting-started.md). It turns the framework basics into a small runnable HTTP service and gives the rest of the documentation a concrete shape. +After the general overview, continue with [Creating Your First Kora Application](../guides/getting-started.md). +It shows the basic application structure with a small `HTTP` service that can be built and run. diff --git a/mkdocs/docs/en/documentation/graalvm-native.md b/mkdocs/docs/en/documentation/graalvm-native.md index 515c70a..617ef51 100644 --- a/mkdocs/docs/en/documentation/graalvm-native.md +++ b/mkdocs/docs/en/documentation/graalvm-native.md @@ -4,20 +4,35 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora GraalVM Native Image notes and native build considerations for Kora applications; key triggers include GraalVM, native-image, reflection config, AOT, native build." --- +GraalVM Native Image is a tool for `AOT compilation` that builds a Java application ahead of time into a standalone native image for the target platform. +Such an image starts without regular JVM warmup, but requires part of the information about code, resources, and reflection to be known at build time. + Kora creates its helper classes at compile time, does not use the Reflection API at runtime, does not use dynamic proxies, -does not generate bytecode at compile time or runtime, -so there are no problems building native image from Kora perspective. +does not generate bytecode at compile time or runtime. +This makes it easier to build Kora applications into a native image that starts faster and usually consumes less memory than a regular JVM application. +The main limitations of this kind of build are usually related not to Kora itself, but to third-party libraries that may require additional reflection, resource, or class initialization settings. + +Therefore, Kora itself usually does not require additional configuration to build a native image. + +## Requirements { #requirements } + +A native build requires a [GraalVM](https://www.graalvm.org/) JDK: **GraalVM Community Edition** or **Oracle GraalVM**, version 21. +The [Gradle plugin](https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html) selects such a toolchain through the `javaLauncher` block shown in [Build](#build) (`JvmVendorSpec.matching("GraalVM Community")`), +so an ordinary JDK can drive the build while `native-image` itself runs on GraalVM. +When building outside the plugin (for example, the `native-image` command inside a [Docker](#docker) builder), the `native-image` tool must be available on `PATH` — the official GraalVM container images already ship it. -Example of building a native image using [plugin](https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html) for `gradle`: +## Build { #build } + +Example of building a native image using the [Gradle plugin](https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html): ===! ":fontawesome-brands-java: `Java`" Plugin `build.gradle`: ```groovy plugins { - id "org.graalvm.buildtools.native" version "0.11.0" + id "org.graalvm.buildtools.native" version "0.11.5" } ``` @@ -32,7 +47,7 @@ Example of building a native image using [plugin](https://graalvm.github.io/nati verbose = true buildArgs.add("--report-unsupported-elements-at-runtime") javaLauncher = javaToolchains.launcherFor { - languageVersion = JavaLanguageVersion.of(17) + languageVersion = JavaLanguageVersion.of(21) vendor = JvmVendorSpec.matching("GraalVM Community") } } @@ -48,7 +63,7 @@ Example of building a native image using [plugin](https://graalvm.github.io/nati Plugin `build.gradle.kts`: ```groovy plugins { - id("org.graalvm.buildtools.native") version("0.11.0") + id("org.graalvm.buildtools.native") version("0.11.5") } ``` @@ -63,7 +78,7 @@ Example of building a native image using [plugin](https://graalvm.github.io/nati verbose.set(true) buildArgs.add("--report-unsupported-elements-at-runtime") javaLauncher = javaToolchains.launcherFor { - languageVersion = JavaLanguageVersion.of(17) + languageVersion = JavaLanguageVersion.of(21) vendor = JvmVendorSpec.matching("GraalVM Community") } } @@ -74,31 +89,269 @@ Example of building a native image using [plugin](https://graalvm.github.io/nati } ``` -Some libraries require additional configuration, some configurations are made in Kora. +Values added to `buildArgs` are passed straight to `native-image`. The most common ones: + +- `--report-unsupported-elements-at-runtime` — defer errors about unsupported features to runtime instead of failing the build (used in the example above). +- `--no-fallback` — never produce a *fallback* image (one that silently bundles a JVM); fail the build instead if something cannot be compiled ahead of time. This flag is used when invoking `native-image` directly (see [Docker](#docker)). +- `debug` / `verbose` — extra build diagnostics; can be dropped for release builds. + +Flags that Kora itself needs are contributed automatically by its modules and do **not** have to be added by hand: + +- `ru.tinkoff.kora:application-graph` ships `--install-exit-handlers` and `--initialize-at-build-time` for the virtual-thread executor holder. +- `ru.tinkoff.kora:common` ships `--initialize-at-run-time` for `Context` and the Reactor context hook. -Tested modules that should work without additional configuration: +These come from `META-INF/native-image` resources inside the module JARs and are merged into the build once the dependency is on the class path (see [Metadata](#metadata)). + +### Fat JAR { #build-jar } + +`native-image` compiles a single class path into the binary, so a Kora application is usually assembled into one *fat JAR* first. +Kora relies on merged `META-INF/services` files (compile-time generated modules and extensions), therefore the JAR must be built with service-file merging — for example with the [Shadow](https://gradleup.com/shadow/) plugin: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + plugins { + id "application" + id "com.gradleup.shadow" version "9.4.1" + } + + jar.enabled = false + shadowJar { + mergeServiceFiles() + manifest { + attributes "Main-Class": application.mainClass + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + plugins { + id("application") + id("com.gradleup.shadow") version("9.4.1") + } + + tasks.jar { + enabled = false + } + tasks.shadowJar { + mergeServiceFiles() + manifest { + attributes["Main-Class"] = "ru.tinkoff.kora.example.Application" + } + } + ``` + +The Shadow plugin produces an `*-all.jar` in `build/libs` that both the Gradle plugin and a direct `native-image` invocation can consume. + +## Docker { #docker } + +In CI and production the native image is usually produced with a two-stage Docker build: a GraalVM *builder* stage compiles the [fat JAR](#build-jar) into a binary, and a slim runtime stage ships only that binary. +This is how the [examples](https://github.com/kora-projects/kora-examples/tree/master/examples/graalvm) build their images, and it is independent of whether the application is written in Java or Kotlin: + +```dockerfile +FROM ghcr.io/graalvm/native-image-community:21 AS builder + +ARG TARGET_DIR=/opt/app +ARG SOURCE_DIR=build/libs +WORKDIR $TARGET_DIR + +COPY $SOURCE_DIR/*-all.jar $TARGET_DIR/application.jar +RUN native-image --no-fallback -classpath $TARGET_DIR/application.jar + +FROM ubuntu:noble AS runner + +ARG TARGET_DIR=/opt/app +WORKDIR $TARGET_DIR + +COPY --from=builder $TARGET_DIR/application $TARGET_DIR/application + +ARG DOCKER_USER=app +RUN groupadd -r $DOCKER_USER && useradd -rg $DOCKER_USER $DOCKER_USER +RUN chmod +x application +USER $DOCKER_USER + +EXPOSE 8080/tcp +EXPOSE 8085/tcp +CMD "/opt/app/application" +``` + +The builder stage compiles `application.jar` into a native binary named `application`, and the runtime stage runs it as a non-root user. +Build the fat JAR first (`./gradlew shadowJar`), then `docker build .`. + +## Metadata { #metadata } + +Some libraries need additional configuration for a native image, and `native-image` can only see what is declared as *reachability metadata*. +Kora ships the metadata for its own modules as `META-INF/native-image///` resources inside each module JAR, so it is applied automatically once the dependency is on the class path. + +Three kinds of files cover the common cases: + +- **`native-image.properties`** — build-time arguments, most importantly the class-initialization flags `--initialize-at-build-time` and `--initialize-at-run-time`. For example, Kora's `common` module initializes `ru.tinkoff.kora.common.Context` *at run time* (its thread/context state must not be baked into the image), while `application-graph` initializes the virtual-thread executor holder *at build time*. +- **`reflect-config.json`** — classes, methods and fields accessed through reflection. For example, Kora registers `Thread.ofVirtual` / `Executors.newVirtualThreadPerTaskExecutor` so Loom virtual threads work in the native image. +- **`resource-config.json`** — resources that must be embedded into the binary. For example, Kora bundles `reference.conf` / `application.conf` so HOCON configuration is readable at runtime. + +### Repository { #metadata-repository } + +If the application uses third-party libraries that need reachability metadata they do not ship themselves, enable loading it from the [GraalVM Reachability Metadata Repository](https://github.com/oracle/graalvm-reachability-metadata): + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + graalvmNative { + metadataRepository { + enabled = true + } + } + + processResources.dependsOn tasks.collectReachabilityMetadata + sourceSets.main { resources.srcDirs += "$buildDir/native-reachability-metadata" } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + graalvmNative { + metadataRepository { + enabled.set(true) + } + } + + tasks.processResources { + dependsOn(tasks.collectReachabilityMetadata) + } + kotlin.sourceSets.main { + resources.srcDir(layout.buildDirectory.dir("native-reachability-metadata")) + } + ``` + +### Custom metadata { #metadata-custom } + +When neither Kora nor the repository covers a class, supply the metadata by hand: drop `native-image.properties`, `reflect-config.json` and/or `resource-config.json` under `src/main/resources/META-INF/native-image///` in your own application — `native-image` merges every such file found on the class path. + +For example, to embed the Logback configuration and the HOCON config file into the binary, an application can ship a `resource-config.json`: + +```json title="src/main/resources/META-INF/native-image/ru.tinkoff.kora.examples/logback/resource-config.json" +{ + "resources": { + "includes": [ + { "pattern": "\\Qlogback.xml\\E" }, + { "pattern": "\\Qapplication.conf\\E" } + ] + } +} +``` + +The `/` path segments are arbitrary but should be unique (usually your application's group and module) so that files from different dependencies do not collide. + +### Agent { #metadata-agent } + +For third-party libraries the repository does not cover, the standard way to discover the required metadata is the GraalVM *tracing agent*. +Run the application on a regular JVM with the agent attached, exercise the code paths that use reflection, resources or proxies, and the agent writes the corresponding config files: + +```bash +java -agentlib:native-image-agent=config-output-dir=src/main/resources/META-INF/native-image// \ + -jar build/libs/application-all.jar +``` + +Commit the generated files as [custom metadata](#metadata-custom). +This is the usual fallback when a native build fails at runtime with a missing-reflection or missing-resource error. + +### Annotation hints { #metadata-hints } + +The official examples generate part of the metadata from annotations with the third-party [GraalVM Hint Processor](https://github.com/GoodforGod/graalvm-hint) library. +This is **not** a Kora API — it is an external, optional convenience that is interchangeable with the hand-written [custom metadata](#metadata-custom) above. + +Add the processor and the annotations: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + dependencies { + annotationProcessor "io.goodforgod:graalvm-hint-processor:1.2.0" + compileOnly "io.goodforgod:graalvm-hint-annotations:1.2.0" + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + plugins { + kotlin("kapt") + } + + dependencies { + kapt("io.goodforgod:graalvm-hint-processor:1.2.0") + compileOnly("io.goodforgod:graalvm-hint-annotations:1.2.0") + } + ``` + +Then annotate the `@KoraApp` interface to declare the entrypoint and the resources to embed — the processor generates the matching `native-image` config at compile time: + +===! ":fontawesome-brands-java: `Java`" + + ```java + import io.goodforgod.graalvm.hint.annotation.NativeImageHint; + import io.goodforgod.graalvm.hint.annotation.ResourceHint; + + @ResourceHint(include = {"openapi/http-server.yaml"}) + @NativeImageHint(name = "application", entrypoint = Application.class) + @KoraApp + public interface Application { + + static void main(String[] args) { + KoraApplication.run(ApplicationGraph::graph); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import io.goodforgod.graalvm.hint.annotation.NativeImageHint + import io.goodforgod.graalvm.hint.annotation.ResourceHint + + @ResourceHint(include = ["openapi/http-server.yaml"]) + @NativeImageHint(name = "application", entrypoint = Application::class) + @KoraApp + interface Application { + + companion object { + @JvmStatic + fun main(args: Array) { + KoraApplication.run(ApplicationGraph::graph) + } + } + } + ``` + +## Modules { #modules } + +Modules for which Kora already provides the required part of native image configuration: - [Configuration](config.md) -- [Json](json.md) -- [Logback](logging-slf4j.md) +- [JSON](json.md) +- [Logback logging](logging-slf4j.md) - [Probes](probes.md) - [Metrics](metrics.md) - [Tracing](tracing.md) - [HTTP server](http-server.md) - [HTTP client](http-client.md) -- [OpenAPI generation](openapi-codegen.md) -- [OpenAPI mapping](openapi-management.md) +- [OpenAPI code generation](openapi-codegen.md) +- [OpenAPI display](openapi-management.md) - [JDBC (Postgres) database](database-jdbc.md) - [R2DBC (Postgres) database](database-r2dbc.md) -- [Database Vertx (Postgres)](database-vertx.md) +- [Vert.x database (Postgres)](database-vertx.md) - [Cassandra database](database-cassandra.md) - [Kafka](kafka.md) -- [gRPC Server](grpc-server.md) +- [gRPC server](grpc-server.md) - [gRPC client](grpc-client.md) -- [resilient](resilient.md) +- [Resilience](resilient.md) - [Cache](cache.md) - [Validation](validation.md) - [Scheduling](scheduling.md) - [Logging](logging-aspect.md) -See examples of working native services at [repository with examples](https://github.com/kora-projects/kora-examples). +Each of these modules ships its `META-INF/native-image` configuration inside its own JAR, so the settings are applied automatically once the dependency is on the class path; the core class-initialization and virtual-thread flags come from `ru.tinkoff.kora:application-graph` and `ru.tinkoff.kora:common`. + +Ready-to-use examples for building with Gradle and Docker are available in the [examples repository](https://github.com/kora-projects/kora-examples/tree/master/examples/graalvm). diff --git a/mkdocs/docs/en/documentation/grpc-client.md b/mkdocs/docs/en/documentation/grpc-client.md index 68814ad..c77a19b 100644 --- a/mkdocs/docs/en/documentation/grpc-client.md +++ b/mkdocs/docs/en/documentation/grpc-client.md @@ -4,7 +4,14 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora gRPC client generation, protobuf Gradle plugin setup, client configuration, generated services, interceptors, and mapping; key triggers include GrpcClientModule, @GrpcClient, @InterceptWith, GrpcClientConfig, GrpcClientInterceptor, protobuf plugin." --- -Module for gRPC client service support based on [grpc.io](https://grpc.io/docs/languages/java/basics/) functionality. +The `gRPC client` calls remote services using a `protobuf` contract and the `HTTP/2` transport. +In Kora, the client is built on top of generated `grpc-java` `stub` classes: the module creates a `ManagedChannel`, attaches interceptors, and registers ready-to-use `stub` instances in the application graph. + +For each service, Kora makes the generated stubs (`BlockingStub`, `FutureStub`, the async `Stub`, and the Kotlin coroutine stub), +the raw `io.grpc.Channel`, and the resolved `GrpcClientConfig` injectable, distinguishing every client by the generated service-class `@Tag` +(for example `@Tag(SimpleServiceGrpc.class)`). + +The gRPC client transport uses Netty, so common `event loop` and transport settings can be configured in the [Netty](netty.md) section. For a step-by-step walkthrough before the reference details, see [gRPC Client](../guides/grpc-client.md) and [Advanced gRPC Client](../guides/grpc-client-advanced.md). @@ -15,7 +22,7 @@ For a step-by-step walkthrough before the reference details, see [gRPC Client](. [Dependency](general.md#dependencies) `build.gradle`: ```groovy implementation "ru.tinkoff.kora:grpc-client" - implementation "io.grpc:grpc-protobuf:1.62.2" + implementation "io.grpc:grpc-protobuf:1.74.0" implementation "javax.annotation:javax.annotation-api:1.3.2" ``` @@ -30,7 +37,7 @@ For a step-by-step walkthrough before the reference details, see [gRPC Client](. [Dependency](general.md#dependencies) `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:grpc-client") - implementation("io.grpc:grpc-protobuf:1.62.2") + implementation("io.grpc:grpc-protobuf:1.74.0") implementation("javax.annotation:javax.annotation-api:1.3.2") ``` @@ -42,7 +49,8 @@ For a step-by-step walkthrough before the reference details, see [gRPC Client](. ### Plugin { #plugin } -The code for the gRPC client is created with [protobuf gradle plugin](https://github.com/google/protobuf-gradle-plugin). +The code for the `gRPC client` is created with the [protobuf Gradle plugin](https://github.com/google/protobuf-gradle-plugin). +The plugin generates Java message classes from the `protobuf` contract and gRPC `stub` classes that are then used by Kora. ===! ":fontawesome-brands-java: `Java`" @@ -55,7 +63,7 @@ The code for the gRPC client is created with [protobuf gradle plugin](https://gi protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { all()*.plugins { grpc {} } @@ -83,7 +91,7 @@ The code for the gRPC client is created with [protobuf gradle plugin](https://gi protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { ofSourceSet("main").forEach { it.plugins { id("grpc") { } } } @@ -100,93 +108,282 @@ The code for the gRPC client is created with [protobuf gradle plugin](https://gi ## Configuration { #configuration } -gRPC service named `SimpleService`, will have configuration with path of `grpcClient.SimpleService`. +A `gRPC client` for the `SimpleService` service will have the `grpcClient.SimpleService` configuration path. -Example of the complete configuration described in the `GrpcClientConfig` class (default or example values are specified): +Basic configuration parameters: ===! ":material-code-json: `Hocon`" ```javascript grpcClient { SimpleService { - url = "grpc://localhost:8090" //(1)! + url = "http://localhost:8090" //(1)! timeout = "10s" //(2)! - keepAliveTime = "0s" //(3)! - keepAliveTimeout = "0s" //(4)! - loadBalancingPolicy = "pick_first" //(5)! - telemetry { - logging { - enabled = false //(6)! + } + } + ``` + + 1. Server `URL` where requests will be sent (`required`, no default). + 2. Maximum request execution time (default: not specified, optional). The value is applied as a `deadline` if the call does not already have its own `deadline`. + +=== ":simple-yaml: `YAML`" + + ```yaml + grpcClient: + SimpleService: + url: "http://localhost:8090" #(1)! + timeout: "10s" #(2)! + ``` + + 1. Server `URL` where requests will be sent (`required`, no default). + 2. Maximum request execution time (default: not specified, optional). The value is applied as a `deadline` if the call does not already have its own `deadline`. + +??? note "Full Configuration" + + Example of the complete configuration described by the `GrpcClientConfig` class: + + ===! ":material-code-json: `Hocon`" + + ```javascript + grpcClient { + SimpleService { + url = "http://localhost:8090" //(1)! + timeout = "10s" //(2)! + keepAliveTime = "0s" //(3)! + keepAliveTimeout = "0s" //(4)! + loadBalancingPolicy = "pick_first" //(5)! + defaultServiceConfig { //(6)! + loadBalancingConfig = [ + { + round_robin = {} + } + ] } - metrics { - enabled = true //(7)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(8)! - tags = { // (9)! - "key1" = "value1" - "key2" = "value2" + telemetry { + logging { + enabled = false //(7)! } - } - tracing { - enabled = true //(10)! - attributes = { // (11)! - "key1" = "value1" - "key2" = "value2" + metrics { + enabled = true //(8)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(9)! + tags = { // (10)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(11)! + attributes = { // (12)! + "key1" = "value1" + "key2" = "value2" + } } } } } + ``` + + 1. Server `URL` where requests will be sent (required, default: not specified). + 2. Maximum request execution time (default: not specified, optional). The value is applied as a `deadline` if the call does not already have its own `deadline`. + 3. Interval between gRPC `PING` frames (default: not specified, optional). + 4. Timeout for acknowledging a `PING` frame (default: not specified, optional). If the acknowledgement is not received within this time, the connection is closed. + 5. Load balancing policy for `ManagedChannelBuilder` (default: not specified, optional). + 6. Standard gRPC service configuration passed to `ManagedChannelBuilder.defaultServiceConfig` (default: not specified, optional). + 7. Enables module logging (default: `false`). + 8. Enables module metrics (default: `true`). + 9. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 10. Additional tags for metrics (default: `{}`). + 11. Enables module tracing (default: `true`). + 12. Additional attributes for tracing (default: `{}`). + + === ":simple-yaml: `YAML`" + + ```yaml + grpcClient: + SimpleService: + url: "http://localhost:8090" #(1)! + timeout: "10s" #(2)! + keepAliveTime: "0s" #(3)! + keepAliveTimeout: "0s" #(4)! + loadBalancingPolicy: "pick_first" #(5)! + defaultServiceConfig: #(6)! + loadBalancingConfig: + - round_robin: {} + telemetry: + logging: + enabled: false #(7)! + metrics: + enabled: true #(8)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! + tags: #(10)! + key1: value1 + key2: value2 + tracing: + enabled: true #(11)! + attributes: #(12)! + key1: value1 + key2: value2 + ``` + + 1. Server `URL` where requests will be sent (required, default: not specified). + 2. Maximum request execution time (default: not specified, optional). The value is applied as a `deadline` if the call does not already have its own `deadline`. + 3. Interval between gRPC `PING` frames (default: not specified, optional). + 4. Timeout for acknowledging a `PING` frame (default: not specified, optional). If the acknowledgement is not received within this time, the connection is closed. + 5. Load balancing policy for `ManagedChannelBuilder` (default: not specified, optional). + 6. Standard gRPC service configuration passed to `ManagedChannelBuilder.defaultServiceConfig` (default: not specified, optional). + 7. Enables module logging (default: `false`). + 8. Enables module metrics (default: `true`). + 9. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 10. Additional tags for metrics (default: `{}`). + 11. Enables module tracing (default: `true`). + 12. Additional attributes for tracing (default: `{}`). + +### Transport and TLS { #transport-tls } + +The `url` scheme selects the transport when the `ManagedChannel` is created (`ManagedChannelLifecycle`): + +- `http` — plaintext transport (`usePlaintext()` on the builder), default port `80` when the port is omitted. +- `https` — `TLS` transport, default port `443` when the port is omitted. +- any other scheme — the port must be specified explicitly, otherwise an `IllegalArgumentException` with the message `Unknown scheme ''` is thrown at startup while resolving the default port. Plaintext is enabled only for the `http` scheme, so other schemes fall back to the default gRPC (`TLS`) transport unless a plaintext port is reachable. + +===! ":material-code-json: `Hocon`" + + ```javascript + grpcClient { + SimpleService { + url = "http://localhost:8090" // plaintext, default port 80 + } } ``` - 1. URL of the server where to make requests (**required**) - 2. Maximum request time (optional) - 3. Sets the interval in milliseconds between PING frames - 4. Sets the timeout in milliseconds for a PING frame to be acknowledged. If sender does not receive an acknowledgment within this time, it will close the connection - 5. Sets the load balancing policy - 6. Enables module logging (default `false`) - 7. Enables module metrics (default `true`) - 8. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 9. Configures tags for metrics (optional) - 10. Enables module tracing (default `true`) - 11. Configures attributes for tracing (optional) - === ":simple-yaml: `YAML`" ```yaml grpcClient: SimpleService: - url: "grpc://localhost:8090" //(1)! - timeout: "10s" //(2)! - keepAliveTime: "0s" //(3)! - keepAliveTimeout: "0s" //(4)! - loadBalancingPolicy: "pick_first" //(5)! - telemetry: - logging: - enabled: false #(6)! - metrics: - enabled: true #(7)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(8)! - tags: #(9)! - key1: value1 - key2: value2 - tracing: - enabled: true #(10)! - attributes: #(11)! - key1: value1 - key2: value2 - ``` - - 1. URL of the server where to make requests (**required**) - 2. Maximum request time (optional) - 3. Sets the interval in milliseconds between PING frames - 4. Sets the timeout in milliseconds for a PING frame to be acknowledged. If sender does not receive an acknowledgment within this time, it will close the connection - 5. Sets the load balancing policy - 6. Enables module logging (default `false`) - 7. Enables module metrics (default `true`) - 8. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 9. Configures tags for metrics (optional) - 10. Enables module tracing (default `true`) - 11. Configures attributes for tracing (optional) + url: "http://localhost:8090" # plaintext, default port 80 + ``` + +For custom `TLS` (`mTLS`, a custom trust store, or non-Netty transport) register your own `GrpcClientChannelFactory` component. +It builds the `ManagedChannelBuilder` and can pass an `io.grpc.ChannelCredentials`; the default implementation is `GrpcNettyClientChannelFactory`, +which binds the channel to Kora's shared Netty `EventLoopGroup`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class TlsGrpcClientChannelFactory implements GrpcClientChannelFactory { + + @Override + public ManagedChannelBuilder forAddress(SocketAddress serverAddress) { + return NettyChannelBuilder.forAddress(serverAddress); + } + + @Override + public ManagedChannelBuilder forAddress(SocketAddress serverAddress, ChannelCredentials creds) { + return NettyChannelBuilder.forAddress(serverAddress, creds); + } + + @Override + public ManagedChannelBuilder forTarget(String target) { + return NettyChannelBuilder.forTarget(target); + } + + @Override + public ManagedChannelBuilder forTarget(String target, ChannelCredentials creds) { + return NettyChannelBuilder.forTarget(target, creds); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class TlsGrpcClientChannelFactory : GrpcClientChannelFactory { + + override fun forAddress(serverAddress: SocketAddress): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forAddress(serverAddress) + } + + override fun forAddress(serverAddress: SocketAddress, creds: ChannelCredentials): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forAddress(serverAddress, creds) + } + + override fun forTarget(target: String): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forTarget(target) + } + + override fun forTarget(target: String, creds: ChannelCredentials): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forTarget(target, creds) + } + } + ``` + +A production override should also bind the builder to Kora's shared Netty `EventLoopGroup` and `NettyChannelFactory` +the way `GrpcNettyClientChannelFactory` does, rather than letting Netty create its own event loop. + +### Timeouts { #timeouts } + +The `timeout` value is applied by the always-on `GrpcClientConfigInterceptor` as a call `deadline`, but **only when the call has no deadline of its own**. +A per-call deadline set through the stub always wins over the configured `timeout`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + // uses the configured grpcClient.SimpleService.timeout as the deadline + var response = stub.createUser(request); + + // overrides the configured timeout for this single call + var responseWithDeadline = stub.withDeadlineAfter(2, TimeUnit.SECONDS).createUser(request); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // uses the configured grpcClient.SimpleService.timeout as the deadline + val response = stub.createUser(request) + + // overrides the configured timeout for this single call + val responseWithDeadline = stub.withDeadlineAfter(2, TimeUnit.SECONDS).createUser(request) + ``` + +When a deadline expires, the call fails with a `StatusRuntimeException` carrying `Status.DEADLINE_EXCEEDED`. + +### Channel config { #channge-config } + +- `keepAliveTime` / `keepAliveTimeout` map to `ManagedChannelBuilder` `PING` settings. `keepAliveTime` is the interval between `HTTP/2` `PING` frames on an idle connection; `keepAliveTimeout` is how long to wait for the `PING` acknowledgement before closing the connection. Both are disabled unless set. +- `loadBalancingPolicy` maps to `ManagedChannelBuilder.defaultLoadBalancingPolicy`. The gRPC default is `pick_first` (a single connection to the first resolved address); `round_robin` distributes calls across all resolved addresses and is typically used with DNS targets that return multiple `A`/`AAAA` records. +- `defaultServiceConfig` is passed as-is to `ManagedChannelBuilder.defaultServiceConfig` and carries the native gRPC [service config](https://github.com/grpc/grpc/blob/master/doc/service_config.md) map (`loadBalancingConfig`, per-method `methodConfig` with retry/hedging policy, etc.). It is described by the `DefaultServiceConfig` wrapper over `Map`. + +### Channel builder configurer { #builder-configurer } + +If file-based configuration is not enough, you can register a `GrpcClientBuilderConfigurer` component. +It receives an already prepared `ManagedChannelBuilder` and lets you configure the channel in code before it is created. +Settings from `GrpcClientConfig` are applied first, then `GrpcClientBuilderConfigurer` is called. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class CustomGrpcClientBuilderConfigurer implements GrpcClientBuilderConfigurer { + @Override + public ManagedChannelBuilder configure(ManagedChannelBuilder builder) { + return builder.maxInboundMessageSize(8 * 1024 * 1024); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class CustomGrpcClientBuilderConfigurer : GrpcClientBuilderConfigurer { + override fun configure(builder: ManagedChannelBuilder<*>): ManagedChannelBuilder<*> { + return builder.maxInboundMessageSize(8 * 1024 * 1024) + } + } + ``` You can also configure [Netty transport](netty.md). @@ -194,7 +391,7 @@ Module metrics are described in the [Metrics Reference](metrics.md#grpc-client) ## Service { #service } -Created gRPC services can be injected as dependency: +Created gRPC `stub` instances can be injected as dependencies: ===! ":fontawesome-brands-java: `Java`" @@ -202,7 +399,7 @@ Created gRPC services can be injected as dependency: @KoraApp public interface Application extends HoconConfigModule, GrpcClientModule { - default SomeService(SimpleServiceGrpc.SimpleServiceBlockingStub grpcService) { + default SomeService someService(SimpleServiceGrpc.SimpleServiceBlockingStub grpcService) { return new SomeService(grpcService); } } @@ -213,12 +410,204 @@ Created gRPC services can be injected as dependency: ```kotlin @KoraApp interface Application : HoconConfigModule, GrpcClientModule { - fun SomeService(grpcService: SimpleServiceGrpc.SimpleServiceBlockingStub?) { + fun someService(grpcService: SimpleServiceGrpc.SimpleServiceBlockingStub): SomeService { return SomeService(grpcService) } } ``` +A stub can also be injected straight into a `@Component` constructor: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final SimpleServiceGrpc.SimpleServiceBlockingStub grpcService; + + public SomeService(SimpleServiceGrpc.SimpleServiceBlockingStub grpcService) { + this.grpcService = grpcService; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val grpcService: SimpleServiceGrpc.SimpleServiceBlockingStub) + ``` + +### Stub types { #stub-types } + +The `protobuf` plugin generates several stub classes for one service (`SimpleService`). Each is injectable by simply declaring the corresponding type; +no `@Tag` is required on the stub itself (Kora resolves the tagged `Channel` behind the scenes): + +| Stub type | Call model | When to use | +|----------------------------------------|------------------------------------------------------------------------|------------------------------------------------------| +| `SimpleServiceBlockingStub` | Synchronous; returns the response directly (or an `Iterator` for server streaming) | Blocking code, simplest call style | +| `SimpleServiceFutureStub` | Asynchronous; returns a `ListenableFuture` (unary only) | Non-blocking code using `ListenableFuture` | +| `SimpleServiceStub` (async) | Asynchronous; delivers results through `StreamObserver` callbacks | Any streaming, callback-style asynchronous calls | +| Kotlin coroutine stub | `suspend` functions and `Flow` | Idiomatic Kotlin coroutines | + +The `BlockingStub`, `FutureStub`, and async `Stub` are wired by the annotation-processor extension (`GrpcClientExtension`), which detects the `@GrpcGenerated` +stub types and calls the generated `newBlockingStub` / `newFutureStub` / `newStub` factory against the tagged `Channel`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends HoconConfigModule, GrpcClientModule { + + default BlockingCaller blockingCaller(SimpleServiceGrpc.SimpleServiceBlockingStub stub) { + return new BlockingCaller(stub); + } + + default FutureCaller futureCaller(SimpleServiceGrpc.SimpleServiceFutureStub stub) { + return new FutureCaller(stub); + } + + default AsyncCaller asyncCaller(SimpleServiceGrpc.SimpleServiceStub stub) { + return new AsyncCaller(stub); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : HoconConfigModule, GrpcClientModule { + + fun blockingCaller(stub: SimpleServiceGrpc.SimpleServiceBlockingStub) = BlockingCaller(stub) + + fun futureCaller(stub: SimpleServiceGrpc.SimpleServiceFutureStub) = FutureCaller(stub) + + fun asyncCaller(stub: SimpleServiceGrpc.SimpleServiceStub) = AsyncCaller(stub) + } + ``` + +For Kotlin coroutine calls, generate the coroutine stub with the [gRPC Kotlin generator](https://github.com/grpc/grpc-kotlin) +(`io.grpc:protoc-gen-grpc-kotlin`). The generated stub extends `io.grpc.kotlin.AbstractCoroutineStub` and is annotated with `@StubFor`; +the KSP symbol processor then generates a Kora module that exposes it as a `@DefaultComponent` bound to the tagged `Channel`, so it is injected the same way: + +===! ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : HoconConfigModule, GrpcClientModule { + + fun coroutineCaller(stub: SimpleServiceGrpcKt.SimpleServiceCoroutineStub) = CoroutineCaller(stub) + } + ``` + +### Call styles { #call-styles } + +The `rpc` shape in the `.proto` contract (single vs `stream` request/response) determines the generated method signature. +The examples below extend the base contract with all four call styles: + +```protobuf +service SimpleService { + rpc unary(RequestEvent) returns (ResponseEvent) {} // unary + rpc serverStream(RequestEvent) returns (stream ResponseEvent) {} // server streaming + rpc clientStream(stream RequestEvent) returns (ResponseEvent) {} // client streaming + rpc biDiStream(stream RequestEvent) returns (stream ResponseEvent) {} // bidirectional streaming +} +``` + +Requests are built with the generated message builders (`RequestEvent.newBuilder()`). +For Java, unary and server-streaming calls are available on the `BlockingStub`, while client-streaming and bidirectional calls require the async `Stub` +(they have no blocking variant). The Kotlin coroutine stub expresses every style with `suspend` functions and `Flow`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var request = RequestEvent.newBuilder().setName("bob").setCode("b1").build(); + + // unary — BlockingStub + ResponseEvent unary = blockingStub.unary(request); + + // unary — FutureStub + ListenableFuture future = futureStub.unary(request); + + // server streaming — BlockingStub returns an iterator + Iterator responses = blockingStub.serverStream(request); + + // server streaming — async Stub delivers results to a StreamObserver + asyncStub.serverStream(request, new StreamObserver<>() { + @Override public void onNext(ResponseEvent value) { /* ... */ } + @Override public void onError(Throwable t) { /* ... */ } + @Override public void onCompleted() { /* ... */ } + }); + + // client streaming — async Stub, write requests, read one response + StreamObserver responseObserver = new StreamObserver<>() { + @Override public void onNext(ResponseEvent value) { /* single response */ } + @Override public void onError(Throwable t) { /* ... */ } + @Override public void onCompleted() { /* ... */ } + }; + StreamObserver requestObserver = asyncStub.clientStream(responseObserver); + requestObserver.onNext(request); + requestObserver.onCompleted(); + + // bidirectional streaming — async Stub, both sides stream + StreamObserver bidi = asyncStub.biDiStream(responseObserver); + bidi.onNext(request); + bidi.onCompleted(); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = RequestEvent.newBuilder().setName("bob").setCode("b1").build() + + // unary — suspend function + val response: ResponseEvent = coroutineStub.unary(request) + + // server streaming — returns a Flow + val serverFlow: Flow = coroutineStub.serverStream(request) + serverFlow.collect { event -> /* ... */ } + + // client streaming — accepts a Flow, returns a single response + val clientResponse: ResponseEvent = coroutineStub.clientStream(flowOf(request)) + + // bidirectional streaming — Flow in, Flow out + val biDiFlow: Flow = coroutineStub.biDiStream(flowOf(request)) + biDiFlow.collect { event -> /* ... */ } + ``` + +### Injecting Channel and config { #inject-channel-config } + +For advanced or manual stub construction you can inject the raw `io.grpc.Channel` and the resolved `GrpcClientConfig` +by tagging them with the generated service class. Both are provided by `GrpcClientExtension`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends HoconConfigModule, GrpcClientModule { + + default SomeService someService(@Tag(SimpleServiceGrpc.class) Channel channel, + @Tag(SimpleServiceGrpc.class) GrpcClientConfig config) { + return new SomeService(SimpleServiceGrpc.newBlockingStub(channel), config.url()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : HoconConfigModule, GrpcClientModule { + + fun someService( + @Tag(SimpleServiceGrpc::class) channel: Channel, + @Tag(SimpleServiceGrpc::class) config: GrpcClientConfig + ): SomeService = SomeService(SimpleServiceGrpc.newBlockingStub(channel), config.url()) + } + ``` + ## Interceptors { #interceptors } [Interceptors](https://grpc.github.io/grpc-java/javadoc/io/grpc/ClientInterceptor.html) allow you to intercept requests before they are passed to services. @@ -227,11 +616,14 @@ Created gRPC services can be injected as dependency: The following interceptors are used at client startup by default: -- `GrpcClientConfigInterceptor`. +- `GrpcClientConfigInterceptor` — applies `timeout` as the call `deadline` when the call has none. +- `GrpcClientTelemetryInterceptor`, if telemetry is available for the client. ### Custom { #custom } -In order to add your custom interceptor, you need to register the interceptor as a component with the service tag: +Unlike the [HTTP client](http-client.md#interceptors), gRPC interceptors have no method/class/global tiers. +Every interceptor is scoped **per client** by tagging the component with the generated service class (`@Tag(SimpleServiceGrpc.class)`). +Register the interceptor as a component with that tag: ===! ":fontawesome-brands-java: `Java`" @@ -253,7 +645,38 @@ In order to add your custom interceptor, you need to register the interceptor as @Tag(SimpleServiceGrpc::class) @Component class MyClientInterceptor : ClientInterceptor { - fun interceptCall( + override fun interceptCall( + method: MethodDescriptor, + callOptions: CallOptions, + next: Channel + ): ClientCall { + return next.newCall(method, callOptions) + } + } + ``` + +To apply one interceptor bean to several clients (a "shared" interceptor), give it multiple `@Tag` values — one generated service class per client: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag({SimpleServiceGrpc.class, OtherServiceGrpc.class}) + @Component + public final class SharedInterceptor implements ClientInterceptor { + @Override + public ClientCall interceptCall(MethodDescriptor method, CallOptions callOptions, Channel next) { + return next.newCall(method, callOptions); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(SimpleServiceGrpc::class, OtherServiceGrpc::class) + @Component + class SharedInterceptor : ClientInterceptor { + override fun interceptCall( method: MethodDescriptor, callOptions: CallOptions, next: Channel @@ -263,4 +686,221 @@ In order to add your custom interceptor, you need to register the interceptor as } ``` -Alternatively you can modify the gRPC service with [GraphInterceptor](container.md#component-inspection). +**Execution order:** + +`ManagedChannelLifecycle` collects all interceptors tagged for the service as `All` and applies them in a fixed order: +your custom interceptors first, then the telemetry interceptor (if telemetry is enabled), then the config/deadline interceptor last. + +``` +Request → Custom interceptors → Telemetry interceptor → Config (deadline) interceptor → gRPC Server +``` + +Because the deadline interceptor runs last, a deadline that a custom interceptor sets on the `CallOptions` is preserved, and the configured `timeout` +is applied only when no earlier interceptor (or per-call `withDeadlineAfter`) provided one. + +Alternatively, you can modify the `stub` with [GraphInterceptor](container.md#component-inspection). + +## Authorization { #authorization } + +gRPC has no dedicated authorization module: authorization is done with a `ClientInterceptor` tagged with the service class that attaches an +`Authorization` (or API-key) header to the outgoing call `Metadata`. The interceptor wraps the call in a `ForwardingClientCall.SimpleForwardingClientCall` +and puts the header in `start(...)`, before the request is sent. + +### Bearer { #bearer } + +A [Bearer](https://swagger.io/docs/specification/authentication/bearer-authentication/) interceptor reads a token from your own provider and +puts it on the `Authorization` header of every call. `TokenProvider` below is your own component: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(SimpleServiceGrpc.class) + @Component + public final class BearerAuthInterceptor implements ClientInterceptor { + + private static final Metadata.Key AUTHORIZATION = + Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER); + + private final TokenProvider tokenProvider; + + public BearerAuthInterceptor(TokenProvider tokenProvider) { + this.tokenProvider = tokenProvider; + } + + @Override + public ClientCall interceptCall(MethodDescriptor method, CallOptions callOptions, Channel next) { + return new ForwardingClientCall.SimpleForwardingClientCall<>(next.newCall(method, callOptions)) { + @Override + public void start(Listener responseListener, Metadata headers) { + headers.put(AUTHORIZATION, "Bearer " + tokenProvider.getToken()); + super.start(responseListener, headers); + } + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(SimpleServiceGrpc::class) + @Component + class BearerAuthInterceptor(private val tokenProvider: TokenProvider) : ClientInterceptor { + + override fun interceptCall( + method: MethodDescriptor, + callOptions: CallOptions, + next: Channel + ): ClientCall { + return object : ForwardingClientCall.SimpleForwardingClientCall(next.newCall(method, callOptions)) { + override fun start(responseListener: Listener, headers: Metadata) { + headers.put(AUTHORIZATION, "Bearer " + tokenProvider.getToken()) + super.start(responseListener, headers) + } + } + } + + companion object { + private val AUTHORIZATION: Metadata.Key = + Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER) + } + } + ``` + +### ApiKey { #apikey } + +An [API-key](https://swagger.io/docs/specification/authentication/api-keys/) interceptor puts a static key on a custom metadata header (for example `X-API-KEY`). +The key is read from a [`@ConfigSource`](config.md) interface injected into the interceptor: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(SimpleServiceGrpc.class) + @Component + public final class ApiKeyInterceptor implements ClientInterceptor { + + private static final Metadata.Key API_KEY = + Metadata.Key.of("X-API-KEY", Metadata.ASCII_STRING_MARSHALLER); + + private final String apiKey; + + public ApiKeyInterceptor(ApiKeyConfig config) { //(1)! + this.apiKey = config.apiKey(); + } + + @Override + public ClientCall interceptCall(MethodDescriptor method, CallOptions callOptions, Channel next) { + return new ForwardingClientCall.SimpleForwardingClientCall<>(next.newCall(method, callOptions)) { + @Override + public void start(Listener responseListener, Metadata headers) { + headers.put(API_KEY, apiKey); + super.start(responseListener, headers); + } + }; + } + } + ``` + + 1. Any `@ConfigSource` interface exposing the API key, for example `String apiKey();` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(SimpleServiceGrpc::class) + @Component + class ApiKeyInterceptor(config: ApiKeyConfig) : ClientInterceptor { //(1)! + + private val apiKey: String = config.apiKey() + + override fun interceptCall( + method: MethodDescriptor, + callOptions: CallOptions, + next: Channel + ): ClientCall { + return object : ForwardingClientCall.SimpleForwardingClientCall(next.newCall(method, callOptions)) { + override fun start(responseListener: Listener, headers: Metadata) { + headers.put(API_KEY, apiKey) + super.start(responseListener, headers) + } + } + } + + companion object { + private val API_KEY: Metadata.Key = + Metadata.Key.of("X-API-KEY", Metadata.ASCII_STRING_MARSHALLER) + } + } + ``` + + 1. Any `@ConfigSource` interface exposing the API key, for example `fun apiKey(): String` + +## Error handling { #error-handling } + +A failed gRPC call throws an `io.grpc.StatusRuntimeException`. Its `getStatus()` carries a `Status.Code` +([status codes](https://grpc.io/docs/guides/status-codes/)) such as `UNAVAILABLE` (server unreachable), `DEADLINE_EXCEEDED` (the `timeout`/deadline expired), +`UNAUTHENTICATED` (rejected credentials), or `INVALID_ARGUMENT`. Response metadata is available through `getTrailers()`. + +**Causes:** + +- `UNAVAILABLE` — wrong `url`, plaintext/TLS mismatch, or the server is down. +- `DEADLINE_EXCEEDED` — the configured `timeout` or a per-call `withDeadlineAfter` was exceeded. +- `UNAUTHENTICATED` / `PERMISSION_DENIED` — missing or invalid authorization metadata. + +**Recommendations:** + +- Branch on `e.getStatus().getCode()` instead of the exception type. +- Use [resilient](resilient.md) aspects (`@Retry`, `@CircuitBreaker`, `@Timeout`) on the wrapping service method for transient failures. + +===! ":fontawesome-brands-java: `Java`" + + ```java + try { + var response = stub.createUser(request); + } catch (StatusRuntimeException e) { + var code = e.getStatus().getCode(); + if (code == Status.Code.DEADLINE_EXCEEDED) { + // timeout / deadline exceeded + } else if (code == Status.Code.UNAVAILABLE) { + // server unreachable + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + try { + val response = stub.createUser(request) + } catch (e: StatusRuntimeException) { + when (e.status.code) { + Status.Code.DEADLINE_EXCEEDED -> { /* timeout / deadline exceeded */ } + Status.Code.UNAVAILABLE -> { /* server unreachable */ } + else -> throw e + } + } + ``` + +## Telemetry { #telemetry } + +Default logging, metrics, and tracing are configured through the `telemetry` block of the [configuration](#configuration) and +described in the [Metrics Reference](metrics.md#grpc-client) section. + +To customize the collected signals, override the telemetry SPI factories as components: `GrpcClientTelemetryFactory` +(the whole telemetry), `GrpcClientLoggerFactory`, `GrpcClientMetricsFactory`, or `GrpcClientTracerFactory`. +The default implementations are wired by `GrpcClientModule`; providing your own component replaces the corresponding default. + +## Telemetry { #telemetry } + +gRPC Client uses a telemetry contract for logging, metrics, and tracing of calls. +Telemetry configuration (section `telemetry { logging / metrics / tracing }`) is described in the [Configuration](#configuration) section. +Extension points are located in `ru.tinkoff.kora.grpc.client.common.telemetry`. + +For each gRPC call, a `GrpcClientTelemetry.GrpcClientTelemetryContext` is created, which is closed upon call completion. +The call is described through telemetry handler parameters, including service, method, response status, and duration. + +The default factory `DefaultGrpcClientTelemetryFactory` combines three factories: +- `GrpcClientLoggerFactory` builds `GrpcClientLogger` for logging call start/end; +- `GrpcClientMetricsFactory` builds `GrpcClientMetrics` for writing call metrics; +- `GrpcClientTracerFactory` builds `GrpcClientTracer` for distributed tracing. + +Metrics and tracing are described in the [Metrics Reference](metrics.md#grpc-client) section. diff --git a/mkdocs/docs/en/documentation/grpc-server.md b/mkdocs/docs/en/documentation/grpc-server.md index b17f67c..9c3e81d 100644 --- a/mkdocs/docs/en/documentation/grpc-server.md +++ b/mkdocs/docs/en/documentation/grpc-server.md @@ -1,10 +1,14 @@ --- -description: "Explains Kora gRPC server generation, protobuf Gradle plugin setup, server configuration, handlers, interceptors, reflection, and debugging. Use when working with GrpcServerModule, @GrpcService, @InterceptWith, GrpcServerConfig, GrpcServerInterceptor, Server Reflection." +description: "Explains Kora gRPC server: protobuf Gradle plugin setup, server configuration, unary and streaming handlers, io.grpc.Status error handling, ServerInterceptor interceptors and their execution order, lifecycle and readiness, telemetry, and reflection. Use when working with GrpcServerModule, GrpcServerConfig, GrpcServerBuilderConfigurer, ServerInterceptor, StreamObserver, reflectionEnabled, Server Reflection." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora gRPC server generation, protobuf Gradle plugin setup, server configuration, handlers, interceptors, reflection, and debugging; key triggers include GrpcServerModule, @GrpcService, @InterceptWith, GrpcServerConfig, GrpcServerInterceptor, Server Reflection." + use_when: "Use this file for Kora docs or implementation questions about the Kora gRPC server: protobuf Gradle plugin setup, server configuration, unary and streaming handlers, io.grpc.Status error handling, ServerInterceptor interceptors and their execution order, scoping and metadata authorization, lifecycle and readiness, telemetry, and reflection; key triggers include GrpcServerModule, GrpcServerConfig, GrpcServerBuilderConfigurer, ServerInterceptor, StreamObserver, reflectionEnabled, Server Reflection. Note: gRPC server interceptors are global io.grpc.ServerInterceptor beans only; there is no @GrpcService or @InterceptWith annotation in this module." --- -Module for gRPC server handlers support based on [grpc.io](https://grpc.io/docs/languages/java/basics/) functionality. +The module starts a `gRPC server` based on [`grpc-java`](https://grpc.io/docs/languages/java/basics/) and connects handlers from the application graph to it. +A handler is a `BindableService`, usually a class that extends a generated `...ImplBase` and implements unary or streaming `RPC` methods. + +Kora creates a `NettyServerBuilder`, adds server services, user-defined and standard `ServerInterceptor`, manages the server lifecycle, and participates in application readiness checks. +If configuration parameters are not enough, the resulting `NettyServerBuilder` can be additionally configured in code through `GrpcServerBuilderConfigurer`. For a step-by-step walkthrough before the reference details, see [gRPC Server](../guides/grpc-server.md) and [Advanced gRPC Server](../guides/grpc-server-advanced.md). @@ -12,10 +16,10 @@ For a step-by-step walkthrough before the reference details, see [gRPC Server](. ===! ":fontawesome-brands-java: `Java`" - [Dependency](general.md#dependencies) `build.gradle`: + [Dependency](general.md#dependencies) in `build.gradle`: ```groovy implementation "ru.tinkoff.kora:grpc-server" - implementation "io.grpc:grpc-protobuf:1.62.2" + implementation "io.grpc:grpc-protobuf:1.74.0" implementation "javax.annotation:javax.annotation-api:1.3.2" ``` @@ -27,10 +31,10 @@ For a step-by-step walkthrough before the reference details, see [gRPC Server](. === ":simple-kotlin: `Kotlin`" - [Dependency](general.md#dependencies) `build.gradle.kts`: + [Dependency](general.md#dependencies) in `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:grpc-server") - implementation "io.grpc:grpc-protobuf:1.62.2" + implementation("io.grpc:grpc-protobuf:1.74.0") implementation("javax.annotation:javax.annotation-api:1.3.2") ``` @@ -42,11 +46,11 @@ For a step-by-step walkthrough before the reference details, see [gRPC Server](. ### Plugin { #plugin } -The code for the gRPC server is created with [protobuf gradle plugin](https://github.com/google/protobuf-gradle-plugin). +The code for the `gRPC server` is generated with the [protobuf gradle plugin](https://github.com/google/protobuf-gradle-plugin). ===! ":fontawesome-brands-java: `Java`" - Plugin `build.gradle`: + Plugin in `build.gradle`: ```groovy plugins { id "com.google.protobuf" version "0.9.4" @@ -55,7 +59,7 @@ The code for the gRPC server is created with [protobuf gradle plugin](https://gi protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { all()*.plugins { grpc {} } @@ -72,7 +76,7 @@ The code for the gRPC server is created with [protobuf gradle plugin](https://gi === ":simple-kotlin: `Kotlin`" - Plugin `build.gradle.kts`: + Plugin in `build.gradle.kts`: ```groovy import com.google.protobuf.gradle.id @@ -83,7 +87,7 @@ The code for the gRPC server is created with [protobuf gradle plugin](https://gi protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { ofSourceSet("main").forEach { it.plugins { id("grpc") { } } } @@ -100,7 +104,29 @@ The code for the gRPC server is created with [protobuf gradle plugin](https://gi ## Configuration { #configuration } -Example of a complete configuration described in the `GrpcServerConfig` class (example values or default values are indicated): +Only `port` typically needs to be set; all other parameters have defaults. +A minimal configuration that binds a port from an environment variable and enables logging looks like this: + +===! ":material-code-json: `Hocon`" + + ```javascript + grpcServer { + port = 8090 + telemetry.logging.enabled = true + } + ``` + +=== ":simple-yaml: `YAML`" + + ```yaml + grpcServer: + port: 8090 + telemetry: + logging: + enabled: true + ``` + +Basic configuration parameters: ===! ":material-code-json: `Hocon`" @@ -109,48 +135,12 @@ Example of a complete configuration described in the `GrpcServerConfig` class (e port = 8090 //(1)! maxMessageSize = "4MiB" //(2)! reflectionEnabled = false //(3)! - shutdownWait = "30s" //(4)! - maxConnectionAge = "0s" //(5)! - maxConnectionAgeGrace = "0s" //(6)! - keepAliveTime = "0s" //(7)! - keepAliveTimeout = "0s" //(8)! - telemetry { - logging { - enabled = false //(9)! - } - metrics { - enabled = true //(10)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(11)! - tags = { // (12)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(13)! - attributes = { // (14)! - "key1" = "value1" - "key2" = "value2" - } - } - } } ``` - 1. gRPC server port - 2. Maximum size of the incoming message (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 3. Enables [gRPC Server Reflection](#reflection) service - 4. Time to wait for processing before shutting down the server in case of [graceful shutdown](container.md#graceful-shutdown) - 5. Sets a custom max connection age, connection lasting longer than which will be gracefully terminated. An unreasonably small value might be increased. A random jitter of +/-10% will be added to it. - 6. Sets a custom grace time for the graceful connection termination. Once the max connection age is reached, RPCs have the grace time to complete. RPCs that do not complete in time will be cancelled, allowing the connection to terminate. - 7. Sets the interval in milliseconds between PING frames - 8. Sets the timeout in milliseconds for a PING frame to be acknowledged. If sender does not receive an acknowledgment within this time, it will close the connection - 9. Enables module logging (default `false`) - 10. Enables module metrics (default `true`) - 11. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 12. Configures tags for metrics (optional) - 13. Enables module tracing (default `true`) - 14. Configures attributes for tracing (optional) + 1. `gRPC server` port (default: `8090`). + 2. Maximum incoming message size (default: `4MiB`). + 3. Enables [`gRPC Server Reflection`](#reflection) service (default: `false`). === ":simple-yaml: `YAML`" @@ -159,82 +149,513 @@ Example of a complete configuration described in the `GrpcServerConfig` class (e port: 8090 #(1)! maxMessageSize: "4MiB" #(2)! reflectionEnabled: false #(3)! - shutdownWait: "30s" #(4)! - maxConnectionAge: "0s" #(5)! - maxConnectionAgeGrace: "0s" #(6)! - keepAliveTime: "0s" #(7)! - keepAliveTimeout: "0s" #(8)! - telemetry: - logging: - enabled: false #(9)! - metrics: - enabled: true #(10)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(11)! - tags: #(12)! - key1: value1 - key2: value2 - tracing: - enabled: true #(13)! - attributes: #(14)! - key1: value1 - key2: value2 - ``` - - 1. gRPC server port - 2. Maximum size of the incoming message (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 3. Enables [gRPC Server Reflection](#reflection) service - 4. Time to wait for processing before shutting down the server in case of [graceful shutdown](container.md#graceful-shutdown) - 5. Sets a custom max connection age, connection lasting longer than which will be gracefully terminated. An unreasonably small value might be increased. A random jitter of +/-10% will be added to it. - 6. Sets a custom grace time for the graceful connection termination. Once the max connection age is reached, RPCs have the grace time to complete. RPCs that do not complete in time will be cancelled, allowing the connection to terminate. - 7. Sets the interval in milliseconds between PING frames - 8. Sets the timeout in milliseconds for a PING frame to be acknowledged. If sender does not receive an acknowledgment within this time, it will close the connection - 9. Enables module logging (default `false`) - 10. Enables module metrics (default `true`) - 11. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 12. Configures tags for metrics (optional) - 13. Enables module tracing (default `true`) - 14. Configures attributes for tracing (optional) + ``` + + 1. `gRPC server` port (default: `8090`). + 2. Maximum incoming message size (default: `4MiB`). + 3. Enables [`gRPC Server Reflection`](#reflection) service (default: `false`). + +??? note "Full Configuration" + + Example of a complete configuration described by `GrpcServerConfig`: + + ===! ":material-code-json: `Hocon`" + + ```javascript + grpcServer { + port = 8090 //(1)! + maxMessageSize = "4MiB" //(2)! + reflectionEnabled = false //(3)! + shutdownWait = "30s" //(4)! + maxConnectionAge = "0s" //(5)! + maxConnectionAgeGrace = "0s" //(6)! + keepAliveTime = "0s" //(7)! + keepAliveTimeout = "0s" //(8)! + telemetry { + logging { + enabled = false //(9)! + } + metrics { + enabled = true //(10)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(11)! + tags = { // (12)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(13)! + attributes = { // (14)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. `gRPC server` port (default: `8090`). + 2. Maximum size of an incoming message (default: `4MiB`). It can be specified as a number of bytes or as `4MiB`, `4MB`, `1000Kb`, and similar values. + 3. Enables the [`gRPC Server Reflection`](#reflection) service (default: `false`). + 4. Time to wait for processing before shutting down the server during [graceful shutdown](container.md#graceful-shutdown) (default: `30s`). + 5. Sets a custom maximum connection age after which the connection is gracefully terminated (default: not specified, optional). A random jitter of +/-10% is added to the value. + 6. Sets additional time for graceful connection termination after the maximum connection age is reached (default: not specified, optional). `RPC` calls that do not finish in time are cancelled so the connection can terminate. + 7. Sets the interval between `PING` frames (default: not specified, optional). + 8. Timeout for acknowledging a `PING` frame (default: not specified, optional). If no acknowledgement is received within this time, the connection is closed. + 9. Enables module logging (default: `false`). + 10. Enables module metrics (default: `true`). + 11. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 12. Metric tags (default: `{}`). + 13. Enables module tracing (default: `true`). + 14. Tracing attributes (default: `{}`). + + === ":simple-yaml: `YAML`" + + ```yaml + grpcServer: + port: 8090 #(1)! + maxMessageSize: "4MiB" #(2)! + reflectionEnabled: false #(3)! + shutdownWait: "30s" #(4)! + maxConnectionAge: "0s" #(5)! + maxConnectionAgeGrace: "0s" #(6)! + keepAliveTime: "0s" #(7)! + keepAliveTimeout: "0s" #(8)! + telemetry: + logging: + enabled: false #(9)! + metrics: + enabled: true #(10)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(11)! + tags: #(12)! + key1: value1 + key2: value2 + tracing: + enabled: true #(13)! + attributes: #(14)! + key1: value1 + key2: value2 + ``` + + 1. `gRPC server` port (default: `8090`). + 2. Maximum size of an incoming message (default: `4MiB`). It can be specified as a number of bytes or as `4MiB`, `4MB`, `1000Kb`, and similar values. + 3. Enables the [`gRPC Server Reflection`](#reflection) service (default: `false`). + 4. Time to wait for processing before shutting down the server during [graceful shutdown](container.md#graceful-shutdown) (default: `30s`). + 5. Sets a custom maximum connection age after which the connection is gracefully terminated (default: not specified, optional). A random jitter of +/-10% is added to the value. + 6. Sets additional time for graceful connection termination after the maximum connection age is reached (default: not specified, optional). `RPC` calls that do not finish in time are cancelled so the connection can terminate. + 7. Sets the interval between `PING` frames (default: not specified, optional). + 8. Timeout for acknowledging a `PING` frame (default: not specified, optional). If no acknowledgement is received within this time, the connection is closed. + 9. Enables module logging (default: `false`). + 10. Enables module metrics (default: `true`). + 11. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 12. Metric tags (default: `{}`). + 13. Enables module tracing (default: `true`). + 14. Tracing attributes (default: `{}`). You can also configure [Netty transport](netty.md). +### Configuration In Code { #builder-configurer } + +If configuration parameters are not enough, register a `GrpcServerBuilderConfigurer` component and additionally configure `NettyServerBuilder` in code. +This component is called after configuration has been applied and after services, user-defined `ServerInterceptor`, and standard `ServerInterceptor` have been added. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MyGrpcServerBuilderConfigurer implements GrpcServerBuilderConfigurer { + + @Override + public NettyServerBuilder configure(NettyServerBuilder builder) { + return builder.permitKeepAliveWithoutCalls(true); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyGrpcServerBuilderConfigurer : GrpcServerBuilderConfigurer { + + override fun configure(builder: NettyServerBuilder): NettyServerBuilder { + return builder.permitKeepAliveWithoutCalls(true) + } + } + ``` + Module metrics are described in the [Metrics Reference](metrics.md#grpc-server) section. ## Handlers { #handlers } -Created gRPC service handlers are required to be tagged with the `@Component` annotation: +A handler is a class that extends the generated `...ImplBase` and is registered in the application graph with the `@Component` annotation. +The `...ImplBase` class is produced from the `proto` contract by the [protobuf gradle plugin](#plugin); you override its `RPC` methods to implement server behavior. +Ordinary Kora components such as services and repositories can be injected into a handler through its constructor. + +Consider a `proto` contract with a single unary method: + +```protobuf title="src/main/proto/message.proto" +syntax = "proto3"; + +package ru.tinkoff.kora.generated.grpc; + +service UserService { + rpc createUser(RequestEvent) returns (ResponseEvent) {} //(1)! +} + +message RequestEvent { + string name = 1; + string code = 2; +} + +message ResponseEvent { + bytes id = 1; +} +``` + +1. A unary `RPC`: one request message produces one response message. + +The plugin generates `UserServiceGrpc.UserServiceImplBase`, and the handler overrides the generated method. +The generated method receives the request message and a [`StreamObserver`](https://grpc.github.io/grpc-java/javadoc/io/grpc/stub/StreamObserver.html) +that is used to send responses back to the client: ===! ":fontawesome-brands-java: `Java`" ```java @Component - public class ExampleService extends ExampleGrpc.ExampleImplBase {} + public final class UserService extends UserServiceGrpc.UserServiceImplBase { + + @Override + public void createUser(Message.RequestEvent request, StreamObserver responseObserver) { //(1)! + var response = Message.ResponseEvent.newBuilder() + .setId(ByteString.copyFromUtf8(UUID.randomUUID().toString())) + .build(); + + responseObserver.onNext(response); //(2)! + responseObserver.onCompleted(); //(3)! + } + } ``` + 1. The generated method receives the request message and a `StreamObserver` for sending the response + 2. Sends a single response message to the client + 3. Signals that the call is complete; for a unary method it is called exactly once, after a single `onNext` + === ":simple-kotlin: `Kotlin`" ```kotlin @Component - class ExampleService : ExampleGrpc.ExampleImplBase {} + class UserService : UserServiceGrpc.UserServiceImplBase() { + + override fun createUser(request: Message.RequestEvent, responseObserver: StreamObserver) { //(1)! + val response = Message.ResponseEvent.newBuilder() + .setId(ByteString.copyFromUtf8(UUID.randomUUID().toString())) + .build() + + responseObserver.onNext(response) //(2)! + responseObserver.onCompleted() //(3)! + } + } + ``` + + 1. The generated method receives the request message and a `StreamObserver` for sending the response + 2. Sends a single response message to the client + 3. Signals that the call is complete; for a unary method it is called exactly once, after a single `onNext` + +### Server streaming { #server-streaming } + +For a server-streaming `RPC` (`returns (stream ...)` in the `proto`), the client sends one request and the server sends back many messages. +Call `onNext` for each message, then `onCompleted` once at the end: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public void getAllUsers(Message.RequestEvent request, StreamObserver responseObserver) { + for (var user : userService.findAll()) { + responseObserver.onNext(toResponse(user)); //(1)! + } + responseObserver.onCompleted(); //(2)! + } + ``` + + 1. Sends one of several response messages + 2. Completes the response stream after the last message + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun getAllUsers(request: Message.RequestEvent, responseObserver: StreamObserver) { + userService.findAll().forEach { responseObserver.onNext(toResponse(it)) } //(1)! + responseObserver.onCompleted() //(2)! + } + ``` + + 1. Sends one of several response messages + 2. Completes the response stream after the last message + +### Client streaming { #client-streaming } + +For a client-streaming `RPC` (`rpc method(stream ...)`), the client sends many messages and the server answers once at the end. +The generated method **returns** a `StreamObserver` that receives the incoming request messages; the final response is produced from `onCompleted`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public StreamObserver createUsers(StreamObserver responseObserver) { + return new StreamObserver<>() { + private final List received = new ArrayList<>(); + + @Override + public void onNext(Message.RequestEvent value) { + received.add(value); //(1)! + } + + @Override + public void onError(Throwable t) { + responseObserver.onError(t); //(2)! + } + + @Override + public void onCompleted() { + responseObserver.onNext(aggregate(received)); //(3)! + responseObserver.onCompleted(); + } + }; + } + ``` + + 1. Collects each incoming request message + 2. Propagates a client-side stream error + 3. Produces the single aggregated response once the client has finished sending + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun createUsers(responseObserver: StreamObserver): StreamObserver { + return object : StreamObserver { + private val received = mutableListOf() + + override fun onNext(value: Message.RequestEvent) { + received += value //(1)! + } + + override fun onError(t: Throwable) { + responseObserver.onError(t) //(2)! + } + + override fun onCompleted() { + responseObserver.onNext(aggregate(received)) //(3)! + responseObserver.onCompleted() + } + } + } + ``` + + 1. Collects each incoming request message + 2. Propagates a client-side stream error + 3. Produces the single aggregated response once the client has finished sending + +### Bidirectional streaming { #bidirectional-streaming } + +For a bidirectional-streaming `RPC` (`rpc method(stream ...) returns (stream ...)`), both sides exchange many messages on the same call. +The method returns a `StreamObserver` for the incoming requests and can send responses at any time through `responseObserver`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public StreamObserver updateUsers(StreamObserver responseObserver) { + return new StreamObserver<>() { + @Override + public void onNext(Message.RequestEvent value) { + responseObserver.onNext(process(value)); //(1)! + } + + @Override + public void onError(Throwable t) { + responseObserver.onError(t); + } + + @Override + public void onCompleted() { + responseObserver.onCompleted(); //(2)! + } + }; + } ``` + 1. Responds to each incoming message as it arrives + 2. Completes the response stream when the client stops sending + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun updateUsers(responseObserver: StreamObserver): StreamObserver { + return object : StreamObserver { + override fun onNext(value: Message.RequestEvent) { + responseObserver.onNext(process(value)) //(1)! + } + + override fun onError(t: Throwable) { + responseObserver.onError(t) + } + + override fun onCompleted() { + responseObserver.onCompleted() //(2)! + } + } + } + ``` + + 1. Responds to each incoming message as it arrives + 2. Completes the response stream when the client stops sending + +### Error handling { #error-handling } + +**Description**: gRPC represents call errors with an [`io.grpc.Status`](https://grpc.github.io/grpc-java/javadoc/io/grpc/Status.html) +code and an optional description rather than with HTTP response codes. +To fail a call, complete the response observer with `responseObserver.onError(status.asRuntimeException())`, +or throw a `StatusRuntimeException` from the handler. +The auto-registered [`TelemetryInterceptor`](#default) observes the terminal `Status` when the call is closed +(on `close`, `onHalfClose`, `onCancel`, and `onComplete`) and records logging, metrics, and tracing accordingly. + +**Causes**: choose the `Status` code that matches the failure — for example `Status.NOT_FOUND` for a missing entity, +`Status.INVALID_ARGUMENT` for invalid input, `Status.UNAUTHENTICATED` or `Status.PERMISSION_DENIED` for authorization failures, +and `Status.INTERNAL` for unexpected server errors. + +**Recommendations**: + +- Attach a human-readable message with `withDescription(...)` and keep the original exception with `withCause(...)` so telemetry can record it. +- Complete a call exactly once: never call `onError` after `onCompleted`, and never call either twice. +- Do not leak internal exception details to clients; map them to an appropriate `Status` first. + +**Handling example**: a unary handler that returns `NOT_FOUND` when an entity is missing and maps unexpected failures to `INTERNAL`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public void getUser(Message.RequestEvent request, StreamObserver responseObserver) { + try { + var user = userService.getUser(request.getName()) + .orElseThrow(() -> Status.NOT_FOUND + .withDescription("User not found: " + request.getName()) + .asRuntimeException()); //(1)! + responseObserver.onNext(toResponse(user)); + responseObserver.onCompleted(); + } catch (StatusRuntimeException e) { + responseObserver.onError(e); //(2)! + } catch (Exception e) { + responseObserver.onError(Status.INTERNAL + .withDescription("Failed to get user") + .withCause(e) //(3)! + .asRuntimeException()); + } + } + ``` + + 1. Builds a `NOT_FOUND` error with a description + 2. Forwards an already-mapped `Status` error to the client + 3. Keeps the original exception as the cause so telemetry can record it + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun getUser(request: Message.RequestEvent, responseObserver: StreamObserver) { + try { + val user = userService.getUser(request.name) + ?: throw Status.NOT_FOUND + .withDescription("User not found: ${request.name}") + .asRuntimeException() //(1)! + responseObserver.onNext(toResponse(user)) + responseObserver.onCompleted() + } catch (e: StatusRuntimeException) { + responseObserver.onError(e) //(2)! + } catch (e: Exception) { + responseObserver.onError( + Status.INTERNAL + .withDescription("Failed to get user") + .withCause(e) //(3)! + .asRuntimeException() + ) + } + } + ``` + + 1. Builds a `NOT_FOUND` error with a description + 2. Forwards an already-mapped `Status` error to the client + 3. Keeps the original exception as the cause so telemetry can record it + +### Signatures { #signatures } + +The shape of a handler method is fixed by the `proto` contract and the generated `...ImplBase`: + +===! ":fontawesome-brands-java: `Java`" + + By `Req` and `Resp` we mean the generated request and response message types. + + - Unary: `void myMethod(Req request, StreamObserver responseObserver)` + - Server streaming: `void myMethod(Req request, StreamObserver responseObserver)` (multiple `onNext`, one `onCompleted`) + - Client streaming: `StreamObserver myMethod(StreamObserver responseObserver)` + - Bidirectional streaming: `StreamObserver myMethod(StreamObserver responseObserver)` + + The generated method returns `void` (or the request `StreamObserver`), so results are delivered asynchronously through the `StreamObserver` callbacks; + responses may be completed from another thread. + +=== ":simple-kotlin: `Kotlin`" + + By `Req` and `Resp` we mean the generated request and response message types. + + - Unary: `myMethod(request: Req, responseObserver: StreamObserver)` + - Server streaming: `myMethod(request: Req, responseObserver: StreamObserver)` (multiple `onNext`, one `onCompleted`) + - Client streaming: `myMethod(responseObserver: StreamObserver): StreamObserver` + - Bidirectional streaming: `myMethod(responseObserver: StreamObserver): StreamObserver` + + When you generate coroutine stubs with the [`grpc-kotlin`](https://github.com/grpc/grpc-kotlin) plugin (`io.grpc:protoc-gen-grpc-kotlin`) + and extend the generated `...CoroutineImplBase`, handler methods can be `suspend` functions (and streaming methods can use `Flow`). + Kora auto-registers [`CoroutineContextInjectInterceptor`](#default), which injects the Kora `Context` into the handler's `CoroutineContext`; + it activates only when `kotlinx-coroutines` is on the classpath. + ## Interceptors { #interceptors } -[Interceptors](https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerInterceptor.html) allow you to intercept requests before they are passed to handlers. +An [`io.grpc.ServerInterceptor`](https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerInterceptor.html) processes a call before it is passed to a `gRPC service`. +Interceptors are suitable for cross-cutting logic: logging, authorization, tracing, working with `Metadata`, and error mapping. + +Unlike the [HTTP server](http-server.md#interceptors), the gRPC server module has **no** `@GrpcService` or `@InterceptWith` annotation: +every `ServerInterceptor` registered as a `@Component` is applied **globally** to all services on the server. +To limit an interceptor to a single service or method, inspect the call at runtime — see [Scoping and authorization](#authorization). ### Default { #default } -The following interceptors are used at server startup by default: +When the server starts, Kora adds standard interceptors: -- `ContextServerInterceptor`. -- `CoroutineContextInjectInterceptor`. -- `MetricCollectorServerInterceptor` -- `LoggingServerInterceptor`. +- `TelemetryInterceptor` — enables server telemetry (logging, metrics, tracing) depending on connected modules and `grpcServer.telemetry` settings, and maps the terminal `Status`/exception when the call closes +- `ContextServerInterceptor` — propagates the Kora `Context` into call processing so it is available inside the handler +- `CoroutineContextInjectInterceptor` — adds `CoroutineContext` support for `Kotlin` coroutine handlers (active only when `kotlinx-coroutines` is on the classpath) -To override the default interceptor list, you can override the `serverBuilder` method from the `GrpcModule` class +User-defined `ServerInterceptor` beans from the application graph are added to `NettyServerBuilder` before the standard interceptors. +For full `NettyServerBuilder` configuration, use [GrpcServerBuilderConfigurer](#builder-configurer). + +### Execution order { #execution-order } + +gRPC invokes interceptors in the **reverse** order of registration, so the last interceptor added runs first (outermost). +Because Kora registers user interceptors first and the standard interceptors last, an incoming call is processed in this order: + +``` +CoroutineContextInjectInterceptor -> ContextServerInterceptor -> TelemetryInterceptor -> user interceptors -> handler +``` + +Consequences of this order: + +- The Kora `Context` and Kotlin `CoroutineContext` are established around your interceptors and the handler, so they are available inside the handler's listener callbacks. +- `TelemetryInterceptor` wraps your interceptors and the handler, so it observes the final `Status` (including errors thrown or reported through the response observer). +- When several user interceptors exist, they run in the reverse of their graph registration order; do not rely on a specific order between them for correctness. ### Custom { #custom } -Adding your custom interceptor requires creating an inheritor of `ServerInterceptor` with the `@Component` annotation: +To add a custom interceptor, create a `ServerInterceptor` implementation with the `@Component` annotation: ===! ":fontawesome-brands-java: `Java`" @@ -248,7 +669,7 @@ Adding your custom interceptor requires creating an inheritor of `ServerIntercep ServerCallHandler serverCallHandler) { // do something - return serverCallHandler.startCall(serverCall, metadata): + return serverCallHandler.startCall(serverCall, metadata); } } ``` @@ -271,37 +692,176 @@ Adding your custom interceptor requires creating an inheritor of `ServerIntercep } ``` +### Scoping and authorization { #authorization } + +Because an interceptor is global, scope it to a specific service or method by inspecting `call.getMethodDescriptor()`: +`getServiceName()` returns the service name (the generated constant `...Grpc.SERVICE_NAME`), and `getFullMethodName()` returns `service/method`. + +Request headers arrive as [`Metadata`](https://grpc.github.io/grpc-java/javadoc/io/grpc/Metadata.html). +Read a header with a `Metadata.Key`, and reject a call by closing it with a `Status` and returning an empty listener so the handler is never invoked. +The example below applies API-key authorization to a single service only: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ApiKeyServerInterceptor implements ServerInterceptor { + + private static final Metadata.Key AUTHORIZATION = + Metadata.Key.of("authorization", Metadata.ASCII_STRING_MARSHALLER); //(1)! + + @Override + public ServerCall.Listener interceptCall(ServerCall call, + Metadata headers, + ServerCallHandler next) { + if (!UserServiceGrpc.SERVICE_NAME.equals(call.getMethodDescriptor().getServiceName())) { //(2)! + return next.startCall(call, headers); + } + + var apiKey = headers.get(AUTHORIZATION); //(3)! + if (apiKey == null || !apiKey.equals("secret")) { + call.close(Status.UNAUTHENTICATED.withDescription("Invalid API key"), new Metadata()); //(4)! + return new ServerCall.Listener<>() {}; //(5)! + } + + return next.startCall(call, headers); + } + } + ``` + + 1. `Metadata.Key` for reading the `authorization` header as an ASCII string + 2. Applies the interceptor only to `UserService`; other services pass through untouched + 3. Reads the header value from the request `Metadata` + 4. Rejects the call with an `UNAUTHENTICATED` status + 5. Returns an empty listener so the handler is never called + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ApiKeyServerInterceptor : ServerInterceptor { + + override fun interceptCall( + call: ServerCall, + headers: Metadata, + next: ServerCallHandler + ): ServerCall.Listener { + if (UserServiceGrpc.SERVICE_NAME != call.methodDescriptor.serviceName) { //(2)! + return next.startCall(call, headers) + } + + val apiKey = headers.get(AUTHORIZATION) //(3)! + if (apiKey != "secret") { + call.close(Status.UNAUTHENTICATED.withDescription("Invalid API key"), Metadata()) //(4)! + return object : ServerCall.Listener() {} //(5)! + } + + return next.startCall(call, headers) + } + + companion object { + private val AUTHORIZATION: Metadata.Key = + Metadata.Key.of("authorization", Metadata.ASCII_STRING_MARSHALLER) //(1)! + } + } + ``` + + 1. `Metadata.Key` for reading the `authorization` header as an ASCII string + 2. Applies the interceptor only to `UserService`; other services pass through untouched + 3. Reads the header value from the request `Metadata` + 4. Rejects the call with an `UNAUTHENTICATED` status + 5. Returns an empty listener so the handler is never called + +## Lifecycle and readiness { #lifecycle } + +The server is managed by the `GrpcNettyServer` component, which is created as a [`@Root`](container.md#root-component) component +and follows the [application lifecycle](container.md#component-lifecycle): + +- On startup it builds and starts the `Netty` server on the configured `port`. If the port is already in use, startup fails with a clear error. +- On shutdown it performs a [graceful shutdown](container.md#graceful-shutdown): it stops accepting new calls and waits up to `shutdownWait` for in-flight calls to finish, then forcibly terminates any remaining calls. + +`GrpcNettyServer` also implements a [readiness probe](probes.md): the server reports **not ready** while it is starting up or shutting down, +and **ready** only while it is running. In a `Kubernetes` deployment this lets the readiness probe reflect the real server state and drain traffic during graceful shutdown. + +## Telemetry { #telemetry } + +Server observability is driven by the `TelemetryInterceptor` through the `GrpcServerTelemetry` facade and is configured under [`grpcServer.telemetry`](#configuration). +Metrics are described in the [Metrics Reference](metrics.md#grpc-server) section. + +Each part of telemetry is a replaceable component: the defaults are registered as default components, so providing your own `@Component` overrides them: + +- `GrpcServerTelemetry` — the aggregate telemetry facade (`createContext` returning a context with `sendMessage`/`receiveMessage`/`close`) +- `GrpcServerLogger` — call logging (`logBegin`/`logEnd`/`logSendMessage`/`logReceiveMessage`); the default is `Slf4jGrpcServerLogger` +- `GrpcServerTracer` — tracing spans around calls +- `GrpcServerMetricsFactory` — per-call metrics collection + +For example, to fully customize logging, register a `@Component` implementing `GrpcServerLogger`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public interface GrpcServerLogger { + + boolean isEnabled(); + + void logBegin(ServerCall call, Metadata headers, String serviceName, String methodName); + + void logEnd(String serviceName, String methodName, @Nullable Status status, @Nullable Throwable exception, long processingTime); + + void logSendMessage(String serviceName, String methodName, Object message); + + void logReceiveMessage(String serviceName, String methodName, Object message); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + interface GrpcServerLogger { + + fun isEnabled(): Boolean + + fun logBegin(call: ServerCall<*, *>, headers: Metadata, serviceName: String, methodName: String) + + fun logEnd(serviceName: String, methodName: String, status: Status?, exception: Throwable?, processingTime: Long) + + fun logSendMessage(serviceName: String, methodName: String, message: Any) + + fun logReceiveMessage(serviceName: String, methodName: String, message: Any) + } + ``` + ## Reflection { #reflection } -Supported by [gRPC Server Reflection](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md) -which provides information about publicly available gRPC services on the server -and helps clients at runtime build RPC requests and responses without pre-compiled service information. -It is used by the gRPC command line tool (gRPC CLI), which can be used to examine server proto-files and send/receive test RPCs. -Reflection is only supported for proto-based services. +[`gRPC Server Reflection`](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md) is supported and provides information about available `gRPC services` on the server. +Reflection helps clients and tools build `RPC` requests at runtime without precompiled service information. +For example, it is used by `gRPC CLI`, which can inspect server `proto` descriptions and send test `RPC` calls. +`gRPC Server Reflection` is supported only for `proto`-based services. -You can learn more about working with gRPC Server Reflection [here](https://github.com/grpc/grpc-java/blob/master/documentation/server-reflection-tutorial.md#enable-server-reflection). +You can learn more about `gRPC Server Reflection` in the [grpc-java guide](https://github.com/grpc/grpc-java/blob/master/documentation/server-reflection-tutorial.md#enable-server-reflection). ### Dependency { #dependency-2 } -An optional gRPC Server Reflection dependency is required. +You must additionally add the [`gRPC Server Reflection`](https://mvnrepository.com/artifact/io.grpc/grpc-services) dependency. ===! ":fontawesome-brands-java: `Java`" - Зависимость `build.gradle`: + [Dependency](general.md#dependencies) in `build.gradle`: ```groovy - implementation "io.grpc:grpc-services:1.62.2" + implementation "io.grpc:grpc-services:1.74.0" ``` === ":simple-kotlin: `Kotlin`" - Зависимость `build.gradle.kts`: + [Dependency](general.md#dependencies) in `build.gradle.kts`: ```groovy - implementation("io.grpc:grpc-services:1.62.2") + implementation("io.grpc:grpc-services:1.74.0") ``` ### Configuration { #configuration-2 } -You must also enable the gRPC Server Reflection service in the configuration: +You must also enable the `gRPC Server Reflection` service in the configuration. +Kora adds it to the server only if the application has the `io.grpc.protobuf.services.ProtoReflectionService` class, so configuration alone is not enough without the dependency. ===! ":material-code-json: `Hocon`" @@ -311,7 +871,7 @@ You must also enable the gRPC Server Reflection service in the configuration: } ``` - 1. Enables gRPC Server Reflection service + 1. Enables the `gRPC Server Reflection` service (default: `false`). === ":simple-yaml: `YAML`" @@ -320,4 +880,36 @@ You must also enable the gRPC Server Reflection service in the configuration: reflectionEnabled: false #(1)! ``` - 1. Enables gRPC Server Reflection service + 1. Enables the `gRPC Server Reflection` service (default: `false`). + +### Usage { #reflection-usage } + +With reflection enabled, tools such as [`grpcurl`](https://github.com/fullstorydev/grpcurl) can discover services and send `RPC` calls without a precompiled client. +For a server listening on port `8090`: + +```bash +grpcurl -plaintext localhost:8090 list #(1)! +grpcurl -plaintext localhost:8090 describe ru.tinkoff.kora.generated.grpc.UserService #(2)! +grpcurl -plaintext -d '{"name": "Bob", "code": "123"}' \ + localhost:8090 ru.tinkoff.kora.generated.grpc.UserService/createUser #(3)! +``` + +1. Lists the services exposed by the server +2. Describes a service and its methods +3. Sends a unary `RPC`; `-plaintext` is used because the example server has no `TLS` + +## Telemetry { #telemetry } + +gRPC Server uses a telemetry contract for logging, metrics, and tracing of calls. +Telemetry configuration (section `telemetry { logging / metrics / tracing }`) is described in the [Configuration](#configuration) section. +Extension points are located in `ru.tinkoff.kora.grpc.server.common.telemetry`. + +For each gRPC call, a `GrpcServerTelemetry.GrpcServerTelemetryContext` is created, which is closed upon call completion. +The call is described through telemetry handler parameters, including service, method, response status, and duration. + +The default factory `DefaultGrpcServerTelemetryFactory` combines three factories: +- `GrpcServerLoggerFactory` builds `GrpcServerLogger` for logging call start/end; +- `GrpcServerMetricsFactory` builds `GrpcServerMetrics` for writing call metrics; +- `GrpcServerTracerFactory` builds `GrpcServerTracer` for distributed tracing. + +Metrics and tracing are described in the [Metrics Reference](metrics.md#grpc-server) section. diff --git a/mkdocs/docs/en/documentation/http-client.md b/mkdocs/docs/en/documentation/http-client.md index 460f11e..06e915b 100644 --- a/mkdocs/docs/en/documentation/http-client.md +++ b/mkdocs/docs/en/documentation/http-client.md @@ -4,15 +4,20 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora HTTP clients, OkHttp, AsyncHttpClient, Java native client, declarative client annotations, request and response mapping, interceptors, and authorization; key triggers include @HttpClient, @HttpRoute, @Path, @Query, @Header, @Cookie, @Json, @InterceptWith, HttpClientModule, OkHttp." --- -Module provides a thin layer of abstraction over HTTP client libraries to create HTTP clients -using declarative-style annotations or using client in imperative-style. +The `HTTP client` module describes outgoing HTTP calls: transport implementation, request mapping, response mapping, +telemetry, and interceptors. In Kora, clients can be described declaratively with `@HttpClient` and `@HttpRoute`, +or used imperatively through the common `HttpClient` interface when a request must be built in code. + +The declarative approach is suitable for most integrations with external services: the method contract becomes the remote call contract, +and Kora creates the implementation at compile time without using `Reflection` at runtime. The imperative approach is useful for low-level +or dynamic scenarios where path, headers, query parameters, or body are easier to assemble manually. ???+ tip "Recommendation" - **We recommend** using an approach where OpenAPI file is primary contract - and clients are created from it using a OpenAPI generator. + **We recommend** using an approach where the `OpenAPI` file is the primary contract + and clients are created from it using the generator. This approach allows you to achieve consistency between the consumer and owner of the contract - and update API faster in case of new version by just updaing contract file. + and update the API faster when the contract changes by replacing the contract file. For more information about the generator, see the [section on generating from OpenAPI](openapi-codegen.md). For a step-by-step walkthrough before the reference details, see [HTTP Client](../guides/http-client.md) and [Advanced HTTP Client](../guides/http-client-advanced.md). @@ -52,131 +57,162 @@ Please note that the implementation is written in Kotlin and uses appropriate de ### Configuration { #configuration } -Example of the complete configuration described in the `OkHttpClientConfig` -and `HttpClientConfig` classes (default or example values are specified): +Basic OkHttp client configuration parameters: ===! ":material-code-json: `Hocon`" ```javascript httpClient { - ok { - followRedirects = true //(1)! - httpVersion = "HTTP_1_1" //(2)! - } - connectTimeout = "5s" //(3)! - readTimeout = "2m" //(4)! - useEnvProxy = false //(5)! - proxy { - host = "localhost" //(6)! - port = 8090 //(7)! - user = "user" //(8)! - password = "password" //(9)! - nonProxyHosts = [ "host1", "host2" ] //(10)! - } - telemetry { - logging { - enabled = false //(11)! - mask = "***" //(12)! - maskQueries = [ ] //(13)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(14)! - pathTemplate = true //(15)! - } - metrics { - enabled = true //(16)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(17)! - tags = { // (18)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(19)! - attributes = { // (20)! - "key1" = "value1" - "key2" = "value2" - } - } - } + connectTimeout = "5s" //(1)! + readTimeout = "2m" //(2)! } ``` - 1. Whether to follow [redirects in HTTP](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) - 2. Maximum HTTP protocol version used (available values: `HTTP_1_1` / `HTTP_2` / `HTTP_3`) - 3. Maximum time to establish a connection - 4. Maximum time to read a response - 5. Whether to use environment variables to configure the proxy - 6. Proxy address (optional) - 7. Proxy port (optional) - 8. User for the proxy (optional) - 9. Password for the proxy (optional) - 10. Hosts that should be excluded from proxying (optional) - 11. Enables module logging (default `false`) - 12. Mask that is used to hide specified headers and request/response parameters - 13. List of request parameters to be hidden - 14. List of request/response headers that should be hidden - 15. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 16. Enables module metrics (default `true`) - 17. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 18. Configures tags for metrics (optional) - 19. Enables module tracing (default `true`) - 20. Configures attributes for tracing (optional) + 1. Maximum time to establish a connection (default: `5s`) + 2. Maximum time to read a response (default: `2m`) === ":simple-yaml: `YAML`" ```yaml httpClient: - ok: - followRedirects: true #(1)! - httpVersion: "HTTP_1_1" #(2)! - connectTimeout: "5s" #(3)! - readTimeout: "2m" #(4)! - useEnvProxy: false #(5)! - proxy: - host: "localhost" #(6)! - port: 8090 #(7)! - user: "user" #(8)! - password: "password" #(9)! - nonProxyHosts: [ "host1", "host2" ] #(10)! - telemetry: - logging: - enabled: false #(11)! - mask: "***" #(12)! - maskQueries: [ ] #(13)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(14)! - pathTemplate: true #(15)! - metrics: - enabled: true #(16)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(17)! - tags: #(18)! - key1: value1 - key2: value2 - tracing: - enabled: true #(19)! - attributes: #(20)! - key1: value1 - key2: value2 - ``` - - 1. Whether to follow [redirects in HTTP](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) - 2. Maximum HTTP protocol version used (available values: `HTTP_1_1` / `HTTP_2` / `HTTP_3`) - 3. Maximum time to establish a connection - 4. Maximum time to read a response - 5. Whether to use environment variables to configure the proxy - 6. Proxy address (optional) - 7. Proxy port (optional) - 8. User for the proxy (optional) - 9. Password for the proxy (optional) - 10. Hosts that should be excluded from proxying (optional) - 11. Enables module logging (default `false`) - 12. Mask that is used to hide specified headers and request/response parameters - 13. List of request parameters to be hidden - 14. List of request/response headers that should be hidden - 15. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 16. Enables module metrics (default `true`) - 17. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 18. Configures tags for metrics (optional) - 19. Enables module tracing (default `true`) - 20. Configures attributes for tracing (optional) + connectTimeout: "5s" #(1)! + readTimeout: "2m" #(2)! + ``` + + 1. Maximum time to establish a connection (default: `5s`) + 2. Maximum time to read a response (default: `2m`) + +??? note "Full Configuration" + + Example of the complete configuration described in the `OkHttpClientConfig` + and `HttpClientConfig` classes (default or example values are specified): + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + ok { + followRedirects = true //(1)! + httpVersion = "HTTP_1_1" //(2)! + retryOnConnectionFailure = true //(3)! + } + connectTimeout = "5s" //(4)! + readTimeout = "2m" //(5)! + useEnvProxy = false //(6)! + proxy { + host = "localhost" //(7)! + port = 8090 //(8)! + user = "user" //(9)! + password = "password" //(10)! + nonProxyHosts = [ "host1", "host2" ] //(11)! + } + telemetry { + logging { + enabled = false //(12)! + mask = "***" //(13)! + maskQueries = [ ] //(14)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(15)! + pathTemplate = true //(16)! + } + metrics { + enabled = true //(17)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(18)! + tags = { // (19)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(20)! + attributes = { // (21)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Whether to follow [HTTP redirects](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) (default: `true`) + 2. Maximum `HTTP` protocol version to use, available values: `HTTP_1_1` / `HTTP_2` / `HTTP_3` (default: `HTTP_1_1`) + 3. Whether to retry a request after a connection failure; this can affect the maximum connection establishment time (default: `true`) + 4. Maximum time to establish a connection (default: `5s`) + 5. Maximum time to read a response (default: `2m`) + 6. Whether to use `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` and `no_proxy` / `NO_PROXY` environment variables for proxy configuration (default: `false`) + 7. Proxy host (`required`, default not specified) + 8. Proxy port (`required`, default not specified) + 9. Proxy user (default not specified, optional) + 10. Proxy password (default not specified, optional) + 11. Hosts to exclude from proxying (default not specified, optional) + 12. Enables module logging (default: `false`) + 13. Mask used to hide specified headers and request or response parameters (default: `***`) + 14. List of request parameters to hide (default: `[]`) + 15. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 16. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 17. Enables module metrics (default: `true`) + 18. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Configures metric tags (default: `{}`) + 20. Enables module tracing (default: `true`) + 21. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + ok: + followRedirects: true #(1)! + httpVersion: "HTTP_1_1" #(2)! + retryOnConnectionFailure: true #(3)! + connectTimeout: "5s" #(4)! + readTimeout: "2m" #(5)! + useEnvProxy: false #(6)! + proxy: + host: "localhost" #(7)! + port: 8090 #(8)! + user: "user" #(9)! + password: "password" #(10)! + nonProxyHosts: [ "host1", "host2" ] #(11)! + telemetry: + logging: + enabled: false #(12)! + mask: "***" #(13)! + maskQueries: [ ] #(14)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(15)! + pathTemplate: true #(16)! + metrics: + enabled: true #(17)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(18)! + tags: #(19)! + key1: value1 + key2: value2 + tracing: + enabled: true #(20)! + attributes: #(21)! + key1: value1 + key2: value2 + ``` + + 1. Whether to follow [HTTP redirects](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) (default: `true`) + 2. Maximum `HTTP` protocol version to use, available values: `HTTP_1_1` / `HTTP_2` / `HTTP_3` (default: `HTTP_1_1`) + 3. Whether to retry a request after a connection failure; this can affect the maximum connection establishment time (default: `true`) + 4. Maximum time to establish a connection (default: `5s`) + 5. Maximum time to read a response (default: `2m`) + 6. Whether to use `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` and `no_proxy` / `NO_PROXY` environment variables for proxy configuration (default: `false`) + 7. Proxy host (`required`, default not specified) + 8. Proxy port (`required`, default not specified) + 9. Proxy user (default not specified, optional) + 10. Proxy password (default not specified, optional) + 11. Hosts to exclude from proxying (default not specified, optional) + 12. Enables module logging (default: `false`) + 13. Mask used to hide specified headers and request or response parameters (default: `***`) + 14. List of request parameters to hide (default: `[]`) + 15. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 16. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 17. Enables module metrics (default: `true`) + 18. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Configures metric tags (default: `{}`) + 20. Enables module tracing (default: `true`) + 21. Configures tracing attributes (default: `{}`) Module metrics are described in the [Metrics Reference](metrics.md#http-client) section. @@ -244,127 +280,154 @@ The `HttpClient` interface implementation is `AsyncHttpClient` and is available ### Configuration { #configuration-2 } -Example of the complete configuration described in the `AsyncHttpClientConfig` -and `HttpClientConfig` classes (default or example values are specified): +Basic AsyncHttpClient configuration parameters: ===! ":material-code-json: `Hocon`" ```javascript httpClient { - async { - followRedirects = true //(1)! - } - connectTimeout = "5s" //(2)! - readTimeout = "2m" //(3)! - useEnvProxy = false //(4)! - proxy { - host = "localhost" //(5)! - port = 8090 //(6)! - user = "user" //(7)! - password = "password" //(8)! - nonProxyHosts = [ "host1", "host2" ] //(9)! - } - telemetry { - logging { - enabled = false //(10)! - mask = "***" //(11)! - maskQueries = [ ] //(12)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(13)! - pathTemplate = true //(14)! - } - metrics { - enabled = true //(15)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(16)! - tags = { // (17)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(18)! - attributes = { // (19)! - "key1" = "value1" - "key2" = "value2" - } - } - } + connectTimeout = "5s" //(1)! + readTimeout = "2m" //(2)! } ``` - 1. Whether to follow [redirects in HTTP](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) - 2. Maximum time to establish a connection - 3. Maximum time to read a response - 4. Whether to use environment variables to configure the proxy - 5. Proxy address (optional) - 6. Proxy port (optional) - 7. User for the proxy (optional) - 8. Password for the proxy (optional) - 9. Hosts that should be excluded from proxying (optional) - 10. Enables module logging (default `false`) - 11. Mask that is used to hide specified headers and request/response parameters - 12. List of request parameters to be hidden - 13. List of request/response headers that should be hidden - 14. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 15. Enables module metrics (default `true`) - 16. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 17. Configures tags for metrics (optional) - 18. Enables module tracing (default `true`) - 19. Configures attributes for tracing (optional) + 1. Maximum time to establish a connection (default: `5s`) + 2. Maximum time to read a response (default: `2m`) === ":simple-yaml: `YAML`" ```yaml httpClient: - async: - followRedirects: true #(1)! - connectTimeout: "5s" #(2)! - readTimeout: "2m" #(3)! - useEnvProxy: false #(4)! - proxy: - host: "localhost" #(5)! - port: 8090 #(6)! - user: "user" #(7)! - password: "password" #(8)! - nonProxyHosts: [ "host1", "host2" ] #(9)! - telemetry: - logging: - enabled: false #(10)! - mask: "***" #(11)! - maskQueries: [ ] #(12)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(13)! - pathTemplate: true #(14)! - metrics: - enabled: true #(15)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(16)! - tags: #(17)! - key1: value1 - key2: value2 - tracing: - enabled: true #(18)! - attributes: #(19)! - key1: value1 - key2: value2 - ``` - - 1. Whether to follow [redirects in HTTP](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) - 2. Maximum time to establish a connection - 3. Maximum time to read a response - 4. Whether to use environment variables to configure the proxy - 5. Proxy address (optional) - 6. Proxy port (optional) - 7. User for the proxy (optional) - 8. Password for the proxy (optional) - 9. Hosts that should be excluded from proxying (optional) - 10. Enables module logging (default `false`) - 11. Mask that is used to hide specified headers and request/response parameters - 12. List of request parameters to be hidden - 13. List of request/response headers that should be hidden - 14. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 15. Enables module metrics (default `true`) - 16. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 17. Configures tags for metrics (optional) - 18. Enables module tracing (default `true`) - 19. Configures attributes for tracing (optional) + connectTimeout: "5s" #(1)! + readTimeout: "2m" #(2)! + ``` + + 1. Maximum time to establish a connection (default: `5s`) + 2. Maximum time to read a response (default: `2m`) + +??? note "Full Configuration" + + Example of the complete configuration described in the `AsyncHttpClientConfig` + and `HttpClientConfig` classes (default or example values are specified): + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + async { + followRedirects = true //(1)! + } + connectTimeout = "5s" //(2)! + readTimeout = "2m" //(3)! + useEnvProxy = false //(4)! + proxy { + host = "localhost" //(5)! + port = 8090 //(6)! + user = "user" //(7)! + password = "password" //(8)! + nonProxyHosts = [ "host1", "host2" ] //(9)! + } + telemetry { + logging { + enabled = false //(10)! + mask = "***" //(11)! + maskQueries = [ ] //(12)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(13)! + pathTemplate = true //(14)! + } + metrics { + enabled = true //(15)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(16)! + tags = { // (17)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(18)! + attributes = { // (19)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Whether to follow [HTTP redirects](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) (default: `true`) + 2. Maximum time to establish a connection (default: `5s`) + 3. Maximum time to read a response (default: `2m`) + 4. Whether to use `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` and `no_proxy` / `NO_PROXY` environment variables for proxy configuration (default: `false`) + 5. Proxy host (`required`, default not specified) + 6. Proxy port (`required`, default not specified) + 7. Proxy user (default not specified, optional) + 8. Proxy password (default not specified, optional) + 9. Hosts to exclude from proxying (default not specified, optional) + 10. Enables module logging (default: `false`) + 11. Mask used to hide specified headers and request or response parameters (default: `***`) + 12. List of request parameters to hide (default: `[]`) + 13. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 14. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 15. Enables module metrics (default: `true`) + 16. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 17. Configures metric tags (default: `{}`) + 18. Enables module tracing (default: `true`) + 19. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + async: + followRedirects: true #(1)! + connectTimeout: "5s" #(2)! + readTimeout: "2m" #(3)! + useEnvProxy: false #(4)! + proxy: + host: "localhost" #(5)! + port: 8090 #(6)! + user: "user" #(7)! + password: "password" #(8)! + nonProxyHosts: [ "host1", "host2" ] #(9)! + telemetry: + logging: + enabled: false #(10)! + mask: "***" #(11)! + maskQueries: [ ] #(12)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(13)! + pathTemplate: true #(14)! + metrics: + enabled: true #(15)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(16)! + tags: #(17)! + key1: value1 + key2: value2 + tracing: + enabled: true #(18)! + attributes: #(19)! + key1: value1 + key2: value2 + ``` + + 1. Whether to follow [HTTP redirects](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) (default: `true`) + 2. Maximum time to establish a connection (default: `5s`) + 3. Maximum time to read a response (default: `2m`) + 4. Whether to use `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` and `no_proxy` / `NO_PROXY` environment variables for proxy configuration (default: `false`) + 5. Proxy host (`required`, default not specified) + 6. Proxy port (`required`, default not specified) + 7. Proxy user (default not specified, optional) + 8. Proxy password (default not specified, optional) + 9. Hosts to exclude from proxying (default not specified, optional) + 10. Enables module logging (default: `false`) + 11. Mask used to hide specified headers and request or response parameters (default: `***`) + 12. List of request parameters to hide (default: `[]`) + 13. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 14. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 15. Enables module metrics (default: `true`) + 16. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 17. Configures metric tags (default: `{}`) + 18. Enables module tracing (default: `true`) + 19. Configures tracing attributes (default: `{}`) You can also configure [Netty transport](netty.md). @@ -404,111 +467,160 @@ The `HttpClient` interface implementation is `JdkHttpClient` and is available fo ### Configuration { #configuration-3 } -Example of the complete configuration described in the `JdkHttpClientConfig` -and `HttpClientConfig` classes (default or example values are specified): +Basic JDK HttpClient configuration parameters: ===! ":material-code-json: `Hocon`" ```javascript httpClient { - jdk { - threads = 2 //(1)! - httpVersion = "HTTP_1_1" //(2)! - } - connectTimeout = "5s" //(3)! - useEnvProxy = false //(4)! - proxy { - host = "localhost" //(5)! - port = 8090 //(6)! - user = "user" //(7)! - password = "password" //(8)! - nonProxyHosts = [ "host1", "host2" ] //(9)! - } - telemetry { - logging { - enabled = false //(10)! - mask = "***" //(11)! - maskQueries = [ ] //(12)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(13)! - pathTemplate = true //(14)! - } - metrics { - enabled = true //(15)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(16)! - } - tracing { - enabled = true //(17)! - } - } + connectTimeout = "5s" //(1)! + readTimeout = "2m" //(2)! } ``` - 1. Number of threads for HTTP client - 2. Which version of HTTP protocol to use (available values: `HTTP_1_1` / `HTTP_2`) - 3. Maximum time to establish a connection - 4. Whether to use environment variables to configure the proxy - 5. Proxy address (optional) - 6. Proxy port (optional) - 7. User for the proxy (optional) - 8. Password for the proxy (optional) - 9. Hosts that should be excluded from proxying (optional) - 10. Enables module logging (default `false`) - 11. Mask that is used to hide specified headers and request/response parameters - 12. List of request parameters to be hidden - 13. List of request/response headers that should be hidden - 14. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 15. Enables module metrics (default `true`) - 16. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 17. Enables module tracing (default `true`) + 1. Maximum time to establish a connection (default: `5s`) + 2. Maximum time to read a response (default: `2m`) === ":simple-yaml: `YAML`" ```yaml httpClient: - jdk: - threads: 2 #(1)! - httpVersion: "HTTP_1_1" #(2)! - connectTimeout: "2s" #(3)! - useEnvProxy: false #(4)! - proxy: - host: "localhost" #(5)! - port: 8090 #(6)! - user: "user" #(7)! - password: "password" #(8)! - nonProxyHosts: [ "host1", "host2" ] #(9)! - telemetry: - logging: - enabled: false #(10)! - mask: "***" #(11)! - maskQueries: [ ] #(12)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(13)! - pathTemplate: true #(14)! - metrics: - enabled: true #(15)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(16)! - telemetry: - enabled: true #(17)! - ``` - - 1. Number of threads for HTTP client - 2. Which version of HTTP protocol to use (available values: `HTTP_1_1` / `HTTP_2`) - 3. Maximum time to establish a connection - 4. Whether to use environment variables to configure the proxy - 5. Proxy address (optional) - 6. Proxy port (optional) - 7. User for the proxy (optional) - 8. Password for the proxy (optional) - 9. Hosts that should be excluded from proxying (optional) - 10. Enables module logging (default `false`) - 11. Mask that is used to hide specified headers and request/response parameters - 12. List of request parameters to be hidden - 13. List of request/response headers that should be hidden - 14. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 15. Enables module metrics (default `true`) - 16. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 17. Enables module tracing (default `true`) - -## Client declarative { #client-declarative } + connectTimeout: "5s" #(1)! + readTimeout: "2m" #(2)! + ``` + + 1. Maximum time to establish a connection (default: `5s`) + 2. Maximum time to read a response (default: `2m`) + +??? note "Full Configuration" + + Example of the complete configuration described in the `JdkHttpClientConfig` + and `HttpClientConfig` classes (default or example values are specified): + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + jdk { + threads = 2 //(1)! + httpVersion = "HTTP_1_1" //(2)! + } + connectTimeout = "5s" //(3)! + readTimeout = "2m" //(4)! + useEnvProxy = false //(5)! + proxy { + host = "localhost" //(6)! + port = 8090 //(7)! + user = "user" //(8)! + password = "password" //(9)! + nonProxyHosts = [ "host1", "host2" ] //(10)! + } + telemetry { + logging { + enabled = false //(11)! + mask = "***" //(12)! + maskQueries = [ ] //(13)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(14)! + pathTemplate = true //(15)! + } + metrics { + enabled = true //(16)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(17)! + tags = { // (18)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(19)! + attributes = { // (20)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Number of threads for the `HTTP` client (default: number of available processors multiplied by `2`) + 2. Which `HTTP` protocol version to use, available values: `HTTP_1_1` / `HTTP_2` (default: `HTTP_1_1`) + 3. Maximum time to establish a connection (default: `5s`) + 4. Maximum time to read a response (default: `2m`) + 5. Whether to use `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` and `no_proxy` / `NO_PROXY` environment variables for proxy configuration (default: `false`) + 6. Proxy host (`required`, default not specified) + 7. Proxy port (`required`, default not specified) + 8. Proxy user (default not specified, optional) + 9. Proxy password (default not specified, optional) + 10. Hosts to exclude from proxying (default not specified, optional) + 11. Enables module logging (default: `false`) + 12. Mask used to hide specified headers and request or response parameters (default: `***`) + 13. List of request parameters to hide (default: `[]`) + 14. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 15. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 16. Enables module metrics (default: `true`) + 17. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Configures metric tags (default: `{}`) + 19. Enables module tracing (default: `true`) + 20. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + jdk: + threads: 2 #(1)! + httpVersion: "HTTP_1_1" #(2)! + connectTimeout: "5s" #(3)! + readTimeout: "2m" #(4)! + useEnvProxy: false #(5)! + proxy: + host: "localhost" #(6)! + port: 8090 #(7)! + user: "user" #(8)! + password: "password" #(9)! + nonProxyHosts: [ "host1", "host2" ] #(10)! + telemetry: + logging: + enabled: false #(11)! + mask: "***" #(12)! + maskQueries: [ ] #(13)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(14)! + pathTemplate: true #(15)! + metrics: + enabled: true #(16)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(17)! + tags: #(18)! + key1: value1 + key2: value2 + tracing: + enabled: true #(19)! + attributes: #(20)! + key1: value1 + key2: value2 + ``` + + 1. Number of threads for the `HTTP` client (default: number of available processors multiplied by `2`) + 2. Which `HTTP` protocol version to use, available values: `HTTP_1_1` / `HTTP_2` (default: `HTTP_1_1`) + 3. Maximum time to establish a connection (default: `5s`) + 4. Maximum time to read a response (default: `2m`) + 5. Whether to use `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` and `no_proxy` / `NO_PROXY` environment variables for proxy configuration (default: `false`) + 6. Proxy host (`required`, default not specified) + 7. Proxy port (`required`, default not specified) + 8. Proxy user (default not specified, optional) + 9. Proxy password (default not specified, optional) + 10. Hosts to exclude from proxying (default not specified, optional) + 11. Enables module logging (default: `false`) + 12. Mask used to hide specified headers and request or response parameters (default: `***`) + 13. List of request parameters to hide (default: `[]`) + 14. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 15. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 16. Enables module metrics (default: `true`) + 17. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Configures metric tags (default: `{}`) + 19. Enables module tracing (default: `true`) + 20. Configures tracing attributes (default: `{}`) + +## Declarative Client { #client-declarative } It is suggested to use special annotations to create a declarative client: @@ -537,10 +649,10 @@ It is suggested to use special annotations to create a declarative client: } ``` -#### Client Configuration { #client-configuration } +### Client Configuration { #client-configuration } -The default configuration of a particular implementation of `@HttpClient` uses the following path `httpClient.{lower case class name}` for configuration lookup, -or specified in the `configPath` parameter in the annotation: +By default, configuration for a particular `@HttpClient` implementation is looked up at `httpClient.{lower case class name}`. +If the path must be specified explicitly, use the `configPath` annotation parameter: ===! ":fontawesome-brands-java: `Java`" @@ -568,7 +680,42 @@ or specified in the `configPath` parameter in the annotation: 1. The path to the configuration of this particular client -Example configuration in the case of the `httpClient.someClient` path described in the `DeclarativeHttpClientConfig` class: +`@HttpClient` can also specify tags for injected components: + +* `httpClientTag` — tag used to select a particular transport `HttpClient` when the graph contains several implementations with different `@Tag` values +* `telemetryTag` — tag used to select a particular client telemetry factory + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient( + configPath = "httpClient.someClient", + httpClientTag = CustomTransport.class, + telemetryTag = CustomTelemetry.class + ) + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + void hello(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient( + configPath = "httpClient.someClient", + httpClientTag = [CustomTransport::class], + telemetryTag = [CustomTelemetry::class] + ) + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + fun hello() + } + ``` + +Basic declarative client configuration parameters: ===! ":material-code-json: `Hocon`" @@ -577,36 +724,12 @@ Example configuration in the case of the `httpClient.someClient` path described someClient { url = "https://localhost:8090" //(1)! requestTimeout = "10s" //(2)! - telemetry { - logging { - enabled = false //(3)! - mask = "***" //(4)! - maskQueries = [ ] //(5)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(6)! - pathTemplate = true //(7)! - } - metrics { - enabled = true //(8)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(9)! - } - tracing { - enabled = true //(10)! - } - } } } ``` - 1. URL of the service where requests will be sent - 2. Maximum request timeout, may spans the entire call: resolving DNS, connecting, writing the request body, server processing, and reading the response body, if call requires redirects or retries all must complete within one timeout period - 3. Enables module logging (default `false`) - 4. Mask that is used to hide specified headers and request/response parameters - 5. List of request parameters to be hidden - 6. List of request/response headers that should be hidden - 7. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 8. Enables module metrics (default `true`) - 9. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 10. Enables module tracing (default `true`) + 1. Base service `URL` where requests will be sent (`required`, no default) + 2. Maximum request time (default: not specified, optional) === ":simple-yaml: `YAML`" @@ -615,110 +738,303 @@ Example configuration in the case of the `httpClient.someClient` path described someClient: url: "https://localhost:8090" #(1)! requestTimeout: "10s" #(2)! - telemetry: - logging: - enabled: false #(3)! - mask: "***" #(4)! - maskQueries: [ ] #(5)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(6)! - pathTemplate: true #(7)! - metrics: - enabled: true #(8)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! - telemetry: - enabled: true #(10)! ``` - 1. URL of the service where requests will be sent - 2. Maximum request timeout, may spans the entire call: resolving DNS, connecting, writing the request body, server processing, and reading the response body, if call requires redirects or retries all must complete within one timeout period - 3. Enables module logging (default `false`) - 4. Mask that is used to hide specified headers and request/response parameters - 5. List of request parameters to be hidden - 6. List of request/response headers that should be hidden - 7. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 8. Enables module metrics (default `true`) - 9. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 10. Enables module tracing (default `true`) + 1. Base service `URL` where requests will be sent (`required`, no default) + 2. Maximum request time (default: not specified, optional) -### Method Configuration { #method-configuration } +??? note "Full Configuration" -Using the above HTTP client example, it is possible to configure separately some of the parameters for a particular method, the configuration path -is determined by the path to the client and the method name, in the example above the configuration is `httpClient.someClient` -and method `hello` the final path will be `httpClient.someClient.hello` + Example configuration in the case of the `httpClient.someClient` path described in the `DeclarativeHttpClientConfig` class: -===! ":material-code-json: `Hocon`" + ===! ":material-code-json: `Hocon`" - ```javascript - httpClient { - someClient { - hello { - requestTimeout = "10s" //(1)! + ```javascript + httpClient { + someClient { + url = "https://localhost:8090" //(1)! + requestTimeout = "10s" //(2)! telemetry { logging { - enabled = false //(2)! - mask = "***" //(3)! - maskQueries = [ ] //(4)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(5)! - pathTemplate = true //(6)! + enabled = false //(3)! + mask = "***" //(4)! + maskQueries = [ ] //(5)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(6)! + pathTemplate = true //(7)! } metrics { - enabled = true //(7)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(8)! + enabled = true //(8)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(9)! + tags = { // (10)! + "key1" = "value1" + "key2" = "value2" + } } tracing { - enabled = true //(9)! + enabled = true //(11)! + attributes = { // (12)! + "key1" = "value1" + "key2" = "value2" + } } } } } + ``` + + 1. Base service `URL` where requests will be sent (`required`, default not specified) + 2. Maximum request time: may include `DNS` resolution, connection, request body write, server processing, and response body read. If the call requires redirects or retries, they must all finish within one period (default not specified, optional) + 3. Enables module logging (default: `false`) + 4. Mask used to hide specified headers and request or response parameters (default: `***`) + 5. List of request parameters to hide (default: `[]`) + 6. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 7. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 8. Enables module metrics (default: `true`) + 9. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 10. Configures metric tags (default: `{}`) + 11. Enables module tracing (default: `true`) + 12. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + someClient: + url: "https://localhost:8090" #(1)! + requestTimeout: "10s" #(2)! + telemetry: + logging: + enabled: false #(3)! + mask: "***" #(4)! + maskQueries: [ ] #(5)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(6)! + pathTemplate: true #(7)! + metrics: + enabled: true #(8)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! + tags: #(10)! + key1: value1 + key2: value2 + tracing: + enabled: true #(11)! + attributes: #(12)! + key1: value1 + key2: value2 + ``` + + 1. Base service `URL` where requests will be sent (`required`, default not specified) + 2. Maximum request time: may include `DNS` resolution, connection, request body write, server processing, and response body read. If the call requires redirects or retries, they must all finish within one period (default not specified, optional) + 3. Enables module logging (default: `false`) + 4. Mask used to hide specified headers and request or response parameters (default: `***`) + 5. List of request parameters to hide (default: `[]`) + 6. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 7. Whether to use the request path template in logging; when not specified, the template is used except at `TRACE`, where the full path is used (default not specified, optional) + 8. Enables module metrics (default: `true`) + 9. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 10. Configures metric tags (default: `{}`) + 11. Enables module tracing (default: `true`) + 12. Configures tracing attributes (default: `{}`) + +### Method Configuration { #method-configuration } + +For a particular method, some parameters can be configured separately. The method configuration path is determined by the client path and the method name: +if the client path is `httpClient.someClient`, the final path for the `hello` method is `httpClient.someClient.hello`. + +Method configuration is applied over client configuration: method `requestTimeout` replaces the client value, and method telemetry settings override +only explicitly specified fields. + +Basic method configuration parameters: + +===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + someClient { + hello { + requestTimeout = "10s" //(1)! + } + } } ``` - 1. Maximum request timeout, may spans the entire call: resolving DNS, connecting, writing the request body, server processing, and reading the response body, if call requires redirects or retries all must complete within one timeout period - 2. Enables module logging (default `false`) - 3. Mask that is used to hide specified headers and request/response parameters - 4. List of request parameters to be hidden - 5. List of request/response headers that should be hidden - 6. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 7. Includes module metrics - 8. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 9. Enables module tracing (default `true`) + 1. Maximum request time (default: not specified, optional) === ":simple-yaml: `YAML`" ```yaml httpClient: someClient: - hello: + hello: requestTimeout: "10s" #(1)! - telemetry: - logging: - enabled: false #(2)! - mask: "***" #(3)! - maskQueries: [ ] #(4)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(5)! - pathTemplate: true #(6)! - metrics: - enabled: true #(7)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(8)! - telemetry: - enabled: true #(9)! ``` - 1. Maximum request timeout, may spans the entire call: resolving DNS, connecting, writing the request body, server processing, and reading the response body, if call requires redirects or retries all must complete within one timeout period - 2. Enables module logging (default `false`) - 3. Mask that is used to hide specified headers and request/response parameters - 4. List of request parameters to be hidden - 5. List of request/response headers that should be hidden - 6. Whether to always use the request path template when logging. Default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 7. Includes module metrics - 8. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 9. Enables module tracing (default `true`) + 1. Maximum request time (default: not specified, optional) + +??? note "Full Configuration" + + Full method configuration example: + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + someClient { + hello { + requestTimeout = "10s" //(1)! + telemetry { + logging { + enabled = false //(2)! + mask = "***" //(3)! + maskQueries = [ ] //(4)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(5)! + pathTemplate = true //(6)! + } + metrics { + enabled = true //(7)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(8)! + tags = { // (9)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(10)! + attributes = { // (11)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + } + ``` + + 1. Maximum request time: may include `DNS` resolution, connection, request body write, server processing, and response body read. If the call requires redirects or retries, they must all finish within one period (default not specified, optional) + 2. Enables module logging (default: `false`) + 3. Mask used to hide specified headers and request or response parameters (default: `***`) + 4. List of request parameters to hide (default: `[]`) + 5. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 6. Whether to use the request path template in logging; when not specified, the client value is inherited (default not specified, optional) + 7. Enables module metrics (default: `true`) + 8. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 9. Configures metric tags (default: `{}`) + 10. Enables module tracing (default: `true`) + 11. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + someClient: + hello: + requestTimeout: "10s" #(1)! + telemetry: + logging: + enabled: false #(2)! + mask: "***" #(3)! + maskQueries: [ ] #(4)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(5)! + pathTemplate: true #(6)! + metrics: + enabled: true #(7)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(8)! + tags: #(9)! + key1: value1 + key2: value2 + tracing: + enabled: true #(10)! + attributes: #(11)! + key1: value1 + key2: value2 + ``` + + 1. Maximum request time: may include `DNS` resolution, connection, request body write, server processing, and response body read. If the call requires redirects or retries, they must all finish within one period (default not specified, optional) + 2. Enables module logging (default: `false`) + 3. Mask used to hide specified headers and request or response parameters (default: `***`) + 4. List of request parameters to hide (default: `[]`) + 5. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 6. Whether to use the request path template in logging; when not specified, the client value is inherited (default not specified, optional) + 7. Enables module metrics (default: `true`) + 8. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 9. Configures metric tags (default: `{}`) + 10. Enables module tracing (default: `true`) + 11. Configures tracing attributes (default: `{}`) ### Request { #request } -The section describes HTTP request transformations at a declarative HTTP client. -It is suggested to use special annotations to specify request parameters. +This section describes `HTTP` request transformations for a declarative `HTTP` client. +Use special annotations to specify request parameters. + +#### String Parameter Conversion { #string-parameter-converter } + +`StringParameterConverter` converts a parameter value to a string before Kora puts it into a path, query parameter, +header, or cookie. The interface has one method: + +```java +public interface StringParameterConverter { + String convert(T value); +} +``` + +The converter is looked up as a regular graph component by the exact parameter type. If the parameter has type `Map`, +the converter is looked up for value type `T`; if `Map>` is used, it is applied to every list item. + +Built-in converters are available for `Boolean`, `Short`, `Integer`, `Long`, `Double`, `Float`, `UUID`, `BigDecimal`, `BigInteger`, +`Duration`, `OffsetTime`, `OffsetDateTime`, `LocalTime`, `LocalDate`, `LocalDateTime`, `ZonedDateTime`, and `Instant`. +Date and time types are written in `ISO` format. For custom types, provide a `StringParameterConverter` component: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserId(long value) {} + + @Module + public interface UserIdModule { + + default StringParameterConverter userIdStringParameterConverter() { + return value -> Long.toString(value.value()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserId(val value: Long) + + @Module + interface UserIdModule { + + fun userIdStringParameterConverter(): StringParameterConverter { + return StringParameterConverter { value -> value.value.toString() } + } + } + ``` + +After that, the type can be used in client parameters: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + User get(@Path("id") UserId id); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + fun get(@Path("id") id: UserId): User + } + ``` #### Path parameter { #path-parameter } @@ -749,7 +1065,9 @@ and the name of the parameter is specified in `value` or is equal to the name of #### Query parameter { #query-parameter } -`@Query` - value of the query parameter, the name of the parameter is specified in `value` or is equal to the name of the method argument by default. +`@Query` - query parameter value, the name is specified in `value` or defaults to the method argument name. +Single values, `List`, `Set`, `Collection`, `Map`, and `Map>` are supported. +For non-string values, an available `StringParameterConverter` is used. ===! ":fontawesome-brands-java: `Java`" @@ -776,8 +1094,9 @@ and the name of the parameter is specified in `value` or is equal to the name of ``` -It is possible to send query parameters in key and value format, for this purpose it is assumed to use `Map` type, -where the key is the parameter name and must be of type `String`, and the parameter value can be of any type and will be processed through `String.valueOf()`: +Query parameters can also be sent in key-value format using `Map`, where the key is the parameter name and must be `String`. +If a `Map` value is a list, every item is sent as a separate value of the same parameter. +If a list item is `null`, the parameter is sent without a value. ===! ":fontawesome-brands-java: `Java`" @@ -804,6 +1123,7 @@ where the key is the parameter name and must be of type `String`, and the parame #### Header { #header } `@Header` - value of [request header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers), parameter name is specified in `value` or defaults to the method argument name. +Single values, `List`, `Set`, `Collection`, `Map`, and a ready `HttpHeaders` object are supported. ===! ":fontawesome-brands-java: `Java`" @@ -829,8 +1149,8 @@ where the key is the parameter name and must be of type `String`, and the parame } ``` -It is possible to send request parameters in key and value format, for this purpose it is supposed to use `HttpHeaders` type or `Map` type, -where the key is the parameter name and must be of type `String`, and the parameter value can be of any type and will be processed through `String.valueOf()`: +Headers can be sent in key-value format using `HttpHeaders` or `Map`, where the key is the header name and must be `String`. +For non-string values, an available `StringParameterConverter` is used: ===! ":fontawesome-brands-java: `Java`" @@ -1044,9 +1364,52 @@ it is possible to use a special `HttpClientRequestMapper` interface to implement } ``` +**Example: Protobuf Serialization** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface ProtobufClient { + + final class ProtobufRequestMapper implements HttpClientRequestMapper { + + @Override + public HttpBodyOutput apply(Context ctx, MyMessage value) { + byte[] protobufBytes = value.toByteArray(); + return HttpBody.of(protobufBytes, "application/x-protobuf"); + } + } + + @HttpRoute(method = HttpMethod.POST, path = "/message") + void sendMessage(@Mapping(ProtobufRequestMapper.class) MyMessage message); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface ProtobufClient { + + class ProtobufRequestMapper : HttpClientRequestMapper { + + override fun apply(Context ctx, MyMessage value): HttpBodyOutput { + val protobufBytes = value.toByteArray() + return HttpBody.of(protobufBytes, "application/x-protobuf") + } + } + + @HttpRoute(method = HttpMethod.POST, path = "/message") + fun sendMessage(@Mapping(ProtobufRequestMapper::class) message: MyMessage) + } + ``` + #### Cookie { #cookie } `@Cookie` - [Cookie](https://developer.mozilla.org/en-US/docs/Glossary/Cookie) value, the parameter name is specified in `value` or defaults to the method argument name. +Single values, `List`, `Set`, `Collection`, `Map`, and a ready `Cookie` object are supported. +Cookies are added to the `Cookie` header; for collections, every value becomes a separate cookie value with the same name. ===! ":fontawesome-brands-java: `Java`" @@ -1248,35 +1611,45 @@ If you need to read the response in a different way, you can use the special `Ht } ``` -#### Response Error { #response-error } +**Example: Error Handling in Mapper** -By default, when no converter tag or converter itself is specified, the conversion will be applied for `2xx` HTTP status codes, -for all others a `HttpClientResponseException` exception will be thrown, which contains [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status), response body and response headers. +===! ":fontawesome-brands-java: `Java`" -#### Conversion by Code { #conversion-by-code } + ```java + @HttpClient + public interface ApiClient { -If specific conversions are required depending on the [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) of the response, you can use the `@ResponseCodeMapper` annotation to specify a -correspondence between the HTTP status code and the `HttpClientResponseMapper` resolver. + record ApiResponse(String status, Object data) {} -You can also use `ResponseCodeMapper.DEFAULT` as an indication of the default behavior for all unlisted status codes. + final class SafeResponseMapper implements HttpClientResponseMapper { -===! ":fontawesome-brands-java: `Java`" + private final JsonReader jsonReader; - ```java - @HttpClient - public interface SomeClient { + public SafeResponseMapper(JsonReader jsonReader) { + this.jsonReader = jsonReader; + } - record UserResponse(UserResponse.Payload payload, UserResponse.Error error) { + @Override + public ApiResponse apply(HttpClientResponse response) throws IOException { + int statusCode = response.statusCode(); + byte[] body = response.body(); - public record Error(int code, String message) {} + if (statusCode >= 400) { + // Handle error: log or throw exception + throw new HttpClientResponseException(statusCode, body, response.headers()); + } - public record Payload(String message) {} + if (body == null || body.length == 0) { + return null; + } + + return jsonReader.read(body); + } } - @ResponseCodeMapper(code = ResponseCodeMapper.DEFAULT, mapper = ResponseErrorMapper.class) - @ResponseCodeMapper(code = 200, mapper = ResponseSuccessMapper.class) - @HttpRoute(method = HttpMethod.GET, path = "/hello/world") - UserResponse hello(); + @HttpRoute(method = HttpMethod.GET, path = "/api/data") + @Mapping(SafeResponseMapper.class) + ApiResponse getData(); } ``` @@ -1284,13 +1657,113 @@ You can also use `ResponseCodeMapper.DEFAULT` as an indication of the default be ```kotlin @HttpClient - interface SomeClient { + interface ApiClient { - data class UserResponse(val payload: Payload, val error: Error) { - - data class Error(val code: Int, val message: String) - - data class Payload(val message: String) + data class ApiResponse(val status: String, val data: Any?) + + class SafeResponseMapper( + private val jsonReader: JsonReader + ) : HttpClientResponseMapper { + + @Throws(IOException::class) + override fun apply(response: HttpClientResponse): ApiResponse { + val statusCode = response.statusCode() + val body = response.body() + + if (statusCode >= 400) { + // Handle error: log or throw exception + throw HttpClientResponseException(statusCode, body, response.headers()) + } + + if (body == null || body.isEmpty()) { + return null + } + + return jsonReader.read(body) + } + } + + @HttpRoute(method = HttpMethod.GET, path = "/api/data") + @Mapping(SafeResponseMapper::class) + fun getData(): ApiResponse + } + ``` + +#### Response Error { #response-error } + +By default, when neither converter tag nor converter is specified, conversion is applied only for `2xx` HTTP response codes. +For all other codes, `HttpClientResponseException` is thrown. It contains the [HTTP response code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status), response body, and response headers. + +#### Client Exceptions { #client-exceptions } + +All standard `HTTP` client exceptions inherit from `HttpClientException`, which is a `RuntimeException`. +This lets you catch a specific error type or all client errors with one common type: + +```java +try { + client.getUser("123"); +} catch (HttpClientResponseException e) { + var code = e.getCode(); + var headers = e.getHeaders(); + var body = e.getBytes(); +} catch (HttpClientException e) { + throw e; +} +``` + +Main exception types: + +* `HttpClientResponseException` — response was received, but its code was not handled as successful. Contains `getCode()`, `getHeaders()`, and `getBytes()`. +* `HttpClientTimeoutException` — request, connection, or read timeout expired. +* `HttpClientConnectionException` — error while establishing or maintaining a connection to the remote host. +* `HttpClientEncoderException` — error while converting a user value into a request body. +* `HttpClientDecoderException` — error while converting a response body into a user type. +* `HttpClientUnknownException` — other transport client error that did not match a more specific category. + +`HttpClientResponseException` is created after reading the response body into a byte array. If the body could not be read fully, +the read error is added as a `suppressed` exception, and `getBytes()` contains the body that could be collected. + +#### Conversion by Code { #conversion-by-code } + +If specific conversions are required depending on the [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) of the response, you can use the `@ResponseCodeMapper` annotation to specify a +correspondence between the HTTP status code and the `HttpClientResponseMapper` resolver. + +You can also use `ResponseCodeMapper.DEFAULT` to define default behavior for all unlisted HTTP codes. +If `mapper` is specified for a code, that particular `HttpClientResponseMapper` is used. +If `type` is specified, Kora selects a response mapper for that type and then casts the result to the method return type. +This is useful for closed response hierarchies where different HTTP statuses correspond to different result subtypes. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface SomeClient { + + record UserResponse(UserResponse.Payload payload, UserResponse.Error error) { + + public record Error(int code, String message) {} + + public record Payload(String message) {} + } + + @ResponseCodeMapper(code = ResponseCodeMapper.DEFAULT, mapper = ResponseErrorMapper.class) + @ResponseCodeMapper(code = 200, mapper = ResponseSuccessMapper.class) + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + UserResponse hello(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface SomeClient { + + data class UserResponse(val payload: Payload, val error: Error) { + + data class Error(val code: Int, val message: String) + + data class Payload(val message: String) } @ResponseCodeMapper(code = ResponseCodeMapper.DEFAULT, mapper = ResponseErrorMapper::class) @@ -1303,9 +1776,51 @@ You can also use `ResponseCodeMapper.DEFAULT` as an indication of the default be In the example above, `ResponseSuccessMapper` will be used for status code `200`, and for all other status codes the `ResponseErrorMapper` will be used. +Example with the `type` parameter: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface SomeClient { + + sealed interface UserResponse permits Success, Error {} + + record Success(String id) implements UserResponse {} + + record Error(String message) implements UserResponse {} + + @Json + @ResponseCodeMapper(code = 200, type = Success.class) + @ResponseCodeMapper(code = 404, type = Error.class) + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + UserResponse get(@Path String id); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface SomeClient { + + sealed interface UserResponse + + data class Success(val id: String) : UserResponse + + data class Error(val message: String) : UserResponse + + @Json + @ResponseCodeMapper(code = 200, type = Success::class) + @ResponseCodeMapper(code = 404, type = Error::class) + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + fun get(@Path id: String): UserResponse + } + ``` + ### Signatures { #signatures } -Available signatures for repository methods out of the box: +Available signatures for declarative `HTTP` client methods out of the box: ===! ":fontawesome-brands-java: `Java`" @@ -1324,8 +1839,69 @@ Available signatures for repository methods out of the box: ## Interceptors { #interceptors } -You can create interceptors to change behavior or create additional behavior using the `HttpClientInterceptor` class. -Interceptors can be applied to specific methods or to the entire `@HttpClient` class: +You can create interceptors to change behavior or create additional behavior using the `HttpClientInterceptor` interface. +Interceptors can be attached to specific methods or the entire `@HttpClient` class using the `@InterceptWith` annotation. + +**Method-level interceptor:** + +### Root URL { #root-uri-interceptor } + +`RootUriInterceptor` is a ready-made interceptor that adds a base `URL` to relative requests. +If the request already contains a scheme (`http://` or `https://`), the interceptor leaves it unchanged. +If the request is relative, `RootUriInterceptor` adds the root address and guarantees one `/` separator between the root and the path. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface ClientModule { + + default RootUriInterceptor rootUriInterceptor() { + return new RootUriInterceptor("https://api.example.com"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface ClientModule { + + fun rootUriInterceptor(): RootUriInterceptor { + return RootUriInterceptor("https://api.example.com") + } + } + ``` + +After registering the interceptor, connect it to the client: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + @InterceptWith(RootUriInterceptor.class) + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + User get(@Path String id); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + @InterceptWith(RootUriInterceptor::class) + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + fun get(@Path id: String): User + } + ``` + +For declarative clients, it is usually more convenient to set the base `URL` through `DeclarativeHttpClientConfig.url`. +`RootUriInterceptor` is useful for imperative `HttpClient` or when a shared root address should be added as separate cross-cutting behavior. ===! ":fontawesome-brands-java: `Java`" @@ -1334,13 +1910,13 @@ Interceptors can be applied to specific methods or to the entire `@HttpClient` c public interface SomeClient { final class MethodInterceptor implements HttpClientInterceptor { - + private final Component1 component1; private MethodInterceptor(Component1 component1) { this.component1 = component1; } - + @Override public CompletionStage processRequest(Context ctx, InterceptChain chain, HttpClientRequest request) throws Exception { component1.doSomething(); @@ -1379,6 +1955,115 @@ Interceptors can be applied to specific methods or to the entire `@HttpClient` c } ``` +**Class-level interceptor:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @InterceptWith(LoggingInterceptor.class) // Applied to all client methods + @HttpClient + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello") + void hello(); + + @HttpRoute(method = HttpMethod.POST, path = "/world") + void world(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @InterceptWith(LoggingInterceptor::class) // Applied to all client methods + @HttpClient + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello") + fun hello() + + @HttpRoute(method = HttpMethod.POST, path = "/world") + fun world() + } + ``` + +**Interceptor execution order:** + +Interceptors are executed in declaration order (left to right). Each interceptor can: +- Modify the request before sending +- Call the next interceptor in the chain (`chain.process()`) +- Modify the response after receiving +- Throw an exception to break the chain + +``` +Request → Interceptor1 → Interceptor2 → Interceptor3 → HTTP Server +Response ← Interceptor1 ← Interceptor2 ← Interceptor3 ← HTTP Server +``` + +### Global interceptor { #interceptor-global } + +To apply an interceptor to all clients, register it as a component without `@InterceptWith`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class GlobalInterceptor implements HttpClientInterceptor { + + @Override + public CompletionStage processRequest(Context ctx, InterceptChain chain, HttpClientRequest request) throws Exception { + // Applied to all HTTP clients + return chain.process(ctx, request); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class GlobalInterceptor : HttpClientInterceptor { + + @Throws(Exception::class) + override fun processRequest( + ctx: Context, + chain: HttpClientInterceptor.InterceptChain, + request: HttpClientRequest + ): CompletionStage { + // Applied to all HTTP clients + return chain.process(ctx, request) + } + } + ``` + +If the interceptor must be applied to all client methods, `@InterceptWith` can be placed on the interface: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + @InterceptWith(ClientInterceptor.class) + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + void hello(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + @InterceptWith(ClientInterceptor::class) + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + fun hello() + } + ``` + +If interceptors are specified on both the client and the method, both interceptor sets are applied for that call. + ### Authorization { #authorization } Kora provides out-of-the-box interceptors that can be used for [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/) authorization. @@ -1590,6 +2275,260 @@ Then add interceptor for the entire HTTP client or specific methods. Authorization by [OAuth](https://swagger.io/docs/specification/authentication/oauth2/) is similar to [Bearer](#bearer), you need to implement `HttpClientTokenProvider` yourself and put it in dependency container. +#### HttpClientTokenProvider { #token-provider } + +`HttpClientTokenProvider` — interface for providing authorization tokens dynamically. +Used when the token needs to be refreshed or obtained from an external source (e.g., OAuth2 token endpoint). + +**Implementation example:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class MyTokenProvider implements HttpClientTokenProvider { + + private final OAuthClient oauthClient; + private volatile String cachedToken; + private volatile long tokenExpiry; + + public MyTokenProvider(OAuthClient oauthClient) { + this.oauthClient = oauthClient; + } + + @Override + public CompletionStage getToken(HttpClientRequest request) { + if (cachedToken != null && System.currentTimeMillis() < tokenExpiry) { + return CompletableFuture.completedFuture(cachedToken); + } + + // Get new token + return oauthClient.refreshToken() + .thenApply(response -> { + this.cachedToken = response.accessToken(); + this.tokenExpiry = System.currentTimeMillis() + response.expiresIn() * 1000; + return this.cachedToken; + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyTokenProvider( + private val oauthClient: OAuthClient + ) : HttpClientTokenProvider { + + private var cachedToken: String? = null + private var tokenExpiry: Long = 0 + + override fun getToken(request: HttpClientRequest): CompletionStage { + if (cachedToken != null && System.currentTimeMillis() < tokenExpiry) { + return CompletableFuture.completedFuture(cachedToken) + } + + // Get new token + return oauthClient.refreshToken() + .thenApply { response -> + cachedToken = response.accessToken() + tokenExpiry = System.currentTimeMillis() + response.expiresIn() * 1000 + cachedToken!! + } + } + } + ``` + +**Usage with BearerAuthHttpClientInterceptor:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface AuthModule { + + default BearerAuthHttpClientInterceptor bearerAuthInterceptor(HttpClientTokenProvider tokenProvider) { + return new BearerAuthHttpClientInterceptor(tokenProvider); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface AuthModule { + + fun bearerAuthInterceptor(tokenProvider: HttpClientTokenProvider): BearerAuthHttpClientInterceptor { + return BearerAuthHttpClientInterceptor(tokenProvider) + } + } + ``` + +## Exception handling { #exception-handling } + +Various exceptions may occur during HTTP requests. All exceptions inherit from the base `HttpClientException`. + +**Exception hierarchy:** + +``` +HttpClientException +├── HttpClientTimeoutException +├── HttpClientConnectionException +├── HttpClientResponseException +├── HttpClientEncoderException +├── HttpClientDecoderException +└── HttpClientUnknownException +``` + +**Handling example:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + class SomeService { + private final SomeClient client; + + public SomeService(SomeClient client) { + this.client = client; + } + + public void call() { + try { + client.hello(); + } catch (HttpClientTimeoutException e) { + // Timeout: log, retry + } catch (HttpClientConnectionException e) { + // Connection error: check service availability + } catch (HttpClientResponseException e) { + // Response error: statusCode, body, headers + int statusCode = e.getStatusCode(); + byte[] body = e.getBody(); + } catch (HttpClientEncoderException e) { + // Serialization error: validate data + } catch (HttpClientDecoderException e) { + // Deserialization error: log + } catch (HttpClientUnknownException e) { + // Unknown error: e.getCause() + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val client: SomeClient + ) { + fun call() { + try { + client.hello() + } catch (e: HttpClientTimeoutException) { + // Timeout: log, retry + } catch (e: HttpClientConnectionException) { + // Connection error: check service availability + } catch (e: HttpClientResponseException) { + // Response error: statusCode, body, headers + val statusCode = e.statusCode + val body = e.body + } catch (e: HttpClientEncoderException) { + // Serialization error: validate data + } catch (e: HttpClientDecoderException) { + // Deserialization error: log + } catch (e: HttpClientUnknownException) { + // Unknown error: e.cause + } + } + } + ``` + +#### HttpClientTimeoutException { #timeout-exception } + +Thrown when a request exceeds the configured timeout (`requestTimeout` or `connectTimeout`). + +**Causes:** +- Server does not respond within `requestTimeout` +- Connection establishment time exceeds `connectTimeout` +- Network delays + +**Recommendations:** +- Configure adequate timeouts in configuration +- Implement retry logic for temporary failures +- Use circuit breaker to protect from cascading failures + +#### HttpClientConnectionException { #connection-exception } + +Thrown when connection to server cannot be established. + +**Causes:** +- DNS resolution fails +- Server unavailable (port closed, firewall) +- Connection refused +- SSL/TLS handshake failed + +**Recommendations:** +- Check service availability (health check) +- Use fallback to backup service +- Configure retry with exponential backoff + +#### HttpClientResponseException { #response-exception } + +Thrown when server returns HTTP error status code (4xx or 5xx) and no custom mapper is specified via `@ResponseCodeMapper`. + +**Available data:** +- `statusCode` — HTTP status code (400, 404, 500, etc.) +- `body` — response body (may contain error details) +- `headers` — response headers + +**Recommendations:** +- Use `@ResponseCodeMapper` for custom status handling +- Log statusCode and body for debugging +- Distinguish client (4xx) and server (5xx) errors + +#### HttpClientEncoderException { #encoder-exception } + +Thrown when serialization of request body fails. + +**Causes:** +- JSON/XML serialization error +- Invalid data in request object +- Missing serializer for type + +**Recommendations:** +- Validate data before sending +- Check for Json annotations on classes +- Log original exception in `cause` + +#### HttpClientDecoderException { #decoder-exception } + +Thrown when deserialization of response body fails. + +**Causes:** +- Invalid JSON/XML in server response +- Schema mismatch (server returned unexpected fields) +- Missing deserializer for type + +**Recommendations:** +- Check API version compatibility +- Log response body for debugging +- Use `@ResponseCodeMapper` for format error handling + +#### HttpClientUnknownException { #unknown-exception } + +Thrown when an unknown error occurs that doesn't fit other categories. + +**Available data:** +- `cause` — original exception + +**Recommendations:** +- Always log `cause` for diagnostics +- Check HTTP client logs at DEBUG/TRACE level +- Report bug if exception is reproducible + ## Client imperative { #client-imperative } The base client represents the `HttpClient` interface and is available for deployment: @@ -1629,3 +2568,207 @@ You can use `HttpClientRequestBuilder` to build requests manually: .body(HttpBody.plaintext("refresh")) .build() ``` + +### HttpClientRequestBuilder { #request-builder } + +`HttpClientRequestBuilder` allows building HTTP requests manually. + +===! ":fontawesome-brands-java: `Java`" + + ```java + HttpClientRequest request = HttpClientRequest.of("POST", "http://localhost:8090/pets/{petId}") + .templateParam("petId", "1") + .queryParam("page", 1) + .header("token", "12345") + .body(HttpBody.plaintext("refresh")) + .build(); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = HttpClientRequest.of("POST", "http://localhost:8090/pets/{petId}") + .templateParam("petId", "1") + .queryParam("page", 1) + .header("token", "12345") + .body(HttpBody.plaintext("refresh")) + .build() + ``` + +### UriQueryBuilder { #uri-query-builder } + +`UriQueryBuilder` helps build URIs with query parameters. + +===! ":fontawesome-brands-java: `Java`" + + ```java + UriQueryBuilder builder = new UriQueryBuilder() + .path("/api/users") + .queryParam("page", 1) + .queryParam("size", 10) + .queryParam("sort", "name"); + + String uri = builder.build(); + // /api/users?page=1&size=10&sort=name + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val builder = UriQueryBuilder() + .path("/api/users") + .queryParam("page", 1) + .queryParam("size", 10) + .queryParam("sort", "name") + + val uri = builder.build() + // /api/users?page=1&size=10&sort=name + ``` + +### HttpBodyInput { #http-body-input } + +`HttpBodyInput` is an interface that describes HTTP request body as a data stream (Flow.Publisher). +Used for streaming large data without loading into memory. + +**Methods:** + +| Method | Returns | Description | +|--------|---------|-------------| +| `asInputStream()` | `InputStream` | Represents body as InputStream for reading | +| `asBufferStage()` | `CompletionStage` | Asynchronously reads entire body to ByteBuffer | +| `asArrayStage()` | `CompletionStage` | Asynchronously reads entire body to byte[] | + +### HttpClientResponse { #http-client-response } + +`HttpClientResponse` is an interface that represents HTTP response from server. + +**Methods:** + +| Method | Returns | Description | +|--------|---------|-------------| +| `statusCode()` | `int` | HTTP status code (200, 404, 500, etc.) | +| `body()` | `byte[]` | Response body as byte array | +| `headers()` | `HttpHeaders` | Response headers | +| `cookies()` | `Cookies` | Cookies from response | + +### HttpHeaders { #http-headers-imperative } + +`HttpHeaders` provides access to request and response headers in the imperative client. + +**Reading headers:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + HttpClientRequest request = HttpClientRequest.of("GET", "http://localhost:8090/api/data") + .build(); + + httpClient.execute(request).thenAccept(response -> { + HttpHeaders headers = response.headers(); + String contentType = headers.getFirst("Content-Type"); + List allValues = headers.get("X-Custom-Header"); + boolean hasHeader = headers.contains("Authorization"); + }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = HttpClientRequest.of("GET", "http://localhost:8090/api/data").build() + + httpClient.execute(request).thenAccept { response -> + val headers = response.headers + val contentType = headers.getFirst("Content-Type") + val allValues = headers.get("X-Custom-Header") + val hasHeader = headers.contains("Authorization") + } + ``` + +**Adding headers:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + MutableHttpHeaders headers = new MutableHttpHeaders(); + headers.add("Authorization", "Bearer token123"); + headers.add("X-Custom-Header", "value"); + headers.set("Content-Type", "application/json"); + + HttpClientRequest request = HttpClientRequest.of("POST", "http://localhost:8090/api/data") + .headers(headers) + .body(HttpBody.plaintext("body")) + .build(); + + httpClient.execute(request); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val headers = MutableHttpHeaders() + headers.add("Authorization", "Bearer token123") + headers.add("X-Custom-Header", "value") + headers.set("Content-Type", "application/json") + + val request = HttpClientRequest.of("POST", "http://localhost:8090/api/data") + .headers(headers) + .body(HttpBody.plaintext("body")) + .build() + + httpClient.execute(request) + ``` + +### Cookies { #cookies-imperative } + +`Cookies` provides access to request and response cookies in the imperative client. + +**Reading cookies:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + HttpClientRequest request = HttpClientRequest.of("GET", "http://localhost:8090/api/profile") + .build(); + + httpClient.execute(request).thenAccept(response -> { + Cookies cookies = response.cookies(); + Cookie sessionCookie = cookies.get("SESSIONID"); + if (sessionCookie != null) { + String value = sessionCookie.value(); + String domain = sessionCookie.domain(); + String path = sessionCookie.path(); + } + }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = HttpClientRequest.of("GET", "http://localhost:8090/api/profile").build() + + httpClient.execute(request).thenAccept { response -> + val cookies = response.cookies + val sessionCookie = cookies.get("SESSIONID") + if (sessionCookie != null) { + val value = sessionCookie.value() + val domain = sessionCookie.domain() + val path = sessionCookie.path() + } + } + ``` + +## Telemetry { #telemetry } + +HTTP Client uses a telemetry contract for logging, metrics, and tracing of requests. +Telemetry configuration (section `telemetry { logging / metrics / tracing }`) is described in the [Configuration](#configuration) section. +Extension points are located in `ru.tinkoff.kora.http.client.common.telemetry`. + +For each HTTP request, an `HttpClientTelemetry.HttpClientTelemetryContext` is created, which is closed upon request completion. +The request is described through telemetry handler parameters, including method, URL, response status, and duration. + +The default factory `DefaultHttpClientTelemetryFactory` combines three factories: +- `HttpClientLoggerFactory` builds `HttpClientLogger` for logging request start/end; +- `HttpClientMetricsFactory` builds `HttpClientMetrics` for writing request metrics; +- `HttpClientTracerFactory` builds `HttpClientTracer` for distributed tracing. + +Metrics and tracing are described in the [Metrics Reference](metrics.md#http-client) section. diff --git a/mkdocs/docs/en/documentation/http-server.md b/mkdocs/docs/en/documentation/http-server.md index 8d74ac5..76f727c 100644 --- a/mkdocs/docs/en/documentation/http-server.md +++ b/mkdocs/docs/en/documentation/http-server.md @@ -4,8 +4,13 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora HTTP server, declarative and imperative controllers, routing, request and response mapping, interceptors, error handling, and Undertow configuration; key triggers include @HttpController, @HttpRoute, @Path, @Query, @Header, @Cookie, @Json, @InterceptWith, HttpServerModule, UndertowHttpServerModule." --- -Module provides a thin layer of abstraction over HTTP server libraries to create HTTP request handlers -using both declarative-style annotations and imperative-style annotations. +The `HTTP server` module describes the incoming HTTP boundary of an application: accepting a request, parsing parameters, +reading the body, selecting a handler, creating a response, telemetry, and interceptors. In Kora, controllers can be described +declaratively with `@HttpController` and `@HttpRoute`, or handlers can be registered imperatively with `HttpServerRequestHandler`. + +The declarative approach fits most APIs: the method signature describes the HTTP contract, and Kora creates the handler at +compile time without using `Reflection` at runtime. The imperative approach is useful for low-level or dynamic routes where +it is easier to process the request manually. ???+ tip "Recommendation" @@ -19,9 +24,9 @@ For a step-by-step walkthrough before the reference details, see [HTTP Server](. ## Dependency { #dependency } -Implementation based on [Undertow](https://undertow.io/). -Undertow is a lightweight open-source web server for Java applications. -It is built on asynchronous and non-blocking I/O operations using NIO, +Implementation is based on [Undertow](https://undertow.io/). +`Undertow` is a lightweight open-source web server for `Java` applications. +It is built on asynchronous and non-blocking I/O operations using `NIO`, which ensures high performance and low resource consumption. ===! ":fontawesome-brands-java: `Java`" @@ -52,7 +57,7 @@ which ensures high performance and low resource consumption. ## Configuration { #configuration } -Example of the complete configuration described in the `HttpServerConfig` class (default or example values are specified): +Basic HTTP server configuration parameters: ===! ":material-code-json: `Hocon`" @@ -60,73 +65,13 @@ Example of the complete configuration described in the `HttpServerConfig` class httpServer { publicApiHttpPort = 8080 //(1)! privateApiHttpPort = 8085 //(2)! - privateApiHttpMetricsPath = "/metrics" //(3)! - privateApiHttpReadinessPath = "/system/readiness" //(4)! - privateApiHttpLivenessPath = "/system/liveness" //(5)! - ignoreTrailingSlash = false //(6)! - ioThreads = 2 //(7)! - blockingThreads = 2 //(8)! - shutdownWait = "30s" //(9)! - threadKeepAliveTimeout = "60s" //(10)! - socketReadTimeout = "0s" //(11)! - socketWriteTimeout = "0s" //(12)! - socketKeepAliveEnabled = false //(13)! - virtualThreadsEnabled = false //(14)! - maxRequestBodySize = "256MiB" //(15)! - telemetry { - logging { - enabled = false //(16)! - stacktrace = true //(17)! - mask = "***" //(18)! - maskQueries = [ ] //(19)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(20)! - pathTemplate = true //(21)! - } - metrics { - enabled = true //(22)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(23)! - tags = { // (24)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(25)! - attributes = { // (26)! - "key1" = "value1" - "key2" = "value2" - } - } - } + maxRequestBodySize = "256MiB" //(3)! } ``` - 1. Public server port - 2. Private server port - 3. Path to get [metrics](metrics.md) on the private server - 4. Path to get [probes](probes.md) status on the private server - 5. Path to get [probes viability](probes.md) status on a private server - 6. Whether to ignore the slash at the end of the path, if enabled `/my/path` and `/my/path/` will be interpreted the same way, default is off - 7. Number of network threads, default is the number of CPU cores or minimum `2`. - 8. Number of worker threads, default is the number of CPU cores multiplied by 2 or a minimum of `2` threads. - 9. Waiting time to shut down the server in case of [normal termination](https://maxilect.ru/blog/pochemu-vazhen-graceful-shutdown-v-oblachnoy-srede-na-pr/) - 10. Maximum lifetime of the request handler thread - 11. Maximum waiting time for reading data from the socket/connection - 12. Maximum waiting time for writing data to the socket/connection - 13. Whether to send `keep-alive' messages during TCP socket/connection lifetime - 14. Includes support for virtual threads for processing requests (instead of `blockingThreads`), requires Java 21+ - 15. Maximum allowed size of the incoming request body - 16. Enables module logging (default `false`) - 17. Enables call stack logging in case of exception - 18. Mask that is used to hide specified headers and request/response parameters - 19. List of request parameters to be hidden - 20. List of request/response headers that should be hidden - 21. Whether to always use the request path template when logging. The default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 22. Enables module metrics (default `true`) - 23. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 24. Configures tags for metrics (optional) - 25. Enables module tracing (default is `true`) - 26. Configures attributes for tracing (optional) + 1. Public `HTTP` server port (default: `8080`) + 2. Private `HTTP` server port (default: `8085`) + 3. Maximum allowed size of incoming request body (default: `256MiB`) === ":simple-yaml: `YAML`" @@ -134,71 +79,162 @@ Example of the complete configuration described in the `HttpServerConfig` class httpServer: publicApiHttpPort: 8080 #(1)! privateApiHttpPort: 8085 #(2)! - privateApiHttpMetricsPath: "/metrics" #(3)! - privateApiHttpReadinessPath: "/system/readiness" #(4)! - privateApiHttpLivenessPath: "/system/liveness" #(5)! - ignoreTrailingSlash: false #(6)! - ioThreads: 2 #(7)! - blockingThreads: 2 #(8)! - shutdownWait: "30s" #(9)! - threadKeepAliveTimeout: "60s" #(10)! - socketReadTimeout: "0s" #(11)! - socketWriteTimeout: "0s" #(12)! - socketKeepAliveEnabled: false #(13)! - virtualThreadsEnabled: false #(14)! - maxRequestBodySize: "256MiB" #(15)! - telemetry: - logging: - enabled: false #(16)! - stacktrace: true #(17)! - mask: "***" #(18)! - maskQueries: [ ] #(19)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(20)! - pathTemplate: true #(21)! - metrics: - enabled: true #(22)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(23)! - tags: #(24)! - key1: value1 - key2: value2 - tracing: - enabled: true #(25)! - attributes: #(26)! - key1: value1 - key2: value2 - ``` - - 1. Public server port - 2. Private server port - 3. Path to get [metrics](metrics.md) on the private server - 4. Path to get [probes](probes.md) status on the private server - 5. Path to get [probes viability](probes.md) status on a private server - 6. Whether to ignore the slash at the end of the path, if enabled `/my/path` and `/my/path/` will be interpreted the same way, default is off - 7. Number of network threads, default is the number of CPU cores or minimum `2`. - 8. Number of worker threads, default is the number of CPU cores multiplied by 2 or a minimum of `2` threads. - 9. Waiting time to shut down the server in case of [normal termination](https://maxilect.ru/blog/pochemu-vazhen-graceful-shutdown-v-oblachnoy-srede-na-pr/) - 10. Maximum lifetime of the request handler thread - 11. Maximum waiting time for reading data from the socket/connection - 12. Maximum waiting time for writing data to the socket/connection - 13. Whether to send `keep-alive' messages during TCP socket/connection lifetime - 14. Includes support for virtual threads for processing requests (instead of `blockingThreads`), requires Java 21+ - 15. Maximum allowed size of the incoming request body - 16. Enables module logging (default `false`) - 17. Enables call stack logging in case of exception - 18. Mask that is used to hide specified headers and request/response parameters - 19. List of request parameters to be hidden - 20. List of request/response headers that should be hidden - 21. Whether to always use the request path template when logging. The default is to always use the path template, except for the `TRACE` logging level, which uses the full path. - 22. Enables module metrics (default `true`) - 23. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 24. Configures tags for metrics (optional) - 25. Enables module tracing (default is `true`) - 26. Configures attributes for tracing (optional) + maxRequestBodySize: "256MiB" #(3)! + ``` + + 1. Public `HTTP` server port (default: `8080`) + 2. Private `HTTP` server port (default: `8085`) + 3. Maximum allowed size of incoming request body (default: `256MiB`) + +??? note "Full Configuration" + + Example of the complete configuration described in the `HttpServerConfig` class (default or example values are specified): + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpServer { + publicApiHttpPort = 8080 //(1)! + privateApiHttpPort = 8085 //(2)! + privateApiHttpMetricsPath = "/metrics" //(3)! + privateApiHttpReadinessPath = "/system/readiness" //(4)! + privateApiHttpLivenessPath = "/system/liveness" //(5)! + ignoreTrailingSlash = false //(6)! + ioThreads = 2 //(7)! + blockingThreads = 2 //(8)! + shutdownWait = "30s" //(9)! + threadKeepAliveTimeout = "60s" //(10)! + socketReadTimeout = "0s" //(11)! + socketWriteTimeout = "0s" //(12)! + socketKeepAliveEnabled = false //(13)! + virtualThreadsEnabled = false //(14)! + maxRequestBodySize = "256MiB" //(15)! + telemetry { + logging { + enabled = false //(16)! + stacktrace = true //(17)! + mask = "***" //(18)! + maskQueries = [ ] //(19)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(20)! + pathTemplate = true //(21)! + } + metrics { + enabled = true //(22)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(23)! + tags = { // (24)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(25)! + attributes = { // (26)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Public `HTTP` server port (default: `8080`) + 2. Private `HTTP` server port (default: `8085`) + 3. Path to get [metrics](metrics.md) on the private server (default: `/metrics`) + 4. Path to get [readiness probe](probes.md) status on the private server (default: `/system/readiness`) + 5. Path to get [liveness probe](probes.md) status on the private server (default: `/system/liveness`) + 6. Whether to ignore a trailing `/` in the path: when enabled, `/my/path` and `/my/path/` are treated as the same route (default: `false`) + 7. Number of network I/O threads (default: number of available processors, but not less than `2`) + 8. Number of threads for blocking request processing (default: `min(max(available processors, 2) * 8, 200)`) + 9. Time to wait for processing before server shutdown during [graceful shutdown](container.md#component-lifecycle) (default: `30s`) + 10. Maximum idle lifetime of a request handler thread (default: `60s`) + 11. Maximum time to wait for reading data from a socket or connection; `0s` disables the timeout (default: `0s`) + 12. Maximum time to wait for writing data to a socket or connection; `0s` disables the timeout (default: `0s`) + 13. Whether to enable `TCP keep-alive` for a socket or connection (default: `false`) + 14. Enables virtual threads for blocking request processing instead of the `blockingThreads` pool, requires `Java 21+` (default: `false`) + 15. Maximum allowed size of an incoming request body (default: `256MiB`) + 16. Enables module logging (default: `false`) + 17. Enables call stack logging on exception (default: `true`) + 18. Mask used to hide specified headers and request or response parameters (default: `***`) + 19. List of request parameters to hide (default: `[]`) + 20. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 21. Whether to use the request path template in logs; when not specified, the template is always used except at `TRACE`, where the full path is used (default not specified, optional) + 22. Enables module metrics (default: `true`) + 23. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 24. Configures metric tags (default: `{}`) + 25. Enables module tracing (default: `true`) + 26. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpServer: + publicApiHttpPort: 8080 #(1)! + privateApiHttpPort: 8085 #(2)! + privateApiHttpMetricsPath: "/metrics" #(3)! + privateApiHttpReadinessPath: "/system/readiness" #(4)! + privateApiHttpLivenessPath: "/system/liveness" #(5)! + ignoreTrailingSlash: false #(6)! + ioThreads: 2 #(7)! + blockingThreads: 2 #(8)! + shutdownWait: "30s" #(9)! + threadKeepAliveTimeout: "60s" #(10)! + socketReadTimeout: "0s" #(11)! + socketWriteTimeout: "0s" #(12)! + socketKeepAliveEnabled: false #(13)! + virtualThreadsEnabled: false #(14)! + maxRequestBodySize: "256MiB" #(15)! + telemetry: + logging: + enabled: false #(16)! + stacktrace: true #(17)! + mask: "***" #(18)! + maskQueries: [ ] #(19)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(20)! + pathTemplate: true #(21)! + metrics: + enabled: true #(22)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(23)! + tags: #(24)! + key1: value1 + key2: value2 + tracing: + enabled: true #(25)! + attributes: #(26)! + key1: value1 + key2: value2 + ``` + + 1. Public `HTTP` server port (default: `8080`) + 2. Private `HTTP` server port (default: `8085`) + 3. Path to get [metrics](metrics.md) on the private server (default: `/metrics`) + 4. Path to get [readiness probe](probes.md) status on the private server (default: `/system/readiness`) + 5. Path to get [liveness probe](probes.md) status on the private server (default: `/system/liveness`) + 6. Whether to ignore a trailing `/` in the path: when enabled, `/my/path` and `/my/path/` are treated as the same route (default: `false`) + 7. Number of network I/O threads (default: number of available processors, but not less than `2`) + 8. Number of threads for blocking request processing (default: `min(max(available processors, 2) * 8, 200)`) + 9. Time to wait for processing before server shutdown during [graceful shutdown](container.md#component-lifecycle) (default: `30s`) + 10. Maximum idle lifetime of a request handler thread (default: `60s`) + 11. Maximum time to wait for reading data from a socket or connection; `0s` disables the timeout (default: `0s`) + 12. Maximum time to wait for writing data to a socket or connection; `0s` disables the timeout (default: `0s`) + 13. Whether to enable `TCP keep-alive` for a socket or connection (default: `false`) + 14. Enables virtual threads for blocking request processing instead of the `blockingThreads` pool, requires `Java 21+` (default: `false`) + 15. Maximum allowed size of an incoming request body (default: `256MiB`) + 16. Enables module logging (default: `false`) + 17. Enables call stack logging on exception (default: `true`) + 18. Mask used to hide specified headers and request or response parameters (default: `***`) + 19. List of request parameters to hide (default: `[]`) + 20. List of request or response headers to hide (default: `[ "authorization", "cookie", "set-cookie" ]`) + 21. Whether to use the request path template in logs; when not specified, the template is always used except at `TRACE`, where the full path is used (default not specified, optional) + 22. Enables module metrics (default: `true`) + 23. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 24. Configures metric tags (default: `{}`) + 25. Enables module tracing (default: `true`) + 26. Configures tracing attributes (default: `{}`) Module metrics are described in the [Metrics Reference](metrics.md#http-server) section. -Kora provides fine-grained control over the Undertow HTTP server through two dedicated configuration interfaces: `UndertowConfigurer` and `HttpHandlerConfigurer`. -These allow you to customize server behavior and request processing pipeline without sacrificing integration with Kora’s modular architecture. +Kora provides fine-grained control over the `Undertow` `HTTP` server through two dedicated configuration interfaces: `UndertowConfigurer` and `HttpHandlerConfigurer`. +They allow configuring server behavior and the request processing pipeline without sacrificing integration with Kora's modular architecture. ## SomeController declarative { #somecontroller-declarative } @@ -224,7 +260,7 @@ The `@HttpRoute` annotation is responsible for specifying the HTTP path and meth 1. Indicates that the class is a component and should be registered in the application dependency container 2. Indicates that the class is a controller and contains HTTP handlers 3. Indicates that the method is a path handler in the controller - 4. Indicates the type of HTTP method handler + 4. Indicates the type of the handler `HTTP` method 5. Indicates the path of the handler method === ":simple-kotlin: `Kotlin`" @@ -246,18 +282,81 @@ The `@HttpRoute` annotation is responsible for specifying the HTTP path and meth 1. Indicates that the class is a component and should be registered in the application dependency container 2. Indicates that the class is a controller and contains HTTP handlers 3. Indicates that the method is a path handler in the controller - 4. Indicates the type of HTTP method handler + 4. Indicates the type of the handler `HTTP` method 5. Indicates the path of the handler method ### Request { #request } -The section describes HTTP request transformations at the controller. -It is suggested to use special annotations to specify the request parameters. +This section describes how an `HTTP` request is converted into controller method arguments. +Special annotations are used for request parts, and the request body is passed as an argument without such an annotation. + +#### String parameter conversion { #string-parameter-reader } + +Values from paths, query parameters, headers, and `cookie` arrive as strings. +Kora uses `StringParameterReader` to convert a string into the target type: + +```java +public interface StringParameterReader { + T read(String string); +} +``` + +`StringParameterReader` is looked up as a graph component by the exact parameter type. If the parameter is declared as `List` or `Set`, +the converter is applied to every value separately. + +Out of the box, Kora supports `String`, `Boolean`, `Integer`, `Long`, `Float`, `Double`, `UUID`, `BigInteger`, `BigDecimal`, +`Duration`, `LocalDate`, `LocalTime`, `LocalDateTime`, `OffsetTime`, `OffsetDateTime`, `ZonedDateTime`, and `enum`. +For `enum`, the default mapping uses the value name via `Enum.name()`. If a value cannot be converted, the request is completed +with a `400` response through `HttpServerResponseException`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserId(long value) {} + + @Module + public interface UserIdModule { + + default StringParameterReader userIdStringParameterReader() { + return StringParameterReader.of( + value -> new UserId(Long.parseLong(value)), + value -> "Invalid user id: " + value + ); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserId(val value: Long) + + @Module + interface UserIdModule { + + fun userIdStringParameterReader(): StringParameterReader { + return StringParameterReader.of( + { value -> UserId(value.toLong()) }, + { value -> "Invalid user id: $value" } + ) + } + } + ``` + +After registering the converter, the custom type can be used in controller parameters: + +```java +@HttpRoute(method = HttpMethod.GET, path = "/users/{id}") +public User get(@Path("id") UserId id) { + return userService.get(id); +} +``` #### Path parameter { #path-parameter } `@Path` - denotes the value of the request path part, the parameter itself is specified in `{path}` in the path and the name of the parameter is specified in `value` or defaults to the name of the method argument. +The value is converted through `StringParameterReader`, so both built-in and custom types can be used. ===! ":fontawesome-brands-java: `Java`" @@ -292,6 +391,8 @@ and the name of the parameter is specified in `value` or defaults to the name of #### Query parameter { #query-parameter } `@Query` - value of the query parameter, the name of the parameter is specified in `value` or is equal to the name of the method argument by default. +Single values, `List`, and `Set` are supported. `List` keeps all parameter values, +while `Set` removes duplicates and preserves the order of first occurrence. ===! ":fontawesome-brands-java: `Java`" @@ -328,6 +429,7 @@ and the name of the parameter is specified in `value` or defaults to the name of #### Request header { #request-header } `@Header` - value of [request header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers), the parameter name is specified in `value` or defaults to the method argument name. +Single values, `List`, and `Set` are supported. `List` and `Set` use all values of the header. ===! ":fontawesome-brands-java: `Java`" @@ -363,13 +465,13 @@ and the name of the parameter is specified in `value` or defaults to the name of #### Request body { #request-body } -Specifying the body of a request requires using a method argument without special annotations, -default supported types are `byte[]`, `ByteBuffer`, `String`. +Specifying the request body requires using a method argument without special annotations. +By default, `byte[]`, `ByteBuffer`, `String`, `FormUrlEncoded`, `FormMultipart`, and custom types through `HttpServerRequestMapper` are supported. -##### Json { #json } +##### JSON { #json } -In order to indicate that the body is Json and needs to automatically create such a reader and embed it, -is required to use the `@Json` annotation: +To indicate that the body is `JSON` and requires an automatically created and injected `JsonReader`, +use the `@Json` annotation: ===! ":fontawesome-brands-java: `Java`" @@ -387,7 +489,7 @@ is required to use the `@Json` annotation: } ``` - 1. Specifies that the body should be written as Json + 1. Specifies that the body should be read as `JSON` === ":simple-kotlin: `Kotlin`" @@ -405,9 +507,9 @@ is required to use the `@Json` annotation: } ``` - 1. Specifies that the body should be written as Json + 1. Specifies that the body should be read as `JSON` -Need to connect [Json](json.md) module. +The [JSON](json.md) module is required. ##### Form UrlEncoded { #form-urlencoded } @@ -476,6 +578,7 @@ You can use `FormMultipart` as the body argument type and it will be treated as #### Cookie { #cookie } `@Cookie` - [Cookie](https://developer.mozilla.org/en-US/docs/Glossary/Cookie) value, the parameter name is specified in `value` or defaults to the method argument name. +The value can be received as `String`, as a `Cookie` type with name, value, and attributes, or as another type through `StringParameterReader`. ===! ":fontawesome-brands-java: `Java`" @@ -509,7 +612,8 @@ You can use `FormMultipart` as the body argument type and it will be treated as #### Custom parameter { #custom-parameter } -In case you need to handle the request in a different way, you can use a special `HttpServerRequestMapper` interface: +If a method argument needs to be assembled from the request manually, use the `HttpServerRequestMapper` interface. +This is useful for user context, authorization, complex header validation, or several request parts at once: ===! ":fontawesome-brands-java: `Java`" @@ -554,7 +658,7 @@ In case you need to handle the request in a different way, you can use a special } @HttpRoute(method = HttpMethod.POST, path = "/hello/world") - @Mapping(UserContextRequestMapper::class) + @Mapping(RequestMapper::class) operator fun get(@Mapping(RequestMapper::class) context: UserContext): String { return "Hello World" } @@ -565,18 +669,20 @@ In case you need to handle the request in a different way, you can use a special ===! ":fontawesome-brands-java: `Java`" - By default, all arguments declared in a method are **required** (*NotNull*). + By default, all arguments declared in a method are **required**. + If a required value is missing in the request, Kora returns a `400` response. === ":simple-kotlin: `Kotlin`" - By default, all arguments declared in a method that do not use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax are **required** (*NotNull*). + By default, all method arguments that do not use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax + are **required**. If a required value is missing in the request, Kora returns a `400` response. #### Optional parameters { #optional-parameters } ===! ":fontawesome-brands-java: `Java`" - In case a method argument is optional, that is, it may not exist then, - `@Nullable` annotation can be used: + If a method argument is optional, meaning it may be missing in the request, + use `@Nullable` or `Optional` for single values: ```java @Component @@ -590,11 +696,11 @@ In case you need to handle the request in a different way, you can use a special } ``` - 1. Any `@Nullable` annotation will do, such as `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / etc. + 1. Any `@Nullable` annotation will do, for example `javax.annotation.Nullable`, `jakarta.annotation.Nullable`, or `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - It is expected to use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark such a parameter as Nullable: + Use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark such a parameter as optional: ```kotlin @Component @@ -610,9 +716,20 @@ In case you need to handle the request in a different way, you can use a special ### Response { #response } -By default, you can use standard return value types, -such as `byte[]`, `ByteBuffer`, `String` which will be processed with status code `200` and corresponding response type header -or `HttpServerResponse` where you will have to fill in all information about HTTP response yourself. +By default, standard return value types can be used: `byte[]`, `ByteBuffer`, `String`. +They are processed with status `200` and the corresponding response content type header. + +If the status, headers, or body must be specified manually, the method can return `HttpServerResponse`. +The main `HttpServerResponse` contract consists of a response code, headers, and an optional body: + +```java +public interface HttpServerResponse { + int code(); + MutableHttpHeaders headers(); + @Nullable + HttpBodyOutput body(); +} +``` ===! ":fontawesome-brands-java: `Java`" @@ -626,13 +743,13 @@ or `HttpServerResponse` where you will have to fill in all information about HTT return HttpServerResponse.of( 200, //(1)! HttpHeaders.of("headerName", "headerValue"), //(2)! - HttpBody.plaintext(body) //(3)! + HttpBody.plaintext("Hello World") //(3)! ); } } ``` - 1. HTTP status response code + 1. `HTTP` response status code 2. Response headers 3. Response body @@ -648,19 +765,20 @@ or `HttpServerResponse` where you will have to fill in all information about HTT return HttpServerResponse.of( 200, //(1)! HttpHeaders.of("headerName", "headerValue"), //(2)! - HttpBody.plaintext(body) //(3)! + HttpBody.plaintext("Hello World") //(3)! ) } } ``` - 1. HTTP status response code + 1. `HTTP` response status code 2. Response headers 3. Response body -#### Json { #json-2 } +#### JSON { #json-2 } -If you intend to respond in Json format, you are required to use the `@Json` annotation over the method: +If the response should be returned as `JSON`, use the `@Json` annotation on the method. +Kora will find or create `JsonWriter` for the response type: ===! ":fontawesome-brands-java: `Java`" @@ -679,7 +797,7 @@ If you intend to respond in Json format, you are required to use the `@Json` ann } ``` - 1. Specifies that the response should be in Json format + 1. Specifies that the response should be in `JSON` format === ":simple-kotlin: `Kotlin`" @@ -698,16 +816,16 @@ If you intend to respond in Json format, you are required to use the `@Json` ann } ``` - 1. Specifies that the response should be in Json format + 1. Specifies that the response should be in `JSON` format -[Json](json.md) module is required. +The [JSON](json.md) module is required. #### Response entity { #response-entity } -If the intention is to read the body and also get the headers and status code of the response, -then the `HttpResponseEntity` is supposed to be used, it is a wrapper over the response body. +If the body, headers, and response status code should be returned together, +use `HttpResponseEntity`, a wrapper around the response body. -Below is an example similar to the Json example along with the `HttpResponseEntity` wrapper: +Below is an example similar to the `JSON` example with the `HttpResponseEntity` wrapper: ===! ":fontawesome-brands-java: `Java`" @@ -745,7 +863,11 @@ Below is an example similar to the Json example along with the `HttpResponseEnti #### Respond exception { #respond-exception } -If you need to respond with an error, you can use `HttpServerResponseException` to throw an exception. +If processing should be interrupted and an error should be returned immediately, throw `HttpServerResponseException`. +It is both an exception and an `HttpServerResponse`, so it can be thrown from a controller, service, or parameter converter. + +The `HttpServerResponseException.of(...)` factory methods allow specifying the status code, response text, cause, and headers. +The response body is written as `text/plain; charset=utf-8`. ===! ":fontawesome-brands-java: `Java`" @@ -783,7 +905,8 @@ If you need to respond with an error, you can use `HttpServerResponseException` #### Custom response { #custom-response } -In case you need to read the response in a different way, you can use the special `HttpServerResponseMapper` interface: +If the response needs to be created in a custom way, use the `HttpServerResponseMapper` interface. +It receives `Context`, the original `HttpServerRequest`, and the controller method result, and returns a ready `HttpServerResponse`: ===! ":fontawesome-brands-java: `Java`" @@ -835,7 +958,7 @@ In case you need to read the response in a different way, you can use the specia ### Signatures { #signatures } -Available signatures for repository methods out of the box: +Available signatures for declarative `HTTP` handler methods out of the box: ===! ":fontawesome-brands-java: `Java`" @@ -854,13 +977,28 @@ Available signatures for repository methods out of the box: ## Interceptors { #interceptors } -You can create interceptors to change behavior or create additional behavior using the `HttpServerInterceptor` class. +Interceptors can be created to change behavior or add shared logic around request processing. +Use the `HttpServerInterceptor` interface: + +```java +public interface HttpServerInterceptor { + CompletionStage intercept(Context context, HttpServerRequest request, InterceptChain chain) throws Exception; + + interface InterceptChain { + CompletionStage process(Context ctx, HttpServerRequest request) throws Exception; + } +} +``` + +An interceptor receives the current `Context`, `HttpServerRequest`, and the chain of further processing. +To pass the request further, call `chain.process(context, request)`. If the interceptor returns a response itself, +the controller handler is not called. Interceptors can be used on: - Specific controller methods - Entire controller -- All controllers at once (requires using `@Tag(HttpServerModule.class)` over the interceptor class) (there can be only one such interceptor). +- All controllers at once: register the interceptor component with the `@Tag(HttpServerModule.class)` tag; there can be several global interceptors ===! ":fontawesome-brands-java: `Java`" @@ -915,8 +1053,8 @@ Interceptors can be used on: ### Error handling { #error-handling } -Error handling at the level of all HTTP responses can also be realized by means of an interceptor, -below is a simple example of such an interceptor. +Error handling for all `HTTP` responses can also be implemented through an interceptor. +Below is a simple example of such an interceptor. ===! ":fontawesome-brands-java: `Java`" @@ -1006,7 +1144,7 @@ The following example shows how to handle all the described declarative request } ``` - 1. Specifies the HTTP method type of the handler method + 1. Specifies the `HTTP` method type of the handler method 2. Indicates the path of the handler method === ":simple-kotlin: `Kotlin`" @@ -1030,5 +1168,485 @@ The following example shows how to handle all the described declarative request } ``` - 1. Specifies the HTTP method type of the handler method + 1. Specifies the `HTTP` method type of the handler method 2. Indicates the path of the handler method + +## Authorization { #authorization } + +Kora provides a mechanism for extracting authorization context from HTTP requests via the `HttpServerPrincipalExtractor` interface. +This interface allows implementing any authentication scheme: [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/). + +### How It Works { #how-it-works } + +`HttpServerPrincipalExtractor` extracts a token from the request (usually from the `Authorization` header) and returns a `Principal` object. +The obtained `Principal` is stored in the request `Context` and can be retrieved anywhere during request processing via `Principal.current()`. + +```java +public interface HttpServerPrincipalExtractor { + CompletionStage extract(HttpServerRequest request, @Nullable String value); +} +``` + +Where: +- `request` — the current HTTP request, from which additional data (headers, parameters) can be extracted +- `value` — the token value extracted from the `Authorization` header (or another source) +- `T extends Principal` — the type of authorization context that will be stored in the `Context` + +### Basic Example { #basic-example } + +Simple example of extracting an API key from the `Authorization` header: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface AuthModule { + + @ConfigSource("auth.apiKey") + interface ApiKeyAuthConfig { + String value(); + } + + default HttpServerPrincipalExtractor apiKeyExtractor(ApiKeyAuthConfig config) { + return (request, value) -> { + if (value == null || !config.value().equals(value)) { + return CompletableFuture.failedFuture( + new IllegalAccessException("Invalid API key") + ); + } + return CompletableFuture.completedFuture( + new Principal() { + @Override + public String name() { + return "api-client"; + } + } + ); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface AuthModule { + + @ConfigSource("auth.apiKey") + interface ApiKeyAuthConfig { + fun value(): String + } + + fun apiKeyExtractor(config: ApiKeyAuthConfig): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || config.value() != value) { + return@HttpServerPrincipalExtractor CompletableFuture.failedFuture( + IllegalAccessException("Invalid API key") + ) + } + CompletableFuture.completedFuture( + object : Principal { + override fun name() = "api-client" + } + ) + } + } + } + ``` + +### Custom Principal { #custom-principal } + +To pass additional authorization information (userId, roles, scope), create a custom `Principal` implementation: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserContext(String userId, List roles) implements Principal {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserContext(val userId: String, val roles: List) : Principal + ``` + +If scope handling is required, use the `PrincipalWithScopes` interface: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record ScopedUser(String userId, Collection scopes) implements PrincipalWithScopes {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class ScopedUser(val userId: String, val scopes: Collection) : PrincipalWithScopes + ``` + +### Bearer Token { #bearer } + +Example of extracting a Bearer token with a custom `Principal` implementation: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface BearerAuthModule { + + default HttpServerPrincipalExtractor bearerExtractor(TokenValidator validator) { + return (request, value) -> { + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + new IllegalAccessException("No Bearer token") + ); + } + + String token = value.substring(7); + return validator.validate(token) + .thenApply(userData -> new UserContext(userData.userId(), userData.roles())); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface BearerAuthModule { + + fun bearerExtractor(validator: TokenValidator): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + IllegalAccessException("No Bearer token") + ) + } + + val token = value.substring(7) + validator.validate(token) + .thenApply { userData -> + UserContext(userData.userId, userData.roles) + } + } + } + } + ``` + +### Getting Principal in Controller { #getting-principal } + +The current authorization context can be obtained anywhere during request processing: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController + public class SecureController { + + @HttpRoute(method = HttpMethod.GET, path = "/secure") + public String getSecureData() { + Principal principal = Principal.current(); + if (principal instanceof UserContext user) { + return "Hello, user: " + user.userId(); + } + throw new SecurityException("Not authenticated"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController + class SecureController { + + @HttpRoute(method = HttpMethod.GET, path = "/secure") + fun getSecureData(): String { + val principal = Principal.current() + return if (principal is UserContext) { + "Hello, user: ${principal.userId}" + } else { + throw SecurityException("Not authenticated") + } + } + } + ``` + +### OAuth2 { #oauth2 } + +For OAuth2 authorization, create an `HttpServerPrincipalExtractor` that validates the token via an OAuth2 provider: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface OAuth2Module { + + default HttpServerPrincipalExtractor oauth2Extractor(OAuth2Client oauth2Client) { + return (request, value) -> { + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + new IllegalAccessException("No OAuth2 token") + ); + } + + String token = value.substring(7); + return oauth2Client.introspect(token) + .thenApply(introspection -> + new ScopedUser( + introspection.subject(), + introspection.scopes() + ) + ); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface OAuth2Module { + + fun oauth2Extractor(oauth2Client: OAuth2Client): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + IllegalAccessException("No OAuth2 token") + ) + } + + val token = value.substring(7) + oauth2Client.introspect(token) + .thenApply { introspection -> + ScopedUser(introspection.subject, introspection.scopes) + } + } + } + } + ``` + +### Scope Checking in Interceptor { #scope-check } + +To check scopes, create an interceptor that validates `PrincipalWithScopes`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ScopeCheckingInterceptor implements HttpServerInterceptor { + + private final String requiredScope; + + public ScopeCheckingInterceptor(@ConfigSource("auth.requiredScope") String requiredScope) { + this.requiredScope = requiredScope; + } + + @Override + public CompletionStage intercept(Context context, + HttpServerRequest request, + InterceptChain chain) { + Principal principal = Principal.current(context); + if (principal instanceof PrincipalWithScopes scoped) { + if (!scoped.scopes().contains(requiredScope)) { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "Insufficient scope") + ); + } + } else { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "No scopes available") + ); + } + + return chain.process(context, request); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ScopeCheckingInterceptor( + @ConfigSource("auth.requiredScope") private val requiredScope: String + ) : HttpServerInterceptor { + + override fun intercept( + context: Context, + request: HttpServerRequest, + chain: HttpServerInterceptor.InterceptChain + ): CompletionStage { + val principal = Principal.current(context) + if (principal is PrincipalWithScopes) { + if (!principal.scopes.contains(requiredScope)) { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "Insufficient scope") + ) + } + } else { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "No scopes available") + ) + } + + return chain.process(context, request) + } + } + ``` + +### OpenAPI Integration { #openapi } + +When using Kora OpenAPI Generator, authorization is configured automatically based on the OpenAPI specification. +The generator creates: + +1. `ApiSecurity` interface with marker classes for each authorization type +2. `HttpServerInterceptor` for each security scheme +3. Requires providing an `HttpServerPrincipalExtractor` with the corresponding `@Tag` + +Example from [kora-examples](https://github.com/kora-projects/kora-examples): + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends + HoconConfigModule, + UndertowHttpServerModule, + JsonModule { + + @Tag(ApiSecurity.ApiKeyAuth.class) + default HttpServerPrincipalExtractor apiKeyExtractor(DataApiAuthConfig config) { + return (request, value) -> { + if (value == null || !config.value().equals(value)) { + throw new SecurityException("Invalid API key"); + } + return CompletableFuture.completedFuture( + new DataApiPrincipal("data-api-client") + ); + }; + } + } + ``` + + where `DataApiPrincipal`: + + ```java + public record DataApiPrincipal(String name) implements Principal {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : + HoconConfigModule, + UndertowHttpServerModule, + JsonModule { + + @Tag(ApiSecurity.ApiKeyAuth::class) + fun apiKeyExtractor(config: DataApiAuthConfig): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || config.value() != value) { + throw SecurityException("Invalid API key") + } + CompletableFuture.completedFuture( + DataApiPrincipal("data-api-client") + ) + } + } + } + ``` + + where `DataApiPrincipal`: + + ```kotlin + data class DataApiPrincipal(val name: String) : Principal + ``` + +Configuration: + +```hocon +auth.apiKey { + value = "secret-api-key-123" +} +``` + +### Error Handling { #error-handling } + +If `HttpServerPrincipalExtractor` throws an exception or returns `null`, the request is rejected with `403 Forbidden`. +For custom authorization error handling, use an interceptor: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(HttpServerModule.class) + @Component + public final class AuthErrorInterceptor implements HttpServerInterceptor { + + @Override + public CompletionStage intercept(Context context, + HttpServerRequest request, + InterceptChain chain) { + return chain.process(context, request).exceptionally(e -> { + if (e instanceof CompletionException) { + e = e.getCause(); + } + if (e instanceof IllegalAccessException) { + return HttpServerResponse.of(401, HttpBody.plaintext("Unauthorized: " + e.getMessage())); + } + if (e instanceof SecurityException) { + return HttpServerResponse.of(403, HttpBody.plaintext("Forbidden: " + e.getMessage())); + } + throw new CompletionException(e); + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(HttpServerModule::class) + @Component + class AuthErrorInterceptor : HttpServerInterceptor { + + override fun intercept( + context: Context, + request: HttpServerRequest, + chain: HttpServerInterceptor.InterceptChain + ): CompletionStage { + return chain.process(context, request).exceptionally { e -> + val error = if (e is CompletionException) e.cause!! else e + when (error) { + is IllegalAccessException -> + HttpServerResponse.of(401, HttpBody.plaintext("Unauthorized: ${error.message}")) + is SecurityException -> + HttpServerResponse.of(403, HttpBody.plaintext("Forbidden: ${error.message}")) + else -> throw CompletionException(error) + } + } + } + } + ``` + +## Telemetry { #telemetry } + +HTTP Server uses a telemetry contract for logging, metrics, and tracing of requests. +Telemetry configuration (section `telemetry { logging / metrics / tracing }`) is described in the [Configuration](#configuration) section. +Extension points are located in `ru.tinkoff.kora.http.server.common.telemetry`. + +For each HTTP request, an `HttpServerTelemetry.HttpServerTelemetryContext` is created, which is closed upon request completion. +The request is described through telemetry handler parameters, including method, path, response status, and duration. + +The default factory `DefaultHttpServerTelemetryFactory` combines three factories: +- `HttpServerLoggerFactory` builds `HttpServerLogger` for logging request start/end; +- `HttpServerMetricsFactory` builds `HttpServerMetrics` for writing request metrics; +- `HttpServerTracerFactory` builds `HttpServerTracer` for distributed tracing. + +Metrics and tracing are described in the [Metrics Reference](metrics.md#http-server) section. diff --git a/mkdocs/docs/en/documentation/json.md b/mkdocs/docs/en/documentation/json.md index 963629d..45a6a9e 100644 --- a/mkdocs/docs/en/documentation/json.md +++ b/mkdocs/docs/en/documentation/json.md @@ -1,10 +1,14 @@ --- -description: "Explains Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration. Use when working with @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonIgnore, JsonNullable, JacksonModule." +description: "Explains Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration. Use when working with @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonSkip, JsonNullable, JacksonModule." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration; key triggers include @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonIgnore, JsonNullable, JacksonModule." + use_when: "Use this file for Kora docs or implementation questions about Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration; key triggers include @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonSkip, JsonNullable, JacksonModule." --- -Module allows you to create productive and reflection-free JSON readers and writers for application classes using annotations. +The `JSON` module creates efficient `JsonReader` and `JsonWriter` implementations for application classes at compile time and without using `Reflection` at runtime. +Generation is controlled by `@Json`, `@JsonReader`, `@JsonWriter`, and related field-level annotations. + +`JsonModule` also provides ready-to-use mappers for `HTTP` client, `HTTP` server, string parameters, and `Kafka`. +This allows the same generated `JsonReader` or `JsonWriter` to be used across different Kora modules. For a step-by-step walkthrough before the reference details, see [JSON](../guides/json.md). @@ -12,7 +16,7 @@ For a step-by-step walkthrough before the reference details, see [JSON](../guide ===! ":fontawesome-brands-java: `Java`" - [Dependency](general.md#dependencies) `build.gradle`: + [Dependency](general.md#dependencies) in `build.gradle`: ```groovy implementation "ru.tinkoff.kora:json-module" ``` @@ -25,7 +29,7 @@ For a step-by-step walkthrough before the reference details, see [JSON](../guide === ":simple-kotlin: `Kotlin`" - [Dependency](general.md#dependencies) `build.gradle.kts`: + [Dependency](general.md#dependencies) in `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:json-module") ``` @@ -38,7 +42,8 @@ For a step-by-step walkthrough before the reference details, see [JSON](../guide ## Writer { #writer } -You can use `@JsonWriter` to create a writer only: +Use `@JsonWriter` to create only a `JsonWriter`. +This option is useful when the type only needs to be written to `JSON`: ===! ":fontawesome-brands-java: `Java`" @@ -56,7 +61,8 @@ You can use `@JsonWriter` to create a writer only: ## Reader { #reader } -You can use `@JsonReader` to create a reader only: +Use `@JsonReader` to create only a `JsonReader`. +This option is useful when the type only needs to be read from `JSON`: ===! ":fontawesome-brands-java: `Java`" @@ -74,8 +80,8 @@ You can use `@JsonReader` to create a reader only: ## Reader & Writer { #reader-and-writer } -You can use `@Json` to create a reader and a writer at once. -In most cases, it is the `@Json` annotation that is preferred: +Use `@Json` to create both `JsonReader` and `JsonWriter`. +In most cases, `@Json` is the preferred annotation: ===! ":fontawesome-brands-java: `Java`" @@ -91,11 +97,71 @@ In most cases, it is the `@Json` annotation that is preferred: data class Dto(val field1: String, val field2: Int) ``` +## Reader And Writer Interfaces { #reader-writer-interfaces } + +`JsonReader` and `JsonWriter` are regular application graph components. +After generation or manual registration, they can be injected by signature like any other dependency. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MyService { + + private final JsonReader reader; + private final JsonWriter writer; + + public MyService(JsonReader reader, JsonWriter writer) { + this.reader = reader; + this.writer = writer; + } + + public Dto read(String json) throws IOException { + return this.reader.read(json); + } + + public byte[] write(Dto dto) throws IOException { + return this.writer.toByteArray(dto); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyService( + private val reader: JsonReader, + private val writer: JsonWriter + ) { + + fun read(json: String): Dto? { + return reader.read(json) + } + + fun write(dto: Dto): ByteArray { + return writer.toByteArray(dto) + } + } + ``` + +`JsonReader` reads a value from `JsonParser`, `byte[]`, `String`, or `InputStream`. +The `readUnchecked(...)` methods do the same, but convert `IOException` to `UncheckedIOException`. + +`JsonWriter` writes a value through `JsonGenerator` and can also return `byte[]`, a string, or a formatted string through `toByteArray(...)`, `toString(...)`, and `toPrettyString(...)`. +The `toByteArrayUnchecked(...)`, `toStringUnchecked(...)`, and `toPrettyStringUnchecked(...)` methods convert `IOException` to `UncheckedIOException`. + +Runtime behavior worth noting when calling the codecs directly: + +- `read(...)` returns `null` when the parser is positioned on a `JSON` `null` token, so a top-level `null` document deserializes to `null`. +- Malformed `JSON` or an unexpected token surfaces as a `Jackson` `JsonParseException`, which is a subtype of `IOException`. +- The `readUnchecked(...)` and `to...Unchecked(...)` variants rethrow any `IOException` (including `JsonParseException`) wrapped in `UncheckedIOException`. + ## Required fields { #required-fields } ===! ":fontawesome-brands-java: `Java`" - By default, all fields declared in an object are considered **required** (*NotNull*). + By default, all fields declared in an object are considered **required** (`NotNull`). ```java @Json @@ -104,7 +170,7 @@ In most cases, it is the `@Json` annotation that is preferred: === ":simple-kotlin: `Kotlin`" - By default, all fields declared in an object that do not use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax are considered **required** (*NotNull*). + By default, all fields declared in an object without [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax are considered **required** (`NotNull`). ```kotlin @Json @@ -115,8 +181,7 @@ In most cases, it is the `@Json` annotation that is preferred: ===! ":fontawesome-brands-java: `Java`" - In case a field in Json is optional, that is, it may not exist then, - you can use the `@Nullable` annotation to match the field in Json and DTO: + If a `JSON` field is optional and can be absent, use the `@Nullable` annotation: ```java @Json @@ -124,11 +189,11 @@ In most cases, it is the `@Json` annotation that is preferred: int field2) { } ``` - 1. Any `@Nullable` annotation will do, such as `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / etc. + 1. Any `@Nullable` annotation is suitable, for example `javax.annotation.Nullable`, `jakarta.annotation.Nullable`, or `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - It is expected to use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark such a parameter as Nullable: + For `Kotlin`, use [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark the parameter as `nullable`: ```kotlin @Json @@ -138,10 +203,10 @@ In most cases, it is the `@Json` annotation that is preferred: ) ``` -## Field naming { #field-naming } +## Field Naming { #field-naming } -In case a field in Json is named differently from what you want to use in a class, -you can use the `@JsonField` annotation to match the field in Json and the DTO. +If a field in `JSON` has a different name than the field in the class, use `@JsonField`. +It sets the key name in `JSON` and also allows specifying separate `JsonReader` and `JsonWriter` implementations for a field. ===! ":fontawesome-brands-java: `Java`" @@ -161,10 +226,36 @@ you can use the `@JsonField` annotation to match the field in Json and the DTO. ) ``` -## Field ignore { #field-ignore } +If a field needs separate mappers, specify them in `reader` and `writer`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record Dto(@JsonField(value = "created_at", + reader = InstantJsonReader.class, + writer = InstantJsonWriter.class) + Instant createdAt) { } + ``` -In case you don't want to read/write a field in DTO, -you can use the `@JsonSkip` annotation and ignore such a field. +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class Dto( + @field:JsonField( + value = "created_at", + reader = InstantJsonReader::class, + writer = InstantJsonWriter::class + ) + val createdAt: Instant + ) + ``` + +## Field Ignore { #field-ignore } + +If a field in a `DTO` should not be read or written, use `@JsonSkip`. +Such a field is ignored when reading and writing `JSON`. ===! ":fontawesome-brands-java: `Java`" @@ -184,29 +275,29 @@ you can use the `@JsonSkip` annotation and ignore such a field. ) ``` -## Serialization levels { #serialization-levels } +## Serialization Levels { #serialization-levels } -The default behavior is not to write fields with `null` values. (1) +By default, fields with `null` values are not written. (1) { .annotate } -1. `IncludeType.NON_NULL` - include the field in the record if not `null`. +1. `IncludeType.NON_NULL` - write the field only if the value is not `null`. -In case you want to change the behavior of the record in these moments, it is suggested to use the `@JsonInclude` annotation. -The annotation can be used not only over a field, but also over a class and then the rule will apply to all fields at once. +To change this behavior, use `@JsonInclude`. +The annotation can be placed not only on a field, but also on a class; in that case, the rule applies to all fields at once. -Various use cases are available: +Available options: -- `IncludeType.ALWAYS` - include the field in the record always -- `IncludeType.NON_NULL` - include the field in the record if it is not `null`. -- `IncludeType.NON_EMPTY` - include the field in the record if it is not `null` and not an empty collection +- `IncludeType.ALWAYS` - always write the field. +- `IncludeType.NON_NULL` - write the field if the value is not `null`. +- `IncludeType.NON_EMPTY` - write the field if the value is not `null` and is not an empty collection or map. -Example of annotation usage: +Example: ===! ":fontawesome-brands-java: `Java`" ```java @Json - @JsonInclude(IncludeType.NOT_NULL) + @JsonInclude(IncludeType.NON_NULL) public record Dto(@JsonInclude(IncludeType.ALWAYS) @Nullable String field1, int field2) { } ``` @@ -221,10 +312,10 @@ Example of annotation usage: ) ``` -## Serialization constructor { #serialization-constructor } +## Serialization Constructor { #serialization-constructor } -If you want to use a specific constructor for serialization, -it can be done by specifying the `@JsonReader` annotation above the constructor or the lower-priority `@Json` annotation: +If a specific constructor should be used for reading `JSON`, annotate it with `@JsonReader`. +You can also use `@Json`, but `@JsonReader` has higher priority: ===! ":fontawesome-brands-java: `Java`" @@ -250,10 +341,58 @@ it can be done by specifying the `@JsonReader` annotation above the constructor } ``` -## JsonNullable wrapper { #jsonnullable-wrapper } +`JsonReader` and `JsonWriter` can be generated for classes, `record`, `enum`, and `sealed` types. +For reading a class, there must be one public constructor or a constructor explicitly annotated with `@JsonReader` or `@Json`. + +### Java Bean and plain classes { #java-bean } -In case you want to distinguish a missing field from a specified `null` value during deserialization, -it is supposed to use a special type `JsonNullable`, which allows interpreting all states of the field. +`@Json`, `@JsonReader`, and `@JsonWriter` are not limited to `record` and `data class`. +A plain class works too: reading requires a single public constructor (or one annotated with `@JsonReader`/`@Json`), and writing uses the field accessors. +`@JsonField` may be placed on private fields to rename the `JSON` key: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @JsonWriter + public class DtoJavaBean { + + @JsonField("string_field") + private String field1; + @JsonField("int_field") + private int field2; + + public DtoJavaBean(String field1, int field2) { + this.field1 = field1; + this.field2 = field2; + } + + public String getField1() { return field1; } + + public int getField2() { return field2; } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @JsonWriter + class DtoJavaBean( + @field:JsonField("string_field") val field1: String, + @field:JsonField("int_field") val field2: Int + ) + ``` + +## JsonNullable Wrapper { #jsonnullable-wrapper } + +If reading `JSON` must distinguish an absent field from a field with a `null` value, use `JsonNullable`. +Main states and factory methods: + +- `JsonNullable.undefined()` - the field is absent in `JSON`. +- `JsonNullable.nullValue()` - the field is present and contains `null`. +- `JsonNullable.of(value)` - the field is present and contains a value. +- `JsonNullable.ofNullable(value)` - creates `nullValue()` if the value is `null`, otherwise `of(value)`. + +When writing `JSON`, `undefined()` is skipped, `nullValue()` is written as `null`, and `of(value)` writes the value itself. ===! ":fontawesome-brands-java: `Java`" @@ -269,16 +408,83 @@ it is supposed to use a special type `JsonNullable`, which allows interpreting a data class Dto(val field1: String, val field2: JsonNullable) ``` -## Sealed classes and interfaces { #sealed-classes-and-interfaces } +### @Nullable vs JsonNullable { #nullable-vs-jsonnullable } -In case you need to write different Json objects depending on the value in a particular field, you are supposed to use an -[isolated class/interface](https://habr.com/ru/companies/otus/articles/720044/) to represent such objects. +A plain [optional field](#optional-fields) (`@Nullable` in `Java` or a nullable type in `Kotlin`) collapses two different `JSON` inputs into the same value: a field that is **absent** and a field that is present with an explicit `null` both read as `null`. +`JsonNullable` keeps these apart, which is what makes it the correct type for `HTTP` `PATCH` bodies where the client sends only the fields it actually wants to change. -Two annotations are added to support isolated classes: +The three read outcomes for a `JsonNullable` field: + +| `JSON` input | Read result | `isDefined()` | `isNull()` | `value()` | +|------------------------|---------------------------|---------------|------------|---------------| +| `{}` (field absent) | `JsonNullable.undefined()`| `false` | `false` | throws | +| `{"field": null}` | `JsonNullable.nullValue()`| `true` | `true` | `null` | +| `{"field": value}` | `JsonNullable.of(value)` | `true` | `false` | `value` | + +Because `value()` throws on `undefined()`, always guard access with `isDefined()` (or check `isNull()`) before calling it. + +### PATCH partial update { #jsonnullable-patch } + +In a `PATCH` request, an absent field means "leave unchanged" while an explicit `null` means "clear the value". +`JsonNullable` lets the handler tell the two apart and apply only the fields the client actually sent: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record UserPatch(JsonNullable name, + JsonNullable email) { } + + public void apply(User user, UserPatch patch) { + if (patch.name().isDefined()) { //(1)! + user.setName(patch.name().value()); + } + if (patch.email().isDefined()) { + user.setEmail(patch.email().value()); //(2)! + } + // fields left as undefined() are not touched + } + ``` + + 1. The field was present in the request body, so it must be applied (even if the value is an explicit `null`). + 2. `value()` returns `null` when the client sent `{"email": null}`, which clears the field. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class UserPatch( + val name: JsonNullable, + val email: JsonNullable + ) + + fun apply(user: User, patch: UserPatch) { + if (patch.name.isDefined()) { //(1)! + user.name = patch.name.value() + } + if (patch.email.isDefined()) { + user.email = patch.email.value() //(2)! + } + // fields left as undefined() are not touched + } + ``` + + 1. The field was present in the request body, so it must be applied (even if the value is an explicit `null`). + 2. `value()` returns `null` when the client sent `{"email": null}`, which clears the field. + +Interaction with [serialization levels](#serialization-levels): `IncludeType.ALWAYS` and `IncludeType.NON_NULL` do **not** change how `JsonNullable` is written (its own `undefined`/`nullValue`/`of` rules apply). +Only `IncludeType.NON_EMPTY` affects `JsonNullable`, treating an `undefined()` or `nullValue()` field as empty so it is omitted from the output. + +## Sealed Classes And Interfaces { #sealed-classes-and-interfaces } + +If different `JSON` objects should be read and written depending on a specific field value, use a +[sealed class or interface](https://kotlinlang.org/docs/sealed-classes.html) to represent those objects. + +Two annotations support sealed types: + +1. `@JsonDiscriminatorField` - specifies the discriminator field in the `DTO` marked as a `sealed` class or interface. +2. `@JsonDiscriminatorValue` - specifies one or more discriminator values for a subclass. -1. `@JsonDiscriminatorField` - specifies the discriminator field in the DTO with which the sealed class/interface is tagged -2. `@JsonDiscriminatorValue` - the value for the above field, marks the inheritor class of the sealed class/interface -3. ===! ":fontawesome-brands-java: `Java`" ```java @@ -315,9 +521,11 @@ Two annotations are added to support isolated classes: } ``` -A `JsonReader` and `JsonWriter` will be created for the inheritor classes using the same rules as if they had the `@Json` annotation on them and a `JsonReader` and `JsonWriter` will be created for the sealed class/interface itself. +Subclasses receive `JsonReader` and `JsonWriter` by the same rules as if they were annotated with `@Json`. +The `sealed` class or interface itself also receives a common `JsonReader` and `JsonWriter`. +Nested `sealed` hierarchies are supported, and `@JsonDiscriminatorValue` can accept multiple values for one subclass. -The Json object below will be written to the `FirstTypeEvent` class: +The `JSON` object below is written to the `FirstTypeEvent` class: ```json { "id": "1", @@ -328,9 +536,139 @@ The Json object below will be written to the `FirstTypeEvent` class: } ``` -## Supported types { #supported-types } +Generic `DTO` types are supported, including generic `sealed` hierarchies. +The codec for each concrete type argument is resolved from the graph like any other field type: -Module provides an extensive list of supported out-of-the-box types that cover most of what you might need. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + @JsonDiscriminatorField("@type") + public sealed interface Response { + + @JsonDiscriminatorValue("ok") + record Ok(T data) implements Response {} + + @JsonDiscriminatorValue("fail") + record Fail(String error) implements Response {} + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + @JsonDiscriminatorField("@type") + sealed interface Response { + + @JsonDiscriminatorValue("ok") + data class Ok(val data: T) : Response + + @JsonDiscriminatorValue("fail") + data class Fail(val error: String) : Response + } + ``` + +## Enums { #enum } + +For `enum`, `JsonReader` and `JsonWriter` can be generated with the same `@Json`, `@JsonReader`, and `@JsonWriter` annotations. +By default, the `enum` value in `JSON` is the result of `toString()`, so it can be overridden: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public enum Status { + CREATED, + DELETED; + + @Override + public String toString() { + return this.name().toLowerCase(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + enum class Status { + CREATED, + DELETED; + + override fun toString(): String { + return name.lowercase() + } + } + ``` + +If a value other than the string from `toString()` is needed, annotate a public parameterless method with `@Json`. +In that case, a corresponding `JsonReader` and `JsonWriter` must be available for the return type: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public enum Status { + CREATED(1), + DELETED(2); + + private final int code; + + Status(int code) { + this.code = code; + } + + @Json + public int code() { + return this.code; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + enum class Status(private val code: Int) { + CREATED(1), + DELETED(2); + + @Json + fun code(): Int = code + } + ``` + +When reading, a `JSON` value that does not match any `enum` constant throws a `Jackson` `JsonParseException` that lists the accepted values. + +## RawJson { #raw-json } + +`RawJson` is used when an object needs to include an already prepared `JSON` fragment without serializing it again. +When written, `RawJson` is passed to the output `JSON` as is, so the value must be a valid `JSON` fragment. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record Dto(String id, RawJson payload) { } + + var dto = new Dto("1", new RawJson("{\"status\":\"ok\"}")); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class Dto(val id: String, val payload: RawJson) + + val dto = Dto("1", RawJson("""{"status":"ok"}""")) + ``` + +## Supported Types { #supported-types } + +The module provides built-in types that cover most common tasks. +For collections and maps, Kora uses the `JsonReader` or `JsonWriter` of the element type. ??? abstract "List of supported types" @@ -351,11 +689,17 @@ Module provides an extensive list of supported out-of-the-box types that cover m * UUID * BigInteger * BigDecimal - * List - * Set + * RawJson + * Object + * Enum + * List + * Set + * SortedSet + * Map * LocalDate * LocalTime * LocalDateTime + * Instant * OffsetTime * OffsetDateTime * ZonedDateTime @@ -367,11 +711,11 @@ Module provides an extensive list of supported out-of-the-box types that cover m * ZoneId * Duration -### Custom types { #custom-types } +### Custom Types { #custom-types } -In case you need to write/read your custom type, it is suggested to register your custom [factory](container.md) for `JsonReader` / `JsonWriter`: +If a custom type must be read or written, register a custom [factory](container.md) for `JsonReader` or `JsonWriter`. -Example of registering a `JsonWriter`: +Example of registering a custom `JsonWriter`: ===! ":fontawesome-brands-java: `Java`" @@ -381,7 +725,7 @@ Example of registering a `JsonWriter`: default JsonWriter zoneOffsetJsonWriter() { return (generator, value) -> { - if(value != null) { + if (value != null) { generator.writeString(value.getId()); } }; @@ -405,35 +749,92 @@ Example of registering a `JsonWriter`: } ``` +Example of registering a custom `JsonReader`. +The reader switches on the current parser token, returns `null` on a `JSON` `null`, reads the expected token, and throws a `JsonParseException` for anything else: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application { + + default JsonReader zoneOffsetJsonReader() { + return parser -> switch (parser.currentToken()) { + case VALUE_NULL -> null; + case VALUE_STRING -> ZoneOffset.of(parser.getValueAsString()); + default -> throw new JsonParseException(parser, + "Expecting VALUE_STRING token, got " + parser.currentToken()); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application { + + fun zoneOffsetJsonReader(): JsonReader = JsonReader { parser -> + when (parser.currentToken()) { + JsonToken.VALUE_NULL -> null + JsonToken.VALUE_STRING -> ZoneOffset.of(parser.valueAsString) + else -> throw JsonParseException(parser, + "Expecting VALUE_STRING token, got ${parser.currentToken()}") + } + } + } + ``` + +A custom `JsonReader` or `JsonWriter` is an ordinary graph component. +Once registered, generated codecs pick it up automatically wherever a field of type `T` occurs, and it can also be pinned to a single field through `@JsonField(reader = ..., writer = ...)` (see [Field Naming](#field-naming)). + ## Jackson { #jackson } -In case one wants to use `Jackson` for writing/reading, one can register [factory](container.md) that -provide `ObjectMapper` and the corresponding `Mappers` that are required in other Kora modules will be provided by the dependency below: +If `Jackson` must be used for reading and writing `JSON` instead of the compile-time generated codecs, use `JacksonModule`. +It replaces the `HTTP` client and `HTTP` server request/response mappers with `Jackson`-backed ones. + +Every `JacksonModule` mapper depends on an `ObjectMapper` component, so a [factory](container.md) that supplies `ObjectMapper` **must** be present in the graph. Without it the graph fails to build. ===! ":fontawesome-brands-java: `Java`" - [Dependency](general.md#dependencies) `build.gradle`: + [Dependency](general.md#dependencies) in `build.gradle`: ```groovy annotationProcessor "ru.tinkoff.kora:json-annotation-processor" implementation "ru.tinkoff.kora:jackson-module" ``` - Module: + Module and `ObjectMapper` factory: ```java @KoraApp - public interface Application extends JacksonModule { } + public interface Application extends JacksonModule { + + default ObjectMapper objectMapper() { //(1)! + return new ObjectMapper(); + } + } ``` + 1. Required by all `JacksonModule` mappers; configure it as needed (modules, features, and so on). + === ":simple-kotlin: `Kotlin`" - [Dependency](general.md#dependencies) `build.gradle.kts`: + [Dependency](general.md#dependencies) in `build.gradle.kts`: ```groovy ksp("ru.tinkoff.kora:json-annotation-processor") implementation("ru.tinkoff.kora:jackson-module") ``` - Module: + Module and `ObjectMapper` factory: ```kotlin @KoraApp - interface Application : JacksonModule + interface Application : JacksonModule { + + fun objectMapper(): ObjectMapper = ObjectMapper() //(1)! + } ``` + + 1. Required by all `JacksonModule` mappers; configure it as needed (modules, features, and so on). + +The `json-annotation-processor` shown above lets `@Json`, `@JsonReader`, and `@JsonWriter` continue to generate codecs, so generated and `Jackson` serialization can coexist (for example, `Jackson` for `HTTP` and generated codecs for [Kafka](kafka.md)). +The `JacksonModule` `HTTP` mappers themselves depend only on the `ObjectMapper`. diff --git a/mkdocs/docs/en/documentation/junit5.md b/mkdocs/docs/en/documentation/junit5.md index 5c8d725..ee73990 100644 --- a/mkdocs/docs/en/documentation/junit5.md +++ b/mkdocs/docs/en/documentation/junit5.md @@ -1,23 +1,23 @@ --- -description: "Explains Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization. Use when working with @KoraAppTest, @TestComponent, @MockComponent, @Tag, @TestConfig, @TestConfigSource, Graph, Mockito." +description: "Explains Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization. Use when working with @KoraAppTest, @TestComponent, @Tag, KoraAppTestConfigModifier, Graph, Mockito." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization; key triggers include @KoraAppTest, @TestComponent, @MockComponent, @Tag, @TestConfig, @TestConfigSource, Graph, Mockito." + use_when: "Use this file for Kora docs or implementation questions about Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization; key triggers include @KoraAppTest, @TestComponent, @Tag, KoraAppTestConfigModifier, Graph, Mockito." --- -Module provides an `Extension` for [JUnit5](https://junit.org/junit5/docs/current/user-guide/) that allows you to easily test your application. +Module provides an extension for [JUnit 5](https://junit.org/junit5/docs/current/user-guide/) that allows testing an application through the same component graph that is used at runtime. -The concept of the JUnit 5 Kora extension is to test the source code that will eventually be used in production. -This implies that dependency container of the main application is involved in the test, -it can be limited or its parts can be replaced by stubs if the test requires. +The Kora extension for `JUnit 5` is intended for component and integration testing of the source code that will later run in the real application. +The test uses the dependency container of the main application: it can be limited to the required components, +extended with test components, or have individual parts replaced with mocks. Module allows you to conduct: -- `Component tests` - testing of a single component -- `Inter-component tests` - testing of several components and their interaction with each other +- `Component tests` - testing of a single component. +- `Inter-component tests` - testing of several components and their interaction with each other. - `Integration tests` - testing of components and interaction with external systems. It is recommended to additionally test the service artifact packaged in the final image, -as black box using [TestContainers library](https://java.testcontainers.org/). +as a black box using the [Testcontainers library](https://java.testcontainers.org/). For a step-by-step walkthrough before the reference details, see [Component Testing](../guides/testing-junit.md), [Integration Testing](../guides/testing-integration.md) and [Black-Box Testing](../guides/testing-black-box.md). @@ -104,15 +104,16 @@ Examples will be shown relative to such an application: ### Test { #test } -The `@KoraAppTest` annotation is supposed to be used to annotate the test class. +To enable the Kora extension, annotate the test class with `@KoraAppTest`. +The annotation connects the `JUnit 5` extension, finds the generated graph of the specified `@KoraApp` application, and prepares the dependency container for the test. Parameters of the `@KoraAppTest` annotation: -- `value` - required parameter that points to the class annotated by `@KoraApp`, representing a graph of all dependencies that will be available within the test. -- `components` - list of components to be initialized within the test, - components that are not declared within the test are specified using special annotation `@TestComponent`. -- `modules` - list of modules with components connected in the application, - which should be additionally included in the dependency container within the test. +- `value` - class annotated with `@KoraApp` whose component graph will be used in the test (`required`, no default). +- `components` - additional component classes that should be included in the test graph in addition to components discovered through `@TestComponent` (default: `{}`). +- `modules` - additional modules with component factory methods that should be connected to the test graph (default: `{}`). + +Only module interfaces can be specified in `modules`. If the whole graph needs to be tested, inject `KoraAppGraph` or do not limit the graph to individual `@TestComponent` components. ===! ":fontawesome-brands-java: `Java`" @@ -137,11 +138,11 @@ Parameters of the `@KoraAppTest` annotation: ### Component { #component } -In order to use components within a test, it is suggested to use the `@TestComponent` annotation -which allows injecting component dependencies into arguments and/or fields of the test class. +To inject and select components for testing, use the `@TestComponent` annotation. +It allows injecting components into test method arguments, the constructor, and/or test class fields, and limits the dependency container to those components. All components listed in the test fields and/or method/constructor arguments annotated `@TestComponent` will be injected as dependencies within the test. -Entire dependency container will be limited to just those components and their dependencies within the test. +The test dependency container will be limited to those components and their dependencies. It is important that components within the test must be used by at least one [@Root component](container.md#root-component) that is also specified within the test. @@ -168,7 +169,7 @@ An example of a test where components are injected in fields: @KoraAppTest(Application::class) class SomeTests { - @TestComponent + @TestComponent lateinit var component1: Supplier @Test @@ -240,6 +241,21 @@ Example of a test where components are injected in method arguments: } ``` +#### Injection Rules { #injection-rules } + +Components can be injected in three ways: into a test class field, into the constructor, or into a test method parameter. +The chosen form affects when the Kora extension can access the test class instance and which additional mechanisms are available. + +- Fields suit most tests and are compatible with `KoraAppTestConfigModifier`, `KoraAppTestGraphModifier`, `PER_METHOD`, and `PER_CLASS`. +- Constructor injection is convenient for immutable fields, but is incompatible with `KoraAppTestConfigModifier` and `KoraAppTestGraphModifier`, because the extension needs a test class instance to call `config()` or `graph()`, while that instance is still being created during constructor injection. +- Method parameters are convenient for dependencies local to a specific test; with `PER_METHOD`, the graph includes parameters of the current method, while with `PER_CLASS`, the extension collects `@TestComponent` parameters from all methods of the class in advance. +- If constructor injection is used, `@TestComponent`, `@Mock`, `@Spy`, `@MockK`, or `@SpyK` cannot also be injected into test method parameters. +- In `PER_CLASS` mode, `@Mock` / `@MockK` cannot be injected into test method parameters because method-level mocks live shorter than the shared test class graph. +- The same element cannot be declared as a regular `@TestComponent`, mock, and spy at the same time: the extension will fail the test with a configuration error. + +If the test needs `KoraAppTestConfigModifier` or `KoraAppTestGraphModifier`, use field injection or method parameters. +If constructor injection is required, it is better to move configuration and graph modification into a separate test `@KoraApp` or connected module. + ### Tag { #tag } In order to inject a dependency/mock that has an `@Tag`, you must specify the appropriate `@Tag` annotation next to the argument for injection: @@ -264,12 +280,56 @@ In order to inject a dependency/mock that has an `@Tag`, you must specify the ap class SomeTests { @Test - fun example(@Tag(Supplier.class) @TestComponent component1: Supplier) { + fun example(@Tag(Supplier::class) @TestComponent component1: Supplier) { assertEquals("?", component1.get()) } } ``` +### Application Graph { #application-graph } + +If a test needs direct access to the prepared graph, inject `KoraAppGraph` into a field, constructor, or test method argument. +It can retrieve one or several components by type and can also account for `@Tag`. + +Main `KoraAppGraph` methods: + +- `getFirst(Type type)` / `getFirst(Class type)` - return the first found component or `null`. +- `getFirst(Type type, Class... tags)` / `getFirst(Class type, Class... tags)` - return the first component with the specified tags or `null`. +- `findFirst(...)` - returns `Optional` instead of `null`. +- `getAll(...)` - returns all components of the specified type, optionally accounting for tags. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SomeTests { + + @Test + void example(KoraAppGraph graph) { + var component = graph.getFirst(Supplier.class, Supplier.class); + + assertNotNull(component); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SomeTests { + + @Test + fun example(graph: KoraAppGraph) { + val component = graph.getFirst(Supplier::class.java, Supplier::class.java) + + assertNotNull(component) + } + } + ``` + +`KoraAppGraph` cannot be used as a target for `@Mock`, `@Spy`, `@MockK`, or `@SpyK`, because it is a service object of the test extension, not an application component. + ### Mock { #mock } ===! ":fontawesome-brands-java: `Java`" @@ -290,7 +350,7 @@ In order to inject a dependency/mock that has an `@Tag`, you must specify the ap annotated component and control the behavior of its methods with `Mockito` or the methods will return default values: `void`, default values for primitives, empty collections and `null` for all other objects. The stub component will be injected as a dependency into the arguments and/or fields of the test class and into all components that required it as a dependency. - All dependent components that are not required anywhere else within the test will be excluded for non-necessity. + All dependent components that are not required anywhere else within the test will be excluded as unnecessary. Example of a test using a `@Mock` component and injecting a mock in a field: @@ -328,7 +388,7 @@ In order to inject a dependency/mock that has an `@Tag`, you must specify the ap class SomeTests { @Test - void example(@MSpy @TestComponent Supplier component1) { + void example(@Spy @TestComponent Supplier component1) { Mockito.when(component1.get()).thenReturn("?"); assertEquals("?", component1.get()); } @@ -338,7 +398,7 @@ In order to inject a dependency/mock that has an `@Tag`, you must specify the ap You can also make a spy from the value of a test class field. The spy component will be injected as a dependency in the arguments and/or fields of the test class and in all components that required it as a dependency. - All dependent components that are not required anywhere else within the test will be excluded for non-necessity. + All dependent components that are not required anywhere else within the test will be excluded as unnecessary. Example of a test using `@Spy` spy component: @@ -383,7 +443,7 @@ In order to inject a dependency/mock that has an `@Tag`, you must specify the ap annotated component and control the behavior of its methods using `MockK`. Mock component will be injected as a dependency into the arguments and/or fields of the test class and into all components that required it as a dependency. - All dependent components that are not required anywhere else within the test will be excluded for non-necessity. + All dependent components that are not required anywhere else within the test will be excluded as unnecessary. Example of a test using `@MockK` component and injecting a mock: @@ -426,7 +486,7 @@ In order to inject a dependency/mock that has an `@Tag`, you must specify the ap You can also make a spy from the value of a test class field. The spy component will be implemented as a dependency in the arguments and/or fields of the test class and in all components that required it as a dependency. - All dependent components that are not required anywhere else within the test will be excluded for non-necessity. + All dependent components that are not required anywhere else within the test will be excluded as unnecessary. An example of a test using the `@SpyK` spy component: @@ -447,15 +507,20 @@ In order to inject a dependency/mock that has an `@Tag`, you must specify the ap #### Mock strictness { #mock-strictness } -You can check usage of `Mockito` mocks in tests by setting the verification level using the `@MockitoStrictness` annotation. +`Mockito` mocks can be checked with the `@MockitoStrictness` annotation. +It sets the verification level for `Mockito` mocks created by the Kora extension within the test class. + +The extension behaves similarly to `MockitoSession`: after the test completes, it passes the created mocks to `Mockito` verification and reports unused or suspicious stubbing. +If `@MockitoStrictness` is not specified, Kora uses `Strictness.WARN`: the test does not fail, but warnings are written to the log. -It works similarly to `MockitoSession` and is an imitation of a session within the Mockito framework, -which usually involves the execution of a single test method. -It provides a mechanism for managing the lifecycle of imitations and ensuring proper cleanup and verification. +Supported levels: -It allows you to maintain strict stub guarantees using the `Strictness` enumeration, -which helps identify unused calls and potentially throw an `UnnecessaryStubbingException` -or write a warning to the log. +- `Strictness.WARN` - default value; writes warnings to the log and does not fail the test. +- `Strictness.STRICT_STUBS` - strict mode; unused stubbing fails the test, for example with `UnnecessaryStubbingException`. +- `Strictness.LENIENT` - lenient mode; disables unused stubbing checks. + +If a specific `@Mock` has its own `strictness` parameter, it applies to that mock's settings. +`@MockitoStrictness` is convenient as a common level for the whole test class, so the setting does not need to be duplicated on every mock. ===! ":fontawesome-brands-java: `Java`" @@ -480,6 +545,9 @@ or write a warning to the log. } ``` +In the example above, `Mockito.when(component1.get()).thenReturn("?")` must be used by the test. +If the `component1.get()` call is removed from the test method, `Strictness.STRICT_STUBS` will fail the test. + === ":simple-kotlin: `Kotlin`" ```kotlin @@ -499,13 +567,15 @@ or write a warning to the log. } ``` +For Kotlin with `Mockito Kotlin`, the same mechanism applies because verification is performed by `Mockito`. +`@MockitoStrictness` does not apply to `MockK` mocks. + ### Test graph { #test-graph } Sometimes you may need to use an extended dependency container as part of your tests. -For example, a test container is an application that extends the main application and adds -some components from common modules that are not used in this application. +For example, a test application can extend the main application and add components that are only needed in tests. -For example, when you have different Read API and Write API applications with common components, +This approach is useful when you have different Read API and Write API applications with common components, which may be required as part of testing one and the other. Or, you may need some save/delete/update functions just for testing as a quick test utility. @@ -547,12 +617,12 @@ Let's imagine that the application looks like this: } ``` -In tests, you can create a graph extending the main application and use it within tests. +In tests, you can create a separate test `@KoraApp` that extends the main application and use that graph. +For this scenario, the generated submodule of the main application is required: without it, the test application cannot inherit and connect the main graph components. ===! ":fontawesome-brands-java: `Java`" - In order to do this, first of all you need to enable the option - to create a sub-module of the main application in `build.gradle`: + First, enable the parameter that creates a submodule of the main application in `build.gradle`: ```groovy compileJava { @@ -564,8 +634,7 @@ In tests, you can create a graph extending the main application and use it withi === ":simple-kotlin: `Kotlin`" - In order to do this, first of all you need to enable the option - to create a sub-module of the main application in `build.gradle.kts`: + First, enable the parameter that creates a submodule of the main application in `build.gradle.kts`: ```groovy ksp { @@ -678,14 +747,67 @@ You can now use the extended application graph in your tests: } ``` +If inheritance from the main `@KoraApp` is not needed and only factory methods from a separate module should be added, +use the `modules` parameter of `@KoraAppTest`. +`modules` accepts module interfaces, not component classes: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public interface TestModule { + + @Root + default Integer testOnlyComponent() { + return 1; + } + } + + @KoraAppTest(value = Application.class, modules = TestModule.class) + class SomeTests { + + @Test + void test(@TestComponent Integer component) { + assertEquals(1, component); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + interface TestModule { + + @Root + fun testOnlyComponent(): Int { + return 1 + } + } + + @KoraAppTest(value = Application::class, modules = [TestModule::class]) + class SomeTests { + + @Test + fun test(@TestComponent component: Int) { + assertEquals(1, component) + } + } + ``` + +Summary: + +- `kora.app.submodule.enabled=true` is needed when a test `@KoraApp` extends the main `@KoraApp`. +- `@KoraAppTest(modules = ...)` suits cases where additional modules simply need to be connected to the test graph. +- Components that should appear in the limited test graph must still be reachable from `@TestComponent`, `components`, or `KoraAppGraph`. + ## Test configuration { #test-configuration } By default, the basic configuration will be used, as in the case of running a real application. -For configuration changes/additions within tests, it is assumed that the test class implements the `KoraAppTestConfigModifier` interface, -where it is required to implement the `KoraConfigModification` method of providing config modification. +To change or add configuration within tests, the test class should implement `KoraAppTestConfigModifier`, +and the `config()` method should return a `KoraConfigModification`. -It is forbidden to use `KoraAppTestConfigModifier` and implementation in the constructor, because in this case it is impossible to get the configuration before implementation. +`KoraAppTestConfigModifier` cannot be used together with component injection into the test class constructor: +the extension needs to obtain the configuration modification before creating the test graph, and for that the test instance must already exist. #### Environment variables { #environment-variables } @@ -753,6 +875,46 @@ In order to use such a config and pass only environment variables, you need to r } ``` +If several values need to be passed at once, use `withSystemProperties(Map)`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SomeTests implements KoraAppTestConfigModifier { + + @NotNull + @Override + public KoraConfigModification config() { + return KoraConfigModification + .ofSystemProperty("POSTGRES_JDBC_URL", "jdbc:postgresql://localhost:5432/postgres") + .withSystemProperties(Map.of( + "POSTGRES_USER", "postgres", + "POSTGRES_PASS", "postgres" + )); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SomeTests : KoraAppTestConfigModifier { + + override fun config(): KoraConfigModification { + return KoraConfigModification + .ofSystemProperty("POSTGRES_JDBC_URL", "jdbc:postgresql://localhost:5432/postgres") + .withSystemProperties( + mapOf( + "POSTGRES_USER" to "postgres", + "POSTGRES_PASS" to "postgres" + ) + ) + } + } + ``` + ### Configuration file { #configuration-file } An example of providing a configuration as a file: @@ -822,12 +984,172 @@ in this case only this configuration will be used without any configuration file } ``` +### Configuration substitution { #configuration-substitution } + +The environment substitution shown in [Environment variables](#environment-variables) also works with an inline configuration: +declare `${ENV}` placeholders directly inside the `ofString(...)` configuration and resolve them with chained `withSystemProperty(...)`. +This is convenient when the whole configuration is described in the test, but some values (ports, hosts, credentials) are only known at runtime: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SomeTests implements KoraAppTestConfigModifier { + + @Override + public @Nonnull KoraConfigModification config() { + return KoraConfigModification.ofString(""" + myconfig { + myinnerconfig { + first = ${ENV_FIRST} + second = ${ENV_SECOND} + } + } + """) + .withSystemProperty("ENV_FIRST", "1") + .withSystemProperty("ENV_SECOND", "2"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SomeTests : KoraAppTestConfigModifier { + + override fun config(): KoraConfigModification { + return KoraConfigModification.ofString( + """ + myconfig { + myinnerconfig { + first = \${ENV_FIRST} + second = \${ENV_SECOND} + } + } + """.trimIndent() + ) + .withSystemProperty("ENV_FIRST", "1") + .withSystemProperty("ENV_SECOND", "2") + } + } + ``` + +### Testcontainers { #testcontainers } + +A common use of `KoraAppTestConfigModifier` is [Testcontainers](https://java.testcontainers.org/) integration: +the test starts a container and passes its runtime connection values into the configuration through `config()`. +Testcontainers assigns a random host port on each run, so the values must not be hardcoded — they are declared as `${...}` placeholders in the inline configuration +and populated from the container getters via `withSystemProperty(...)`. + +Because `config()` runs **before** the test graph is built, the configuration is ready before any component is created. +For the same reason `KoraAppTestConfigModifier` is incompatible with [constructor injection](#injection-rules): use field or method-parameter injection as shown below. + +===! ":fontawesome-brands-java: `Java`" + + Add the [Testcontainers](https://java.testcontainers.org/) dependencies in `build.gradle`: + ```groovy + testImplementation "org.testcontainers:junit-jupiter:1.21.4" + testImplementation "org.testcontainers:postgresql:1.21.4" + ``` + + ```java + @Testcontainers + @KoraAppTest(Application.class) + class SomeIntegrationTests implements KoraAppTestConfigModifier { + + @Container + private static final PostgreSQLContainer POSTGRES = new PostgreSQLContainer<>("postgres:16"); + + @TestComponent + private SomeService service; + + @NotNull + @Override + public KoraConfigModification config() { + return KoraConfigModification.ofString(""" + db { + jdbcUrl = ${POSTGRES_JDBC_URL} + username = ${POSTGRES_USER} + password = ${POSTGRES_PASS} + poolName = "kora" + } + """) + .withSystemProperty("POSTGRES_JDBC_URL", POSTGRES.getJdbcUrl()) + .withSystemProperty("POSTGRES_USER", POSTGRES.getUsername()) + .withSystemProperty("POSTGRES_PASS", POSTGRES.getPassword()); + } + + @Test + void example() { + // interact with the service backed by the container + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + Add the [Testcontainers](https://java.testcontainers.org/) dependencies in `build.gradle.kts`: + ```groovy + testImplementation("org.testcontainers:junit-jupiter:1.21.4") + testImplementation("org.testcontainers:postgresql:1.21.4") + ``` + + ```kotlin + @Testcontainers + @KoraAppTest(Application::class) + class SomeIntegrationTests : KoraAppTestConfigModifier { + + companion object { + @Container + @JvmStatic + val POSTGRES = PostgreSQLContainer("postgres:16") + } + + @TestComponent + lateinit var service: SomeService + + override fun config(): KoraConfigModification { + return KoraConfigModification.ofString( + """ + db { + jdbcUrl = \${POSTGRES_JDBC_URL} + username = \${POSTGRES_USER} + password = \${POSTGRES_PASS} + poolName = "kora" + } + """.trimIndent() + ) + .withSystemProperty("POSTGRES_JDBC_URL", POSTGRES.jdbcUrl) + .withSystemProperty("POSTGRES_USER", POSTGRES.username) + .withSystemProperty("POSTGRES_PASS", POSTGRES.password) + } + + @Test + fun example() { + // interact with the service backed by the container + } + } + ``` + +For a full walkthrough — dependencies, a test `@KoraApp`, migrations and repository setup — see the [Integration Testing](../guides/testing-integration.md) guide. + ## Container modification { #container-modification } -In order to add/replace/mock components within an unannotated application dependency container requires implementing the `KoraAppTestGraphModifier` interface and -Implement a method to provide a dependency container modifier. +To add, replace, or programmatically create mocks in the application container without annotations, implement `KoraAppTestGraphModifier` +and return a `KoraGraphModification` from the `graph()` method. + +`KoraAppTestGraphModifier` cannot be used together with component injection into the test class constructor: +the extension needs to obtain the graph modification before creating the graph and injecting components. -It is forbidden to use `KoraAppTestGraphModifier` and embedding in the constructor because then you can't get the graph before embedding. +`KoraGraphModification` supports these operations: + +- `addComponent(...)` - adds a new component to the test graph. +- `replaceComponent(...)` - replaces an existing component, while its dependencies remain in the graph. +- `mockComponent(...)` - replaces an existing component with a mock and removes the replaced component's real dependencies from the graph if they are no longer needed by the test. + +`addComponent(...)` and `replaceComponent(...)` have overloads with `Function` if the new component should be built from already initialized graph components. +For components with `@Tag`, use overloads with `List> tags`. ### Adding { #adding } @@ -921,7 +1243,7 @@ In case it is required to add components using a real component from the graph, ### Replacement { #replacement } -An example of replacing a component in a dependency container: +An example of replacing a component in a dependency container, this mechanism can also be used to create custom mocks: ===! ":fontawesome-brands-java: `Java`" @@ -960,7 +1282,7 @@ An example of replacing a component in a dependency container: } ``` -In case it is required to add components using a real component from the graph, this is also available through another method signature: +In case it is required to replace components using a real component from the graph, this is also available through another method signature: ===! ":fontawesome-brands-java: `Java`" @@ -1009,9 +1331,56 @@ In case it is required to add components using a real component from the graph, } ``` +### Programmatic Mock { #programmatic-mock } + +If a component should be replaced specifically as a mock, use `mockComponent(...)`. +Unlike `replaceComponent(...)`, this method tells the extension that the real dependencies of the replaced component are not needed and can be excluded from the test graph. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(value = Application.class) + class SomeTests implements KoraAppTestGraphModifier { + + @Override + public @Nonnull KoraGraphModification graph() { + return KoraGraphModification.create() + .mockComponent(TypeRef.of(Supplier.class, String.class), () -> Mockito.mock(Supplier.class)); + } + + @Test + void example(@TestComponent Supplier supplier) { + Mockito.when(supplier.get()).thenReturn("?"); + + assertEquals("?", supplier.get()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(value = Application::class) + class SomeTests : KoraAppTestGraphModifier { + + override fun graph(): KoraGraphModification { + return KoraGraphModification.create() + .mockComponent(TypeRef.of(Supplier::class.java, String::class.java), Supplier { mockk>() }) + } + + @Test + fun example(@TestComponent supplier: Supplier) { + every { supplier.get() } returns "?" + + assertEquals("?", supplier.get()) + } + } + ``` + ## Initialization { #initialization } -In case you want to initialize the dependency container once within the entire test class, you should annotate the test class with `@TestInstance(TestInstance.Lifecycle.PER_CLASS)`: +By default, `JUnit 5` uses `TestInstance.Lifecycle.PER_METHOD`, so Kora creates and cleans up the test graph for each test method. +If the container should be initialized once for the whole test class, annotate the test class with `@TestInstance(TestInstance.Lifecycle.PER_CLASS)`: ===! ":fontawesome-brands-java: `Java`" @@ -1033,4 +1402,12 @@ In case you want to initialize the dependency container once within the entire t } ``` -The default behavior is to initialize the container every time of every test method. +With `PER_CLASS`, one graph instance is used by all test methods in the class, and cleanup runs after the whole class completes. +This speeds up heavy integration tests, but mutable component and mock state should be handled more carefully. + +Lifecycle restrictions: + +- When components are injected into the constructor, `@TestComponent` or mocks cannot also be injected into test method parameters. +- When components are injected into the constructor, `KoraAppTestConfigModifier` and `KoraAppTestGraphModifier` cannot be used. +- In `PER_CLASS` mode, `@Mock` / `@MockK` cannot be injected into test method parameters; use fields or the constructor. +- For `@Nested` classes, field injection into the inner class cannot be used if the outer test class runs in `PER_CLASS` mode; use method parameters or a separate lifecycle for the nested class. diff --git a/mkdocs/docs/en/documentation/kafka.md b/mkdocs/docs/en/documentation/kafka.md index ef9cdbd..f5ded4b 100644 --- a/mkdocs/docs/en/documentation/kafka.md +++ b/mkdocs/docs/en/documentation/kafka.md @@ -4,7 +4,13 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora Kafka consumers and producers, listener and publisher annotations, configuration, serialization, error handling, rebalance events, transactions, and telemetry tags; key triggers include @KafkaListener, @KafkaPublisher, @Topic, @Json, @Tag, KafkaModule, KafkaConsumer, KafkaProducer, KafkaSkipRecordException." --- -Module for creating declarative [Apache Kafka](https://kafka.apache.org/) `Consumer` and `Producer` using annotations. +The `Kafka` module provides declarative integration with [Apache Kafka](https://kafka.apache.org/): reading messages through +`@KafkaListener`, sending messages through `@KafkaPublisher`, serialization, deserialization, transactions, processing errors, +and telemetry. + +`Apache Kafka` is a distributed event streaming platform. Applications write events to a `topic`, while other applications read +them through a `consumer group` or directly assigned partitions. Kora creates the required `Consumer` and `Producer` at compile time, +binds them to the dependency graph, and lets most of the contract be described through method signatures. For a step-by-step walkthrough before the reference details, see [Kafka Messaging](../guides/messaging-kafka.md). @@ -38,7 +44,9 @@ For a step-by-step walkthrough before the reference details, see [Kafka Messagin ## Consumer { #consumer } -Descriptions of working with [Kafka Consumer](https://docs.confluent.io/platform/current/clients/consumer.html) +`Consumer` reads records from a `topic` and passes them to an application method. Kora creates the consumer container, +calls `poll()`, applies deserialization, invokes the handler, and commits the offset unless the method signature requires +manual `Consumer` control. Creating a `Consumer` requires using the `@KafkaListener` annotation over a method: @@ -109,13 +117,14 @@ each with its own individual configuration. It looks like this: } ``` -The value in the annotation indicates from which part of the configuration file the settings should be taken. As far as getting the configuration is concerned - works similarly to `@ConfigSource` +The value in the annotation indicates which part of the configuration file should be used. +Conceptually, it is similar to `@ConfigSource`: the annotation value selects the configuration branch for a specific container. -### Configuration { #configuration } +### Configuration { #config-consumer } Configuration describes the settings of a particular `@KafkaListener` and an example for the configuration at path `kafka.someConsumer` is given below. -Example of the complete configuration described in the `KafkaListenerConfig` class (default or example values are specified): +Basic configuration parameters: ===! ":material-code-json: `Hocon`" @@ -123,116 +132,185 @@ Example of the complete configuration described in the `KafkaListenerConfig` cla kafka { someConsumer { topics = ["topic1", "topic2"] //(1)! - topicsPattern = "topic*" //(2)! - allowEmptyRecords = false //(3)! - offset = "latest" //(4)! - pollTimeout = "5s" //(5)! - backoffTimeout = "15s" //(6)! - partitionRefreshInterval = "1m" //(7)! - threads = 1 //(8)! - shutdownWait = "30s" //(9)! - driverProperties { //(10)! + offset = "latest" //(2)! + pollTimeout = "5s" //(3)! + threads = 1 //(4)! + driverProperties { //(5)! "bootstrap.servers": "localhost:9093" "group.id": "my-group-id" } - telemetry { - logging { - enabled = false //(11)! - } - metrics { - enabled = true //(12)! - slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(13)! - tags = { // (14)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(15)! - attributes = { // (16)! - "key1" = "value1" - "key2" = "value2" - } - } - } } } ``` - 1. Specifies the topics to which Consumer will subscribe (**required** or specify `topicsPattern`) - 2. Specifies the pattern of topics to which the Consumer will subscribe (**required** or `topics` is specified). - 3. Whether to process empty records in case the signature accepts `ConsumerRecords` - 4. Works only if `group.id` is not specified. Specifies which position in the topics the Consumer should use.Valid values are: - 1. `earliest` - earliest available offset - 2. `latest` - latest available offset - 3. String in `Duration` format, e.g. `5m` - shift back a certain time. - 5. Maximal waiting time for messages from a topic within one call - 6. Maximum waiting time between unexpected exceptions during processing - 7. Time interval within which it is required to update partitions in case of `assign` method - 8. Number of threads on which the consumer will be started for parallel processing (if it is equal to 0 then no consumer will be started at all) - 9. Waiting time for processing before switching off the consumer in case of [gracefull shutdown](container.md#graceful-shutdown) - 10. *Properties* from the official kafka client, documentation on them can be found at [link](https://kafka.apache.org/documentation/#consumerconfigs) (**required**) - 11. Enables module logging (default `false`) - 12. Enables module metrics (default `true`) - 13. Configuring [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 14. Configures tags for metrics (optional) - 15. Enables module tracing (default `true`) - 16. Configures attributes for tracing (optional) + 1. List of `topic`s to subscribe to (`required` to specify either `topics` or `topicsPattern`) + 2. Initial read position (default: `latest`). Allowed values: `earliest`, `latest`, or time offset (e.g. `5m`) + 3. Maximum time to wait for messages (default: `5s`) + 4. Number of threads for the consumer (default: `1`) + 5. Official `Kafka Consumer` `Properties` (`required`, no default) === ":simple-yaml: `YAML`" ```yaml kafka: someConsumer: - topics: #(1)! + topics: - "topic1" - - "topic2" - topicsPattern: "topic*" #(2)! - allowEmptyRecords: false #(3)! - offset: "latest" #(4)! - pollTimeout: "5s" #(5)! - backoffTimeout: "15s" #(6)! - partitionRefreshInterval: "1m" #(7)! - threads: 1 #(8)! - shutdownWait: "30s" #(9)! - driverProperties: #(10)! - bootstrap.servers: "localhost:9093" - group.id: "my-group-id" - telemetry: - logging: - enabled: false #(11)! - metrics: - enabled: true #(12)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! - tags: #(14)! - key1: value1 - key2: value2 - tracing: - enabled: true #(15)! - attributes: #(16)! - key1: value1 - key2: value2 - ``` - - 1. Specifies the topics to which Consumer will subscribe (**required** or specify `topicsPattern`) - 2. Specifies the pattern of topics to which the Consumer will subscribe (**required** or `topics` is specified). - 3. Whether to process empty records in case the signature accepts `ConsumerRecords` - 4. Works only if `group.id` is not specified. Specifies which position in the topics the Consumer should use.Valid values are: - 1. `earliest` - earliest available offset - 2. `latest` - latest available offset - 3. String in `Duration` format, e.g. `5m` - shift back a certain time. - 5. Maximal waiting time for messages from a topic within one call - 6. Maximum waiting time between unexpected exceptions during processing - 7. Time interval within which it is required to update partitions in case of `assign` method - 8. Number of threads on which the consumer will be started for parallel processing (if it is equal to 0 then no consumer will be started at all) - 9. Waiting time for processing before switching off the consumer in case of [gracefull shutdown](container.md#graceful-shutdown) - 10. *Properties* from the official kafka client, documentation on them can be found at [link](https://kafka.apache.org/documentation/#consumerconfigs) (**required**) - 11. Enables module logging (default `false`) - 12. Enables module metrics (default `true`) - 13. Configuring [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 14. Configures tags for metrics (optional) - 15. Enables module tracing (default `true`) - 16. Configures attributes for tracing (optional) + - "topic2" #(1)! + offset: "latest" #(2)! + pollTimeout: "5s" #(3)! + threads: 1 #(4)! + driverProperties: #(5)! + "bootstrap.servers": "localhost:9093" + "group.id": "my-group-id" + ``` + + 1. List of `topic`s to subscribe to (`required` to specify either `topics` or `topicsPattern`) + 2. Initial read position (default: `latest`). Allowed values: `earliest`, `latest`, or time offset (e.g. `5m`) + 3. Maximum time to wait for messages (default: `5s`) + 4. Number of threads for the consumer (default: `1`) + 5. Official `Kafka Consumer` `Properties` (`required`, no default) + +??? note "Full Configuration" + + Example of the complete configuration described in the `KafkaListenerConfig` class (default or example values are specified): + + In a real configuration, either `topics` or `topicsPattern` is usually specified. + + ===! ":material-code-json: `Hocon`" + + ```javascript + kafka { + someConsumer { + topics = ["topic1", "topic2"] //(1)! + topicsPattern = "topic*" //(2)! + partitions = ["0", "1"] //(3)! + allowEmptyRecords = false //(4)! + offset = "latest" //(5)! + pollTimeout = "5s" //(6)! + backoffTimeout = "15s" //(7)! + partitionRefreshInterval = "1m" //(8)! + threads = 1 //(9)! + shutdownWait = "30s" //(10)! + driverProperties { //(11)! + "bootstrap.servers": "localhost:9093" + "group.id": "my-group-id" + } + telemetry { + logging { + enabled = false //(12)! + } + metrics { + enabled = true //(13)! + slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(14)! + tags = { // (15)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(16)! + attributes = { // (17)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + ``` + + 1. List of `topic` values the `Consumer` subscribes to (not set by default, optional; either `topics` or `topicsPattern` must be specified) + 2. `topic` pattern the `Consumer` subscribes to (not set by default, optional; either `topics` or `topicsPattern` must be specified) + 3. List of partitions used only for consumer name construction when `group.id`, `topics`, and `topicsPattern` are not specified; partition assignment is controlled by the `assign` container (not set by default, optional) + If specified, consumer will read only from specified partitions. Example: `["0", "1", "2"]`. + 4. Whether to process empty batches when the signature accepts `ConsumerRecords` (default: `false`) + If `false` and `ConsumerRecords` is empty (no messages), consumer method will not be called. + If `true`, method will be called with empty `ConsumerRecords` (useful for periodic checks). + 5. Initial read position for the `assign` strategy when `group.id` is not specified (default: `latest`). Valid values: + 1. `earliest` - earliest available `offset` + 2. `latest` - latest available `offset` + 3. string in `Duration` format, for example `5m`, - shift back by the specified duration + Format: number + unit (ms, s, m, h, d). Examples: `5m` = 5 minutes ago, `1h` = 1 hour ago. + 6. Maximum time to wait for messages from a `topic` within one `poll()` call (default: `5s`) + 7. Initial delay between unexpected processing errors; with repeated errors the delay increases up to `60s` (default: `15s`) + If consumer throws unexpected exception (not `KafkaSkipRecordException`), + Kora will restart consumer with `backoffTimeout` delay to prevent cyclic errors. + 8. Partition list refresh period for the `assign` strategy (default: `1m`) + 9. Number of threads the consumer starts on; if set to `0`, the consumer is not started (default: `1`) + 10. Time to wait for processing before stopping the consumer during [graceful shutdown](container.md#graceful-shutdown) (default: `30s`) + 11. Official `Kafka Consumer` `Properties`; see [Apache Kafka Consumer Configs](https://kafka.apache.org/documentation/#consumerconfigs) (`required`, not set by default) + 12. Enables module logging (default: `false`) + 13. Enables module metrics (default: `true`) + 14. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Configures metric tags (default: `{}`) + 16. Enables module tracing (default: `true`) + 17. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + kafka: + someConsumer: + topics: #(1)! + - "topic1" + - "topic2" + topicsPattern: "topic*" #(2)! + partitions: #(3)! + - "0" + - "1" + allowEmptyRecords: false #(4)! + offset: "latest" #(5)! + pollTimeout: "5s" #(6)! + backoffTimeout: "15s" #(7)! + partitionRefreshInterval: "1m" #(8)! + threads: 1 #(9)! + shutdownWait: "30s" #(10)! + driverProperties: #(11)! + bootstrap.servers: "localhost:9093" + group.id: "my-group-id" + telemetry: + logging: + enabled: false #(12)! + metrics: + enabled: true #(13)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(14)! + tags: #(15)! + key1: value1 + key2: value2 + tracing: + enabled: true #(16)! + attributes: #(17)! + key1: value1 + key2: value2 + ``` + + 1. List of `topic` values the `Consumer` subscribes to (not set by default, optional; either `topics` or `topicsPattern` must be specified) + 2. `topic` pattern the `Consumer` subscribes to (not set by default, optional; either `topics` or `topicsPattern` must be specified) + 3. List of partitions used only for consumer name construction when `group.id`, `topics`, and `topicsPattern` are not specified; partition assignment is controlled by the `assign` container (not set by default, optional) + If specified, consumer will read only from specified partitions. Example: `["0", "1", "2"]`. + 4. Whether to process empty batches when the signature accepts `ConsumerRecords` (default: `false`) + If `false` and `ConsumerRecords` is empty (no messages), consumer method will not be called. + If `true`, method will be called with empty `ConsumerRecords` (useful for periodic checks). + 5. Initial read position for the `assign` strategy when `group.id` is not specified (default: `latest`). Valid values: + 1. `earliest` - earliest available `offset` + 2. `latest` - latest available `offset` + 3. string in `Duration` format, for example `5m`, - shift back by the specified duration + Format: number + unit (ms, s, m, h, d). Examples: `5m` = 5 minutes ago, `1h` = 1 hour ago. + 6. Maximum time to wait for messages from a `topic` within one `poll()` call (default: `5s`) + 7. Initial delay between unexpected processing errors; with repeated errors the delay increases up to `60s` (default: `15s`) + If consumer throws unexpected exception (not `KafkaSkipRecordException`), + Kora will restart consumer with `backoffTimeout` delay to prevent cyclic errors. + 8. Partition list refresh period for the `assign` strategy (default: `1m`) + 9. Number of threads the consumer starts on; if set to `0`, the consumer is not started (default: `1`) + 10. Time to wait for processing before stopping the consumer during [graceful shutdown](container.md#graceful-shutdown) (default: `30s`) + 11. Official `Kafka Consumer` `Properties`; see [Apache Kafka Consumer Configs](https://kafka.apache.org/documentation/#consumerconfigs) (`required`, not set by default) + 12. Enables module logging (default: `false`) + 13. Enables module metrics (default: `true`) + 14. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Configures metric tags (default: `{}`) + 16. Enables module tracing (default: `true`) + 17. Configures tracing attributes (default: `{}`) Module metrics are described in the [Metrics Reference](metrics.md#kafka) section. @@ -270,6 +348,8 @@ Example of `subscribe` strategy configuration: `assign` connection strategy implies that each instance of the application reads messages from the topic simultaneously with others, i.e., messages are duplicated between all instances of the application within the topic. +This strategy is useful, for example, when all application replicas must receive the same message at once: to reset a local cache, +update local reference data, or handle a service event. To use this strategy, simply **do not specify** `group.id` in the consumer configuration. However, only one topic can be specified at a time in this strategy. @@ -300,50 +380,124 @@ Example of `assign` strategy configuration: ### Signatures { #signatures } -Available signatures for Kafka consumer out-of-the-box methods, where `K` refers to the key type and `V` to the message value type. +Available signatures for out-of-the-box `Kafka Consumer` methods, where `K` refers to the key type and `V` to the message value type. +The generator supports three signature families: separate `key`/`value` arguments, a single `ConsumerRecord`, or a whole `ConsumerRecords` batch. +These families cannot be mixed in the same method. + +#### Key and value { #key-value-signature } + +A signature with separate arguments accepts `value`, optional `key`, optional `Headers`, optional `Consumer`, and optional deserialization errors. +One user argument is treated as `value`; two user arguments are treated as `key` and `value` in that exact order. +If `key` is not declared, the key deserialization type is considered to be `byte[]`. -Calls `poll()` for the `ConsumerRecords` bundle and passes each event individually to a handler. -The handler accepts `value` (mandatory), `key` (optional), `Headers` (optional) from `ConsumerRecord`, -`Exception` (optional) in case of serialization/connection error and after processing **each** event, `commitSync()` is called: +To handle deserialization errors, add `Exception`, `RecordKeyDeserializationException`, or `RecordValueDeserializationException`. +When such an argument is present, Kora passes the deserialization error to it, and the corresponding `key` or `value` is passed as `null`. +Without such an argument, the deserialization error is thrown from the handler, and the record is read again without committing the current offset. ===! ":fontawesome-brands-java: `Java`" ```java - @KafkaListener("kafka.someConsumer1") - void process1(K key, V value, Headers headers) { - // some handler code + @KafkaListener("kafka.someConsumer") + void process(K key, V value, Headers headers) { + // some value handling work + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaListener("kafka.someConsumer") + fun process(key: K, value: V, headers: Headers) { + // some value handling work } + ``` - @KafkaListener("kafka.someConsumer2") - void process2(@Nullable V value, @Nullable Exception exception) { - // some handler code +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaListener("kafka.someOtherConsumer") + void process(@Nullable V value, @Nullable Exception exception) { + if (exception != null) { + // do deserialization handling work + } else { + // some value handling work + } } ``` === ":simple-kotlin: `Kotlin`" ```kotlin - @KafkaListener("kafka.someConsumer1") - fun process1(key: K, value: V, headers: Headers) { - // some handler code + @KafkaListener("kafka.someOtherConsumer") + fun process(value: V?, exception: Exception?) { + if (exception != null) { + // do deserialization handling work + } else { + // some value handling work + } } + ``` + +#### Whole record { #record-signature } + +A signature with `ConsumerRecord` accepts one whole record, optional `Consumer`, and optional deserialization errors: +`Exception`, `RecordKeyDeserializationException`, or `RecordValueDeserializationException`. +`Headers`, separate `key`/`value` arguments, and the telemetry context are not supported in this signature. + +If error arguments are not declared, the deserialization error can be thrown when calling `record.key()` or `record.value()`. +If error arguments are declared, Kora calls `key()` and/or `value()` beforehand, catches the deserialization error, and passes it to the method. + +===! ":fontawesome-brands-java: `Java`" - @KafkaListener("kafka.someConsumer2") - fun process2(value: V?, exception: Exception?) { - // some handler code + ```java + @KafkaListener("kafka.someConsumer") + void process(ConsumerRecord record) { + try { + var key = record.key(); + var value = record.value(); + + // some value handling work + } catch (RecordKeyDeserializationException e) { + // do deserialization handling work + } catch (RecordValueDeserializationException e) { + // do deserialization handling work + } } ``` -Calls `poll()` for the `ConsumerRecords` bundle and passes each event individually to handler. -The handler accepts `ConsumerRecord` and `KafkaConsumerRecordsTelemetryContext`/`KafkaConsumerRecordTelemetryContext` (optional) -and `commitSync()` is called after processing **each event**: +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaListener("kafka.someConsumer") + fun process(record: ConsumerRecord) { + try { + val key = record.key() + val value = record.value() + + // some value handling work + } catch (e: RecordKeyDeserializationException) { + // do deserialization handling work + } catch (e: RecordValueDeserializationException) { + // do deserialization handling work + } + } + ``` ===! ":fontawesome-brands-java: `Java`" ```java @KafkaListener("kafka.someConsumer") - void process(ConsumerRecord record) { - // some handler code + void process(ConsumerRecord record, + @Nullable RecordKeyDeserializationException keyException, + @Nullable RecordValueDeserializationException valueException) { + if (keyException != null || valueException != null) { + // do deserialization handling work + return; + } + + var key = record.key(); + var value = record.value(); + // some value handling work } ``` @@ -351,21 +505,45 @@ and `commitSync()` is called after processing **each event**: ```kotlin @KafkaListener("kafka.someConsumer") - fun process(record: ConsumerRecord) { - // some handler code + fun process( + record: ConsumerRecord, + keyException: RecordKeyDeserializationException?, + valueException: RecordValueDeserializationException?, + ) { + if (keyException != null || valueException != null) { + // do deserialization handling work + return + } + + val key = record.key() + val value = record.value() + // some value handling work } ``` -Calls `poll()` for the `ConsumerRecords` bundle and passes the entire batch to handler. -Handler accepts `ConsumerRecords` and `KafkaConsumerRecordsTelemetryContext`/`KafkaConsumerRecordsTelemetryContext` (optional) -and `commitSync()` is called after processing **whole batch** of events: +#### Batch of records { #records-signature } + +A signature with `ConsumerRecords` accepts the whole batch of records from one `poll()`. +Together with it, only `Consumer` and `KafkaConsumerRecordsTelemetryContext` can be declared. +Separate `key`/`value` arguments, `Headers`, and deserialization error arguments are not supported in this signature; deserialization errors should be handled while iterating over records. ===! ":fontawesome-brands-java: `Java`" ```java @KafkaListener("kafka.someConsumer") - void process(ConsumerRecords record) { - // some handler code + void process(ConsumerRecords records) { + for (var record : records) { + try { + var key = record.key(); + var value = record.value(); + + // some value handling work + } catch (RecordKeyDeserializationException e) { + // do deserialization handling work + } catch (RecordValueDeserializationException e) { + // do deserialization handling work + } + } } ``` @@ -373,19 +551,51 @@ and `commitSync()` is called after processing **whole batch** of events: ```kotlin @KafkaListener("kafka.someConsumer") - fun process(record: ConsumerRecords) { - // some handler code + fun process(records: ConsumerRecords) { + for (record in records) { + try { + val key = record.key() + val value = record.value() + + // some value handling work + } catch (e: RecordKeyDeserializationException) { + // do deserialization handling work + } catch (e: RecordValueDeserializationException) { + // do deserialization handling work + } + } } ``` -In case `Consumer` is taken as an argument, `commit` must be **called manually**. +#### Offset commit { #manual-commit } + +If the signature does not declare a `Consumer` argument, Kora commits the offset automatically: after each record for `key`/`value` and `ConsumerRecord` signatures, or after the whole batch for `ConsumerRecords`. +It does this by calling `commitSync()`. + +If the signature declares a `Consumer` argument, automatic offset commit is disabled, and the handler is fully responsible for calling `commitSync()` or `commitAsync()`. +This mode is useful when the offset should be committed only after an external operation, several records should be committed together, or the read position should be controlled manually. + +In `subscribe` mode, a manual `commit` commits the offset inside the consumer group. +In `assign` mode, partitions are not coordinated through a consumer group, so it is usually more important to manage the position manually with `seek()`, `pause()`, and `resume()` instead of relying on a group offset commit. +If the handler fails before the manual commit, the record or batch will be read again according to the current consumer position. ===! ":fontawesome-brands-java: `Java`" ```java @KafkaListener("kafka.someConsumer") - void process(ConsumerRecord record, Consumer consumer) { - // some handler code + void process(ConsumerRecord record, Consumer consumer) { + try { + var key = record.key(); + var value = record.value(); + + // some value handling work + } catch (RecordKeyDeserializationException e) { + // do deserialization handling work + } catch (RecordValueDeserializationException e) { + // do deserialization handling work + } finally { + consumer.commitSync(); + } } ``` @@ -393,14 +603,27 @@ In case `Consumer` is taken as an argument, `commit` must be **called manu ```kotlin @KafkaListener("kafka.someConsumer") - fun process(record: ConsumerRecord, consumer: Consumer) { - // some handler code + fun process(record: ConsumerRecord, consumer: Consumer) { + try { + val key = record.key() + val value = record.value() + + // some value handling work + } catch (e: RecordKeyDeserializationException) { + // do deserialization handling work + } catch (e: RecordValueDeserializationException) { + // do deserialization handling work + } finally { + consumer.commitSync() + } } ``` ### Deserialization { #deserialization } -`Deserializer` - used to deserialize `ConsumerRecord` keys and values. +`Deserializer` is used to deserialize `ConsumerRecord` keys and values. +Kora provides `Deserializer` components for basic types: `String`, `UUID`, `byte[]`, `Bytes`, `ByteBuffer`, `Double`, `Float`, +`Integer`, `Long`, `Short`, and `Void`. Tags are supported to better customize the `Deserializer`. Tags can be set on parameter-key, parameter-value, as well as on parameters of type `ConsumerRecord` and `ConsumerRecords`. @@ -441,7 +664,8 @@ These tags will be set on container dependencies. } ``` -In case deserialization from `Json` is required, the `@Json` tag can be used: +If deserialization from `JSON` is required, use the `@Json` tag. +In this case, Kora uses `JsonReader` and `JsonKafkaDeserializer` from the [JSON](json.md) module: ===! ":fontawesome-brands-java: `Java`" @@ -487,7 +711,145 @@ In case deserialization from `Json` is required, the `@Json` tag can be used: For non-key handlers, the default is `Deserializer` since it simply returns unhandled bytes. -### Exception handling { #exception-handling } +### Custom Deserializer { #custom-deserializer } + +If custom deserialization is required, you can implement your own `Deserializer`. + +**Option 1: Default deserializer for type** + +If you provide `Deserializer` as a component without a tag, it will be used for all consumers of this type: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public static class MyEventDeserializer implements Deserializer { + + private final JsonReader reader; + + public MyEventDeserializer(JsonReader reader) { + this.reader = reader; + } + + @Override + public MyEvent deserialize(String topic, byte[] data) { + try { + return reader.read(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + @Component + final class SomeConsumer { + + @KafkaListener("kafka.someConsumer") + void process(MyEvent value) { // Uses MyEventDeserializer + // event handling + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyEventDeserializer( + private val reader: JsonReader + ) : Deserializer { + + override fun deserialize(topic: String, data: ByteArray): MyEvent { + return try { + reader.read(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + @Component + class SomeConsumer { + + @KafkaListener("kafka.someConsumer") + fun process(value: MyEvent) { // Uses MyEventDeserializer + // event handling + } + } + ``` + +**Option 2: Point deserializer for specific consumer** + +If you need to use different deserialization for different consumers of the same type, you can use tags: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + final class SomeConsumer { + + @Json + public record MyEvent(String username, int code) {} + + @Tag(MyEvent.class) + @Component + public static class MyDeserializer implements Deserializer { + + private final JsonReader reader; + + public MyDeserializer(JsonReader reader) { + this.reader = reader; + } + + @Override + public MyEvent deserialize(String topic, byte[] data) { + try { + return reader.read(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + @KafkaListener("kafka.someConsumer") + void process(@Tag(MyEvent.class) MyEvent value) { + // event handling + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeConsumer { + + @Json + data class MyEvent(val username: String, val code: Int) + + @Tag(MyEvent::class) + @Component + class MyDeserializer( + private val reader: JsonReader + ) : Deserializer { + + override fun deserialize(topic: String, data: ByteArray): MyEvent { + return try { + reader.read(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + @KafkaListener("kafka.someConsumer") + fun process(@Tag(MyEvent::class) value: MyEvent) { + // event handling + } + } + ``` + +### Exception handling { #exception-handling-consumer } If the method labeled `@KafkaListener` throws an exception, Consumer will be restarted, because there is no general solution on how to handle this and the developer **must** decide how to handle it. @@ -554,10 +916,52 @@ At that point, it is worth handling it in the way you want. The following exceptions are thrown: -* ` `ru.tinkoff.kora.kafka.common.exceptions.RecordKeyDeserializationException`. +* `ru.tinkoff.kora.kafka.common.exceptions.RecordKeyDeserializationException`. * `ru.tinkoff.kora.kafka.common.exceptions.RecordValueDeserializationException`. -From these exceptions, you can get a raw `ConsumerRecord`. +From these exceptions, you can get a raw `ConsumerRecord` using `getRecord()` method: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + final class ConsumerService { + + @KafkaListener("kafka.someConsumer") + public void process(String key, String value) { + try { + // Access key/value which may throw deserialization exception + } catch (RecordKeyDeserializationException e) { + ConsumerRecord rawRecord = e.getRecord(); + // Handle raw record (log, send to DLQ, etc.) + } catch (RecordValueDeserializationException e) { + ConsumerRecord rawRecord = e.getRecord(); + // Handle raw record (log, send to DLQ, etc.) + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ConsumerService { + + @KafkaListener("kafka.someConsumer") + fun process(key: String, value: String) { + try { + // Access key/value which may throw deserialization exception + } catch (e: RecordKeyDeserializationException) { + val rawRecord = e.getRecord() + // Handle raw record (log, send to DLQ, etc.) + } catch (e: RecordValueDeserializationException) { + val rawRecord = e.getRecord() + // Handle raw record (log, send to DLQ, etc.) + } + } + } + ``` If you use a signature with unpacked `key`/`value`/`headers`, you can add `Exception`, `Throwable`, `RecordKeyDeserializationException`, `RecordKeyDeserializationException` with the last argument or `RecordValueDeserializationException`. @@ -644,12 +1048,22 @@ it should be provided as a component by the consumer tag: @Override public void onPartitionsRevoked(Consumer consumer, Collection partitions) { - + // Called before partitions are revoked from this consumer. + // Use this to commit offsets or cleanup state. } @Override public void onPartitionsAssigned(Consumer consumer, Collection partitions) { + // Called when partitions are assigned to this consumer. + // Use this to initialize state for assigned partitions. + } + @Override + public void onPartitionsLost(Consumer consumer, Collection partitions) { + // Called when partitions are lost (e.g., consumer failure, group rebalance). + // Unlike onPartitionsRevoked, this is called when the consumer is no longer + // part of the group and cannot commit offsets. + // Use this to cleanup local state for lost partitions. } } ``` @@ -662,11 +1076,20 @@ it should be provided as a component by the consumer tag: class SomeListener : ConsumerAwareRebalanceListener { override fun onPartitionsRevoked(consumer: Consumer<*, *>, partitions: Collection) { - + // Called before partitions are revoked from this consumer. + // Use this to commit offsets or cleanup state. } override fun onPartitionsAssigned(consumer: Consumer<*, *>, partitions: Collection) { - + // Called when partitions are assigned to this consumer. + // Use this to initialize state for assigned partitions. + } + + override fun onPartitionsLost(consumer: Consumer<*, *>, partitions: Collection) { + // Called when partitions are lost (e.g., consumer failure, group rebalance). + // Unlike onPartitionsRevoked, this is called when the consumer is no longer + // part of the group and cannot commit offsets. + // Use this to cleanup local state for lost partitions. } } ``` @@ -692,12 +1115,28 @@ public interface BaseKafkaRecordsHandler { } ``` +### Telemetry { #telemetry } + +Kafka uses a telemetry contract for logging, metrics, and tracing of messages. +Telemetry configuration (the `telemetry { logging / metrics / tracing }` section) is described in the [Configuration](#config-consumer) section. + +For each batch & message, `KafkaListener` creates a separate telemetry context, which is closed upon completion of processing. +The message is described via telemetry handler parameters, including topic, partition, offset, and processing duration. + +The default factory, `DefaultKafkaListenerTelemetryFactory`, combines three factories: +- `KafkaListenerLoggerFactory` builds a `KafkaListenerLogger` to log the start and end of message processing; +- `KafkaListenerMetricsFactory` builds a `KafkaListenerMetrics` to record message metrics; +- `KafkaListenerTracerFactory` builds a `KafkaListenerTracer` for distributed tracing. + +Metrics and tracing are described in the [Metrics Reference](metrics.md#kafka) section. + ## Producer { #producer } -Descriptions of working with [Kafka Producer](https://docs.confluent.io/platform/current/clients/producer.html) +`Producer` sends records to a `topic`. Kora creates an implementation of the interface annotated with `@KafkaPublisher`, +selects a `Serializer` for the key and value, calls `KafkaProducer#send`, and connects sending with telemetry. -Assume to use the `@KafkaPublisher` annotation on the interface to create `Kafka Producer`, -in order to send messages to any topic it is supposed to create a method with the signature `ProducerRecord`: +To create a `Producer`, use the `@KafkaPublisher` annotation on an interface. +To send messages to an arbitrary `topic`, declare a method with a `ProducerRecord` parameter: ===! ":fontawesome-brands-java: `Java`" @@ -717,12 +1156,11 @@ in order to send messages to any topic it is supposed to create a method with th } ``` -The annotation parameter indicates the path to the configuration. +The annotation parameter indicates the path to the producer configuration. ### Topic { #topic } -In case it is required to use typed contracts for specific topics, the `@KafkaPublisher.Topic` annotation is supposed to be used -to create such contracts: +If typed methods are required for specific `topic` values, use the `@KafkaPublisher.Topic` annotation: ===! ":fontawesome-brands-java: `Java`" @@ -746,13 +1184,13 @@ to create such contracts: } ``` -The annotation parameter indicates the path for the configuration of the topic. +The annotation parameter indicates the path for the `topic` configuration. -### Configuration { #configuration-2 } +### Configuration { #config-producer } -Configuration describes the settings of a particular `@KafkaPublisher` and an example is given below for the configuration on the `kafka.someConsumer` path. +Configuration describes the settings of a particular `@KafkaPublisher`; below is an example for the `kafka.someProducer` configuration path. -Example of the complete configuration described in the `KafkaPublisherConfig` class (default or example values are specified): +Basic configuration parameters: ===! ":material-code-json: `Hocon`" @@ -762,27 +1200,11 @@ Example of the complete configuration described in the `KafkaPublisherConfig` cl driverProperties { //(1)! "bootstrap.servers": "localhost:9093" } - telemetry { - logging { - enabled = false //(2)! - } - metrics { - enabled = true //(3)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(4)! - } - tracing { - enabled = true //(5)! - } - } } } ``` - 1. *Properties* from the official kafka client, documentation on them can be found at [link](https://kafka.apache.org/documentation/#producerconfigs) (**required**) - 2. Enables module logging (default `false`) - 3. Enables module metrics (default `true`) - 4. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 5. Enables module tracing (default `true`) + 1. Official `Kafka Producer` `Properties` (`required`, no default) === ":simple-yaml: `YAML`" @@ -790,24 +1212,87 @@ Example of the complete configuration described in the `KafkaPublisherConfig` cl kafka: someProducer: driverProperties: #(1)! - bootstrap.servers: "localhost:9093" - telemetry: - logging: - enabled: true #(2)! - metrics: - enabled: true #(3)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(4)! - telemetry: - enabled: true #(5)! + "bootstrap.servers": "localhost:9093" ``` - 1. *Properties* from the official kafka client, documentation on them can be found at [link](https://kafka.apache.org/documentation/#producerconfigs) (**required**) - 2. Enables module logging (default `false`) - 3. Enables module metrics (default `true`) - 4. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 5. Enables module tracing (default `true`) + 1. Official `Kafka Producer` `Properties` (`required`, no default) + +??? note "Full Configuration" -Topic configuration describes the settings of a particular `@KafkaPublisher.Topic` and an example for the configuration at path `path.to.topic.config` is given below. + Example of the complete configuration described in the `KafkaPublisherConfig` class (default or example values are specified): + + ===! ":material-code-json: `Hocon`" + + ```javascript + kafka { + someProducer { + driverProperties { //(1)! + "bootstrap.servers": "localhost:9093" + } + telemetry { + logging { + enabled = false //(2)! + } + metrics { + enabled = true //(3)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(4)! + tags = { //(5)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(6)! + attributes = { //(7)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + ``` + + 1. Official `Kafka Producer` `Properties`; see [Apache Kafka Producer Configs](https://kafka.apache.org/documentation/#producerconfigs) (`required`, not set by default) + 2. Enables module logging (default: `false`) + 3. Enables module metrics (default: `true`) + 4. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 5. Configures metric tags (default: `{}`) + 6. Enables module tracing (default: `true`) + 7. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + kafka: + someProducer: + driverProperties: #(1)! + bootstrap.servers: "localhost:9093" + telemetry: + logging: + enabled: true #(2)! + metrics: + enabled: true #(3)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(4)! + tags: #(5)! + key1: value1 + key2: value2 + tracing: + enabled: true #(6)! + attributes: #(7)! + key1: value1 + key2: value2 + ``` + + 1. Official `Kafka Producer` `Properties`; see [Apache Kafka Producer Configs](https://kafka.apache.org/documentation/#producerconfigs) (`required`, not set by default) + 2. Enables module logging (default: `false`) + 3. Enables module metrics (default: `true`) + 4. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 5. Configures metric tags (default: `{}`) + 6. Enables module tracing (default: `true`) + 7. Configures tracing attributes (default: `{}`) + +`topic` configuration describes the settings of a particular `@KafkaPublisher.Topic`; below is an example for the `kafka.someProducer.someTopic` configuration path. Example of the complete configuration described in the `KafkaPublisherConfig.TopicConfig` class (default or example values are specified): @@ -824,8 +1309,10 @@ Example of the complete configuration described in the `KafkaPublisherConfig.Top } ``` - 1. Topic where method will send data (**required**) - 2. Partition of the topic where method will send data (optional) + 1. `topic` where the method sends data (`required`, not set by default) + 2. `topic` partition where the method sends data (not set by default, optional) + If specified, all messages will be sent to the specified partition. + If not specified, standard Kafka partitioning is used (by key or random). === ":simple-yaml: `YAML`" @@ -837,8 +1324,10 @@ Example of the complete configuration described in the `KafkaPublisherConfig.Top partition: 1 #(2)! ``` - 1. Topic where method will send data (**required**) - 2. Partition of the topic where method will send data (optional) + 1. `topic` where the method sends data (`required`, not set by default) + 2. `topic` partition where the method sends data (not set by default, optional) + If specified, all messages will be sent to the specified partition. + If not specified, standard Kafka partitioning is used (by key or random). ### Signatures { #signatures-2 } @@ -936,7 +1425,11 @@ It is possible to send `ProducerRecord` with or without `Callback` and combine t ### Serialization { #serialization } -In order to specify which `Serializer` to take from a container, there is an option to use tags. +`Serializer` is used to serialize `ProducerRecord` keys and values. +Kora provides `Serializer` components for basic types: `String`, `UUID`, `byte[]`, `Bytes`, `ByteBuffer`, `Double`, `Float`, +`Integer`, `Long`, `Short`, and `Void`. + +To specify which `Serializer` to take from the container, tags can be used. Tags should be set on `ProducerRecord` or `key`/`value` parameters of methods: ===! ":fontawesome-brands-java: `Java`" @@ -965,7 +1458,8 @@ Tags should be set on `ProducerRecord` or `key`/`value` parameters of methods: } ``` -If you want to serialize as Json, you should use `@Json` annotation: +If serialization to `JSON` is required, use the `@Json` tag. +In this case, Kora uses `JsonWriter` and `JsonKafkaSerializer` from the [JSON](json.md) module: ===! ":fontawesome-brands-java: `Java`" @@ -999,22 +1493,195 @@ If you want to serialize as Json, you should use `@Json` annotation: } ``` -### Exception handling { #exception-handling-2 } +### Custom Serializer { #custom-serializer } + +If custom serialization is required, you can implement your own `Serializer`. + +**Option 1: Default serializer for type** + +If you provide `Serializer` as a component without a tag, it will be used for all producers of this type: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public static class MyEventSerializer implements Serializer { + + private final JsonWriter writer; + + public MyEventSerializer(JsonWriter writer) { + this.writer = writer; + } + + @Override + public byte[] serialize(String topic, MyEvent data) { + try { + return writer.toByteArray(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + @KafkaPublisher("kafka.someProducer") + public interface MyPublisher { + + @KafkaPublisher.Topic("kafka.someProducer.topic") + void send(MyEvent value); // Uses MyEventSerializer + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyEventSerializer( + private val writer: JsonWriter + ) : Serializer { + + override fun serialize(topic: String, data: MyEvent): ByteArray { + return try { + writer.toByteArray(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + @KafkaPublisher("kafka.someProducer") + interface MyPublisher { + + @KafkaPublisher.Topic("kafka.someProducer.topic") + fun send(value: MyEvent) // Uses MyEventSerializer + } + ``` + +**Option 2: Point serializer for specific producer** + +If you need to use different serialization for different producers of the same type, you can use tags: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaPublisher("kafka.someProducer") + public interface MyKafkaProducer { + + @Json + record MyEvent(String username, int code) {} + + @Tag(MyEvent.class) + @Component + class MySerializer implements Serializer { + + private final JsonWriter writer; + + public MySerializer(JsonWriter writer) { + this.writer = writer; + } + + @Override + public byte[] serialize(String topic, MyEvent data) { + try { + return writer.toByteArray(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + void send(ProducerRecord record); + } + ``` + +=== ":simple-kotlin: `Kotlin`" -In case of a submission error in a method annotated `@Topic` and which does not return `Future` a `ru.tinkoff.kora.kafka.kora.kafka.common.exceptions.KafkaPublishException` will be thrown. -where in `cause` will lie the actual error from `KafkaProducer`. + ```kotlin + @KafkaPublisher("kafka.someProducer") + interface MyKafkaProducer { + + @Json + data class MyEvent(val username: String, val code: Int) + + @Tag(MyEvent::class) + @Component + class MySerializer( + private val writer: JsonWriter + ) : Serializer { + + override fun serialize(topic: String, data: MyEvent): ByteArray { + return try { + writer.toByteArray(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + fun send(record: ProducerRecord) + } + ``` + +### Exception handling { #exception-handling-producer } + +If a send error happens in a method annotated with `@KafkaPublisher.Topic` that does not return `Future`, +`ru.tinkoff.kora.kafka.common.exceptions.KafkaPublishException` is thrown. +The original error from `KafkaProducer` is available in `cause`. + +To get the failed record, use the `getRecord()` method: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + class SomeService { + + private final MyPublisher publisher; + + public SomeService(MyPublisher publisher) { + this.publisher = publisher; + } + + void sendMessage() { + try { + publisher.send("key", "value"); + } catch (KafkaPublishException e) { + ProducerRecord failedRecord = e.getRecord(); + // Handle the failed record (log, retry, etc.) + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val publisher: MyPublisher + ) { + + fun sendMessage() { + try { + publisher.send("key", "value") + } catch (e: KafkaPublishException) { + val failedRecord = e.getRecord() + // Handle the failed record (log, retry, etc.) + } + } + } + ``` #### Serialization errors { #serialization-errors } -In case of a key/value serialization error in a method annotated with `@Topic`, `org.apache.kafka.common.errors.SerializationException` will be thrown -similar to what would happen in the case of `org.apache.kafka.kafka.clients.producer.Producer#send`. +If a key or value serialization error happens in a method annotated with `@KafkaPublisher.Topic`, +`org.apache.kafka.common.errors.SerializationException` is thrown, just like with a direct `org.apache.kafka.clients.producer.Producer#send` call. ### Transactions { #transactions } -It is possible to send a message to Kafka in [within a transaction](https://www.confluent.io/blog/transactions-apache-kafka/), this is supposed to use the -`@KafkaPublisher` annotation and inherit `TransactionalPublisher` interface to create such a `KafkaProducer`. +Messages can be sent to `Kafka` [within a transaction](https://www.confluent.io/blog/transactions-apache-kafka/). +For this, use the `@KafkaPublisher` annotation and extend `TransactionalPublisher`. -It is required to first create a regular `KafkaProducer` and then use it to create a transactional Producer: +First, describe a regular `KafkaProducer`, and then use its type to create a transactional `Producer`: ===! ":fontawesome-brands-java: `Java`" @@ -1047,7 +1714,8 @@ It is required to first create a regular `KafkaProducer` and then use it to crea interface MyTransactionalPublisher : TransactionalPublisher ``` -It is expected to use `inTx` methods to send such messages, all messages within Lambda will be applied if it is successful and canceled if it fails. +Use `inTx` methods to send messages in a transaction: all messages inside the `lambda` are committed on successful execution +and aborted on error. ===! ":fontawesome-brands-java: `Java`" @@ -1067,7 +1735,7 @@ It is expected to use `inTx` methods to send such messages, all messages within }) ``` -It is also possible to manually perform all manipulations with `KafkaProducer`: +It is also possible to manage the transaction manually through `begin()`: ===! ":fontawesome-brands-java: `Java`" @@ -1093,7 +1761,7 @@ It is also possible to manually perform all manipulations with `KafkaProducer`: } ``` -#### Configuration { #configuration-3 } +#### Configuration { #config-producer-tx } `KafkaPublisherConfig.TransactionConfig` is used to configure `@KafkaPublisher` with the `TransactionalPublisher` interface: @@ -1102,45 +1770,203 @@ It is also possible to manually perform all manipulations with `KafkaProducer`: ```javascript kafka { someTransactionalProducer { - idPrefix = "kafka-app-" //(1)! + idPrefix = "kora-app-" //(1)! maxPoolSize = 10 //(2)! maxWaitTime = "10s" //(3)! } } ``` - 1. Transaction identifier prefix - 2. Connection set size for transactions - 3. Maximum transaction waiting time + 1. Transaction identifier prefix; a random `UUID` will be appended to it (default: `kora-app-`) + Format: `{idPrefix}-{uuid}`. Example: `kafka-app-550e8400-e29b-41d4-a716-446655440000`. + 2. Maximum size of the transactional `Producer` pool (default: `10`) + 3. Maximum time to wait for a free `Producer` from the pool (default: `10s`) === ":simple-yaml: `YAML`" ```yaml kafka: someTransactionalProducer: - idPrefix: "kafka-app-" #(1)! + idPrefix: "kora-app-" #(1)! maxPoolSize: 10 #(2)! maxWaitTime: "10s" #(3)! ``` - 1. Transaction identifier prefix - 2. Connection set size for transactions - 3. Maximum transaction waiting time + 1. Transaction identifier prefix; a random `UUID` will be appended to it (default: `kora-app-`) + Format: `{idPrefix}-{uuid}`. Example: `kafka-app-550e8400-e29b-41d4-a716-446655440000`. + 2. Maximum size of the transactional `Producer` pool (default: `10`) + 3. Maximum time to wait for a free `Producer` from the pool (default: `10s`) + +### Advanced Transaction Usage { #advanced-transactions } + +#### Transaction Interface { #transaction-interface } + +The `begin()` method returns a `Transaction

` object that provides advanced transaction management capabilities: + +===! ":fontawesome-brands-java: `Java`" + + ```java + try (var tx = transactionalPublisher.begin()) { + // Sending messages + tx.publisher().send("key1", "value1"); + tx.publisher().send("key2", "value2"); + + // Commit consumer offsets within the transaction (exactly-once semantics) + Map offsets = ...; + ConsumerGroupMetadata groupMetadata = ...; + tx.sendOffsetsToTransaction(offsets, groupMetadata); + + // Explicit flush to guarantee sending before commit + tx.flush(); + + // commit() is called automatically on try-with-resources close + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + transactionalPublisher.begin().use { tx -> + // Sending messages + tx.publisher().send("key1", "value1") + tx.publisher().send("key2", "value2") + + // Commit consumer offsets within the transaction (exactly-once semantics) + val offsets: Map = ... + val groupMetadata: ConsumerGroupMetadata = ... + tx.sendOffsetsToTransaction(offsets, groupMetadata) + + // Explicit flush to guarantee sending before commit + tx.flush() + + // commit() is called automatically on use close + } + ``` + +**`Transaction

` methods:** + +| Method | Description | +|--------|-------------| +| `publisher()` | Returns typed publisher for sending messages | +| `producer()` | Returns raw `Producer` for low-level operations | +| `sendOffsetsToTransaction(offsets, groupMetadata)` | Commits consumer offsets within the same transaction | +| `flush()` | Guarantees all messages are sent before commit | +| `abort()` | Aborts the transaction | +| `abort(cause)` | Aborts the transaction with specified cause | +| `close()` | Closes the transaction (commits if no abort) | + +#### Transaction methods { #tx-methods } + +`TransactionalPublisher` provides 4 methods for working with transactions: + +| Method | Passes to callback | Returns value | +|--------|-------------------|---------------| +| `inTx(TransactionalConsumer)` | `P publisher` | `void` | +| `inTx(TransactionalFunction)` | `P publisher` | `R` | +| `withTx(TransactionConsumer)` | `Transaction

tx` | `void` | +| `withTx(TransactionFunction)` | `Transaction

tx` | `R` | + +**Example with return value:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + // inTx with return value + Long messageId = transactionalPublisher.inTx(producer -> { + producer.send("key", "value"); + return System.currentTimeMillis(); + }); + + // withTx with Transaction access + transactionalPublisher.withTx(tx -> { + tx.publisher().send("key", "value"); + tx.sendOffsetsToTransaction(offsets, groupMetadata); + tx.flush(); // Explicit flush + }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // inTx with return value + val messageId = transactionalPublisher.inTx { producer -> + producer.send("key", "value") + System.currentTimeMillis() + } + + // withTx with Transaction access + transactionalPublisher.withTx { tx -> + tx.publisher().send("key", "value") + tx.sendOffsetsToTransaction(offsets, groupMetadata) + tx.flush() // Explicit flush + } + ``` + +#### Default Serializers and Deserializers { #default-serializers } + +`KafkaModule` automatically provides serializers and deserializers for base types via `KafkaSerializersModule` and `KafkaDeserializersModule`. + +These serializers/deserializers are provided as components **without tags** and are used by default for all consumers/producers of corresponding types. -### Сигнатуры { #signatures-3 } +**Supported types out of the box:** -Signatures available for Kafka producer methods out of the box, where `K` refers to the key type and `V` refers to the message value type. +| Type | Serializer | Deserializer | +|------|------------|--------------| +| `String` | `StringSerializer` | `StringDeserializer` | +| `byte[]` | `ByteArraySerializer` | `ByteArrayDeserializer` | +| `ByteBuffer` | `ByteBufferSerializer` | `ByteBufferDeserializer` | +| `Bytes` | `BytesSerializer` | `BytesDeserializer` | +| `UUID` | `UUIDSerializer` | `UUIDDeserializer` | +| `Integer` | `IntegerSerializer` | `IntegerDeserializer` | +| `Long` | `LongSerializer` | `LongDeserializer` | +| `Short` | `ShortSerializer` | `ShortDeserializer` | +| `Double` | `DoubleSerializer` | `DoubleDeserializer` | +| `Float` | `FloatSerializer` | `FloatDeserializer` | +| `Void` | `VoidSerializer` | `VoidDeserializer` | -Allows sending `value` (required), `key` (optional), and `headers` (optional) from `ProducerRecord`: +To use, simply specify the type in the publisher/consumer method: ===! ":fontawesome-brands-java: `Java`" ```java @KafkaPublisher("kafka.someProducer") public interface MyPublisher { + @KafkaPublisher.Topic("kafka.someProducer.topic") + void send(UUID key, String value); + } + ``` - @KafkaPublisher.Topic("kafka.someProducer.someTopic") - void send(K key, V value, Headers headers); +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaPublisher("kafka.someProducer") + interface MyPublisher { + @KafkaPublisher.Topic("kafka.someProducer.topic") + fun send(key: UUID, value: String) + } + ``` + +### Signatures { #signatures-3 } + +Available signatures for out-of-the-box `Kafka Producer` methods, where `K` refers to the key type and `V` to the message value type. +The generator supports two signature families: sending a ready `ProducerRecord` and sending through a method annotated with `@KafkaPublisher.Topic`. +These families cannot be mixed in the same method. + +#### Prepared event { #producer-record-signature } + +A method with `ProducerRecord` is used when the `topic`, partition, timestamp, or `Headers` should be set by the calling code. +Such a method cannot be annotated with `@KafkaPublisher.Topic`, because all send details are already contained in the `ProducerRecord`. +One `Callback` can be passed additionally. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaPublisher("kafka.someProducer") + public interface MyPublisher { + + void send(ProducerRecord record); + + void send(ProducerRecord record, Callback callback); } ``` @@ -1150,47 +1976,67 @@ Allows sending `value` (required), `key` (optional), and `headers` (optional) fr @KafkaPublisher("kafka.someProducer") interface MyPublisher { - @KafkaPublisher.Topic("kafka.someProducer.someTopic") - fun send(key: K, value: V, headers: Headers) + fun send(record: ProducerRecord) + + fun send(record: ProducerRecord, callback: Callback) } ``` -===! ":fontawesome-brands-java: `Java`" +#### Methods per topic { #topic-signature } + +A method with `key`, `value`, and `Headers` must be annotated with `@KafkaPublisher.Topic`. +One user argument is treated as `value`; two user arguments are treated as `key` and `value` in that exact order. +`Headers` and `Callback` can be declared additionally, but only one argument of each type is allowed. +If `Headers` is not passed, Kora creates empty headers. - Result of the `RecordMetadata` operation can be either `Future` or `CompletionStage`: +===! ":fontawesome-brands-java: `Java`" ```java @KafkaPublisher("kafka.someProducer") public interface MyPublisher { @KafkaPublisher.Topic("kafka.someProducer.someTopic") - RecordMetadata send(V value); + void send(V value); @KafkaPublisher.Topic("kafka.someProducer.someTopic") - Future sendFuture(V value); + void send(K key, V value); @KafkaPublisher.Topic("kafka.someProducer.someTopic") - CompletionStage sendStage(V value); + void send(K key, V value, Headers headers); + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + void send(K key, V value, Headers headers, Callback callback); } ``` === ":simple-kotlin: `Kotlin`" - You can get it as a result of the `RecordMetadata` operation or have the `suspend` modifier: - ```kotlin @KafkaPublisher("kafka.someProducer") interface MyPublisher { @KafkaPublisher.Topic("kafka.someProducer.someTopic") - fun send(value: V): RecordMetadata + fun send(value: V) @KafkaPublisher.Topic("kafka.someProducer.someTopic") - suspend fun sendSuspend(value: V): RecordMetadata - } + fun send(key: K, value: V) + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(key: K, value: V, headers: Headers) + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(key: K, value: V, headers: Headers, callback: Callback) + } ``` -It is possible to send `ProducerRecord` and `Callback` (optional) and combine response signatures: +#### Send result { #publisher-result } + +For a synchronous method, the return type can be `void`/`Unit` or `RecordMetadata`. +In this case, Kora calls `KafkaProducer#send`, waits for send completion through `Future#get()`, and only then returns control to the caller. + +For asynchronous sending, the return type can be `Future`, `CompletionStage`, or `CompletableFuture`. +In `Kotlin`, `suspend` methods and `Deferred` are also supported. +If the signature contains a `Callback`, Kora first completes its own send telemetry and then calls the user `Callback`. ===! ":fontawesome-brands-java: `Java`" @@ -1198,7 +2044,17 @@ It is possible to send `ProducerRecord` and `Callback` (optional) and combine re @KafkaPublisher("kafka.someProducer") public interface MyPublisher { - void send(ProducerRecord record, Callback callback); + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + RecordMetadata send(V value); + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + Future sendFuture(V value); + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + CompletionStage sendStage(V value); + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + CompletableFuture sendCompletableFuture(V value); } ``` @@ -1208,6 +2064,39 @@ It is possible to send `ProducerRecord` and `Callback` (optional) and combine re @KafkaPublisher("kafka.someProducer") interface MyPublisher { - fun send(record: ProducerRecord, callback: Callback) - } + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: V): RecordMetadata + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + suspend fun sendSuspend(value: V): RecordMetadata + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: String): Future + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: String): CompletionStage + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: String): CompletableFuture + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: String): Deferred + } ``` + +Invalid combinations are: `ProducerRecord` together with `@KafkaPublisher.Topic`, `ProducerRecord` together with separate `key`/`value`/`Headers`, more than one `Headers`, more than one `Callback`, and a method with separate `key`/`value` without `@KafkaPublisher.Topic`. + +### Telemetry { #telemetry } + +Kafka uses a telemetry contract for logging, metrics, and tracing of messages. +Telemetry configuration (the `telemetry { logging / metrics / tracing }` section) is described in the [Configuration](#config-producer) section. + +For each message, `KafkaPublisher` creates a telemetry context, which is closed upon completion of processing. +The message is described via telemetry handler parameters, including topic, partition, offset, and processing duration. + +The default factory, `DefaultKafkaPublisherTelemetryFactory`, combines three factories: +- `KafkaPublisherLoggerFactory` builds a `KafkaPublisherLogger` to log the start and end of message processing; +- `KafkaPublisherMetricsFactory` builds a `KafkaPublisherMetrics` to record message metrics; +- `KafkaPublisherTracerFactory` builds a `KafkaPublisherTracer` for distributed tracing. + +Metrics and tracing are described in the [Metrics Reference](metrics.md#kafka) section. diff --git a/mkdocs/docs/en/documentation/logging-aspect.md b/mkdocs/docs/en/documentation/logging-aspect.md index 5e4b161..219bae9 100644 --- a/mkdocs/docs/en/documentation/logging-aspect.md +++ b/mkdocs/docs/en/documentation/logging-aspect.md @@ -4,13 +4,16 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora logging aspects for argument and result logging, selective logging, MDC enrichment, structured parameters, conversion, and signatures; key triggers include @Log, @Log.in, @Log.out, @Log.off, @Mdc, @StructuredArgument, MDC, LogAspect." --- -Module for declarative logging of method arguments and result using annotations. +The declarative logging module lets you describe method logging with `@Log` and `@Mdc` annotations. +At compile time, Kora creates an aspect wrapper for the method; the wrapper logs method entry, method exit, result, error, and `MDC` values without manual code in business logic. +This is useful for consistent call diagnostics, especially when you need to quickly understand which method was called, with which arguments, and how it completed. For a step-by-step walkthrough before the reference details, see [Observability](../guides/observability.md). ## Dependency { #dependency } -Most likely already transitively connected from other dependencies or from [Logback](logging-slf4j.md#logback), otherwise it needs to be added: +Annotations and helper classes are provided by the `logging-common` dependency. +Usually it is already brought by other Kora modules or by [Logback](logging-slf4j.md#logback), but when using the annotations directly, the dependency can be added explicitly: ===! ":fontawesome-brands-java: `Java`" @@ -38,143 +41,419 @@ Most likely already transitively connected from other dependencies or from [Logb interface Application : LoggingModule ``` +Aspect generation also requires the common [annotation processors](general.md#annotation-processor) or [`KSP` processors](general.md#ksp). +In a regular Kora application, they are already connected as part of the basic project setup. + ## Logging { #logging } -It is expected to use special combinations of annotations to customize method logging. +Method logging is configured with annotation combinations: -### Argument { #argument } +- `@Log` - logs method entry and exit (default: `INFO`). +- `@Log.in` - logs only method entry (default: `INFO`). +- `@Log.out` - logs only method exit (default: `INFO`). +- `@Log.result` - sets the level from which the result value is added to the log (default: `DEBUG`). +- `@Log.off` - disables logging of the method result or a specific parameter. +- `@Log(Level)` on a parameter - sets the level from which the parameter is added to structured data (default: `DEBUG` for a parameter without a separate annotation). -```java -@Log.in -public String methodWithReturnAndOnlyLogArgs(@Log.off String strParam,int numParam) { - return"testResult"; -} -``` +The entry or exit event itself is written at the level specified in `@Log`, `@Log.in`, or `@Log.out`. +Argument and result values are added to structured data only when the corresponding detail level is enabled. +Which detail level is active depends on the effective logger level configured through `logging.level` / `logging.levels` — see [logging levels configuration](logging-slf4j.md#configuration). + +### Arguments { #argument } + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log.in + public String doWork(@Log.off String strParam, int numParam) { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log.`in` + fun doWork(@Log.off strParam: String?, numParam: Int): String { + return "testResult" + } + ``` - - + + - + + + + +
Logging LevelLoggingLogging levelLog
TRACE, DEBUGDEBUG +

INFO [main] r.t.e.e.Example.doWork: > {data: {numParam: "4"}}

+
TRACE -

DEBUG [] r.t.e.e.Example.methodWithArgs: > {data: {numParam: "4"}}

+

INFO [main] r.t.e.e.Example.doWork: > {data: {numParam: "4"}}
INFO -

INFO [] r.t.e.e.Example.methodWithArgs: >

+

INFO [main] r.t.e.e.Example.doWork: >
### Result { #result } -```java -@Log.out -public String methodWithOnlyLogReturnAndArgs(String strParam, int numParam) { - return "testResult" -} -``` +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log.out + public String doWork(String strParam, int numParam) { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log.out + fun doWork(strParam: String, numParam: Int): String { + return "testResult" + } + ``` - + - + + + + +
Logging levelLoggingLog
TRACE, DEBUGDEBUG +

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}

+
TRACE -

DEBUG [] r.t.e.e.Example.methodWithArgs: < {data: {out: "testResult"}}

+

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}
INFO -

INFO [] r.t.e.e.Example.methodWithArgs: <

+

INFO [main] r.t.e.e.Example.doWork: <
-### Argument and result { #argument-and-result } +### Arguments And Result { #argument-and-result } -```java -@Log -public String methodWithArgs(String strParam, int numParam) { - return "testResult"; -} -``` +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log + public String doWork(String strParam, int numParam) { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log + fun doWork(strParam: String, numParam: Int): String { + return "testResult" + } + ``` - - + + - + + + + +
Logging LevelLoggingLogging levelLog
TRACE, DEBUGDEBUG +

INFO [main] r.t.e.e.Example.doWork: > {data: {strParam: "s", numParam: "4"}}

+

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}

+
TRACE -

DEBUG [] r.t.e.e.Example.methodWithArgs: > {data: {strParam: "s", numParam: "4"}}

-

DEBUG [] r.t.e.e.Example.methodWithArgs: < {data: {out: "testResult"}}

+

INFO [main] r.t.e.e.Example.doWork: > {data: {strParam: "s", numParam: "4"}}

+

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}
INFO -

INFO [] r.t.e.e.Example.methodWithArgs: >

-

INFO [] r.t.e.e.Example.methodWithArgs: <

+

INFO [main] r.t.e.e.Example.doWork: >

+

INFO [main] r.t.e.e.Example.doWork: <
-### Selective logging { #selective-logging } +If a method completes with an error, the aspect logs method exit with error data: `errorType` and `errorMessage`. +When `DEBUG` is enabled, the exception object is also passed to the log. -```java -@Log.out -@Log.off -public String methodWithOnlyLogReturnAndArgs(String strParam,int numParam) { - return"testResult"; -} -``` +### Selective Logging { #selective-logging } + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log.out + @Log.off + public String doWork(String strParam, int numParam) { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log.out + @Log.off + fun doWork(strParam: String, numParam: Int): String { + return "testResult" + } + ``` - - + + + + + + + +
Logging LevelLoggingLogging levelLog
TRACE, DEBUG -

INFO [] r.t.e.e.Example.methodWithArgs: <

+

INFO [main] r.t.e.e.Example.doWork: <

+
INFO +

INFO [main] r.t.e.e.Example.doWork: <

+
+ +In this example, `@Log.off` on the method disables writing the result value, but does not disable the method exit event itself. +To exclude a specific argument from the log, put `@Log.off` on the parameter. + +Parameter detail level can be configured separately: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log.in + public void doWork(@Log(Level.INFO) String id, @Log(Level.TRACE) String payload) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log.`in` + fun doWork(@Log(Level.INFO) id: String, @Log(Level.TRACE) payload: String) { } + ``` + +At `INFO`, only `id` is added to structured data, while `payload` appears only when `TRACE` is enabled. + +The result value can be emitted already at `INFO` by explicitly setting `@Log.result(Level.INFO)`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log.out + @Log.result(Level.INFO) + public String doWork() { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log.out + @Log.result(Level.INFO) + fun doWork(): String { + return "testResult" + } + ``` + +### Structured Parameter { #structured-parameter } + +If a string representation of a parameter is not suitable for the log, the parameter type can implement the `StructuredArgument` interface. +In that case, the object defines the field name through `fieldName()` and writes the value to `JsonGenerator` through `writeTo(...)`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record Entity(String name, String code) implements StructuredArgument { + + @Override + public String fieldName() { + return "name"; + } + + @Override + public void writeTo(JsonGenerator generator) throws IOException { + generator.writeString(name); + } + } + + @Log.in + public String doWork(Entity entity) { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class Entity(val name: String, val code: String) : StructuredArgument { + + override fun writeTo(generator: JsonGenerator) = generator.writeString(name) + + override fun fieldName(): String = "name" + } + + @Log.`in` + fun doWork(entity: Entity): String { + return "testResult" + } + ``` + + + + + + + + + + + + + + +
Logging levelLog
DEBUG, TRACE +

INFO [main] r.t.e.e.Example.doWork: >

+

     data={"entity":"Bob"}

+
INFO +

INFO [main] r.t.e.e.Example.doWork: >

+
+ +When you need a structured value without introducing a dedicated type, the `StructuredArgument` interface exposes static factory helpers: +`arg(fieldName, value)` / `arg(fieldName, value, JsonWriter)` build a structured argument (overloads accept `String`, `Integer`, `Long`, `Boolean`, `Map`, a `JsonWriter`, or a raw `StructuredArgumentWriter`), +while `marker(fieldName, value)` builds an `org.slf4j.Marker` for a single log call. The resulting `StructuredArgument` can also be passed straight into `MDC.put`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + // ad-hoc structured value fed into MDC + MDC.put("order", StructuredArgument.arg("orderId", orderId)); + + // or as an SLF4J marker on a single log line + log.info(StructuredArgument.marker("orderId", orderId), "order accepted"); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // ad-hoc structured value fed into MDC + MDC.put("order", StructuredArgument.arg("orderId", orderId)) + + // or as an SLF4J marker on a single log line + log.info(StructuredArgument.marker("orderId", orderId), "order accepted") + ``` + +### Parameter Conversion { #parameter-conversion } + +If the parameter type cannot be changed, describe an external `StructuredArgumentMapper` and specify it through `@Mapping` on the required argument. +The mapper receives the original parameter value and writes the structured value to `JsonGenerator`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record Entity(String name, String code) { } + + public final class EntityLogMapper implements StructuredArgumentMapper { + public void write(JsonGenerator gen, Entity value) throws IOException { + gen.writeString(value.name()); + } + } + + @Log.in + public String doWork(@Mapping(EntityLogMapper.class) Entity entity) { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class Entity(val name: String, val code: String) + + class EntityLogMapper : StructuredArgumentMapper { + + @Throws(IOException::class) + override fun write(gen: JsonGenerator, value: Entity) = gen.writeString(value.name) + } + + @Log.`in` + fun doWork(@Mapping(EntityLogMapper::class) entity: Entity): String { + return "testResult" + } + ``` + + + + + + + + +
Logging levelLog
DEBUG, TRACE +

INFO [main] r.t.e.e.Example.doWork: >

+

     data={"entity":"Bob"}
INFO -

INFO [] r.t.e.e.Example.methodWithArgs: <

+

INFO [main] r.t.e.e.Example.doWork: >
### MDC (Mapped Diagnostic Context) { #mdc-mapped-diagnostic-context } -The `@Mdc` annotation allows adding key-value pairs to MDC (Mapped Diagnostic Context) for structured logging. -MDC allows adding contextual information to each log message. +The `@Mdc` annotation adds key-value pairs to `MDC` (`Mapped Diagnostic Context`). +`MDC` stores execution context and lets you add it to log messages: for example, request, user, or operation identifiers. + +The annotation can be applied to methods and method parameters. +Repeated `@Mdc` usage is supported on methods. +Values added without `global = true` are restored after method execution. -The annotation can be applied to methods and method parameters. Multiple application is supported. +**`@Mdc` annotation parameters:** -**Parameters of the `@Mdc` annotation:** +- `key()` - `MDC` entry key (default: `""`). +- `value()` - `MDC` entry value (default: `""`). +- `global()` - keep the value in `MDC` after method exit (default: `false`). -- `key()` - Key for the MDC entry. If not specified, the name of the annotated parameter is used. -- `value()` - Value for the MDC entry. If not specified, the value of the annotated parameter is used. -- `global()` - If true, the MDC value will be available globally within the thread, not just during method execution. +For `@Mdc` on a method, non-empty `key` and `value` are required. +For `@Mdc` on a parameter, the key is taken from `key`, then from `value`; if both values are empty, the parameter name is used. +The entry value is the parameter value. -#### Parameter annotation { #parameter-annotation } +#### Parameter Annotation { #parameter-annotation } ===! ":fontawesome-brands-java: `Java`" - ```java + ```java public String test(@Mdc String s) { return "1"; } @@ -188,13 +467,13 @@ The annotation can be applied to methods and method parameters. Multiple applica } ``` -In this case, the MDC key will match the parameter name ("s"), and the value will be the parameter value. +In this case, the `MDC` key matches the parameter name `s`, and the value is the parameter value. -#### Parameter annotation with key { #parameter-annotation-with-key } +#### Parameter Annotation With Key { #parameter-annotation-with-key } ===! ":fontawesome-brands-java: `Java`" - ```java + ```java public String test(@Mdc(key = "123") String s) { return "1"; } @@ -208,13 +487,13 @@ In this case, the MDC key will match the parameter name ("s"), and the value wil } ``` -Here, the MDC key will be "123", and the value will be the value of parameter "s". +Here, the `MDC` key is `123`, and the value is the parameter value `s`. -#### Method use { #method-use } +#### Method Annotation { #method-use } ===! ":fontawesome-brands-java: `Java`" - ```java + ```java @Mdc(key = "key1", value = "value2") public String test(String s) { return "1"; @@ -230,17 +509,17 @@ Here, the MDC key will be "123", and the value will be the value of parameter "s } ``` -This example demonstrates: -- Method annotation with local MDC value +In this example, the `key1=value2` entry is added to `MDC` before the method call. +After the method completes, the previous `key1` value is restored. #### Combined { #combined } ===! ":fontawesome-brands-java: `Java`" - ```java + ```java @Mdc(key = "key", value = "value", global = true) @Mdc(key = "key1", value = "value2") - public Integer test(@Mdc(key = "123") String s) { + public String test(@Mdc(key = "123") String s) { return "1"; } ``` @@ -255,15 +534,19 @@ This example demonstrates: } ``` -In this example, two MDC annotations are applied to the method, and one annotation is applied to the parameter. +In this example, two `@Mdc` annotations are applied to the method, and one is applied to the parameter. +The `key=value` entry remains in `MDC` after method execution because of `global = true`; the other entries are restored or removed. -#### Generated value for MDC value { #generated-value-for-mdc-value } +Under the hood, non-global entries are snapshotted before the call and restored in a `finally` block once the method returns, so they never leak beyond the method scope. +Entries added with `global = true` (and any value set through the imperative `MDC.put`, see below) stay in the `Context` for the remainder of the request/thread scope and are therefore visible to every subsequent log line. + +#### Generated Value From Code { #generated-value-for-mdc-value } ===! ":fontawesome-brands-java: `Java`" - ```java + ```java @Mdc(key = "key", value = "${java.util.UUID.randomUUID().toString()}") - public Integer test(String s) { + public String test(String s) { return "1"; } ``` @@ -273,39 +556,105 @@ In this example, two MDC annotations are applied to the method, and one annotati ```kotlin @Mdc(key = "key", value = "\${java.util.UUID.randomUUID().toString()}") fun test(s: String): String { - return "1" + return "1"; } ``` -When calling the method, an MDC entry will be added with the key "key" and a value as generated random UUID. +When the method is called, an `MDC` entry with key `key` is added, and the value is a random `UUID`. +For `Java`, a value in the `${...}` format is inserted into the generated code as an expression. -**Example log with MDC:** +**Example log with `MDC`:** ``` INFO [main] r.t.e.e.Example.test: > {data: {s: "testValue"}} key=some-uuid-value key1=value2 123=testValue ``` +`@Mdc` is not supported for methods that return `CompletionStage`, `Mono`, or `Flux`. +For `Kotlin`, regular methods and `suspend` methods are supported, but `global = true` cannot be used in `suspend` methods. + +### Imperative MDC { #imperative-mdc } + +Where an annotation does not fit — inside interceptors, filters, or plain service code — use the imperative `ru.tinkoff.kora.logging.common.MDC` API. +It is the programmatic counterpart of `@Mdc`: entries are bound to Kora's `Context`, so they propagate across async boundaries exactly like `@Mdc(global = true)` entries and appear in every log line emitted for the remainder of the current `Context` scope. + +The static `put` method has overloads for `String`, `Integer`, `Long`, and `Boolean` values, plus a `StructuredArgumentWriter` overload for structured values. +`remove(key)` drops a single entry, and `get().values()` returns the current entries as an unmodifiable `Map`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.logging.common.MDC; + + @Component + public final class OrderService { + + public void process(String orderId) { + MDC.put("orderId", orderId); // String + MDC.put("attempt", 1); // Integer + MDC.put("bytes", 1024L); // Long + MDC.put("retryable", true); // Boolean + MDC.put("payload", gen -> gen.writeString(orderId)); // StructuredArgumentWriter + + // ... business logic; every log line in this Context now carries the keys + + MDC.remove("attempt"); // drop a single key + var current = MDC.get().values(); // read current entries + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.logging.common.MDC + + @Component + class OrderService { + + fun process(orderId: String) { + MDC.put("orderId", orderId) // String + MDC.put("attempt", 1) // Integer + MDC.put("bytes", 1024L) // Long + MDC.put("retryable", true) // Boolean + MDC.put("payload") { gen -> gen.writeString(orderId) } // StructuredArgumentWriter + + // ... business logic; every log line in this Context now carries the keys + + MDC.remove("attempt") // drop a single key + val current = MDC.get().values() // read current entries + } + } + ``` + +When you already hold a `Context` (for example inside an interceptor), address it explicitly through `MDC.get(ctx)` and `MDC.put(ctx, key, writer)` instead of the current-`Context` shortcuts. +Unlike `@Mdc`, the imperative API has no reactive/`suspend` restriction, because it writes directly to the `Context` rather than wrapping the method call. + +!!! warning "Use Kora's `MDC`, not SLF4J's" + + Always import `ru.tinkoff.kora.logging.common.MDC` — never `org.slf4j.MDC`. + The SLF4J class writes to a separate `ThreadLocal` that is not tied to Kora's `Context`: values placed there will not appear in Kora structured logs and will not propagate across async boundaries (reactive operators, `suspend` functions, thread hand-offs). + ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Method signatures supported for logging aspects: ===! ":fontawesome-brands-java: `Java`" - Class must be non `final` in order for aspects to work. + The class must not be `final` so that aspects can create a subclass. - The `T` refers to the type of the return value, either `Void`. + `T` means the return value type or `Void`. - `T myMethod()` - `Optional myMethod()` - - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) (only for `@Log`) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (only for `@Log`, requires [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (only for `@Log`, requires [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - Class must be `open` in order for aspects to work. + The class must be `open` so that aspects can create a subclass. - By `T` we mean the type of the return value, either `T?`, either `Unit`. + `T` means the return value type, `T?`, or `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `myMethod(): Flow` [Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) diff --git a/mkdocs/docs/en/documentation/logging-slf4j.md b/mkdocs/docs/en/documentation/logging-slf4j.md index a3abf2e..79316b1 100644 --- a/mkdocs/docs/en/documentation/logging-slf4j.md +++ b/mkdocs/docs/en/documentation/logging-slf4j.md @@ -1,66 +1,97 @@ --- -description: "Explains Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC. Use when working with Slf4jModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." +description: "Explains Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC. Use when working with LoggingModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC; key triggers include Slf4jModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." + use_when: "Use this file for Kora docs or implementation questions about Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC; key triggers include LoggingModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." --- -Kora uses [slf4j-api](https://www.slf4j.org/) as the logging engine for the entire framework, -it is expected that an implementation based on [Logback](#logback) will be used. +Kora uses [`slf4j-api`](https://www.slf4j.org/) as the common logging facade across the framework. +`SLF4J` separates application code from the concrete logging implementation, and Kora expects [`Logback`](#logback) to be used as the main implementation. + +The logging module is responsible for obtaining a `Logger` through the standard `SLF4J` factory, managing logging levels through Kora configuration, and passing structured data to log records. +Structured data can be added through `StructuredArgument`, `Marker`, and `MDC` so that it is emitted together with the regular text message. For a step-by-step walkthrough before the reference details, see [Observability](../guides/observability.md). ## Usage { #usage } -Loggers are required to be provided through the [SLF4J](https://www.slf4j.org/manual.html#hello_world) factory. +A `Logger` is created through the [`SLF4J`](https://www.slf4j.org/manual.html#hello_world) factory: ===! ":fontawesome-brands-java: `Java`" ```java - Logger logger = LoggerFactory.getLogger(SomeService.class) + Logger logger = LoggerFactory.getLogger(SomeService.class); ``` === ":simple-kotlin: `Kotlin`" ```kotlin - val logger = LoggerFactory.getLogger(SomeService::class.java); + val logger = LoggerFactory.getLogger(SomeService::class.java) ``` ## Configuration { #configuration } -Logging levels described in the `LoggingConfig` class: +Logging levels are described by the `LoggingConfig` class. +The configuration sets a level for `ROOT`, a package, or a specific class: ===! ":material-code-json: `Hocon`" ```javascript logging { levels { //(1)! + "ROOT": "WARN" + "ru.tinkoff.kora": "INFO" "ru.tinkoff.kora.http.server.common.telemetry": "INFO" "ru.tinkoff.kora.http.client.common.telemetry.DefaultHttpClientTelemetry": "INFO" } } ``` - 1. Logging levels for classes and packages are specified + 1. Logging levels for `ROOT`, classes, and packages (default: not specified, optional). === ":simple-yaml: `YAML`" ```yaml logging: levels: #(1)! + ROOT: "WARN" + ru.tinkoff.kora: "INFO" ru.tinkoff.kora.http.server.common.telemetry: "INFO" ru.tinkoff.kora.http.client.common.telemetry.DefaultHttpClientTelemetry: "INFO" - } ``` - 1. Logging levels for classes and packages are specified + 1. Logging levels for `ROOT`, classes, and packages (default: not specified, optional). + +The section key may be written as either `levels` or `level` — both are accepted as aliases. +Logger names may be listed as flat dotted strings (as above) or as a nested object; Kora flattens nested objects into dotted logger names. +The `ROOT` logger name is matched case-insensitively, so `ROOT` and `root` are equivalent. +For example, the shipped [examples](https://github.com/kora-projects/kora-examples) use the singular `level` alias with a nested lowercase `root`: + +```javascript +logging.level { + "root": "WARN" + "ru.tinkoff.kora": "INFO" + "ru.tinkoff.kora.example": "INFO" +} +``` + +!!! note + + When the `logging` section is absent, Kora applies no level map of its own. + The [Logback](#logback) implementation, however, resets all loggers on every (re)apply: `ROOT` is normalized to `INFO` and every other per-logger level is cleared so that it inherits from its parent, after which the configured levels are applied on top. + As a result, the `` value from `logback.xml` is effectively replaced by `INFO` at startup unless a `ROOT` level is set in the configuration. + +### Runtime level refresh { #levels-refresh } + +Configured levels are applied by the `LoggingLevelRefresher` — a root component that on startup resets all loggers and re-applies the levels from the `logging` section through the `LoggingLevelApplier`. +It re-runs on every configuration refresh, so when the [Config Watcher](config.md#config-watcher) is active, changing a level in the configuration file takes effect at runtime without restarting the application. -Logback configuration parameters are described in the modules that include logback, e.g. [HTTP server](http-server.md), [HTTP client](http-client.md), etc. +Logging parameters for specific modules are described in the documentation for those modules, for example [HTTP server](http-server.md), [HTTP client](http-client.md), [gRPC client](grpc-client.md). -### Module { #module } +### Modules { #module } -Enabling/disabling logging of certain modules is specified in the configuration of the modules themselves. +Logging for specific modules is enabled and disabled in the configuration of those modules through `telemetry.logging.enabled`. -Logging of **all modules is disabled** by default, for convenience below is a separate configuration to enable logging of most modules. +By default, logging is **disabled for all modules**, so the configuration below shows how to enable logging for most modules: ===! ":material-code-json: `Hocon`" @@ -77,16 +108,16 @@ Logging of **all modules is disabled** by default, for convenience below is a se SomePathToConfigKafkaProducer.telemetry.logging.enabled = true //(10)! ``` - 1. Database [JDBC](database-jdbc.md) / [R2DBC](database-jdbc.md) / [Vertx](database-vertx.md) - 2. Database [Cassandra](database-cassandra.md) - 3. [gRPC server](grpc-server.md) - 4. [HTTP server](http-server.md) - 5. [Scheduler](scheduling.md) - 6. [gRPC client](grpc-client.md) (Specified for a specific service) - 7. [SOAP client](soap-client.md) (Specified for a specific service) - 8. [HTTP client](http-client.md) (Specified for a specific client) - 9. Kafka [consumer](kafka.md#consumer) (Specified for a specific consumer) - 10. Kafka [producer](kafka.md#producer) (Specified for a specific producer) + 1. Logging for [JDBC](database-jdbc.md), `R2DBC`, or `Vertx` database requests (default: `false`). + 2. Logging for [Cassandra](database-cassandra.md) database requests (default: `false`). + 3. Logging for [gRPC server](grpc-server.md) requests (default: `false`). + 4. Logging for [HTTP server](http-server.md) requests (default: `false`). + 5. Logging for [scheduler](scheduling.md) executions (default: `false`). + 6. Logging for [gRPC client](grpc-client.md) requests, specified for a particular service (default: `false`). + 7. Logging for [SOAP client](soap-client.md) requests, specified for a particular service (default: `false`). + 8. Logging for [HTTP client](http-client.md) requests, specified for a particular client (default: `false`). + 9. Logging for a Kafka [consumer](kafka.md#config-consumer), specified for a particular consumer (default: `false`). + 10. Logging for a Kafka [producer](kafka.md#config-producer), specified for a particular producer (default: `false`). === ":simple-yaml: `YAML`" @@ -103,20 +134,20 @@ Logging of **all modules is disabled** by default, for convenience below is a se SomePathToConfigKafkaProducer.telemetry.logging.enabled: true #(10)! ``` - 1. Database [JDBC](database-jdbc.md) / [R2DBC](database-jdbc.md) / [Vertx](database-vertx.md) - 2. Database [Cassandra](database-cassandra.md) - 3. [gRPC server](grpc-server.md) - 4. [HTTP server](http-server.md) - 5. [Scheduler](scheduling.md) - 6. [gRPC client](grpc-client.md) (Specified for a specific service) - 7. [SOAP client](soap-client.md) (Specified for a specific service) - 8. [HTTP client](http-client.md) (Specified for a specific client) - 9. Kafka [consumer](kafka.md#consumer) (Specified for a specific consumer) - 10. Kafka [producer](kafka.md#producer) (Specified for a specific producer) + 1. Logging for [JDBC](database-jdbc.md), `R2DBC`, or `Vertx` database requests (default: `false`). + 2. Logging for [Cassandra](database-cassandra.md) database requests (default: `false`). + 3. Logging for [gRPC server](grpc-server.md) requests (default: `false`). + 4. Logging for [HTTP server](http-server.md) requests (default: `false`). + 5. Logging for [scheduler](scheduling.md) executions (default: `false`). + 6. Logging for [gRPC client](grpc-client.md) requests, specified for a particular service (default: `false`). + 7. Logging for [SOAP client](soap-client.md) requests, specified for a particular service (default: `false`). + 8. Logging for [HTTP client](http-client.md) requests, specified for a particular client (default: `false`). + 9. Logging for a Kafka [consumer](kafka.md#config-consumer), specified for a particular consumer (default: `false`). + 10. Logging for a Kafka [producer](kafka.md#config-producer), specified for a particular producer (default: `false`). ## Logback { #logback } -The module provides a logging implementation based on [Logback](https://www.baeldung.com/logback), adds support for structured logs and the ability to configure logging levels via [config file](config.md). +The module provides a logging implementation based on [`Logback`](https://www.baeldung.com/logback), adds support for structured logs, and allows logging levels to be managed through the [configuration file](config.md). ### Dependency { #dependency } @@ -148,16 +179,15 @@ The module provides a logging implementation based on [Logback](https://www.bael ### Configuration { #configuration-2 } -It is assumed that [Logback](https://logback.qos.ch/manual/configuration.html) will be configured via `logback.xml`, and only logging levels will be specified in the Kora configuration, example `logback.xml`: +`Logback` is configured through `logback.xml`, while Kora configuration usually contains only logging levels. +Example `logback.xml`: ```xml - + + - - UTF-8 - %d{HH:mm:ss.SSS} %-5level [%thread] %logger{36} - %msg%n - + @@ -170,14 +200,47 @@ It is assumed that [Logback](https://logback.qos.ch/manual/configuration.html) w ``` -## Other implementation { #other-implementation } +`ConsoleTextRecordEncoder` writes a text log record and adds structured data from `StructuredArgument`, `Marker`, `SLF4J` key-value pairs, and `MDC`. +This is the only encoder shipped by the module: it produces text with structured fields appended, not a single JSON document. +A record is emitted as a plain text line — `timestamp level [thread] logger - message` — followed, when structured fields are present, by tab-indented `fieldName={json}` lines: + +```text +2026-07-02 10:15:30.123 INFO [main] r.t.k.example.SomeService - userId=42 user logged in + role="admin" +``` + +`KoraAsyncAppender` is used for asynchronous log writing: it stores `MDC` values from the current context in `KoraLoggingEvent` so they are not lost when the record is passed to another thread. + +### Custom pattern { #custom-pattern } + +Instead of `ConsoleTextRecordEncoder`, a standard `PatternLayoutEncoder` can be used together with the converters that render Kora structured data. +`KoraMdcConverter` renders the Kora context `MDC` and `KoraLoggingMarkerConverter` renders a `StructuredArgument` marker; register them as conversion words and reference them in the pattern: + +```xml + + + + + + + %d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %logger - %koraMdc%msg %koraMarker%n + + + + + + + +``` + +## Other Implementation { #other-implementation } -Kora uses [slf4j-api](https://www.slf4j.org/) as the logging engine, you can plug in any custom compatible implementation. -The base module adds support for structured logs and the ability to configure logging levels via [config file](config.md). +Kora uses [`slf4j-api`](https://www.slf4j.org/) as the logging facade, so any compatible implementation can be connected. +The base module adds common components for structured logs and logging-level management through the [configuration file](config.md). ### Dependency { #dependency-2 } -A generic logging implementation will need to be connected: +The common logging module must be connected: ===! ":fontawesome-brands-java: `Java`" @@ -207,31 +270,32 @@ A generic logging implementation will need to be connected: ### Usage { #usage-2 } -When using your custom implementation, you would need to provide an implementation of `LoggingLevelApplier` that implements the -setting the logging level and resetting it. +When using a custom implementation, provide a `LoggingLevelApplier` component that can apply a logging level for the specified `Logger` and reset levels to their initial state. -It will also be necessary for the implementation to independently support `StructuredArgument`, `StructuredArgumentWriter` and `MDC` if they are to be used. +If the application uses structured data, the custom implementation must also support writing `StructuredArgument`, `StructuredArgumentWriter`, and `MDC`. ## Structured Logs { #structured-logs } -You can pass structured data to a log record in two ways via: +Structured logs make it possible to pass not only text but also named fields to a log record. +These fields are convenient for log collection tools and can be used for search, filtering, and views. -- Marker -- Parameter +Structured data can be passed to a log record in two ways: -The marker and parameter methods also take `Long`, `Integer`, `String`, `Boolean` and `Map` as arguments. +- through `Marker`; +- through a message parameter. + +The `marker` and `arg` methods also accept `Long`, `Integer`, `String`, `Boolean`, and `Map` values. +For more complex objects, pass a custom `StructuredArgumentWriter` or `JsonWriter`. ### Marker { #marker } -You can pass structured data to the log via a marker: +`Marker` adds a structured field to a log record and does not take a parameter slot in the text message: ===! ":fontawesome-brands-java: `Java`" ```java var logger = LoggerFactory.getLogger(getClass()); - var marker = StructuredArgument.marker("key", gen -> { - gen.writeString("value"); - }); + var marker = StructuredArgument.marker("key", "value"); logger.info(marker, "message"); ``` @@ -239,47 +303,114 @@ You can pass structured data to the log via a marker: ```kotlin val logger = LoggerFactory.getLogger(javaClass) - val marker = StructuredArgument.marker("key") { it.writeString("value") } + val marker = StructuredArgument.marker("key", "value") logger.info(marker, "message") ``` ### Parameter { #parameter } -You can transfer structured data to the log via parameters: +A message parameter adds a structured field through the regular `SLF4J` argument array: ===! ":fontawesome-brands-java: `Java`" ```java var logger = LoggerFactory.getLogger(getClass()); - var parameter = StructuredArgument.arg("key", gen -> { - gen.writeString("value"); - }); - log.info("message", parameter); + var parameter = StructuredArgument.arg("key", "value"); + logger.info("message", parameter); ``` === ":simple-kotlin: `Kotlin`" ```kotlin val logger = LoggerFactory.getLogger(javaClass) - val parameter = StructuredArgument.arg("key") { it.writeString("value") } + val parameter = StructuredArgument.arg("key", "value") logger.info("message", parameter) ``` +### Complex object { #complex-object } + +For values that are not a `String`, number, `Boolean`, or `Map`, pass a `JsonWriter` (the same [`@Json`](json.md) writer generated for the type) or a raw `StructuredArgumentWriter` lambda that writes the field value directly to the `JsonGenerator`. +Both `arg` and `marker` provide these overloads: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var logger = LoggerFactory.getLogger(getClass()); + var parameter = StructuredArgument.arg("user", gen -> { + gen.writeStartObject(); + gen.writeStringField("id", "42"); + gen.writeStringField("role", "admin"); + gen.writeEndObject(); + }); + logger.info("user logged in", parameter); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val logger = LoggerFactory.getLogger(javaClass) + val parameter = StructuredArgument.arg("user") { gen -> + gen.writeStartObject() + gen.writeStringField("id", "42") + gen.writeStringField("role", "admin") + gen.writeEndObject() + } + logger.info("user logged in", parameter) + ``` + ### MDC { #mdc } -Structured data can be attached to all records within a context using the `ru.tinkoff.kora.logging.common.MDC` class: +Structured data can be attached to all records within the current context using the `ru.tinkoff.kora.logging.common.MDC` class. +The value will be added to every log record until it is removed from `MDC`: -=== ":fontawesome-brands-java: ``Java``" +!!! warning "Import" + + Use `ru.tinkoff.kora.logging.common.MDC`, not `org.slf4j.MDC`. Kora keeps its `MDC` inside the Kora context rather than in a thread-local, so values placed into `org.slf4j.MDC` are not rendered by the Kora encoders and do not propagate across asynchronous boundaries. For a declarative alternative see [`@Mdc`](logging-aspect.md). + +===! ":fontawesome-brands-java: `Java`" ```java - MDC.put("key", gen -> gen.writeString("value")); + MDC.put("key", "value"); + try { + logger.info("message"); + } finally { + MDC.remove("key"); + } ``` === ":simple-kotlin: `Kotlin`" - ````kotlin - MDC.put("key") { it.writeString("value") } + ```kotlin + MDC.put("key", "value") + try { + logger.info("message") + } finally { + MDC.remove("key") + } + ``` + +`put` accepts `String`, `Integer`, `Long`, and `Boolean` values, as well as a raw `StructuredArgumentWriter` for arbitrary JSON; typed values are rendered as their JSON type rather than as text. +There is also a `put(Context, key, value)` overload for writing into an explicitly provided context instead of the current one: + +===! ":fontawesome-brands-java: `Java`" + + ```java + MDC.put("userId", 42); //(1)! + logger.info("user resolved"); ``` -If you are using `AsyncAppender` to send logs, you need to use `ru.tinkoff.kora.logging.logback.KoraAsyncAppender` to correctly pass MDC parameters, -which will pass to the delegate `ru.tinkoff.kora.kora.logging.logging.logback.KoraLoggingEvent` containing, among other things, a structured MDC. + 1. Rendered as a JSON number (`userId=42`), not as a string. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + MDC.put("userId", 42) //(1)! + logger.info("user resolved") + ``` + + 1. Rendered as a JSON number (`userId=42`), not as a string. + +Because it lives in the Kora context, the `MDC` propagates across asynchronous and reactive boundaries together with the context. + +If `AsyncAppender` is used, use `ru.tinkoff.kora.logging.logback.KoraAsyncAppender` to pass `MDC` parameters correctly. +It snapshots the current context `MDC` at append time and passes `ru.tinkoff.kora.logging.logback.KoraLoggingEvent` to the delegate, so the structured `MDC` is preserved when the record is handed to the async worker thread. diff --git a/mkdocs/docs/en/documentation/mapstruct.md b/mkdocs/docs/en/documentation/mapstruct.md index dd5bad3..4596140 100644 --- a/mkdocs/docs/en/documentation/mapstruct.md +++ b/mkdocs/docs/en/documentation/mapstruct.md @@ -1,13 +1,16 @@ --- -description: "Explains Kora MapStruct integration for generated mappers and dependency injection of mapper components. Use when working with @Mapper, MapStruct, MapStructModule, @Component, annotation processor." +description: "Explains Kora MapStruct integration: MapStruct-generated @Mapper implementations become injectable Kora components, dependency injection of mapper helpers via uses, tags, and the compile-time extension. Use when working with @Mapper, @Mapping, MapStruct, generated Impl, uses, injectionStrategy, componentModel, @Tag, annotation processor, KSP." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora MapStruct integration for generated mappers and dependency injection of mapper components; key triggers include @Mapper, MapStruct, MapStructModule, @Component, annotation processor." + use_when: "Use this file for Kora docs or implementation questions about Kora MapStruct integration where MapStruct-generated @Mapper implementations become injectable Kora components; key triggers include @Mapper, @Mapping, MapStruct, generated Impl, uses, injectionStrategy, componentModel, @Tag, annotation processor, KSP." --- Module allows you to integrate the [MapStruct](https://mapstruct.org/) library to convert classes between each other. ## Dependency { #dependency } +The Kora integration is a compile-time extension that is auto-activated as soon as the `mapstruct-processor` is on the +annotation-processor classpath — no extra Kora artifact or module import is required. + ===! ":fontawesome-brands-java: `Java`" [Dependency](general.md#dependencies) `build.gradle`: @@ -15,11 +18,11 @@ Module allows you to integrate the [MapStruct](https://mapstruct.org/) library t annotationProcessor "org.mapstruct:mapstruct-processor:1.5.5.Final" implementation "org.mapstruct:mapstruct:1.5.5.Final" ``` - + === ":simple-kotlin: `Kotlin`" [MapStruct](https://mapstruct.org/) in Kotlin works with [kapt](https://kotlinlang.org/docs/kapt.html), so you are required to configure kapt plugin `build.gradle.kts`: - ``groovy + ```groovy plugins { kotlin("kapt") version ("1.9.10") } @@ -46,32 +49,76 @@ Module allows you to integrate the [MapStruct](https://mapstruct.org/) library t implementation("org.mapstruct:mapstruct:1.5.5.Final") ``` -## Usage +## Usage { #usage } + +The creation of the mappers themselves falls to the [MapStruct](https://mapstruct.org/) library; Kora only contributes a +compile-time extension that makes the generated mappers available in the dependency container. + +The extension is registered automatically through `ServiceLoader` and activates as soon as the `org.mapstruct.Mapper` +annotation is present on the classpath (an annotation-processor extension for Java, a KSP extension for Kotlin). For every +requested `@Mapper` interface or abstract class it locates the MapStruct-generated `Impl` in the same package and +exposes its public constructor as a component. Because of this you do **not** need any Kora module or configuration, and you +do **not** need `componentModel = "kora"` — the default `componentModel` works out of the box. -The creation of the transducers themselves falls to the [MapStruct](https://mapstruct.org/) library, -Kora in this case just provides the classes created by the library as dependencies in the dependency container. +Declare a mapper the standard MapStruct way and it becomes injectable: ===! ":fontawesome-brands-java: `Java`" ```java - @KoraApp - public interface Application { + public enum CarType { TYPE1, TYPE2 } + + public record Car(String make, int numberOfSeats, CarType type) { } + + public record CarDto(String make, int seatCount, String type) { } + + @Mapper + public interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + CarDto map(Car car); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + enum class CarType { TYPE1, TYPE2 } + + data class Car(val make: String, val numberOfSeats: Int, val type: CarType) + + data class CarDto(val make: String, val seatCount: Int, val type: String) - public enum CarType {TYPE1, TYPE2} + @Mapper + interface CarMapper { - public record Car(String make, int numberOfSeats, CarType type) { } + @Mapping(source = "numberOfSeats", target = "seatCount") + fun map(car: Car): CarDto + } + ``` + +`@Mapper` is supported both on interfaces and on abstract classes, and on mappers nested inside an enclosing type — in the +nested case the extension resolves the generated implementation by joining the enclosing names with `$` +(for example `SomeInterface.CarMapper` becomes `SomeInterface$CarMapperImpl`). - public record CarTO(String make, int seatCount, String type) { } +### Usage in a service { #service } - @Mapper - public interface CarMapper { +An injected mapper is an ordinary Kora component, so you constructor-inject it into a [@Component](container.md#components) +service like any other dependency: + +===! ":fontawesome-brands-java: `Java`" - @Mapping(source = "numberOfSeats", target = "seatCount") - CarTO map(Car car); + ```java + @Component + public final class CarService { + + private final CarMapper carMapper; + + public CarService(CarMapper carMapper) { + this.carMapper = carMapper; } - - default SomeService someService(CarMapper carMapper) { - return new SomeService(carMapper); + + public CarDto convert(Car car) { + return carMapper.map(car); } } ``` @@ -79,24 +126,109 @@ Kora in this case just provides the classes created by the library as dependenci === ":simple-kotlin: `Kotlin`" ```kotlin - @KoraApp - interface Application { + @Component + class CarService(private val carMapper: CarMapper) { - enum class CarType { TYPE1, TYPE2 } + fun convert(car: Car): CarDto { + return carMapper.map(car) + } + } + ``` + +### Mapper dependencies { #dependencies } - data class Car(val make: String, val numberOfSeats: Int, val type: CarType) +A mapper often delegates to helper mappers or services. MapStruct wires those helpers through the `uses` attribute of +`@Mapper`. To have Kora supply them from the dependency container (rather than MapStruct instantiating them itself), generate +the implementation with constructor injection: set `injectionStrategy = InjectionStrategy.CONSTRUCTOR` and +`componentModel = "jakarta"`. The generated `Impl` then receives every `uses` type through its public constructor, and +Kora resolves each of them from the graph — so the helper must be available as a component (for example annotated with +[@Component](container.md#components) or provided by a factory). - data class CarTO(val make: String, val seatCount: Int, val type: String) +===! ":fontawesome-brands-java: `Java`" - @Mapper - interface CarMapper { + ```java + @Component + public final class DateMapper { - @Mapping(source = "numberOfSeats", target = "seatCount") - fun map(car: Car): CarTO + public String asString(Date date) { + return date != null ? new SimpleDateFormat("yyyy-MM-dd").format(date) : null; } - fun someService(carMapper: CarMapper): SomeService { - return SomeService(carMapper) + public Date asDate(String date) throws ParseException { + return date != null ? new SimpleDateFormat("yyyy-MM-dd").parse(date) : null; } } + + @Mapper(uses = DateMapper.class, + injectionStrategy = InjectionStrategy.CONSTRUCTOR, + componentModel = "jakarta") + public interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + CarDto map(Car car); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class DateMapper { + + fun asString(date: Date?): String? = + date?.let { SimpleDateFormat("yyyy-MM-dd").format(it) } + + fun asDate(date: String?): Date? = + date?.let { SimpleDateFormat("yyyy-MM-dd").parse(it) } + } + + @Mapper(uses = [DateMapper::class], + injectionStrategy = InjectionStrategy.CONSTRUCTOR, + componentModel = "jakarta") + interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + fun map(car: Car): CarDto + } + ``` + +### Tag { #tag } + +A `@Mapper` may be qualified with a [@Tag](container.md#tags), and the extension provides the mapper only when the requested +tags equal the tags declared on the mapper type. This lets you register several mappers of the same type and disambiguate them +at the injection point: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(MyTag.class) + @Mapper + public interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + CarDto map(Car car); + } + + @Component + public final class CarService { + + public CarService(@Tag(MyTag.class) CarMapper carMapper) { + // ... + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(MyTag::class) + @Mapper + interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + fun map(car: Car): CarDto + } + + @Component + class CarService(@Tag(MyTag::class) private val carMapper: CarMapper) ``` diff --git a/mkdocs/docs/en/documentation/metrics.md b/mkdocs/docs/en/documentation/metrics.md index 524d129..027569b 100644 --- a/mkdocs/docs/en/documentation/metrics.md +++ b/mkdocs/docs/en/documentation/metrics.md @@ -5,8 +5,10 @@ agent: --- Module for collecting application metrics using [Micrometer](https://micrometer.io/docs/concepts#_purpose). +It creates a `PrometheusMeterRegistry`, connects Kora component metrics to it, and exposes the result in the `Prometheus` format through the private `HTTP` server. +This lets you collect application, `JVM`, process, and built-in integration metrics in one place and scrape them with an external observability system. -Requires [private HTTP server](http-server.md) module added to provide metrics in [prometheus](https://prometheus.io/docs/concepts/data_model/) format. +Publishing metrics requires the [private HTTP server](http-server.md), which exposes them in the [Prometheus](https://prometheus.io/docs/concepts/data_model/) format. For a step-by-step walkthrough before the reference details, see [Observability](../guides/observability.md). @@ -40,7 +42,7 @@ For a step-by-step walkthrough before the reference details, see [Observability] ## Configuration { #configuration } -Example of HTTP server path configuration for retrieving metrics described in the `HttpServerConfig` class (default values are specified): +Example of private `HTTP` server path configuration for retrieving metrics described in the `HttpServerConfig` class (default values are specified): ===! ":material-code-json: `Hocon`" @@ -50,7 +52,7 @@ Example of HTTP server path configuration for retrieving metrics described in th } ``` - 1. Path to get metrics in `prometheus` format (if [HTTP server](http-server.md) module is added): + 1. Path for retrieving metrics in the `Prometheus` format (default: `"/metrics"`). === ":simple-yaml: `YAML`" @@ -59,7 +61,7 @@ Example of HTTP server path configuration for retrieving metrics described in th privateApiHttpMetricsPath: "/metrics" #(1)! ``` - 1. Path to get metrics in `prometheus` format (if [HTTP server](http-server.md) module is added): + 1. Path for retrieving metrics in the `Prometheus` format (default: `"/metrics"`). Example of the complete configuration described in the `MetricsConfig` class (default values are specified): @@ -71,7 +73,7 @@ Example of the complete configuration described in the `MetricsConfig` class (de } ``` - 1. OpenTelemetry standard metrics format (available values: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/)) + 1. Metrics format according to the `OpenTelemetry` standard (available values: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/), default: `V120`). === ":simple-yaml: `YAML`" @@ -80,19 +82,181 @@ Example of the complete configuration described in the `MetricsConfig` class (de opentelemetrySpec: "V120" #(1)! ``` - 1. OpenTelemetry standard metrics format (available values: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/)) + 1. Metrics format according to the `OpenTelemetry` standard (available values: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/), default: `V120`). -Metrics collection configuration parameters are described in modules where metrics collection is present, e.g. [HTTP server](http-server.md), [HTTP client](http-client.md), etc. +### Module Metrics { #module-metrics } + +The `metrics` block above configures the registry globally. Each metric-collecting module additionally exposes a per-module +`telemetry.metrics` block described in `TelemetryConfig.MetricsConfig`, letting you toggle metrics, tune histogram buckets, +and attach extra tags for that module only. The example below uses the [HTTP Server](http-server.md) module as the host, but +the same `telemetry.metrics` fields apply verbatim to [HTTP Client](http-client.md), [Database](database-common.md), +[Kafka](kafka.md), [gRPC Server](grpc-server.md), [gRPC Client](grpc-client.md), [Scheduling](scheduling.md), +[Cache](cache.md), and every other integration that reports metrics: + +===! ":material-code-json: `Hocon`" + + ```javascript + httpServer { + telemetry { + metrics { + enabled = true //(1)! + slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(2)! + tags { //(3)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Enables metrics collection for the module (default: `true`) + 2. [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) histogram buckets for `DistributionSummary`/`Timer` metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO` in milliseconds for `V120` / `#DEFAULT_SLO_V123` in seconds for `V123`) + 3. Extra common tags added to every metric the module reports (default: `{}`) + +=== ":simple-yaml: `YAML`" + + ```yaml + httpServer: + telemetry: + metrics: + enabled: true #(1)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(2)! + tags: #(3)! + key1: value1 + key2: value2 + ``` + + 1. Enables metrics collection for the module (default: `true`) + 2. [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) histogram buckets for `DistributionSummary`/`Timer` metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO` in milliseconds for `V120` / `#DEFAULT_SLO_V123` in seconds for `V123`) + 3. Extra common tags added to every metric the module reports (default: `{}`) + +Setting `enabled = false` disables metric creation for that module entirely (the module's `MetricsFactory` returns no +metrics), which is the recommended way to silence a noisy integration. The default `slo` bucket values per standard are +listed in the [Personalization](#personalization) section. + +Metrics collection configuration parameters are also described in the modules that collect metrics: [HTTP Server](http-server.md), [HTTP Client](http-client.md), [gRPC Server](grpc-server.md), [gRPC Client](grpc-client.md), [Scheduling](scheduling.md), [Cache](cache.md), and other integrations. ## Usage { #usage } -We follow and encourage to use the notation described in the [specification](https://prometheus.io/docs/concepts/data_model/). +Kora follows the notation described in the [`Prometheus` specification](https://prometheus.io/docs/concepts/data_model/). + +After the module is connected, `PrometheusMeterRegistry` is registered in `Metrics.globalRegistry` and used by all components that collect metrics. +When the application stops, this registry is removed from `Metrics.globalRegistry` and closed. + +The `PrometheusMeterRegistryWrapper` component is a `Root` component and implements `Wrapped`, so user code can inject either the generic `MeterRegistry` or the concrete `PrometheusMeterRegistry`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + private final MeterRegistry meterRegistry; + + public SomeService(MeterRegistry meterRegistry) { + this.meterRegistry = meterRegistry; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val meterRegistry: MeterRegistry + ) + ``` + +The registry automatically gets standard `Micrometer` binders: `ClassLoaderMetrics`, `JvmMemoryMetrics`, `JvmGcMetrics`, `JvmThreadMetrics`, `ProcessorMetrics`, `FileDescriptorMetrics`, `UptimeMetrics`. +Kora also registers the `kora.up` metric with value `1` and the `version` tag. + +Kora additionally bridges the `Micrometer` registry to an `OpenTelemetry` `MeterProvider` (`MicrometerMeterProvider` from `io.opentelemetry.contrib.metrics.micrometer`), so libraries instrumented with the `OpenTelemetry` metrics API publish through the same `PrometheusMeterRegistry`. -Once the `Metrics.globalRegistry` module is connected, the `PrometheusMeterRegistry` will be registered and used in all components that collect metrics. +A runnable baseline that wires `MetricsModule` alongside `HoconConfigModule`, `LogbackModule`, `UndertowHttpServerModule`, and the `OpenTelemetry` exporter is available in the [kora-java-telemetry](https://github.com/kora-projects/kora-examples/tree/master/examples/java/kora-java-telemetry) example. + +### Prometheus Export { #prometheus-export } + +Metrics are exposed in the [Prometheus](https://prometheus.io/docs/concepts/data_model/) text format by the [private HTTP server](http-server.md) on the `privateApiHttpMetricsPath` (default `/metrics`) served at `privateApiHttpPort`. +The private server must have a port configured for the endpoint to be reachable. +With the example configuration (`privateApiHttpPort = 8085`), the current metric snapshot can be scraped like this: + +```shell +curl http://localhost:8085/metrics +``` + +Point your `Prometheus` scrape target (or any compatible collector) at the same host, port, and path. + +### Custom Metric { #custom-metric } + +For a custom metric, it is better to create a separate component, inject `MeterRegistry`, and reuse created `Meter` instances. +Do not create a new metric on every method call: if the tag set depends on the operation, use a key with limited cardinality and cache the metric in `ConcurrentHashMap`. +The `register(...)` call is needed for initial metric registration in `MeterRegistry`; on the hot path, prefer using an already created `Timer` / `Counter` / `Gauge` and only call `record(...)` or `increment(...)`. +Kora uses the same approach for its internal metrics. + +For example, a duration metric for an external operation: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ExternalOperationMetrics { + private record Key(String operation, String status) {} + + private final MeterRegistry meterRegistry; + private final ConcurrentHashMap timers = new ConcurrentHashMap<>(); + + public ExternalOperationMetrics(MeterRegistry meterRegistry) { + this.meterRegistry = meterRegistry; + } + + public void record(String operation, String status, long durationNanos) { + var key = new Key(operation, status); + var timer = this.timers.computeIfAbsent(key, k -> Timer.builder("external.operation.duration") + .tag("operation", k.operation()) + .tag("status", k.status()) + .register(this.meterRegistry)); + + timer.record(durationNanos, TimeUnit.NANOSECONDS); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ExternalOperationMetrics( + private val meterRegistry: MeterRegistry + ) { + private data class Key( + val operation: String, + val status: String + ) + + private val timers = ConcurrentHashMap() + + fun record(operation: String, status: String, durationNanos: Long) { + val key = Key(operation, status) + val timer = timers.computeIfAbsent(key) { + Timer.builder("external.operation.duration") + .tag("operation", it.operation) + .tag("status", it.status) + .register(meterRegistry) + } + + timer.record(durationNanos, TimeUnit.NANOSECONDS) + } + } + ``` + +Tag values must have a limited number of variants. +Do not use user identifiers, request numbers, full error text, or other high-cardinality values as tags. ## Personalization { #personalization } -In order to make changes to the `PrometheusMeterRegistry` configuration, you need to add to the `PrometheusMeterRegistryInitializer` container. +To change `PrometheusMeterRegistry` configuration, add a `PrometheusMeterRegistryInitializer` to the container. +The initializer receives the created registry before standard system metrics are registered, so it can add common tags, `MeterFilter`, renaming rules, or custom `PrometheusMeterRegistry` settings. **Important**, `PrometheusMeterRegistryInitializer` is applied only once when the application is initialized. @@ -126,14 +290,34 @@ For example, we want to add a common tag for all metrics: } ``` -Standard metrics have some configurations such as `ServiceLayerObjectives` for Distribution summary metrics. -The configuration field names can be viewed in `ru.tinkoff.kora.micrometer.module.MetricsConfig`. +Standard metrics also have their own settings, for example `slo` histogram buckets for `DistributionSummary`/`Timer` metrics, configured per module under [`telemetry.metrics`](#module-metrics). +When `slo` is not overridden, the defaults depend on the selected `OpenTelemetry` standard: + +- `V120` — `DEFAULT_SLO` in **milliseconds**: `1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000` +- `V123` — `DEFAULT_SLO_V123` in **seconds**: `0.001, 0.010, 0.050, 0.100, 0.200, 0.500, 1, 2, 5, 10, 20, 30, 60, 90` + +Both arrays are declared in `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig`; the global registry field names are in `ru.tinkoff.kora.micrometer.module.MetricsConfig`. + +### Tag Providers { #tag-providers } + +The tag set attached to framework metrics is produced by per-module tag providers registered as `@DefaultComponent`. +To change which tags are emitted for a given integration, supply your own implementation of the corresponding interface as a `@DefaultComponent` override: + +- `MicrometerHttpServerTagsProvider` (package `ru.tinkoff.kora.micrometer.module.http.server.tag`) — HTTP server metrics +- `MicrometerHttpClientTagsProvider` (package `ru.tinkoff.kora.micrometer.module.http.client.tag`) — HTTP client metrics +- `MicrometerGrpcServerTagsProvider` / `MicrometerGrpcClientTagsProvider` (packages `...grpc.server.tag` / `...grpc.client.tag`) — gRPC metrics +- `MicrometerKafkaConsumerTagsProvider` / `MicrometerKafkaProducerTagsProvider` (packages `...kafka.consumer.tag` / `...kafka.producer.tag`) — Kafka metrics + +The default provider is selected from `metrics.opentelemetrySpec`, so an override replaces the tag mapping for both standards. ## Standard { #standard } -The original metrics format used the OpenTelemetry `V120` standard, after Kora `1.1.0` it became possible to provide metrics -in the OpenTelemetry `V123` standard, a partial list of changes can be seen [in the OpenTelemetry documentation](https://opentelemetry.io/blog/2023/http-conventions-declared-stable/) -and [OpenTelemetry migration guidelines](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/) +The original metrics format used the `OpenTelemetry` `V120` standard; after Kora `1.1.0`, metrics can also be provided +in the `OpenTelemetry` `V123` standard. A partial list of changes is available in the [OpenTelemetry documentation](https://opentelemetry.io/blog/2023/http-conventions-declared-stable/) +and [OpenTelemetry migration guidelines](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/). + +The `metrics.opentelemetrySpec` parameter affects some metric names, units, and tag sets. +The reference below lists both `V120` and `V123` variants for such metrics; if no variant is specified, the name is the same for both standards. ## Metrics Reference { #metrics-reference } @@ -145,13 +329,14 @@ All Kora metrics use [OpenTelemetry semantic conventions](https://opentelemetry. This metric type enables efficient data visualization across buckets and percentile calculation. - [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) — monotonically increasing counter - [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) — current metric value +- [Timer](https://docs.micrometer.io/micrometer/reference/concepts/timers.html) — operation duration with count, sum, max, and buckets support ### HTTP Server { #http-server } | Metric | Prometheus | Type | Description | Tags | |--------|------------|------|-------------|------| -| `http.server.request.duration` | `http_server_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | HTTP server request processing duration | `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | -| `http.server.active_requests` | `http_server_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Number of active HTTP requests | `http.request.method`, `http.route`, `server.address`, `url.scheme` | +| `http.server.duration` (`V120`), `http.server.request.duration` (`V123`) | `http_server_duration_milliseconds` (`V120`) / `http_server_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | `HTTP` server request processing duration | `V120`: `http.request.method`, `http.response.status_code`, `http.route`, `server.address`, `url.scheme`, `http.target`, `http.method`, `http.status_code`; `V123`: `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | +| `http.server.active_requests` | `http_server_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Number of active `HTTP` requests | `V120`: `http.route`, `http.request.method`, `server.address`, `url.scheme`, `http.target`, `http.method`; `V123`: `http.route`, `http.request.method`, `server.address`, `url.scheme` | See [HTTP Server](http-server.md) module documentation for more details. @@ -159,7 +344,7 @@ See [HTTP Server](http-server.md) module documentation for more details. | Metric | Prometheus | Type | Description | Tags | |--------|------------|------|-------------|------| -| `http.client.request.duration` | `http_client_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | HTTP client request duration | `http.request.method`, `http.response.status_code`, `server.address`, `url.scheme`, `http.route`, `error.type` | +| `http.client.duration` (`V120`), `http.client.request.duration` (`V123`) | `http_client_duration_milliseconds` (`V120`) / `http_client_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | `HTTP` client request duration | `V120`: `http.request.method`, `http.response.status_code`, `server.address`, `url.scheme`, `http.route`, `http.status_code`, `http.method`, `http.target`, `error.type`; `V123`: `http.request.method`, `http.response.status_code`, `server.address`, `url.scheme`, `http.route`, `http.status_code`, `error.type` | See [HTTP Client](http-client.md) module documentation for more details. @@ -167,7 +352,7 @@ See [HTTP Client](http-client.md) module documentation for more details. | Metric | Prometheus | Type | Description | Tags | |--------|------------|------|-------------|------| -| `db.client.request.duration` | `db_client_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Database operation/query duration | `db.pool.name`, `db.statement`, `db.operation`, `error.type` | +| `database.client.request.duration` (`V120`), `db.client.request.duration` (`V123`) | `database_client_request_duration_milliseconds` (`V120`) / `db_client_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Database operation/query duration | `V120`: `pool`, `query.id`, `query.operation`, `error`; `V123`: `db.pool.name`, `db.statement`, `db.operation`, `error.type` | See [Database](database-common.md) module documentation for more details. @@ -222,10 +407,11 @@ See [Scheduling](scheduling.md) module documentation for more details. | Metric | Prometheus | Type | Description | Tags | |--------|------------|------|-------------|------| -| `cache.duration` | `cache_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Cache operation duration (GET, SET, DELETE, etc.) | `cache`, `operation`, `origin`, `status` | +| `cache.duration` | `cache_duration_seconds` / `_count` / `_sum` / `_bucket` / `_max` | [Timer](https://docs.micrometer.io/micrometer/reference/concepts/timers.html) | Cache operation duration (`GET`, `SET`, `DELETE`, and others) | `cache`, `operation`, `origin`, `status` | | `cache.ratio` | `cache_ratio_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Cache hit/miss counter | `cache`, `origin`, `type` | +| `cache.hit`, `cache.miss` | `cache_hit_total`, `cache_miss_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Deprecated hit/miss counters kept for compatibility | `cache`, `origin` | -Standard Micrometer metrics are automatically registered when using Caffeine: +Standard `Micrometer` metrics are automatically registered when using `Caffeine`: | Metric | Prometheus | Type | Description | |--------|------------|------|-------------| @@ -285,7 +471,7 @@ See [Camunda 7 BPMN](camunda7-bpmn.md) module documentation for more details. | Metric | Prometheus | Type | Description | Tags | |--------|------------|------|-------------|------| -| `camunda.rest.server.request.duration` | `camunda_rest_server_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Camunda REST request duration | `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | +| `camunda.rest.server.duration` (`V120`), `camunda.rest.server.request.duration` (`V123`) | `camunda_rest_server_duration_milliseconds` (`V120`) / `camunda_rest_server_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | `Camunda REST` request duration | `V120`: `http.request.method`, `http.response.status_code`, `http.route`, `server.address`, `url.scheme`, `http.target`, `http.method`, `http.status_code`; `V123`: `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | | `camunda.rest.server.active_requests` | `camunda_rest_server_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Number of active Camunda REST requests | `http.route`, `http.request.method`, `server.address`, `url.scheme` | See [Camunda 7 REST](camunda7-rest.md) module documentation for more details. @@ -294,9 +480,9 @@ See [Camunda 7 REST](camunda7-rest.md) module documentation for more details. | Metric | Prometheus | Type | Description | Tags | |--------|------------|------|-------------|------| -| `zeebe.worker.handler.duration` | `zeebe_worker_handler_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Zeebe worker job handler duration | `job.name`, `job.type`, `status`, `error`, `error.code` | -| `zeebe.worker.handler` | `zeebe_worker_handler_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Zeebe worker error counter | `job.name`, `job.type`, `status`, `error.code` | -| `zeebe.client.worker.job` | `zeebe_client_worker_job_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Number of activated/handled Zeebe jobs | `action`, `type` | +| `zeebe.worker.handler` (`V120`), `zeebe.worker.handler.duration` (`V123`) | `zeebe_worker_handler_seconds` (`V120`) / `zeebe_worker_handler_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | `Zeebe Worker` job handler duration | `job.name`, `job.type`, `status`, `error`, `error.code` | +| `zeebe.worker.handler` | `zeebe_worker_handler_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | `Zeebe Worker` error counter | `job.name`, `job.type`, `status`, `error.code` | +| `zeebe.client.worker.job` | `zeebe_client_worker_job_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Number of activated and handled `Zeebe` jobs | `action`, `type` | See [Camunda 8 Worker](camunda8-worker.md) module documentation for more details. diff --git a/mkdocs/docs/en/documentation/netty.md b/mkdocs/docs/en/documentation/netty.md index e83d03d..56d7586 100644 --- a/mkdocs/docs/en/documentation/netty.md +++ b/mkdocs/docs/en/documentation/netty.md @@ -1,21 +1,63 @@ --- -description: "Explains Kora Netty customization and transport configuration used by HTTP clients, gRPC clients and servers, and Vert.x integrations. Use when working with NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." +description: "Explains Kora Netty customization and transport configuration used by HTTP Async clients, gRPC clients and gRPC servers. Use when working with NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Netty customization and transport configuration used by HTTP clients, gRPC clients and servers, and Vert.x integrations; key triggers include NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." + use_when: "Use this file for Kora docs or implementation questions about Kora Netty customization and transport configuration used by HTTP Async clients, gRPC clients and gRPC servers; key triggers include NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." --- -Functionality customizing Netty components used by other modules like [Vertx](database-vertx.md), [HTTP Async client](http-client.md#asynchttpclient), [gRPC client](grpc-client.md), [gRPC server](grpc-server.md). +Netty is a networking library built around non-blocking I/O and the `event loop` model. +In Kora, it is used as a low-level network `transport` mechanism for modules that need to process connections and network events efficiently. + +This functionality customizes shared Netty components used by other modules: [HTTP Async client](http-client.md#asynchttpclient), [gRPC client](grpc-client.md), [gRPC server](grpc-server.md). +These settings are useful when an application needs to control the network `transport`, I/O thread count, or native transport selection. +Default values are usually suitable for most services, but you can set them explicitly for high network load or specific environment requirements. Module itself does not provide any utility on its own, but only serves to configure [Netty transport and Netty event loop](https://netty.io/4.1/api/io/netty/channel/EventLoop.html) within Kora. ## Connection { #connection } -The module will be transitively provided to dependencies that use it. +Usually you do not need to connect this module manually: it is added as a transitive dependency by Kora modules that require Netty. + +## What it provides { #what-it-provides } + +When the module is connected, `NettyCommonModule` contributes the following shared components to the dependency container. +Consumer modules ([HTTP Async client](http-client.md#asynchttpclient), [gRPC client](grpc-client.md), [gRPC server](grpc-server.md)) inject them instead of creating their own Netty threads: + +- **`NettyTransportConfig`** - configuration bound to the `netty` section (preferred [transport](#transport) and [worker thread count](#configuration)). +- **Worker `EventLoopGroup`** with tag `@Tag(NettyCommonModule.WorkerLoopGroup.class)` - the shared `event loop` that processes connections and network I/O. Its size is set by the `threads` parameter, and both clients and servers use it. +- **Boss `EventLoopGroup`** with tag `@Tag(NettyCommonModule.BossLoopGroup.class)` - a separate group fixed at `1` thread that only server components (for example [gRPC server](grpc-server.md)) use to accept incoming connections; the `threads` parameter does not affect it. +- **[`NettyChannelFactory`](#channel-factory)** - a factory that creates Netty channels matching the selected [transport](#transport). + +Both `event loop` groups are managed by the Kora [lifecycle](container.md#component-lifecycle): they are shut down gracefully after all dependent components are released, so no manual management is required. + +Advanced modules that build a custom Netty `transport` can inject these components directly: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MyNettyTransport { + + public MyNettyTransport(@Tag(NettyCommonModule.WorkerLoopGroup.class) EventLoopGroup workerGroup, + NettyChannelFactory channelFactory) { + // build a client or server bootstrap on the shared event loop + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyNettyTransport( + @Tag(NettyCommonModule.WorkerLoopGroup::class) workerGroup: EventLoopGroup, + channelFactory: NettyChannelFactory, + ) + ``` ## Configuration { #configuration } -An example of the configuration described in the `NettyTransportConfig` class: +An example of the configuration described by the `NettyTransportConfig` class: ===! ":material-code-json: `Hocon`" @@ -26,11 +68,8 @@ An example of the configuration described in the `NettyTransportConfig` class: } ``` - 1. Preferred [trasnport](https://netty.io/wiki/native-transports.html) if available on the path as a dependency, is selected by default in order of availability: - 1. `Epoll` (require [dependency](https://mvnrepository.com/artifact/io.netty/netty-transport-native-epoll)) - 2. `KQueue` (require [dependency](https://mvnrepository.com/artifact/io.netty/netty-transport-native-kqueue)) - 3. `Nio` - 2. Number of threads [Netty event loop](https://netty.io/4.1/api/io/netty/channel/EventLoop.html), defaults to the number of CPU cores multiplied by 2. + 1. Preferred [transport](https://netty.io/wiki/native-transports.html): `NIO`, `EPOLL` or `KQUEUE` (default: not specified, optional). + 2. Number of `worker event loop` threads (default: number of available CPU cores multiplied by `2`). Server components also create a `boss event loop` with `1` thread, and the `threads` value does not affect it. === ":simple-yaml: `YAML`" @@ -40,8 +79,74 @@ An example of the configuration described in the `NettyTransportConfig` class: threads: 2 #(2)! ``` - 1. Preferred [trasnport](https://netty.io/wiki/native-transports.html) if available on the path as a dependency, is selected by default in order of availability: - 1. `Epoll` (require [dependency](https://mvnrepository.com/artifact/io.netty/netty-transport-native-epoll)) - 2. `KQueue` (require [dependency](https://mvnrepository.com/artifact/io.netty/netty-transport-native-kqueue)) - 3. `Nio` - 2. Number of threads [Netty event loop](https://netty.io/4.1/api/io/netty/channel/EventLoop.html), defaults to the number of CPU cores multiplied by 2. + 1. Preferred [transport](https://netty.io/wiki/native-transports.html): `NIO`, `EPOLL` or `KQUEUE` (default: not specified, optional). + 2. Number of `worker event loop` threads (default: number of available CPU cores multiplied by `2`). Server components also create a `boss event loop` with `1` thread, and the `threads` value does not affect it. + +## Transport { #transport } + +The `transport` parameter sets the preferred Netty `transport`: + +- `NIO` - standard Java NIO `transport`, always available. +- `EPOLL` - Linux `native transport`. +- `KQUEUE` - macOS / BSD `native transport`. + +If `transport` is not set, Kora selects the first available transport in this order: + +1. `EPOLL` +2. `KQUEUE` +3. `NIO` + +If the configured `native transport` is not available at runtime, Kora uses the first available `transport` from the same order. + +## Native Transport { #native-transport } + +To use `EPOLL` or `KQUEUE`, the corresponding Netty native dependency must be available in the `runtime classpath`: + +- [`io.netty:netty-transport-native-epoll`](https://mvnrepository.com/artifact/io.netty/netty-transport-native-epoll) for Linux. +- [`io.netty:netty-transport-native-kqueue`](https://mvnrepository.com/artifact/io.netty/netty-transport-native-kqueue) for macOS / BSD. + +When adding a native dependency, choose the `classifier` for the target platform, for example `linux-x86_64`, `osx-x86_64` or `osx-aarch_64`. + +???+ tip "Recommendation" + + Usually it is enough to leave `transport` unset and let Kora select it automatically. Add `native transport` intentionally: for example, when you need it for performance or Netty features unavailable in `NIO`. + +## Channel factory { #channel-factory } + +`NettyChannelFactory` is a shared injectable component that produces Netty [`ChannelFactory`](https://netty.io/4.1/api/io/netty/channel/ChannelFactory.html) instances matching the selected [transport](#transport). +It is an advanced injection point for modules that build their own Netty client or server `bootstrap` and want channels consistent with the chosen `transport`: + +- `getClientFactory()` / `getClientFactory(boolean domainSocket)` - a factory for client channels. +- `getServerFactory()` / `getServerFactory(boolean domainSocket)` - a factory for server channels. + +The no-argument overloads create standard `TCP` socket channels. +Passing `domainSocket = true` requests a [Unix domain socket](https://en.wikipedia.org/wiki/Unix_domain_socket) channel: this is supported by the `EPOLL` and `KQUEUE` `native transports`, while the `NIO` implementation currently falls back to standard socket channels. + +## Thread factory { #thread-factory } + +Both the worker and boss `event loop` groups accept an optional [`ThreadFactory`](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadFactory.html). +To customize Netty thread naming or priority, provide a `ThreadFactory` component tagged with `@Tag(NettyCommonModule.class)`; when present, Kora uses it for both groups: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends AsyncHttpClientModule { + + @Tag(NettyCommonModule.class) + default ThreadFactory nettyThreadFactory() { + return new DefaultThreadFactory("netty-io"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : AsyncHttpClientModule { + + @Tag(NettyCommonModule::class) + fun nettyThreadFactory(): ThreadFactory = DefaultThreadFactory("netty-io") + } + ``` diff --git a/mkdocs/docs/en/documentation/openapi-codegen.md b/mkdocs/docs/en/documentation/openapi-codegen.md index 45941aa..223f095 100644 --- a/mkdocs/docs/en/documentation/openapi-codegen.md +++ b/mkdocs/docs/en/documentation/openapi-codegen.md @@ -1,19 +1,21 @@ --- -description: "Explains Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support. Use when working with openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth." +description: "Explains Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support. Use when working with openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth, prefixPath, requestInDelegateParams, HttpClientTokenProvider, PrincipalWithScopes, ApiSecurity." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support; key triggers include openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth." + use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support; key triggers include openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth, prefixPath, requestInDelegateParams, HttpClientTokenProvider, PrincipalWithScopes, ApiSecurity." --- -Module for creating declarative HTTP handlers [HTTP server](http-server.md) -or create declarative [HTTP clients](http-client.md) from OpenAPI contracts using [OpenAPI Generator plugin](https://openapi-generator.tech/docs/plugins#gradle). +This module generates Kora code from an `OpenAPI` contract using [OpenAPI Generator](https://openapi-generator.tech/docs/plugins#gradle). +From a single API description, it can create declarative [HTTP server](http-server.md) handlers or declarative [HTTP clients](http-client.md), +as well as request and response models, mappers, authorization handling, and additional annotations. +This approach is useful when `OpenAPI` is the source of truth for the transport contract and application code must follow it automatically. -For a step-by-step walkthrough before the reference details, see [OpenAPI HTTP Server](../guides/openapi-http-server.md), [Advanced OpenAPI HTTP Server](../guides/openapi-http-server-advanced.md) and [OpenAPI HTTP Client](../guides/openapi-http-client.md). +For a step-by-step walkthrough before the reference documentation, see [OpenAPI HTTP Server](../guides/openapi-http-server.md), [Advanced OpenAPI HTTP Server](../guides/openapi-http-server-advanced.md), and [OpenAPI HTTP Client](../guides/openapi-http-client.md). ## Dependency { #dependency } ===! ":fontawesome-brands-java: `Java`" - [Dependency](general.md#dependencies) `build.gradle`: + Generator dependency in `build.gradle`: ```groovy buildscript { dependencies { @@ -22,18 +24,18 @@ For a step-by-step walkthrough before the reference details, see [OpenAPI HTTP S } ``` - Plugin dependency `build.gradle`: + Plugin dependency in `build.gradle`: ```groovy plugins { id "org.openapi.generator" version "7.14.0" } - - Use of other versions of the plugin is not guaranteed as it may not be compatible at the code level. ``` + Other plugin versions are not guaranteed to work because the `OpenAPI Generator` API can be incompatible at code level. + === ":simple-kotlin: `Kotlin`" - [Dependency](general.md#dependencies) `build.gradle.kts`: + [Dependency](general.md#dependencies) in `build.gradle.kts`: ```groovy buildscript { dependencies { @@ -42,56 +44,330 @@ For a step-by-step walkthrough before the reference details, see [OpenAPI HTTP S } ``` - Plugin dependency `build.gradle.kts`: + Plugin dependency in `build.gradle.kts`: ```groovy plugins { id("org.openapi.generator") version("7.14.0") } - - Use of other versions of the plugin is not guaranteed as it may not be compatible at the code level. ``` -Requires [HTTP server](http-server.md) or [HTTP client](http-client.md) module. + Other plugin versions are not guaranteed to work because the `OpenAPI Generator` API can be incompatible at code level. + +Generated code also requires the [HTTP server](http-server.md) or [HTTP client](http-client.md) module, depending on the selected generation mode. ## Configuration { #configuration } -Configuration is required for [OpenAPI Generator plugin](https://openapi-generator.tech/docs/plugins#gradle) parameters: +Configure the [OpenAPI Generator plugin](https://openapi-generator.tech/docs/plugins#gradle) parameters: + +- `Gradle` plugin parameters are described in the [plugin documentation](https://github.com/OpenAPITools/openapi-generator/blob/v7.14.0/modules/openapi-generator-gradle-plugin/README.adoc). +- The `configOptions` plugin parameter is described in the [configuration documentation](https://openapi-generator.tech/docs/configuration/). +- The `openapiNormalizer` plugin parameter is described in the [customization documentation](https://openapi-generator.tech/docs/customization/#normalizer-opts). + +### Common `OpenAPI Generator` Options { #common-opts } + +In addition to Kora-specific `configOptions`, `GenerateTask` accepts common `OpenAPI Generator` parameters. +They define where to read the contract from, where to put generated files, which packages to use, and how to preprocess the `OpenAPI` description. +For Kora projects, these parameters are usually set explicitly because generated code is then added to normal project compilation. + +| Parameter | Description | +| -------- | -------- | +| `generatorName` | Generator name (`required`, no default). Always set it to `kora` for Kora. | +| `inputSpec` | Path to the `OpenAPI` file (`required`, no default). Usually this is a file under `src/main/resources/openapi`, for example `$projectDir/src/main/resources/openapi/openapi.yaml`. | +| `outputDir` | Directory for generated files (not specified by default, optional). In Kora projects, this is usually a directory under `build`, for example `$buildDir/generated/openapi`, and it is added to the main source set. | +| `apiPackage` | Package for generated API interfaces, controllers, `delegate` classes, and mappers (default: `org.openapitools.api`). It is recommended to set it explicitly, for example `ru.tinkoff.kora.example.openapi.api`. | +| `modelPackage` | Package for models generated from `OpenAPI` schemas (default: `org.openapitools.model`). It is recommended to set it explicitly, for example `ru.tinkoff.kora.example.openapi.model`. | +| `invokerPackage` | Auxiliary generator package (default: `org.openapitools.api`). It is recommended to set it explicitly next to `apiPackage` and `modelPackage`, for example `ru.tinkoff.kora.example.openapi.invoker`. | +| `configOptions` | Generator-specific parameters (default: `{}`). For Kora, this is where `mode`, `clientConfigPrefix`, `enableServerValidation`, `interceptors`, and the other parameters described below are set. | +| `globalProperties` | Limits which entities are generated (default: `{}`). Useful when you need to generate only `apis`, only `models`, or specific models and operations. Use carefully: normal Kora clients and servers usually need API classes, models, and mappers together. | +| `openapiNormalizer` | Preprocesses the `OpenAPI` contract before generation (default: `{}`). Often used to disable standard transformations with `DISABLE_ALL`, generate only selected operations with `FILTER`, or control rules such as `SIMPLIFY_ONEOF_ANYOF`. | +| `importMappings` | Maps a schema name to an existing class (default: `{}`). Useful when a model is written manually or comes from another module, for example `Money: "com.example.Money"`. | +| `typeMappings` | Maps an `OpenAPI Generator` type to a language type (default: `{}`). Used for targeted type replacement, for example replacing `OffsetDateTime` with a project-specific time type. | +| `schemaMappings` | Maps an `OpenAPI` schema to an external type without generating the model (default: `{}`). Similar to `importMappings`, but configured at schema level and useful for reusing shared DTOs. | +| `skipValidateSpec` | Skips `OpenAPI` contract validation before generation (default: `false`). In normal builds it is better to keep validation enabled; use `true` only temporarily for external contracts that cannot be fixed quickly. | +| `cleanupOutput` | Cleans `outputDir` before generation (default: `false`). Useful when the contract changes often and files from removed operations or models must disappear. Do not point `outputDir` to a directory with handwritten code. | + +Example with common options: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + def openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient", GenerateTask) { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/openapi.yaml" + outputDir = "$buildDir/generated/openapi/client" + + def corePackage = "ru.tinkoff.kora.example.openapi" + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + + skipValidateSpec = false + cleanupOutput = true + openapiNormalizer = [ + DISABLE_ALL: "true", + FILTER: "tag:public|billing" + ] + configOptions = [ + mode: "java-client", + clientConfigPrefix: "httpClient.billing", + filterWithModels: "true" + ] + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + val openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient") { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/openapi.yaml" + outputDir = "$buildDir/generated/openapi/client" + + val corePackage = "ru.tinkoff.kora.example.openapi" + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + + skipValidateSpec = false + cleanupOutput = true + openapiNormalizer = mapOf( + "DISABLE_ALL" to "true", + "FILTER" to "tag:public|billing" + ) + configOptions = mapOf( + "mode" to "kotlin-client", + "clientConfigPrefix" to "httpClient.billing", + "filterWithModels" to "true" + ) + } + ``` + +Use `globalProperties` only for narrow generation tasks, for example when extracting a few models into an intermediate module: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + globalProperties = [ + models: "User,Order", + apis: "false", + supportingFiles: "false" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + globalProperties = mapOf( + "models" to "User,Order", + "apis" to "false", + "supportingFiles" to "false" + ) + ``` + +### Useful `openapiNormalizer` Rules { #normalizer-opts } + +`openapiNormalizer` changes the input `OpenAPI` contract before generation. It is not a Kora parameter, but a general `OpenAPI Generator` mechanism. +For Kora, it is especially useful when one large contract is used by several applications or when the contract contains ambiguous shapes for code generation. + +| Rule | Description | +| -------- | -------- | +| `DISABLE_ALL` | Disables standard normalization rules (default: `false`). Starting with `OpenAPI Generator 7`, some rules are enabled by default, so predictable generation often starts with `DISABLE_ALL: "true"` and then enables only the needed rules explicitly. | +| `FILTER` | Keeps only selected operations for generation (not specified by default, optional). Supports one filter at a time: `operationId:name1\|name2`, `method:get\|post`, or `tag:public\|billing`. Operations that do not match are marked as `x-internal: true` and are not generated. | +| `KEEP_ONLY_FIRST_TAG_IN_OPERATION` | Keeps only the first tag on an operation (default: `false`). Useful when operations have several tags and are split into several API classes differently from what you expect. | +| `SET_TAGS_FOR_ALL_OPERATIONS` | Replaces tags on all operations with one provided value (not specified by default, optional). Useful when you want to force one generated API class. | +| `SET_TAGS_TO_OPERATIONID` | Sets an operation tag to `operationId`, or to `default` when `operationId` is empty (default: `false`). Useful for contracts without usable tags when predictable operation grouping is needed. | +| `SET_TAGS_TO_VENDOR_EXTENSION` | Reads operation tags from the specified extension, for example `x-tags` (not specified by default, optional). Useful when an external contract cannot be changed but already has custom operation grouping. | +| `FIX_DUPLICATED_OPERATIONID` | Adds a numeric suffix to duplicated `operationId` values (default: `false`). It is better to fix the contract, but this rule helps generate code for an external description temporarily. | +| `SET_BEARER_AUTH_FOR_NAME` | Converts the specified security scheme to `bearerAuth` (not specified by default, optional). Useful for external contracts where a bearer token is described in a non-standard way but should be handled as a normal bearer scheme in the application. | +| `REF_AS_PARENT_IN_ALLOF` | Marks a `$ref` inside `allOf` as a parent schema with `x-parent: true` (default: `false`). Can help contracts that model inheritance through `allOf`. | +| `SIMPLIFY_ONEOF_ANYOF` | Simplifies some `oneOf`/`anyOf` constructs, for example by moving a `null` variant to `nullable: true` and removing single wrappers (enabled by default in `OpenAPI Generator 7` unless `DISABLE_ALL` is set). For Kora, this can change generated model shapes, so enable it deliberately. | +| `SIMPLIFY_ANYOF_STRING_AND_ENUM_STRING` | Simplifies `anyOf` made from `string` and a string enum to `string` (default: `false`). This can help with contracts where the enum restriction is not important for code. | +| `SIMPLIFY_BOOLEAN_ENUM` | Converts a boolean enum to a plain `boolean` (enabled by default in `OpenAPI Generator 7` unless `DISABLE_ALL` is set). | +| `REFACTOR_ALLOF_WITH_PROPERTIES_ONLY` | Moves properties from a schema that has both `allOf` and `properties` into a separate schema inside `allOf` (enabled by default in `OpenAPI Generator 7` unless `DISABLE_ALL` is set). This can help inheritance, but strict contracts should be checked after generation. | +| `NORMALIZE_31SPEC` | Normalizes some `OpenAPI 3.1` constructs into a form better understood by the generator (default: `false`). Useful for `3.1` contracts when generation fails on newer schema forms. | +| `REMOVE_X_INTERNAL` | Removes `x-internal: true` from operations and models (default: `false`). Use only when the contract already contains `x-internal`, but a specific generation task must force such operations back in. | +| `SET_CONTAINER_TO_NULLABLE` | Marks container types `array`, `set`, or `map` as `nullable` (not specified by default, optional). Use only when an external contract systematically misses `nullable` on such fields. | +| `SET_PRIMITIVE_TYPES_TO_NULLABLE` | Marks primitive types `string`, `integer`, `number`, or `boolean` as `nullable` (not specified by default, optional). This significantly changes model signatures, so apply it only to problematic external contracts. | + +Example of generating only the public part of a contract: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + openapiNormalizer = [ + DISABLE_ALL: "true", + FILTER: "tag:public|billing" + ] + configOptions = [ + mode: "java-client", + filterWithModels: "true" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + openapiNormalizer = mapOf( + "DISABLE_ALL" to "true", + "FILTER" to "tag:public|billing" + ) + configOptions = mapOf( + "mode" to "kotlin-client", + "filterWithModels" to "true" + ) + ``` + +`FILTER` excludes only operations by itself. If unused models should also be removed after filtering, enable the Kora `filterWithModels` parameter. +For more complex selection, usually create separate generation tasks with different `FILTER` values, for example one with `tag:billing` and another with `operationId:createUser|getUser`. + +Example of normalizing tags for a contract without convenient grouping: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + openapiNormalizer = [ + DISABLE_ALL: "true", + SET_TAGS_TO_VENDOR_EXTENSION: "x-kora-tag", + FIX_DUPLICATED_OPERATIONID: "true" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + openapiNormalizer = mapOf( + "DISABLE_ALL" to "true", + "SET_TAGS_TO_VENDOR_EXTENSION" to "x-kora-tag", + "FIX_DUPLICATED_OPERATIONID" to "true" + ) + ``` + +### Common `JSON` and Model Options { #model-opts } + +Kora also supports several `configOptions` that control `JSON` mappers and common model generation. +They do not depend on whether a client or a server is generated. + +| Parameter | Description | +| -------- | -------- | +| `jsonAnnotation` | Annotation tag used to inject `JSON` mappers into generated request and response mappers (default: `ru.tinkoff.kora.json.common.annotation.Json`). | +| `objectType` | Type for `type: object` schemas without a more precise description. `Java` uses `java.lang.Object` by default, and `Kotlin` uses `kotlin.Any`. For example, set it to `com.fasterxml.jackson.databind.JsonNode` if the application wants to handle arbitrary `JSON` as a tree. | +| `disableHtmlEscaping` | Disables HTML character escaping in `JSON` strings (default: `false`). Usually the default value is kept. | +| `ignoreAnyOfInEnum` | Ignores `anyOf` when generating enums (default: `false`). Can help with contracts where an enum is described through mixed `anyOf` constructs. | +| `discriminatorCaseSensitive` | Controls case sensitivity of the discriminator value lookup for polymorphic (`oneOf`) models with a discriminator (default: `true`). Set to `false` when incoming discriminator values may differ in case from the schema definition. | +| `additionalModelTypeAnnotations` | Additional annotations on model types (not specified by default, optional). Several annotations are separated by `;`, for example `@Deprecated;@MyAnnotation`. | +| `additionalEnumTypeAnnotations` | Additional annotations on enum types (not specified by default, optional). Several annotations are separated by `;`. | + +Example: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-client", + jsonAnnotation: "ru.tinkoff.kora.json.common.annotation.Json", + objectType: "com.fasterxml.jackson.databind.JsonNode", + additionalModelTypeAnnotations: "@Deprecated" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "jsonAnnotation" to "ru.tinkoff.kora.json.common.annotation.Json", + "objectType" to "com.fasterxml.jackson.databind.JsonNode", + "additionalModelTypeAnnotations" to "@Deprecated" + ) + ``` + +### Multiple Generation Tasks { #multiple-gens } + +Several `GenerateTask` tasks can be registered in one module, for example to generate two independent contracts, +or to generate a client for one contract and a server for another. Each task writes into the same `outputDir` and is added to the same source set, +so the only requirement is that generated packages do not collide. Give every task its own `apiPackage`/`modelPackage`/`invokerPackage`. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + def openApiGeneratePetV2 = tasks.register("openApiGeneratePetV2", GenerateTask) { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV2.yaml" + outputDir = "$buildDir/generated/openapi" + def corePackage = "ru.tinkoff.kora.example.openapi.petV2" //(1)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = [mode: "java-client", clientConfigPrefix: "httpClient.petV2"] + } + sourceSets.main { java.srcDirs += openApiGeneratePetV2.get().outputDir } + compileJava.dependsOn openApiGeneratePetV2 + + def openApiGeneratePetV3 = tasks.register("openApiGeneratePetV3", GenerateTask) { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV3.yaml" + outputDir = "$buildDir/generated/openapi" + def corePackage = "ru.tinkoff.kora.example.openapi.petV3" //(2)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = [mode: "java-reactive-client", clientConfigPrefix: "httpClient.petV3"] + } + sourceSets.main { java.srcDirs += openApiGeneratePetV3.get().outputDir } + compileJava.dependsOn openApiGeneratePetV3 + ``` + + 1. Isolated package for the first contract + 2. Different package for the second contract, so class names cannot clash + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + val openApiGeneratePetV2 = tasks.register("openApiGeneratePetV2") { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV2.yaml" + outputDir = "$buildDir/generated/openapi" + val corePackage = "ru.tinkoff.kora.example.openapi.petV2" //(1)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = mapOf("mode" to "kotlin-client", "clientConfigPrefix" to "httpClient.petV2") + } + kotlin.sourceSets.main { kotlin.srcDir(openApiGeneratePetV2.get().outputDir) } + tasks.withType { dependsOn(openApiGeneratePetV2) } + + val openApiGeneratePetV3 = tasks.register("openApiGeneratePetV3") { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV3.yaml" + outputDir = "$buildDir/generated/openapi" + val corePackage = "ru.tinkoff.kora.example.openapi.petV3" //(2)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = mapOf("mode" to "kotlin-suspend-client", "clientConfigPrefix" to "httpClient.petV3") + } + kotlin.sourceSets.main { kotlin.srcDir(openApiGeneratePetV3.get().outputDir) } + tasks.withType { dependsOn(openApiGeneratePetV3) } + ``` -- Configuring Gradle plugin parameters in [documentation](https://github.com/OpenAPITools/openapi-generator/blob/v7.14.0/modules/openapi-generator-gradle-plugin/README.adoc). -- Configuring `configOptions` plugin parameter in [documentation](https://openapi-generator.tech/docs/generators/java/#config-options). -- Configuring `openapiNormalizer` plugin parameter in [documentation](https://openapi-generator.tech/docs/customization/#openapi-normalizer). + 1. Isolated package for the first contract + 2. Different package for the second contract, so class names cannot clash ## Client { #client } -A minimal example of configuring a plugin to create a declarative HTTP client: +A minimal plugin configuration for creating a declarative HTTP client: ===! ":fontawesome-brands-java: `Java`" - Kora's available plugin options: - - - `clientConfigPrefix` - configuration prefix of created HTTP clients - - `tags` - possibility to put additional tags on created HTTP-clients - - `interceptors` - ability to specify interceptors for HTTP clients - - `primaryAuth` - specify which [authorization mechanism](http-client.md#authorization) to use as the primary one if several [securitySchemes]((https://swagger.io/docs/specification/authentication/)) are specified in OpenAPI - - `securityConfigPrefix` - prefix of authorization mechanism configuration [Basic](http-client.md#basic)/[ApiKey](http-client.md#apikey) (configuration path will be specified prefix + name [securitySchemes]((https://swagger.io/docs/specification/authentication/)) in OpenAPI, or just name in OpenAPI if prefix is not specified). - - `authAsMethodArgument` - ability to specify authorization as an argument of an HTTP client method rather than through an interceptor - - `authAllowMultiple` - generate interceptors for [multi-authentication](https://swagger.io/docs/specification/v3_0/authentication/#using-multiple-authentication-types) if it is specified in the specification. Values: `true`, `false` - - `additionalContractAnnotations` - ability to specify additional annotations over HTTP client methods - - `enableJsonNullable` - Treat `nullable=true` and `required=false` schema fields as a [JsonNullable](json.md#jsonnullable-wrapper) wrapper - - `forceIncludeOptional` - Force to set `@JsonInclude(Always)` for fields with `nullable=true` and `required=false` instead of `enableJsonNullable`. Values: `true`, `false`. - - `forceIncludeNonRequired` - Force to set [@JsonInclude(Always)](json.md#serialization-levels) for fields with `required=false` only. Values: `true`, `false`. - - `filterWithModels` - filter and exclude also unnecessary models from generation when the [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) option in `openapiNormalizer` is specified - - `mode` in which mode the generator should operate, available values: - * `java-client` - create synchronous client - * `java-async-client` - create [CompletionStage](https://www.baeldung.com/java-completablefuture) client - * `java-reactive-client` - create [reactive](https://projectreactor.io/docs/core/release/reference/) client, you need to connect [Project Reactor](https://mvnrepository.com/artifact/io.projectreactor/reactor-core) yourself. + For clients, `configOptions.mode` supports `java-client`, `java-async-client`, and `java-reactive-client`. + Other client parameters are described below in the authorization, interceptors, tags, models, and implicit headers sections. ```groovy def openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient", GenerateTask) { generatorName = "kora" group = "openapi tools" inputSpec = "$projectDir/src/main/resources/openapi/openapi.yaml" //(1)! - outputDir = "$buildDir/generated/openapi" //(2)! + outputDir = "$buildDir/generated/openapi" //(2)! def corePackage = "ru.tinkoff.kora.example.openapi" apiPackage = "${corePackage}.api" //(3)! modelPackage = "${corePackage}.model" //(4)! @@ -108,36 +384,20 @@ A minimal example of configuring a plugin to create a declarative HTTP client: compileJava.dependsOn openApiGenerateHttpClient //(9)! ``` - 1. Path to OpenAPI file from which classes will be created - 2. Directory where the files will be created - 3. Package from classes of delegates, controllers, converters, etc. - 4. Package from classes of models, DTOs, etc. - 5. Package from calling classes - 6. Mode of plugin operation (creating Java client / Kotlin / Java server, etc.) - 7. Prefix path to client configuration file - 8. Register the generated classes as the source code of the project - 9. Make code compilation dependent on HTTP client class generation (first generate, then compile) + 1. Path to the `OpenAPI` file used to create classes + 2. Directory where generated files are created + 3. Package for delegates, controllers, and mappers + 4. Package for models and DTOs + 5. Auxiliary generator package + 6. Plugin mode + 7. Client configuration path prefix + 8. Register generated classes as project source code + 9. Make code compilation depend on HTTP client class generation: generate first, compile after === ":simple-kotlin: `Kotlin`" - Kora's available plugin options: - - - `clientConfigPrefix` - configuration prefix of created HTTP clients - - `tags` - possibility to put additional tags on created HTTP-clients - - `interceptors` - ability to specify interceptors for HTTP clients - - `primaryAuth` - specify which [authorization mechanism](http-client.md#authorization) to use as the primary one if several [securitySchemes]((https://swagger.io/docs/specification/authentication/)) are specified in OpenAPI - - `securityConfigPrefix` - prefix of authorization mechanism configuration [Basic](http-client.md#basic)/[ApiKey](http-client.md#apikey) (configuration path will be specified prefix + name [securitySchemes]((https://swagger.io/docs/specification/authentication/)) in OpenAPI, or just name in OpenAPI if prefix is not specified). - - `authAsMethodArgument` - ability to specify authorization as an argument of an HTTP client method rather than through an interceptor - - `authAllowMultiple` - generate interceptors for [multi-authentication](https://swagger.io/docs/specification/v3_0/authentication/#using-multiple-authentication-types) if it is specified in the specification. Values: `true`, `false` - - `additionalContractAnnotations` - ability to specify additional annotations over HTTP client methods - - `additionalContractAnnotations` - ability to specify additional annotations over HTTP client methods - - `enableJsonNullable` - Treat `nullable=true` and `required=false` schema fields as a [JsonNullable](json.md#jsonnullable-wrapper) wrapper - - `forceIncludeOptional` - Force to set `@JsonInclude(Always)` for fields with `nullable=true` and `required=false` instead of `enableJsonNullable`. Values: `true`, `false`. - - `forceIncludeNonRequired` - Force to set [@JsonInclude(Always)](json.md#serialization-levels) for fields with `required=false` only. Values: `true`, `false`. - - `filterWithModels` - filter and exclude also unnecessary models from generation when the [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) option in `openapiNormalizer` is specified - - `mode` in which mode the generator should operate, available values: - * `kotlin-client` - create synchronous client - * `kotlin-suspend-client` - create suspend client + For clients, `configOptions.mode` supports `kotlin-client` and `kotlin-suspend-client`. + Other client parameters are described below in the authorization, interceptors, tags, models, and implicit headers sections. ```groovy val openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient") { @@ -161,29 +421,271 @@ A minimal example of configuring a plugin to create a declarative HTTP client: tasks.withType { dependsOn(openApiGenerateHttpClient) } //(9)! ``` - 1. Path to OpenAPI file from which classes will be created - 2. Directory where the files will be created - 3. Package from classes of delegates, controllers, converters, etc. - 4. Package from classes of models, DTOs, etc. - 5. Package from calling classes - 6. Mode of plugin operation (creating Java client / Kotlin / Java server, etc.) - 7. Prefix path to client configuration file - 8. Register the generated classes as the source code of the project - 9. Make code compilation dependent on HTTP client class generation (first generate, then compile) + 1. Path to the `OpenAPI` file used to create classes + 2. Directory where generated files are created + 3. Package for delegates, controllers, and mappers + 4. Package for models and DTOs + 5. Auxiliary generator package + 6. Plugin mode + 7. Client configuration path prefix + 8. Register generated classes as project source code + 9. Make code compilation depend on HTTP client class generation: generate first, compile after -Once created, the HTTP client will be available for deployment as a dependency on the created interface. +After generation, the HTTP client is available for dependency injection through the generated interface. -### Interceptors { #interceptors } +### Generated Client Usage { #client-usage } + +For every API tag, the generator produces an interface annotated with [`@HttpClient`](http-client.md), named after the tag (for example `PetApi`). +It is injected into components like any other Kora client, without extra registration: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class RootService { + + private final PetApi petApi; //(1)! + + public RootService(PetApi petApi) { + this.petApi = petApi; + } + } + ``` + + 1. The generated `@HttpClient` interface, injected directly + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class RootService( + private val petApi: PetApi, //(1)! + ) + ``` + + 1. The generated `@HttpClient` interface, injected directly + +The generated client reads its configuration from the path given by `clientConfigPrefix`, followed by the generated interface name. +For `clientConfigPrefix = "httpClient.petV2"` and interface `PetApi`, the configuration block is `httpClient.petV2.PetApi`. +The full set of client options (`url`, `requestTimeout`, per-operation blocks, `telemetry`) is described in the [HTTP client](http-client.md#configuration) documentation: + +===! ":material-code-json: `Hocon`" + + ```javascript + httpClient.petV2.PetApi { + url = "https://localhost:8443" //(1)! + requestTimeout = "10s" //(2)! + getValuesConfig { //(3)! + requestTimeout = "20s" + } + telemetry.logging.enabled = true + } + ``` + + 1. Base URL of the target service + 2. Default request timeout for all operations + 3. Per-operation override block, named after the `operationId` (here `getValues`) + +=== ":simple-yaml: `YAML`" + + ```yaml + httpClient: + petV2: + PetApi: + url: "https://localhost:8443" #(1)! + requestTimeout: "10s" #(2)! + getValuesConfig: #(3)! + requestTimeout: "20s" + telemetry: + logging: + enabled: true + ``` + + 1. Base URL of the target service + 2. Default request timeout for all operations + 3. Per-operation override block, named after the `operationId` (here `getValues`) + +The client method signatures depend on the selected `mode`: + +| Mode | Return type example | +| -------- | -------- | +| `java-client` | `PetApiResponses.GetPetByIdApiResponse` (blocking value) | +| `java-async-client` | `CompletionStage` | +| `java-reactive-client` | `Mono` (requires `reactor-core`) | +| `kotlin-client` | `PetApiResponses.GetPetByIdApiResponse` (blocking value) | +| `kotlin-suspend-client` | `suspend fun ...: PetApiResponses.GetPetByIdApiResponse` | + +Every method returns a sealed `*ApiResponses` envelope whose subtypes encode the HTTP status, the same way [server delegates](#delegate-response-types) do. + +### Client Authorization { #client-authorization } + +If the `OpenAPI` contract describes `securitySchemes`, the generator creates an `ApiSecurity` module with components for client authorization. +For `apiKey` and `basic`, configuration-reading components are generated. For `bearer` and `oauth`, a matching tagged `HttpClientTokenProvider` component is expected. + +`securityConfigPrefix` sets a common authorization configuration prefix. If the prefix is not specified, the configuration path is the `securitySchemes` name. +If an operation has several authorization schemes, `primaryAuth` can be specified; otherwise the generator picks one of the schemes and logs a warning. +If `authAllowMultiple` is enabled, the generator creates a composite interceptor that applies several authorization schemes sequentially. +If `authAsMethodArgument` is enabled, authorization data is added to the client method signature instead of a generated interceptor. + +#### apiKey and basic { #client-authorization-config } + +For `apiKey` and `basic` schemes, the generator produces `@DefaultComponent` config readers and interceptors, so no beans are required — only configuration values. +The configuration path is `securityConfigPrefix` followed by the scheme name (or just the scheme name when `securityConfigPrefix` is not set). +An `apiKey` scheme reads a single string; a `basic` scheme reads a `username`/`password` object: + +===! ":material-code-json: `Hocon`" + + ```javascript + openapiAuth { + apiKeyAuth = "MyAuthApiKey" //(1)! + basicAuth { //(2)! + username = "user" + password = "password" + } + } + ``` + + 1. `apiKey` scheme `apiKeyAuth`: value sent by the generated `ApiKeyHttpClientInterceptor` in the header/query/cookie declared by the scheme + 2. `basic` scheme `basicAuth`: credentials wrapped by the generated `BasicAuthHttpClientInterceptor` + +=== ":simple-yaml: `YAML`" + + ```yaml + openapiAuth: + apiKeyAuth: "MyAuthApiKey" #(1)! + basicAuth: #(2)! + username: "user" + password: "password" + ``` + + 1. `apiKey` scheme `apiKeyAuth`: value sent by the generated `ApiKeyHttpClientInterceptor` in the header/query/cookie declared by the scheme + 2. `basic` scheme `basicAuth`: credentials wrapped by the generated `BasicAuthHttpClientInterceptor` + +#### bearer and oauth { #client-authorization-token } + +For `bearer` and `oauth` schemes, the generator expects an [`HttpClientTokenProvider`](http-client.md#token-provider) component tagged with the generated `ApiSecurity` marker class +(for example `ApiSecurity.BearerAuth`). The generator wraps it in a `BearerAuthHttpClientInterceptor` automatically, so only the token provider must be supplied: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface ClientAuthModule { + + @Tag(ApiSecurity.BearerAuth.class) //(1)! + default HttpClientTokenProvider bearerTokenProvider() { + return request -> CompletableFuture.completedFuture("my-token"); //(2)! + } + } + ``` + + 1. Tag must match the generated marker class for the scheme + 2. Real implementations usually fetch or refresh the token here + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface ClientAuthModule { + + @Tag(ApiSecurity.BearerAuth::class) //(1)! + fun bearerTokenProvider(): HttpClientTokenProvider { + return HttpClientTokenProvider { CompletableFuture.completedFuture("my-token") } //(2)! + } + } + ``` + + 1. Tag must match the generated marker class for the scheme + 2. Real implementations usually fetch or refresh the token here + +#### Multiple schemes { #client-authorization-multiple } + +When an operation declares several security schemes, `primaryAuth` selects which one to apply; otherwise the generator picks one and logs a warning. +To apply several schemes on the same request, enable `authAllowMultiple` — the generator builds a composite interceptor that runs each scheme sequentially. +To pass the credentials explicitly per call instead of through an interceptor, enable `authAsMethodArgument` — the authorization value becomes a client method argument: + +===! ":fontawesome-brands-java: `Java`" -It is possible to put [interceptors](http-client.md#interceptors) on created clients with `@HttpClient` annotation. + ```groovy + configOptions = [ + mode: "java-client", + securityConfigPrefix: "openapiAuth", + primaryAuth: "apiKeyAuth", //(1)! + authAllowMultiple: "false", //(2)! + authAsMethodArgument: "false" //(3)! + ] + ``` + + 1. Scheme applied when an operation lists several + 2. Apply every declared scheme with a composite interceptor + 3. Add the auth value as a method argument instead of an interceptor -The value is a Json object whose key is the api tag from the contract, and the value is an object with `type` and `tag` fields, -it is possible to specify both fields at the same time, or optionally one of them: +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "securityConfigPrefix" to "openapiAuth", + "primaryAuth" to "apiKeyAuth", //(1)! + "authAllowMultiple" to "false", //(2)! + "authAsMethodArgument" to "false" //(3)! + ) + ``` -- `type` - the implementation class of a particular interceptor -- `tag` - tags of the interceptor (can be specified as an array of strings). + 1. Scheme applied when an operation lists several + 2. Apply every declared scheme with a composite interceptor + 3. Add the auth value as a method argument instead of an interceptor -In order to do this, set the `configOptions.interceptors` parameter: +### Additional Annotations { #additional-contract-annotations } + +The `additionalContractAnnotations` parameter adds annotations above generated client or server controller methods. +The value is a `JSON` object where the key is the API tag from the contract, or `*` for all operations, and the value is an array of objects with the `annotation` field. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-client", + additionalContractAnnotations: """ + { + "*": [ + { "annotation": "ru.tinkoff.example.CommonAnnotation" } + ], + "pet": [ + { "annotation": "ru.tinkoff.example.PetAnnotation" } + ] + } + """ + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "additionalContractAnnotations" to """{ + "*": [ + { "annotation": "ru.tinkoff.example.CommonAnnotation" } + ], + "pet": [ + { "annotation": "ru.tinkoff.example.PetAnnotation" } + ] + } + """ + ) + ``` + +### Interceptors { #interceptors } + +Generated clients annotated with `@HttpClient` can also be annotated with [interceptors](http-client.md#interceptors). +The value is a `JSON` object where the key is an API tag from the contract and the value is an array of objects with `type` and `tag` fields. +Both fields can be specified together, or only one of them can be specified: + +- `type` - implementation class of a concrete interceptor +- `tag` - interceptor tags, either a string or an array of strings + +Set `configOptions.interceptors`: ===! ":fontawesome-brands-java: `Java`" @@ -242,10 +744,10 @@ In order to do this, set the `configOptions.interceptors` parameter: ### Tags { #tags } -It is possible to put parameters `httpClientTag` and `telemetryTag` on created clients with `@HttpClient` annotation. -The value is a Json object, the key of which is the api tag from the contract, and the value is the object with the fields `httpClientTag` and `telemetryTag`. +Generated clients annotated with `@HttpClient` can receive `httpClientTag` and `telemetryTag` parameters. +The value is a `JSON` object where the key is an API tag from the contract and the value is an object with `httpClientTag` and `telemetryTag` fields. -For this purpose it is necessary to set the `configOptions.tags` parameter: +Set `configOptions.tags`: ===! ":fontawesome-brands-java: `Java`" @@ -254,11 +756,11 @@ For this purpose it is necessary to set the `configOptions.tags` parameter: mode: "java-client", tags: """ { - "*": { // применится для всех тегов, кроме явно указанных (в данном случае instrument) + "*": { "httpClientTag": "some.tag.Common", "telemetryTag": "some.tag.Common" }, - "instrument": { // применится для instrument + "instrument": { "httpClientTag": "some.tag.Instrument", "telemetryTag": "some.tag.Instrument" } @@ -273,11 +775,11 @@ For this purpose it is necessary to set the `configOptions.tags` parameter: configOptions = mapOf( "mode" to "kotlin-client", "tags" to """{ - "*": { // применится для всех тегов, кроме явно указанных (в данном случае instrument) + "*": { "httpClientTag": "some.tag.Common", "telemetryTag": "some.tag.Common" }, - "instrument": { // применится для instrument + "instrument": { "httpClientTag": "some.tag.Instrument", "telemetryTag": "some.tag.Instrument" } @@ -286,36 +788,75 @@ For this purpose it is necessary to set the `configOptions.tags` parameter: ) ``` +## Implicit Headers { #implicit-headers } + +By default, headers from an `OpenAPI` operation become generated method arguments. +If some headers are supplied by infrastructure rather than application code, they can be made implicit. + +- `implicitHeaders = true` makes all headers from `OpenAPI` operations implicit. +- `implicitHeadersRegex` makes only headers whose names match the regular expression implicit. + +An implicit header is removed from the method signature but remains in `OpenAPI` annotations in generated code. +This keeps the header in contract documentation without requiring application code to pass it manually. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-client", + implicitHeadersRegex: "X-Request-.*" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "implicitHeadersRegex" to "X-Request-.*" + ) + ``` + +## Models { #models } + +The generator creates request and response models from `OpenAPI` schemas. +Optional fields use `@Nullable` in `Java` and nullable type `T?` in `Kotlin`. +For schemas with inheritance and a discriminator, `Java` can generate `sealed interface`, and `Kotlin` can generate `sealed interface` / classes depending on the schema. + +### Optional Nullable Fields { #json-nullable } + +If a field is both `nullable: true` and absent from the `required` list, it is generated as a normal optional field by default. +If you need to distinguish three states - the field is absent in `JSON`, the field is present with `null`, and the field is present with a value - enable `enableJsonNullable`. +In that case, the field is generated as [JsonNullable](json.md#jsonnullable-wrapper). + +`forceIncludeOptional` and `forceIncludeNonRequired` control serialization of optional fields: + +- `forceIncludeOptional` sets `@JsonInclude(Always)` for fields with `nullable: true` and `required: false` instead of using `JsonNullable`. +- `forceIncludeNonRequired` sets `@JsonInclude(Always)` for all fields with `required: false`. + +`forceIncludeOptional` cannot be enabled together with `enableJsonNullable` because both modes solve the same problem in different ways. + +### Model Filtering { #filter-with-models } + +`OpenAPI Generator` can filter operations through `openapiNormalizer.FILTER`. +If `filterWithModels` is additionally enabled, the Kora generator tries to exclude unused models that remain after operation filtering. +This is useful for large contracts where an application generates only part of the API. + ## Server { #server } -A minimal example of configuring a plugin to create HTTP server handlers: +A minimal plugin configuration for creating HTTP server handlers: ===! ":fontawesome-brands-java: `Java`" - Available Kora plugin parameters: - - - `enableServerValidation` - whether to create validators according to the OpenAPI secification description for the server and whether to enable validation on HTTP handlers: `true, false`. - - `enableServerValidationInterceptor` - whether add validator interceptor for separate validation exception mapping to HTTP responses: `true, false` - - `requestInDelegateParams` - whether to expected `HttpServerRequest` as a method argument: `true, false` - - `interceptors` - ability to specify interceptors for HTTP controllers - - `additionalContractAnnotations` - ability to specify additional annotations for controller methods - - `enableJsonNullable` - Treat `nullable=true` and `required=false` schema fields as a [JsonNullable](json.md#jsonnullable-wrapper) wrapper - - `forceIncludeOptional` - Force to set [@JsonInclude(Always)](json.md#serialization-levels) for fields with `nullable=true` and `required=false` instead of `enableJsonNullable`. Values: `true`, `false`. - - `forceIncludeNonRequired` - Force to set [@JsonInclude(Always)](json.md#serialization-levels) for fields with `required=false` only. Values: `true`, `false`. - - `filterWithModels` - filter and exclude also unnecessary models from generation when the [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) option in `openapiNormalizer` is specified - - `prefixPath` - path prefix for HTTP-server controllers - - `delegateMethodBodyMode` - behavior for method body generation in delegate class. `none` - do not generate method body, `throw-exception` - throw exception in method body. For `throw-exception` additionally generates module with default Delegate class implementation if not exists another implementation in application graph - - `mode` in which mode the generator should operate, available values: - * `java-server` - create a synchronous server - * `java-async-server` - create a [CompletionStage](https://www.baeldung.com/java-completablefuture) server - * `java-reactive-server` - create a [reactive](https://projectreactor.io/docs/core/release/reference/) server, you need to connect [Project Reactor](https://mvnrepository.com/artifact/io.projectreactor/reactor-core) yourself. + For servers, `configOptions.mode` supports `java-server`, `java-async-server`, and `java-reactive-server`. + Other server parameters are described below in the validation, `delegate` classes, interceptors, models, and implicit headers sections. ```groovy def openApiGenerateHttpServer = tasks.register("openApiGenerateHttpServer", GenerateTask) { generatorName = "kora" group = "openapi tools" inputSpec = "$projectDir/src/main/resources/openapi/openapi.yaml" //(1)! - outputDir = "$buildDir/generated/openapi" //(2)! + outputDir = "$buildDir/generated/openapi" //(2)! def corePackage = "ru.tinkoff.kora.example.openapi" apiPackage = "${corePackage}.api" //(3)! modelPackage = "${corePackage}.model" //(4)! @@ -324,40 +865,26 @@ A minimal example of configuring a plugin to create HTTP server handlers: DISABLE_ALL: "true" ] configOptions = [ - mode: "java-server" //(6)! + mode: "java-server", //(6)! ] } sourceSets.main { java.srcDirs += openApiGenerateHttpServer.get().outputDir } //(7)! compileJava.dependsOn openApiGenerateHttpServer //(8)! ``` - 1. Path to OpenAPI file from which classes will be created - 2. Directory where the files will be created - 3. Package from classes of delegates, controllers, converters, etc. - 4. Package from classes of models, DTOs, etc. - 5. Package from calling classes - 6. Mode of plugin operation (creating Java client / Kotlin / Java server, etc.) - 7. Register the generated classes as the source code of the project - 8. Make code compilation dependent on HTTP client class generation (first generate, then compile) + 1. Path to the `OpenAPI` file used to create classes + 2. Directory where generated files are created + 3. Package for delegates, controllers, and mappers + 4. Package for models and DTOs + 5. Auxiliary generator package + 6. Plugin mode + 7. Register generated classes as project source code + 8. Make code compilation depend on HTTP server class generation: generate first, compile after === ":simple-kotlin: `Kotlin`" - Available Kora plugin parameters: - - - `enableServerValidation` - whether to create validators according to the OpenAPI secification description for the server and whether to enable validation on HTTP handlers: `true, false`. - - `enableServerValidationInterceptor` - whether add validator interceptor for separate validation exception mapping to HTTP responses: `true, false` - - `requestInDelegateParams` - whether to expected `HttpServerRequest` as a method argument: `true, false` - - `interceptors` - ability to specify interceptors for HTTP controllers - - `additionalContractAnnotations` - ability to specify additional annotations for controller methods - - `enableJsonNullable` - Treat `nullable=true` and `required=false` schema fields as a [JsonNullable](json.md#jsonnullable-wrapper) wrapper - - `forceIncludeOptional` - Force to set [@JsonInclude(Always)](json.md#serialization-levels) for fields with `nullable=true` and `required=false` instead of `enableJsonNullable`. Values: `true`, `false`. - - `forceIncludeNonRequired` - Force to set [@JsonInclude(Always)](json.md#serialization-levels) for fields with `required=false` only. Values: `true`, `false`. - - `filterWithModels` - filter and exclude also unnecessary models from generation when the [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) option in `openapiNormalizer` is specified - - `prefixPath` - path prefix for HTTP-server controllers - - `delegateMethodBodyMode` - behavior for method body generation in delegate class. `none` - do not generate method body, `throw-exception` - throw exception in method body. For `throw-exception` additionally generates module with default Delegate class implementation if not exists another implementation in application graph - - `mode` in which mode the generator should operate, available values: - * `kotlin-server` - create synchronous server - * `kotlin-suspend-server` - create suspend server + For servers, `configOptions.mode` supports `kotlin-server` and `kotlin-suspend-server`. + Other server parameters are described below in the validation, `delegate` classes, interceptors, models, and implicit headers sections. ```groovy val openApiGenerateHttpServer = tasks.register("openApiGenerateHttpServer") { @@ -380,20 +907,20 @@ A minimal example of configuring a plugin to create HTTP server handlers: tasks.withType { dependsOn(openApiGenerateHttpServer) } //(8)! ``` - 1. Path to OpenAPI file from which classes will be created - 2. Directory where the files will be created - 3. Package from classes of delegates, controllers, converters, etc. - 4. Package from classes of models, DTOs, etc. - 5. Package from calling classes - 6. Mode of plugin operation (creating Java client / Kotlin / Java server, etc.) - 7. Register the generated classes as the source code of the project - 8. Make code compilation dependent on HTTP client class generation (first generate, then compile) + 1. Path to the `OpenAPI` file used to create classes + 2. Directory where generated files are created + 3. Package for delegates, controllers, and mappers + 4. Package for models and DTOs + 5. Auxiliary generator package + 6. Plugin mode + 7. Register generated classes as project source code + 8. Make code compilation depend on HTTP server class generation: generate first, compile after -Once created, the handlers will be automatically registered. +After generation, handlers are registered automatically. ### Validation { #validation } -In order to generate models and controllers with annotations from the [validation](validation.md) module, the `enableServerValidation` option must be set: +To generate models and controllers with annotations from the [validation](validation.md) module, set `enableServerValidation`: ===! ":fontawesome-brands-java: `Java`" @@ -404,7 +931,7 @@ In order to generate models and controllers with annotations from the [validatio ] ``` - 1. Enabling validation on the HTTP server controller side + 1. Enables validation on the HTTP server controller side === ":simple-kotlin: `Kotlin`" @@ -415,19 +942,170 @@ In order to generate models and controllers with annotations from the [validatio ) ``` - 1. Enabling validation on the HTTP server controller side + 1. Enables validation on the HTTP server controller side -### Interceptors { #interceptors-2 } +When `enableServerValidation` is enabled, the generator adds validation annotations to models and server method parameters, +and also adds `@Validate` to controller methods with validated parameters. +`enableServerValidationInterceptor` controls adding `ValidationHttpServerInterceptor`, which converts validation errors to HTTP responses. +If `enableServerValidationInterceptor` is not specified explicitly, it is considered enabled when server validation is enabled. +If `enableServerValidationInterceptor = false` is specified, validation annotations remain, but the standard response interceptor is not added. + +### Delegate Implementation { #delegate-method-body } + +The server generator creates a controller and a `delegate` contract where the user implements application logic. +By default, `delegateMethodBodyMode = none`, so `delegate` contract methods do not get a standard body and must be implemented by the application. + +If `delegateMethodBodyMode = throwException` is set, methods get a body that throws an exception, and the generator also creates a module +with a default `delegate` contract implementation. This mode is useful when the application must be built before all operations are implemented, or when custom implementations are connected gradually. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-server", + delegateMethodBodyMode: "throwException" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-server", + "delegateMethodBodyMode" to "throwException" + ) + ``` + +#### Delegate Response Types { #delegate-response-types } + +Each generated `delegate` method returns a sealed `*ApiResponses` envelope whose subtypes encode the HTTP status declared in the contract. +For an operation `getPetById` with responses `200` and `404`, the generator produces `PetApiResponses.GetPetByIdApiResponse` with subtypes +`GetPetById200ApiResponse` (carrying the body via `content()`) and `GetPetById404ApiResponse`. The implementation returns the subtype matching the outcome: + +===! ":fontawesome-brands-java: `Java`" + + Return type depends on `mode`: `java-server` returns the value directly (shown here), `java-async-server` returns `CompletionStage<...>`, `java-reactive-server` returns `Mono<...>`: + + ```java + @Component + public final class PetDelegate implements PetApiDelegate { + + private final Map petMap = new ConcurrentHashMap<>(); + + @Override + public PetApiResponses.GetPetByIdApiResponse getPetById(long petId) { + var pet = petMap.get(petId); + if (pet == null) { + return new PetApiResponses.GetPetByIdApiResponse.GetPetById404ApiResponse(); //(1)! + } + return new PetApiResponses.GetPetByIdApiResponse.GetPetById200ApiResponse(pet); //(2)! + } + + @Override + public PetApiResponses.AddPetApiResponse addPet(Pet body) { + petMap.put(body.id(), body); + return new PetApiResponses.AddPetApiResponse.AddPet200ApiResponse(body); + } + } + ``` + + 1. Status `404` subtype, no body + 2. Status `200` subtype carrying the response body + +=== ":simple-kotlin: `Kotlin`" + + Return type depends on `mode`: `kotlin-server` returns the value directly (shown here), `kotlin-suspend-server` uses a `suspend` method: + + ```kotlin + @Component + class PetDelegate : PetApiDelegate { + + private val petMap = ConcurrentHashMap() + + override fun getPetById(petId: Long): PetApiResponses.GetPetByIdApiResponse { + val pet = petMap[petId] + return if (pet == null) { + PetApiResponses.GetPetByIdApiResponse.GetPetById404ApiResponse() //(1)! + } else { + PetApiResponses.GetPetByIdApiResponse.GetPetById200ApiResponse(pet) //(2)! + } + } + + override fun addPet(pet: Pet): PetApiResponses.AddPetApiResponse { + petMap[pet.id] = pet + return PetApiResponses.AddPetApiResponse.AddPet200ApiResponse(pet) + } + } + ``` + + 1. Status `404` subtype, no body + 2. Status `200` subtype carrying the response body + +#### Raw Request in Delegate { #request-in-delegate } + +By default, a `delegate` method receives only the parameters declared in the contract. If an implementation needs access to the raw request +(for example to read an infrastructure header or the remote address), enable `requestInDelegateParams`. The generator then adds an +`HttpServerRequest` as the first parameter of every `delegate` method. This is a server-only option. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-server", + requestInDelegateParams: "true" //(1)! + ] + ``` + + 1. Adds `HttpServerRequest _serverRequest` as the first argument of each delegate method + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-server", + "requestInDelegateParams" to "true" //(1)! + ) + ``` + + 1. Adds `HttpServerRequest _serverRequest` as the first argument of each delegate method + +#### Controller Path Prefix { #prefix-path } + +`prefixPath` prepends a base path to every generated HTTP server controller route. It is useful when all operations should be served under a common +segment (for example `/api/v1`) that is not part of the `OpenAPI` paths. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-server", + prefixPath: "/api/v1" //(1)! + ] + ``` -It is possible to put [interceptors](http-server.md#interceptors) on created controllers with `@HttpController` annotation. + 1. A contract path `/pet/{id}` becomes `/api/v1/pet/{id}` -The value is a Json object whose key is the api tag from the contract, and the value is an object with `type` and `tag` fields, -it is possible to specify both fields at the same time, or optionally one of them: +=== ":simple-kotlin: `Kotlin`" -- `type` - the implementation class of a particular interceptor -- `tag` - tags of the interceptor (can be specified as an array of strings). + ```groovy + configOptions = mapOf( + "mode" to "kotlin-server", + "prefixPath" to "/api/v1" //(1)! + ) + ``` -In order to do this, set the `configOptions.interceptors` parameter: + 1. A contract path `/pet/{id}` becomes `/api/v1/pet/{id}` + +### Interceptors { #interceptors-2 } + +Generated controllers annotated with `@HttpController` can also be annotated with [interceptors](http-server.md#interceptors). +The value is a `JSON` object where the key is an API tag from the contract and the value is an object with `type` and `tag` fields. +Both fields can be specified together, or only one of them can be specified: + +- `type` - implementation class of a concrete interceptor +- `tag` - interceptor tags, either a string or an array of strings + +Set `configOptions.interceptors`: ===! ":fontawesome-brands-java: `Java`" @@ -486,23 +1164,43 @@ In order to do this, set the `configOptions.interceptors` parameter: ### Authorization { #authorization } -Kora provides an interface to extract authorization information within the interceptor, -created for the server from OpenAPI, you can pull any type of authorization [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/) +When the `OpenAPI` contract describes `securitySchemes`, the server generator creates an `ApiSecurity` module with one marker class per scheme: +`ApiSecurity.BearerAuth`, `ApiSecurity.BasicAuth`, `ApiSecurity.ApiKeyAuth`, and `ApiSecurity.OAuth` +(handling [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/)). +For each scheme, the application must provide an `HttpServerPrincipalExtractor` component tagged with the matching marker class. +The extractor receives the request and the parsed credential value and returns the authenticated `Principal`: -===! ":fontawesome-brands-java: ``Java``" +===! ":fontawesome-brands-java: `Java`" ```java @Module public interface AuthModule { - + @Tag(ApiSecurity.BearerAuth.class) default HttpServerPrincipalExtractor bearerHttpServerPrincipalExtractor() { - return (request, value) -> CompletableFuture.completedFuture(new MyPrincipal(request.headers().getFirst("Authorization"))); + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); + } + + @Tag(ApiSecurity.BasicAuth.class) + default HttpServerPrincipalExtractor basicHttpServerPrincipalExtractor() { + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); + } + + @Tag(ApiSecurity.ApiKeyAuth.class) + default HttpServerPrincipalExtractor apiKeyHttpServerPrincipalExtractor() { + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); + } + + @Tag(ApiSecurity.OAuth.class) + default HttpServerPrincipalExtractor oauthHttpServerPrincipalExtractor() { //(1)! + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); } } ``` -=== ":simple-kotlin: ``Kotlin``" + 1. `OAuth` schemes declare scopes, so the extractor returns a `PrincipalWithScopes` + +=== ":simple-kotlin: `Kotlin`" ```kotlin @Module @@ -510,22 +1208,65 @@ created for the server from OpenAPI, you can pull any type of authorization [Bas @Tag(ApiSecurity.BearerAuth::class) fun bearerHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { - return HttpServerPrincipalExtractor { request, value -> - CompletableFuture.completedFuture( - MyPrincipal(request.headers().getFirst("Authorization"))) - ) - } + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + + @Tag(ApiSecurity.BasicAuth::class) + fun basicHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + + @Tag(ApiSecurity.ApiKeyAuth::class) + fun apiKeyHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + + @Tag(ApiSecurity.OAuth::class) + fun oauthHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { //(1)! + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + } + ``` + + 1. `OAuth` schemes declare scopes, so the extractor returns a `PrincipalWithScopes` + +For `OAuth`, the returned principal must implement `PrincipalWithScopes` so the generated controller can enforce the scopes declared on each operation. +Only the schemes that the contract actually uses need an extractor; a marker class exists for every declared scheme: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserPrincipal(String name) implements PrincipalWithScopes { + + @Override + public Collection scopes() { + return List.of("read", "write"); //(1)! + } + } + ``` + + 1. Scopes granted to this principal, matched against the operation's required scopes + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserPrincipal(val name: String) : PrincipalWithScopes { + + override fun scopes(): Collection { + return listOf("read", "write") //(1)! } } ``` + 1. Scopes granted to this principal, matched against the operation's required scopes + ## Recommendations { #recommendations } -???+ warning "Advice" +???+ tip "Advice" - In case you have something that is not created by the plugin, or the behavior is different from what you want or other versions, - you should carefully check the [plugin configuration](#configuration) settings and examine them, - as they may affect the results of how classes are created. + If something is not generated by the plugin, or behavior differs from expectations or from other versions, + carefully check the [plugin configuration](#configuration) and study the settings, + because they can affect how classes are generated. - Starting with `7.0.0` version of the plugin, the `SIMPLIFY_ONEOF_ANYOF` rule enabled by default at the `openapiNormalizer` parameter - may lead to some not obvious generator results. + Starting with plugin version `7.0.0`, the `SIMPLIFY_ONEOF_ANYOF` rule enabled by default in `openapiNormalizer` + can lead to some non-obvious generator results. diff --git a/mkdocs/docs/en/documentation/openapi-management.md b/mkdocs/docs/en/documentation/openapi-management.md index 7041c67..a2d10ac 100644 --- a/mkdocs/docs/en/documentation/openapi-management.md +++ b/mkdocs/docs/en/documentation/openapi-management.md @@ -1,11 +1,14 @@ --- -description: "Explains Kora OpenAPI management module for serving generated OpenAPI specifications through the management HTTP server. Use when working with OpenApiManagementModule, OpenAPI, management endpoint, private HTTP server." +description: "Explains Kora OpenAPI management module for serving generated OpenAPI specifications, Swagger UI, and RapiDoc pages through the public HTTP server. Use when working with OpenApiManagementModule, OpenApiManagementConfig, OpenAPI endpoint, Swagger UI, RapiDoc." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI management module for serving generated OpenAPI specifications through the management HTTP server; key triggers include OpenApiManagementModule, OpenAPI, management endpoint, private HTTP server." + use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI management module for serving generated OpenAPI specifications, Swagger UI, and RapiDoc pages through the public HTTP server; key triggers include OpenApiManagementModule, OpenApiManagementConfig, OpenAPI endpoint, Swagger UI, RapiDoc, /openapi, /swagger-ui, /rapidoc." --- -A module to provide an OpenAPI file from an application, -as well as [Swagger UI](https://swagger.io/tools/swagger-ui/) and [Rapidoc](https://rapidocweb.com/) for displaying OpenAPI. +The `openapi-management` module serves ready-made `OpenAPI` files from an application, along with [Swagger UI](https://swagger.io/tools/swagger-ui/) and [RapiDoc](https://rapidocweb.com/) pages for viewing them. +`OpenAPI` is a machine-readable HTTP API contract: it helps inspect available operations, data models, and request parameters. + +The module does not create a contract from code; it only publishes existing files from application resources. +This is useful for local development, test environments, and operational access to API documentation without a separate documentation server. For a step-by-step walkthrough before the reference details, see [OpenAPI HTTP Server](../guides/openapi-http-server.md). @@ -37,11 +40,12 @@ For a step-by-step walkthrough before the reference details, see [OpenAPI HTTP S interface Application : OpenApiManagementModule ``` -Requires [HTTP server](http-server.md) module. +Requires the [HTTP server](http-server.md) module because it registers its own `GET` handlers for serving files and viewer pages. +These are ordinary `HttpServerRequestHandler` beans collected by the **public** HTTP server, so `/openapi`, `/swagger-ui`, and `/rapidoc` are exposed on the public HTTP port, not on the private (management) port. ## Configuration { #configuration } -An example of the configuration described in the `OpenApiManagementConfig` class: +An example of the configuration described by the `OpenApiManagementConfig` class: ===! ":material-code-json: `Hocon`" @@ -63,22 +67,23 @@ An example of the configuration described in the `OpenApiManagementConfig` class } ``` - 1. Relative path to OpenAPI files in the `resources` directory, either a single file or multiple files can be specified - 2. The on/off switch of the controller that gives the OpenAPI - 3. Path where OpenAPI will be available - 1. If a single OpenAPI file is specified, then represent entire path where file is available - 2. If multiple OpenAPI files are specified, is a path prefix to the file name `/openapi/{fileName}`, taking the specified path and appending the file name to it without the directories and its extension, example of the file `someDirectory/my-openapi-1.yaml` the file path will be `/openapi/my-openapi-1`. - 4. On/Off of the controller that gives SwaggerUI - 5. Path where the SwaggerUI will be accessed - 6. On/Off of the controller that gives Rapidoc - 7. Path where Rapidoc will be available + 1. Path to an `OpenAPI` file or a list of paths relative to application resources (required, default: not specified). + 2. Enables serving `OpenAPI` files through the HTTP handler (default: `false`). + 3. Path where `OpenAPI` files are available (default: `/openapi`). + If one file is specified, it is available exactly at this path. + If multiple files are specified, the path becomes a prefix of the `/openapi/{file}` form. + The `{file}` value is taken from the file name without directories and without the `.json`, `.yml`, or `.yaml` extension: `someDirectory/my-openapi-1.yaml` will be available at `/openapi/my-openapi-1`. + 4. Enables the `Swagger UI` page (default: `false`). + 5. Path where the `Swagger UI` page is available (default: `/swagger-ui`). + 6. Enables the `RapiDoc` page (default: `false`). + 7. Path where the `RapiDoc` page is available (default: `/rapidoc`). === ":simple-yaml: `YAML`" ```yaml openapi: management: - file = [ "my-openapi-1.yaml", "my-openapi-2.yaml" ] #(1)! + file: [ "my-openapi-1.yaml", "my-openapi-2.yaml" ] #(1)! enabled: false #(2)! endpoint: "/openapi" #(3)! swaggerui: @@ -89,22 +94,45 @@ An example of the configuration described in the `OpenApiManagementConfig` class endpoint: "/rapidoc" #(7)! ``` - 1. Relative path to OpenAPI files in the `resources` directory, either a single file or multiple files can be specified - 2. The on/off switch of the controller that gives the OpenAPI - 3. Path where OpenAPI will be available - 1. If a single OpenAPI file is specified, then represent entire path where file is available - 2. If multiple OpenAPI files are specified, is a path prefix to the file name `/openapi/{fileName}`, taking the specified path and appending the file name to it without the directories and its extension, example of the file `someDirectory/my-openapi-1.yaml` the file path will be `/openapi/my-openapi-1`. - 4. On/Off of the controller that gives SwaggerUI - 5. Path where the SwaggerUI will be accessed - 6. On/Off of the controller that gives Rapidoc - 7. Path where Rapidoc will be available + 1. Path to an `OpenAPI` file or a list of paths relative to application resources (required, default: not specified). + 2. Enables serving `OpenAPI` files through the HTTP handler (default: `false`). + 3. Path where `OpenAPI` files are available (default: `/openapi`). + If one file is specified, it is available exactly at this path. + If multiple files are specified, the path becomes a prefix of the `/openapi/{file}` form. + The `{file}` value is taken from the file name without directories and without the `.json`, `.yml`, or `.yaml` extension: `someDirectory/my-openapi-1.yaml` will be available at `/openapi/my-openapi-1`. + 4. Enables the `Swagger UI` page (default: `false`). + 5. Path where the `Swagger UI` page is available (default: `/swagger-ui`). + 6. Enables the `RapiDoc` page (default: `false`). + 7. Path where the `RapiDoc` page is available (default: `/rapidoc`). + +Files are read from application resources on the first request and then cached in memory (subsequent requests return the cached bytes). +Files with the `.json` extension use the `text/json; charset=utf-8` response type; all other files use `text/x-yaml; charset=utf-8`. + +With multiple files, `Swagger UI` shows the list of available contracts, and `RapiDoc` opens the first file from the list. + +When multiple files are configured, a request to `/openapi/{file}` with an unknown `{file}` name returns `404` (`OpenAPI file not registered`), and a request with an empty `{file}` value returns `400` (`OpenAPI file not specified`). +If a configured resource cannot be located or read at request time, the handler returns `404` or `500` respectively; otherwise it responds with `200` and the file content. + +## Endpoints { #endpoints } + +With serving enabled, the module registers the following `GET` routes on the public HTTP server (paths shown with default `endpoint` values): + +| Route | Backing handler | Enabled by | +|-------|-----------------|------------| +| `GET /openapi` (single file) or `GET /openapi/{file}` (multiple files) | `OpenApiHttpServerHandler` | `enabled = true` | +| `GET /swagger-ui` | `SwaggerUIHttpServerHandler` | `swaggerui.enabled = true` | +| `GET /swagger-ui/oauth2-redirect` | `SwaggerOauthHttpServerHandler` | registered automatically together with `Swagger UI` | +| `GET /rapidoc` | `RapidocHttpServerHandler` | `rapidoc.enabled = true` | + +Each route uses the `endpoint` value from its configuration section, so overriding an `endpoint` moves the corresponding route. +The `OAuth2` redirect path is always `swaggerui.endpoint` plus the `/oauth2-redirect` suffix. ## Recommendations { #recommendations } -???+ tip "Recommendation" +???+ warning "Recommendation" - We recommend using [contract first approach](openapi-codegen.md) and generate code using this contract, - in this approach same contract file is displayed. + We recommend creating the [contract first and then generating code from it](openapi-codegen.md). + In this case, the module publishes the same contract file that is used for generation. - In the case where the code is first and the contract file is supposed to be created from it, you can use the [Swagger Gradle Plugin](https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-gradle-plugin/README.md). - together with [Swagger annotation set](https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations), which will be used to create the contract file. + If code is written first and the contract should be created from it, you can use the [Swagger Gradle Plugin](https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-gradle-plugin/README.md) + together with [Swagger annotations](https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations). diff --git a/mkdocs/docs/en/documentation/probes.md b/mkdocs/docs/en/documentation/probes.md index 49c5612..b1bf904 100644 --- a/mkdocs/docs/en/documentation/probes.md +++ b/mkdocs/docs/en/documentation/probes.md @@ -1,85 +1,281 @@ --- -description: "Explains Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting. Use when working with ReadinessProbe, LivenessProbe, ProbeFailure, ProbesModule, CircuitBreaker." +description: "Explains Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting. Use when working with ReadinessProbe, LivenessProbe, LivenessProbeFailure, ReadinessProbeFailure, CircuitBreaker." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting; key triggers include ReadinessProbe, LivenessProbe, ProbeFailure, ProbesModule, CircuitBreaker." + use_when: "Use this file for Kora docs or implementation questions about Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting; key triggers include ReadinessProbe, LivenessProbe, LivenessProbeFailure, ReadinessProbeFailure, CircuitBreaker." --- -Functionality that gives the application two methods for obtaining probes on a private port about service readiness/liveness. +Probes let you check application `liveness` and `readiness` through the private HTTP port. +They are usually used by orchestrators and load balancers to decide whether requests can be sent to the application and whether its instance should be restarted. +Having two separate probes helps distinguish temporary inability to receive traffic from a state where the process itself should be considered unhealthy. -Provided by adding a [private HTTP server](http-server.md) module. +Probes are handled by the [private HTTP server](http-server.md). By default, it runs on port `8085`. +The `LivenessProbe` and `ReadinessProbe` interfaces come from the core `ru.tinkoff.kora:common` module (a transitive dependency of every Kora application), +and the endpoints that expose them are provided by the [HTTP server](http-server.md) module, so no additional dependency is required to add a probe. + +Both probe endpoints are always present on the private server, even when the application registers no custom probe of that kind — in that case the endpoint simply reports success. For a step-by-step walkthrough before the reference details, see [Observability](../guides/observability.md). ## Liveness { #liveness } -This sample is responsible for indicating whether the application is currently alive. Kora tries to start giving this sample as early as possible, so that orchestrators know for sure that there are no problems at startup and don't try to restart the application. +This probe indicates that the application is alive and should not be restarted. Kora tries to expose this probe as early as possible so the orchestrator does not restart the application during normal startup. -Example of the HTTP server path configuration for probe ping described in the `HttpServerConfig` class (default values are specified): +Example of the private HTTP server path configuration described in the `HttpServerConfig` class (default value is shown): ===! ":material-code-json: `Hocon`" ```javascript httpServer { - privateApiHttpLivenessPath = "/system/liveness" + privateApiHttpLivenessPath = "/system/liveness" //(1)! } ``` + 1. `Liveness` probe path on the private HTTP server (default: `/system/liveness`). + === ":simple-yaml: `YAML`" ```yaml httpServer: - privateApiHttpLivenessPath: "/system/liveness" + privateApiHttpLivenessPath: "/system/liveness" #(1)! ``` -Creating your custom viability sample requires the component to implement the interface: + 1. `Liveness` probe path on the private HTTP server (default: `/system/liveness`). + +To create a custom `liveness` probe, register a [component](container.md) that implements the `LivenessProbe` interface: + ```java public interface LivenessProbe { @Nullable - LivenessProbeFailure probe(); + LivenessProbeFailure probe() throws Exception; } ``` -The sample shall return `LivenessProbeFailure` on error, and `null` on success. +The probe must return `null` on success or a `LivenessProbeFailure` describing the problem. +`LivenessProbeFailure` is a record whose single `message` field becomes the `503` response body: + +```java +public record LivenessProbeFailure(String message) {} +``` + +The `probe()` method is declared `throws Exception`, so the implementation may call checked-exception APIs directly. +A thrown exception is treated as a failure — see [Response](#response) for the exact status codes and bodies. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.common.Component; + import ru.tinkoff.kora.common.liveness.LivenessProbe; + import ru.tinkoff.kora.common.liveness.LivenessProbeFailure; + + @Component + public final class ApplicationHealthProbe implements LivenessProbe { + + @Override + public LivenessProbeFailure probe() { + return null; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.Component + import ru.tinkoff.kora.common.liveness.LivenessProbe + import ru.tinkoff.kora.common.liveness.LivenessProbeFailure + + @Component + class ApplicationHealthProbe : LivenessProbe { + override fun probe(): LivenessProbeFailure? = null + } + ``` ## Readiness { #readiness } -This sample is responsible for indicating whether the application is currently ready to run. +This probe indicates that the application is ready to receive workload. -Example of the HTTP server path configuration for probe ping described in the `HttpServerConfig` class (default values are specified): +Example of the private HTTP server path configuration described in the `HttpServerConfig` class (default value is shown): ===! ":material-code-json: `Hocon`" ```javascript httpServer { - privateApiHttpReadinessPath = "/system/readiness" + privateApiHttpReadinessPath = "/system/readiness" //(1)! } ``` + 1. `Readiness` probe path on the private HTTP server (default: `/system/readiness`). + === ":simple-yaml: `YAML`" ```yaml httpServer: - privateApiHttpReadinessPath: "/system/readiness" + privateApiHttpReadinessPath: "/system/readiness" #(1)! ``` -Creating your custom viability sample requires the component to implement the interface: + 1. `Readiness` probe path on the private HTTP server (default: `/system/readiness`). + +To create a custom `readiness` probe, register a [component](container.md) that implements the `ReadinessProbe` interface: + ```java public interface ReadinessProbe { @Nullable - ReadinessProbeFailure probe(); + ReadinessProbeFailure probe() throws Exception; } ``` -The sample shall return `ReadinessProbeFailure` in case of error, and `null` in case of success. +The probe must return `null` on success or a `ReadinessProbeFailure` describing the problem. +`ReadinessProbeFailure` is a record whose single `message` field becomes the `503` response body: + +```java +public record ReadinessProbeFailure(String message) {} +``` + +As with `LivenessProbe`, the `probe()` method is declared `throws Exception` and a thrown exception is treated as a failure. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.common.Component; + import ru.tinkoff.kora.common.readiness.ReadinessProbe; + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure; + + import java.time.Duration; + import java.time.Instant; + + @Component + public final class CustomReadinessProbe implements ReadinessProbe { + + private static final Duration WARMUP_PERIOD = Duration.ofMillis(500); + + private final Instant startedAt = Instant.now(); + + @Override + public ReadinessProbeFailure probe() { + var readyAt = startedAt.plus(WARMUP_PERIOD); + if (Instant.now().isBefore(readyAt)) { + return new ReadinessProbeFailure("Service is warming up"); + } + return null; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.Component + import ru.tinkoff.kora.common.readiness.ReadinessProbe + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure + import java.time.Duration + import java.time.Instant + + @Component + class CustomReadinessProbe : ReadinessProbe { + private val startedAt = Instant.now() + + override fun probe(): ReadinessProbeFailure? { + val readyAt = startedAt.plus(Duration.ofMillis(500)) + return if (Instant.now().isBefore(readyAt)) { + ReadinessProbeFailure("Service is warming up") + } else { + null + } + } + } + ``` + +## Multiple probes { #multiple-probes } + +Kora collects **every** registered [component](container.md) that implements `LivenessProbe` (or `ReadinessProbe`) automatically — +you do not wire them together manually. Each endpoint runs all probes of its kind and aggregates the result: + +- The endpoint returns `200 OK` only when **all** probes of that kind succeed. +- A single failing probe makes the whole endpoint report `503`, with the failing probe's message as the body. +- When no probe of that kind is registered, the endpoint returns `200 OK` — the private server always exposes both paths. + +This lets you split independent readiness or liveness conditions across several small, focused probe components. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.common.Component; + import ru.tinkoff.kora.common.readiness.ReadinessProbe; + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure; + + @Component + public final class ComponentReadinessProbe implements ReadinessProbe { //(1)! + + private final SomeComponent component; + + public ComponentReadinessProbe(SomeComponent component) { + this.component = component; + } + + @Override + public ReadinessProbeFailure probe() { + if (component.isInitialized()) { //(2)! + return null; + } + return new ReadinessProbeFailure("SomeComponent is not initialized yet"); + } + } + ``` + + 1. Any number of `ReadinessProbe` components can coexist; the endpoint fails if any of them fails + 2. Check the state of an **internal** component, not an external dependency + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.Component + import ru.tinkoff.kora.common.readiness.ReadinessProbe + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure + + @Component + class ComponentReadinessProbe( + private val component: SomeComponent + ) : ReadinessProbe { //(1)! + + override fun probe(): ReadinessProbeFailure? { + return if (component.isInitialized) { //(2)! + null + } else { + ReadinessProbeFailure("SomeComponent is not initialized yet") + } + } + } + ``` + + 1. Any number of `ReadinessProbe` components can coexist; the endpoint fails if any of them fails + 2. Check the state of an **internal** component, not an external dependency + +## Response { #response } + +Each probe endpoint is served by the [private HTTP server](http-server.md) and returns a `text/plain` body together with a status code: + +- `200 OK` — body `OK` — all registered probes returned `null`, or no probe of that kind is registered. +- `503 Service Unavailable` — body is the `message` of the returned `LivenessProbeFailure` / `ReadinessProbeFailure` — at least one probe reported a failure. +- `503 Service Unavailable` — body `Probe failed: ` — a probe threw an exception; the thrown exception is treated as a failure. +- `503 Service Unavailable` — body `Probe is not ready yet` — a probe component has not been initialized in the dependency container yet. +- `408 Request Timeout` — body `Probe failed: timeout` — probe execution did not finish within `30` seconds. + +The endpoint responds as soon as the aggregated result is known; orchestrator and load-balancer health checks can therefore match on either the status code or the plaintext body. ## Recommendations { #recommendations } ???+ warning "Recommendation" - **We strongly discourage runs that test external dependencies such as databases or other services.** + **Probes that directly check external dependencies, such as databases, queues, or other services, are not recommended.** + + Temporary unavailability of an external dependency should not automatically restart the application. For such cases, use the [CircuitBreaker](resilient.md#circuitbreaker) pattern. - If external dependencies are not available, we recommend using the [CircuitBreaker](resilient.md#circuitbreaker) pattern. +A probe should reflect the state of the application **itself**, not of the systems it talks to. +Good examples are a `ReadinessProbe` that reports a failure while the service is warming up, +or one that checks whether an internal component has finished its initialization. -A good example for `ReadinessProbe` is prob that returns an error while the service is warming up. +Each probe runs on a dedicated executor (a virtual-thread executor when available, otherwise `ForkJoinPool.commonPool()`), +so a probe body may block without stalling the private HTTP server. The whole endpoint is still bounded by a `30` second timeout, +after which it responds with `408`, so keep probe logic fast and avoid long-running or unbounded work. diff --git a/mkdocs/docs/en/documentation/resilient.md b/mkdocs/docs/en/documentation/resilient.md index 6d93f5d..84192db 100644 --- a/mkdocs/docs/en/documentation/resilient.md +++ b/mkdocs/docs/en/documentation/resilient.md @@ -1,10 +1,14 @@ --- -description: "Explains Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, telemetry, configuration, and supported signatures. Use when working with @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." +description: "Explains Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, imperative managers, exceptions, telemetry, configuration, and supported signatures. Use when working with @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, telemetry, configuration, and supported signatures; key triggers include @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." + use_when: "Use this file for Kora docs or implementation questions about Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, imperative managers, exceptions, telemetry, configuration, and supported signatures; key triggers include @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerManager, RetryManager, TimeoutManager, FallbackManager, RetryState, CallNotPermittedException, RetryExhaustedException, TimeoutExhaustedException, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." --- -Module for creating a fault-tolerant application using approaches such as *CircuitBreaker, Fallback, Timeout, Retry* using aspect annotations. +Module for building a fault-tolerant application using mechanisms such as [CircuitBreaker](#circuitbreaker), +[Fallback](#fallback), [Retry](#retry), and [Timeout](#timeout). +These mechanisms can be applied declaratively through aspect annotations or directly through manager components when protection is needed in imperative code. + +`ResilientModule` combines `CircuitBreakerModule`, `RetryModule`, `TimeoutModule`, and `FallbackModule`. For a step-by-step walkthrough before the reference details, see [Resilience](../guides/resilient.md). @@ -38,26 +42,29 @@ For a step-by-step walkthrough before the reference details, see [Resilience](.. ## CircuitBreaker { #circuitbreaker } -CircuitBreaker is a proxy that controls the flow to requests of a particular method -and can temporarily stop execution of this method if the method throws many exceptions that meet the specified filter requirements (`CircuitBreakerPredicate`). +`CircuitBreaker` is a proxy that controls the request flow to a particular method +and can temporarily prohibit execution of this method if it throws many exceptions matching the configured filter (`CircuitBreakerPredicate`). The purpose of applying CircuitBreaker is to give the system time to correct the error that caused the failure before allowing the application to attempt the operation again. -The CircuitBreaker pattern provides stability while the system recovers from the failure and reduces the impact on performance. +The `CircuitBreaker` pattern provides stability while the system recovers from the failure and reduces the impact on performance. +`CircuitBreaker` can be in one of several states: `CLOSED`, `OPEN`, `HALF_OPEN`. -- `CLOSED`: An application request is redirected to an operation. The proxy keeps a count of the number of recent failures within a set number of operations (`slidingWindowSize`) coming through the proxy, and if the operation call did not complete successfully, the proxy increments this number. - If the number of requests exceeds the specified minimum ceiling required for counts (`minimumRequiredCalls`) and the number of recent failures exceeds the specified threshold (`failureRateThreshold`) for the specified period of time, the proxy is placed in the `OPEN` state. +- `CLOSED`: an application request is passed to the protected operation. The proxy counts recent failures within the configured number of operations (`slidingWindowSize`) passing through it, and increments this count when the operation does not complete successfully. + If the number of requests exceeds the minimum amount required for calculation (`minimumRequiredCalls`) and the number of recent failures exceeds the configured threshold (`failureRateThreshold`), the proxy moves to `OPEN`. - `OPEN`: While in this status, the request from the application immediately terminates with an error and an exception is returned to the application. - At this point, the proxy starts a wait time timer (`waitDurationInOpenState`), and when the time of this timer expires, the proxy is placed in the `HALF-OPEN` state. -- `HALF-OPEN`: A limited number of requests (`permittedCallsInHalfOpenState`) from the application are allowed to pass through and invoke the operation. If these requests are successful, it is assumed that the error that previously caused the - failure has been resolved, and the circuit breaker enters the `CLOSED` state (the failure counter is reset). If any request terminates with a fault, the circuit breaker assumes that the + At this point, the proxy starts a wait timer (`waitDurationInOpenState`), and when it expires, the proxy moves to `HALF_OPEN`. +- `HALF_OPEN`: a limited number of requests (`permittedCallsInHalfOpenState`) from the application are allowed to pass through and invoke the operation. If these requests are successful, it is assumed that the error that previously caused the + failure has been resolved, and `CircuitBreaker` enters the `CLOSED` state (the failure counter is reset). If any request terminates with a failure, `CircuitBreaker` assumes that the fault is still present, so it returns to the `OPEN` state and restarts the wait time timer (`waitDurationInOpenState`) to give the system additional time to recover from the failure. -The `Half-Open` state helps prevent requests to the service from growing rapidly. Because once a service is started, it may be able to handle a limited number of requests for some time before full recovery. +The `HALF_OPEN` state helps prevent requests to the service from growing rapidly: after recovery starts, the service may be able to handle only a limited number of requests for some time. Initially it has the `CLOSED` state. ### Declarative usage { #declarative-usage } +If `CircuitBreaker` is in the `OPEN` state, the call fails with `CallNotPermittedException`. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -103,17 +110,19 @@ Example of a complete configuration described in the `CircuitBreakerConfig` clas waitDurationInOpenState = "25s" //(4)! permittedCallsInHalfOpenState = 15 //(5)! enabled = true //(6)! + failurePredicateName = "MyPredicate" //(7)! } } } ``` - 1. Limit the number of requests within which `failureRateThreshold` is calculated to determine the state (**required**) - 2. Minimum number of queries required to start state calculation (**required**) - 3. Percentage of unsuccessful requests required to transition to `OPEN` states (has values from *1 to 100*) (**required**) - 4. Waiting time in `OPEN` status, after which the transition to `HALF-OPEN` status is realized (**required**) - 5. Required number of requests in `HALF-OPEN` status that must be successful for transition to `CLOSED` status (**required**) - 6. Enable or disable circuit breaker (default `true`) + 1. Maximum number of requests used to calculate `failureRateThreshold` and determine the state (`required`, default not specified). + 2. Minimum number of requests required to start state calculation (`required`, default not specified). + 3. Percentage of failed requests required to transition to `OPEN`; the value must be from `1` to `100` (`required`, default not specified). + 4. Waiting time in `OPEN`, after which the transition to `HALF_OPEN` is performed (`required`, default not specified). + 5. Number of requests in `HALF_OPEN` that must complete successfully to transition to `CLOSED` (`required`, default not specified). + 6. Enable or disable `CircuitBreaker` (default: `true`). + 7. Exception filter name from `CircuitBreakerPredicate#name()` (all errors are recorded by default). === ":simple-yaml: `YAML`" @@ -128,14 +137,16 @@ Example of a complete configuration described in the `CircuitBreakerConfig` clas waitDurationInOpenState: "25s" #(4)! permittedCallsInHalfOpenState: 15 #(5)! enabled: true #(6)! + failurePredicateName: "MyPredicate" #(7)! ``` - 1. Limit the number of requests within which `failureRateThreshold` is calculated to determine the state (**required**) - 2. Minimum number of queries required to start state calculation (**required**) - 3. Percentage of unsuccessful requests required to transition to `OPEN` states (has values from *1 to 100*) (**required**) - 4. Waiting time in `OPEN` status, after which the transition to `HALF-OPEN` status is realized (**required**) - 5. Required number of requests in `HALF-OPEN` status that must be successful for transition to `CLOSED` status (**required**) - 6. Enable or disable circuit breaker (default `true`) + 1. Maximum number of requests used to calculate `failureRateThreshold` and determine the state (`required`, default not specified). + 2. Minimum number of requests required to start state calculation (`required`, default not specified). + 3. Percentage of failed requests required to transition to `OPEN`; the value must be from `1` to `100` (`required`, default not specified). + 4. Waiting time in `OPEN`, after which the transition to `HALF_OPEN` is performed (`required`, default not specified). + 5. Number of requests in `HALF_OPEN` that must complete successfully to transition to `CLOSED` (`required`, default not specified). + 6. Enable or disable `CircuitBreaker` (default: `true`). + 7. Exception filter name from `CircuitBreakerPredicate#name()` (all errors are recorded by default). An example of overriding named settings for a particular CircuitBreaker: @@ -160,6 +171,17 @@ An example of overriding named settings for a particular CircuitBreaker: waitDurationInOpenState: "50s" ``` +!!! warning "Constraints" + + The following are validated at application startup — violating any of them fails the graph build: + `failureRateThreshold` must be in range `1..100`; `slidingWindowSize` ≥ `1`; `minimumRequiredCalls` ≥ `1` **and** ≤ `slidingWindowSize`; `permittedCallsInHalfOpenState` ≥ `1`. + Either the named or the `default` configuration **must** be present for every `@CircuitBreaker`, otherwise startup fails. + +!!! note + + Setting `enabled = false` turns the aspect into a transparent pass-through — the method is invoked directly with no circuit-breaking. + `failurePredicateName` defaults to `KoraCircuitBreakerPredicate` (records every error); a custom `CircuitBreakerPredicate` can be reused by several breakers by referencing its `name()`. + Module metrics are described in the [Metrics Reference](metrics.md#resilience) section. ### Exception filtering { #exception-filtering } @@ -167,7 +189,7 @@ Module metrics are described in the [Metrics Reference](metrics.md#resilience) s In order to register which errors should be recorded as CircuitBreaker errors, you can override the default filter, you need to implement `CircuitBreakerPredicate` and register your component in the context and specify in the CircuitBreaker configuration its name returned in the `name()` method. -CircuitBreaker register all errors by default. +`CircuitBreaker` records all errors by default. ===! ":fontawesome-brands-java: `Java`" @@ -213,7 +235,7 @@ Configuration: } ``` - 1. Name of the predicate from the `name()` method + 1. Exception filter name from `CircuitBreakerPredicate#name()` (all errors are recorded by default). === ":simple-yaml: `YAML`" @@ -224,12 +246,12 @@ Configuration: failurePredicateName: "MyPredicate" #(1)! ``` - 1. Name of the predicate from the `name()` method + 1. Exception filter name from `CircuitBreakerPredicate#name()` (all errors are recorded by default). ### Imperative usage { #imperative-usage } -You can use a breaker in imperative code, for this you need to implement `CircuitBreakerManager` as a dependency and take `CircuitBreaker` from it by the name of the configuration that would be specified in the annotation. -and take `CircuitBreaker` from it by the name of the configuration that would be specified in the annotation: +You can use a breaker in imperative code: inject `CircuitBreakerManager` +and get `CircuitBreaker` from it by the configuration name that would be specified in the annotation: ===! ":fontawesome-brands-java: `Java`" @@ -271,13 +293,75 @@ and take `CircuitBreaker` from it by the name of the configuration that would be } ``` +To return a fallback value instead of throwing `CallNotPermittedException` when the breaker is `OPEN`, use the `accept` overload that takes a second `Supplier`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public String doWork() { + var circuitBreaker = manager.get("custom"); + return circuitBreaker.accept(this::doSomeWork, () -> "fallback"); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun doWork(): String { + val circuitBreaker = manager["custom"] + return circuitBreaker.accept({ doSomeWork() }, { "fallback" }) + } + ``` + +When the protected call cannot be wrapped in a single `Supplier`, acquire and release the permit manually. +Call `acquire()` (throws `CallNotPermittedException` when the breaker is `OPEN`, or `HALF_OPEN` with no test calls left) to obtain a permit, then **always** report the outcome with `releaseOnSuccess()` or `releaseOnError(Throwable)` — otherwise the breaker leaks a permit and its accounting becomes incorrect: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public String doWork() { + var circuitBreaker = manager.get("custom"); + circuitBreaker.acquire(); // throws CallNotPermittedException when the call is not permitted + try { + var result = doSomeWork(); + circuitBreaker.releaseOnSuccess(); + return result; + } catch (Throwable e) { + circuitBreaker.releaseOnError(e); + throw e; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun doWork(): String { + val circuitBreaker = manager["custom"] + circuitBreaker.acquire() // throws CallNotPermittedException when the call is not permitted + try { + val result = doSomeWork() + circuitBreaker.releaseOnSuccess() + return result + } catch (e: Throwable) { + circuitBreaker.releaseOnError(e) + throw e + } + } + ``` + +`tryAcquire()` is the non-throwing alternative: it returns `false` when the call is not permitted, so you can branch without catching `CallNotPermittedException`. +When `acquire()` does throw, the current breaker [state](#circuitbreaker) (`OPEN` or `HALF_OPEN`) is available via `CallNotPermittedException#state()`. + ## Retry { #retry } -Retry - provides the ability to customize the policy of repeated invocation of annotated methods. -It allows you to specify when you want to retry a method, customize retry parameters in case the method threw an exception that meets the specified filter requirements (*RetryPredicate*). +`Retry` provides the ability to configure repeated invocation of annotated methods. +It allows you to specify when a method should be retried and configure retry parameters when the method throws an exception matching the configured filter (`RetryPredicate`). ### Declarative usage { #declarative-usage-2 } +If all attempts are exhausted, the call fails with `RetryExhaustedException`. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -321,15 +405,17 @@ Example of the complete configuration described in the `RetryConfig` class (defa attempts = 2 //(2)! delayStep = "100ms" //(3)! enabled = true //(4)! + failurePredicateName = "MyPredicate" //(5)! } } } ``` - 1. Initial delay time for the operation at Retry (**required**) - 2. Number of Retry attempts for the operation (**required**) - 3. Delay step which is accumulated in consequence of subsequent Retry attempts - 4. Enable or disable retrier (default `true`) + 1. Initial delay before a repeated call (`required`, default not specified). + 2. Number of retry attempts (`required`, default not specified). + 3. Delay increment for subsequent attempts (default: `0`). + 4. Enable or disable `Retry` (default: `true`). + 5. Exception filter name from `RetryPredicate#name()` (all errors are recorded by default). === ":simple-yaml: `YAML`" @@ -341,18 +427,33 @@ Example of the complete configuration described in the `RetryConfig` class (defa attempts: 2 #(2)! delayStep: "100ms" #(3)! enabled: true #(4)! + failurePredicateName: "MyPredicate" #(5)! ``` - 1. Initial delay time for the operation at Retry (**required**) - 2. Number of Retry attempts for the operation (**required**) - 3. Delay step which is accumulated in consequence of subsequent Retry attempts - 4. Enable or disable retrier (default `true`) + 1. Initial delay before a repeated call (`required`, default not specified). + 2. Number of retry attempts (`required`, default not specified). + 3. Delay increment for subsequent attempts (default: `0`). + 4. Enable or disable `Retry` (default: `true`). + 5. Exception filter name from `RetryPredicate#name()` (all errors are recorded by default). + +!!! warning "Constraints & delay progression" + + `delay` and `attempts` are required (resolved from the named or `default` config) and `attempts` must be `≥ 0`; a missing `delay`/`attempts` or a negative `attempts` fails application startup. + `attempts` counts the retries **after** the initial call, so `attempts = 2` allows up to `3` executions in total. + Each retry waits `delayStep` (default `0`) longer than the previous one, so the delays are `delay`, `delay + delayStep`, `delay + 2·delayStep`, … . + +!!! note + + Setting `enabled = false` turns `@Retry` into a transparent pass-through (the method runs once). + `failurePredicateName` defaults to `KoraRetryPredicate` (retries on every error); a custom `RetryPredicate` can be reused by several retriers by referencing its `name()`. ### Exception filtering { #exception-filtering-2 } In order to register which errors should be recorded as errors on the Retry side, you can override the default filter, it is required to implement `RetryPredicate` and register its component in the context and specify in the Retry configuration its name returned in the `name()` method. +`Retry` records all errors by default. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -397,7 +498,7 @@ Configuration: } ``` - 1. Name of the predicate from the `name()` method + 1. Exception filter name from `RetryPredicate#name()` (all errors are recorded by default). === ":simple-yaml: `YAML`" @@ -408,12 +509,12 @@ Configuration: failurePredicateName: "MyPredicate" #(1)! ``` - 1. Name of the predicate from the `name()` method + 1. Exception filter name from `RetryPredicate#name()` (all errors are recorded by default). ### Imperative usage { #imperative-usage-2 } -It is possible to use a repeater in imperative code, for this you need to implement `RetryManager` as a dependency and take `Retry` from it by the name of the configuration that would be specified in the annotation. -and take `Retry` from it by the name of the configuration that would be specified in the annotation: +You can use a retrier in imperative code: inject `RetryManager` +and get `Retry` from it by the configuration name that would be specified in the annotation: ===! ":fontawesome-brands-java: `Java`" @@ -455,12 +556,86 @@ and take `Retry` from it by the name of the configuration that would be specifie } ``` +To return a fallback value instead of throwing `RetryExhaustedException` once all attempts are used up, pass a second `Supplier`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var retry = manager.get("custom"); + return retry.retry(this::doSomeWork, () -> "fallback"); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val retry = manager["custom"] + return retry.retry({ doSomeWork() }, { "fallback" }) + ``` + +For asynchronous imperative code there is an overload that retries a `Supplier>` and returns a `CompletionStage`, scheduling each attempt after the configured delay without blocking the calling thread. + +#### Manual retry state { #manual-retry-state } + +For full control over the retry loop, use `retry.asState()`, which returns a `RetryState`. +It is `AutoCloseable`, so wrap it in try-with-resources (Java) or `use` (Kotlin) to record metrics on completion. +On each caught exception call `onException(Throwable)`, which returns a `RetryStatus`: + +- `ACCEPTED` — another attempt is allowed; call `doDelay()` (blocks for the current backoff) and retry. +- `REJECTED` — the exception was rejected by the `RetryPredicate` and must not be retried; rethrow it. +- `EXHAUSTED` — all attempts are used up; throw `RetryExhaustedException` (or fall back to a default). + +`getAttempts()` / `getAttemptsMax()` report progress and `getDelayNanos()` returns the next delay. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public String doWork() { + var retry = manager.get("custom"); + try (var state = retry.asState()) { + while (true) { + try { + return doSomeWork(); + } catch (Exception e) { + switch (state.onException(e)) { + case ACCEPTED -> state.doDelay(); // wait, then loop and retry + case REJECTED -> throw e; // not retryable + case EXHAUSTED -> throw new RetryExhaustedException("custom", state.getAttemptsMax(), e); + } + } + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun doWork(): String { + val retry = manager["custom"] + retry.asState().use { state -> + while (true) { + try { + return doSomeWork() + } catch (e: Exception) { + when (state.onException(e)) { + Retry.RetryState.RetryStatus.ACCEPTED -> state.doDelay() // wait, then loop and retry + Retry.RetryState.RetryStatus.REJECTED -> throw e // not retryable + Retry.RetryState.RetryStatus.EXHAUSTED -> throw RetryExhaustedException("custom", state.attemptsMax, e) + } + } + } + } + } + ``` + ## Timeout { #timeout } -Timeout - provides the ability to set the maximum time of operation of the annotated method. +`Timeout` sets the maximum execution time of the annotated method. ### Declarative usage { #declarative-usage-3 } +If the method does not complete within `duration`, the call fails with `TimeoutExhaustedException`. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -517,8 +692,8 @@ Example of the complete configuration described in the `TimeoutConfig` class (de } ``` - 1. The time limit of the operation after which a `TimeoutExhaustedException` will be thrown. - 2. Enable or disable timeouter (default `true`) + 1. Operation time limit after which `TimeoutExhaustedException` will be thrown (`required`, default not specified). + 2. Enable or disable `Timeout` (default: `true`). === ":simple-yaml: `YAML`" @@ -526,17 +701,22 @@ Example of the complete configuration described in the `TimeoutConfig` class (de resilient: timeout: default: - delay: "1s" #(1)! + duration: "1s" #(1)! enabled: true #(2)! ``` - 1. The time limit of the operation after which a `TimeoutExhaustedException` will be thrown. - 2. Enable or disable timeouter (default `true`) + 1. Operation time limit after which `TimeoutExhaustedException` will be thrown (`required`, default not specified). + 2. Enable or disable `Timeout` (default: `true`). + +!!! note + + `duration` is required (resolved from the named or `default` config) and startup fails without it. + Setting `enabled = false` turns `@Timeout` into a transparent pass-through — the method runs with no time limit. ### Imperative usage { #imperative-usage-3 } -You can use a time limiter in imperative code, for this you need to implement `TimeoutManager` as a dependency and take `Timeout` from it by the name of the configuration that would be specified in the annotation. -and take `Timeout` from it by the name of the configuration that would be specified in the annotation: +You can use a time limiter in imperative code: inject `TimeoutManager` +and get `Timeout` from it by the configuration name that would be specified in the annotation: ===! ":fontawesome-brands-java: `Java`" @@ -578,10 +758,29 @@ and take `Timeout` from it by the name of the configuration that would be specif } ``` +`Timeout` also exposes `execute(Runnable)` for operations that return nothing, and `timeout()` returns the configured `Duration`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var timeout = manager.get("custom"); + Duration limit = timeout.timeout(); // configured duration + timeout.execute(() -> { /* do some work */ }); // Runnable variant, throws TimeoutExhaustedException on timeout + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val timeout = manager["custom"] + val limit: Duration = timeout.timeout() // configured duration + timeout.execute(Runnable { /* do some work */ }) // Runnable variant, throws TimeoutExhaustedException on timeout + ``` + ## Fallback { #fallback } -Fallback - provides an opportunity to specify a method that will be called in the case of -if the exception thrown by the annotated method is satisfied by filters (*FallbackPredicate*). +`Fallback` allows you to specify a method that will be called when an exception thrown by the annotated method matches the configured filters (`FallbackPredicate`). + +The fallback method **must match** the return type of the annotated method. ### Declarative usage { #declarative-usage-4 } @@ -613,7 +812,7 @@ An example of a backup method with no arguments: @Fallback(value = "custom", method = "getFallback()") fun value(): String = "value" - fun fallback(): String = "fallback" + fun getFallback(): String = "fallback" } ``` @@ -672,8 +871,8 @@ Example of the complete configuration described in the `FallbackConfig` class (d } ``` - 1. Name of the predicate from the `name()` method - 2. Enable or disable fallback (default `true`) + 1. Exception filter name from `FallbackPredicate#name()` (all errors are recorded by default). + 2. Enable or disable `Fallback` (default: `true`). === ":simple-yaml: `YAML`" @@ -685,14 +884,22 @@ Example of the complete configuration described in the `FallbackConfig` class (d enabled: true #(2)! ``` - 1. Name of the predicate from the `name()` method - 2. Enable or disable fallback (default `true`) + 1. Exception filter name from `FallbackPredicate#name()` (all errors are recorded by default). + 2. Enable or disable `Fallback` (default: `true`). + +!!! note + + Unlike the other aspects, `@Fallback` has no required properties — with no configuration it uses defaults. + Setting `enabled = false` disables the fallback so the original exception propagates. + `failurePredicateName` defaults to `KoraFallbackPredicate` (triggers the fallback for every error); a custom `FallbackPredicate` can be reused by several fallbacks by referencing its `name()`. ### Exception filtering { #exception-filtering-3 } In order to register which errors should be recorded as Fallback errors, you can override the default filter, you need to implement `FallbackPredicate` and register your component in the context and specify in the Fallback configuration its name returned in the `name()` method. +`Fallback` records all errors by default. + ===! ":fontawesome-brands-java: `Java`" ```java @@ -725,8 +932,8 @@ you need to implement `FallbackPredicate` and register your component in the con ### Imperative usage { #imperative-usage-4 } -You can use the fallback method in imperative code, for this you need to implement as a dependency `FallbackManager` -and take `Fallback` from it by the name of the configuration that would be specified in the annotation: +You can use the fallback method in imperative code: inject `FallbackManager` +and get `Fallback` from it by the configuration name that would be specified in the annotation: ===! ":fontawesome-brands-java: `Java`" @@ -768,6 +975,38 @@ and take `Fallback` from it by the name of the configuration that would be speci } ``` +For operations that return nothing, use the `Runnable` overload; and `canFallback(Throwable)` reports whether a given exception would trigger the fallback according to the configured `FallbackPredicate`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var fallback = manager.get("custom"); + + // canFallback tells whether the exception would trigger the fallback + if (fallback.canFallback(exception)) { + // exception matches the configured FallbackPredicate + } + + // Runnable variant for operations that return nothing + fallback.fallback( + () -> { /* primary action */ }, + () -> { /* fallback action */ }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val fallback = manager["custom"] + + // canFallback tells whether the exception would trigger the fallback + if (fallback.canFallback(exception)) { + // exception matches the configured FallbackPredicate + } + + // Runnable variant for operations that return nothing + fallback.fallback(Runnable { /* primary action */ }, Runnable { /* fallback action */ }) + ``` + ## Combination { #combination } It is possible to combine all of the above annotations simultaneously over a single method. @@ -813,10 +1052,12 @@ You can change the order as you wish and combine it with other annotations that In the example above: -1. Applies `@Timeout` which says that the method should not run longer than the time specified in the configuration -2. Applies `@Retry` which will attempt to retry the method the number of times specified in the configuration if the method throws an exception along the chain (including `@Timeout`). -3. Applies `@CircuitBreaker` which will operate according to the configuration and [state](#circuitbreaker) depending on the successful result of the method or if the method threw a chain exception (including `@Timeout` & `@Retry`). -4. Applies `@Fallback` which will call *getFallback* method with argument *arg1* in case the method threw an exception along the chain (including `@Timeout` & `@Retry` & `@CircuitBreaker`) +1. `@Timeout` is applied and checks that the method does not run longer than the time specified in the configuration. +2. `@Retry` is applied and attempts to repeat method execution the configured number of times if the method throws an exception in the chain, including an exception from `@Timeout`. +3. `@CircuitBreaker` is applied and works according to its configuration and [state](#circuitbreaker), depending on the successful method result or an exception in the chain, including exceptions from `@Timeout` and `@Retry`. +4. `@Fallback` is applied and calls the `getFallback` method with the `arg1` argument if the method throws an exception in the chain, including exceptions from `@Timeout`, `@Retry`, and `@CircuitBreaker`. + +Aspect invocation order follows the annotation order on the method: from top to bottom. Example configuration for all aspects: @@ -867,20 +1108,77 @@ Example configuration for all aspects: attempts: 2 ``` +## Exceptions { #exceptions } + +All resilience exceptions extend `ru.tinkoff.kora.resilient.ResilientException` (a `RuntimeException`), which exposes `name()` — the configuration name of the aspect that raised it. + +| Exception | Thrown by | Additional API | +|------------------------------|---------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------| +| `ResilientException` | base type for all of the below | `name()` | +| `CallNotPermittedException` | `@CircuitBreaker` / `CircuitBreaker#acquire()` when the breaker is `OPEN`, or `HALF_OPEN` with no test calls left | `state()` returns the `CircuitBreaker.State` (`OPEN` / `HALF_OPEN`) | +| `RetryExhaustedException` | `@Retry` / `Retry#retry(...)` when every attempt failed | `name()`; message carries the number of attempts, the last failure is the `getCause()` | +| `TimeoutExhaustedException` | `@Timeout` / `Timeout#execute(...)` when the method exceeds `duration` | `name()` | + +**Description** — resilience aspects signal failure by throwing one of these unchecked exceptions out of the protected method. + +**Causes** + +- `CallNotPermittedException` — the circuit breaker is short-circuiting calls because the failure rate reached `failureRateThreshold`; the call was rejected without invoking the method. +- `RetryExhaustedException` — the method kept throwing a retryable exception until `attempts` was reached; the underlying failure is available via `getCause()`. +- `TimeoutExhaustedException` — the method did not complete within `duration`. + +**Recommendations** + +- Catch `ResilientException` to handle any resilience failure uniformly, or catch the concrete type when the handling differs. +- When aspects are [combined](#combination), a downstream aspect's exception propagates up the chain: e.g. a `TimeoutExhaustedException` from `@Timeout` is observed by `@Retry`, then `@CircuitBreaker`, and finally `@Fallback`. Prefer a `@Fallback` method or an imperative fallback over turning these into user-facing errors. + +Handling example: + +===! ":fontawesome-brands-java: `Java`" + + ```java + try { + return service.getValue(); + } catch (CallNotPermittedException e) { + log.warn("CircuitBreaker '{}' is {}", e.name(), e.state()); + return cachedValue(); + } catch (TimeoutExhaustedException | RetryExhaustedException e) { + log.warn("Resilient '{}' failed", e.name(), e); + return cachedValue(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + try { + return service.value() + } catch (e: CallNotPermittedException) { + log.warn("CircuitBreaker '{}' is {}", e.name(), e.state()) + return cachedValue() + } catch (e: ResilientException) { // TimeoutExhaustedException, RetryExhaustedException, ... + log.warn("Resilient '{}' failed", e.name(), e) + return cachedValue() + } + ``` + ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Available method signatures supported by these annotations out of the box: +All four annotations support regular synchronous methods, asynchronous types, and reactive types, but the actual set depends on the language and processor. ===! ":fontawesome-brands-java: `Java`" Class must be non `final` in order for aspects to work. - The `T` refers to the type of the return value, either `Void`. + `T` means the return value type. + - `void myMethod()` - `T myMethod()` - `Optional myMethod()` - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` / `CompletableFuture myMethod()` ([CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html)) + - `Mono myMethod()` ([Project Reactor](https://projectreactor.io/docs/core/release/reference/), requires [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` ([Project Reactor](https://projectreactor.io/docs/core/release/reference/), requires [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" @@ -889,5 +1187,5 @@ Available signatures for repository methods out of the box: By `T` we mean the type of the return value, either `T?`, or `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `suspend myMethod(): T` ([Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine), requires [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `myMethod(): Flow` ([Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine), requires [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) diff --git a/mkdocs/docs/en/documentation/s3-client.md b/mkdocs/docs/en/documentation/s3-client.md index c7b701a..12cc758 100644 --- a/mkdocs/docs/en/documentation/s3-client.md +++ b/mkdocs/docs/en/documentation/s3-client.md @@ -6,17 +6,19 @@ agent: ??? warning "Experimental module" - **Experimental** module is fully working and tested, but requires additional approbation and usage analytics, - for this reason, API may potentially undergo minor changes before fully stable. + **Experimental** module is fully working and tested, but requires additional approbation and usage analytics, + therefore API may potentially undergo minor changes before it becomes fully stable. -Module provides a thin layer of abstraction for creating S3-clients -using declarative-style annotations, or using imperative-style clients to work with [S3 storage](https://aws.amazon.com/s3/faqs/). +The module provides an abstraction layer for working with [S3-compatible object storage](https://aws.amazon.com/s3/faqs/): +you can create declarative `S3` clients using annotations or inject ready-to-use imperative clients. +A declarative client is convenient for typical object and key operations, while an imperative client is useful when operations +need to be controlled directly in code. For a step-by-step walkthrough before the reference details, see [S3](../guides/s3.md). ## AWS { #aws } -S3 client implementation based on the [AWS library](https://github.com/aws/aws-sdk-java-v2). +`S3` client implementation based on the [AWS library](https://github.com/aws/aws-sdk-java-v2). Components available for injection: @@ -57,124 +59,159 @@ Requires any [HTTP client](http-client.md) module to be added. ### Configuration { #configuration } -Complete configuration described in the `AwsS3ClientConfig` and `S3Config` classes (example values or default values are specified): +Basic S3 client configuration parameters: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript s3client { - aws { - addressStyle = "PATH" //(1)! - requestTimeout = "45s" //(2)! - checksumValidationEnabled = false //(3)! - chunkedEncodingEnabled = true //(4)! - upload { - bufferSize = "32MiB" //(5)! - partSize = "8MiB" //(6)! - } - } - - url = "http://localhost:9000" //(7)! - accessKey = "someKey" //(8)! - secretKey = "someSecret" //(9)! - region = "aws-global" //(10)! - telemetry { - logging { - enabled = false //(11)! - } - metrics { - enabled = true //(12)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(13)! - tags = { // (14)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(15)! - attributes = { // (16)! - "key1" = "value1" - "key2" = "value2" - } - } - } + url = "http://localhost:9000" //(1)! + accessKey = "someKey" //(2)! + secretKey = "someSecret" //(3)! + region = "aws-global" //(4)! } ``` - 1. Which type of [file access to use](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), can have values `PATH` or `VIRTUAL_HOSTED` - 2. The maximum execution time of the operation - 3. Whether to check the checksum [MD5 of files before uploading and when retrieving](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) from AWS - 4. Whether to encode in chunks when [signing file data](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#chunkedEncodingEnabled(java.lang.Boolean)) when uploading to AWS - 5. Maximum buffer size when loading files (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 6. Maximum file chunk size when loading a file at a time (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 7. S3 storage URL - 8. S3 access key - 9. S3 access secret - 10. S3 storage region - 11. Enables module logging (default is `false`) - 12. Enables module metrics (default is `true`) - 13. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 14. Configures tags for metrics (optional) - 15. Enables module tracing (default is `true`) - 16. Configures attributes for tracing (optional) + 1. `URL` of the `S3` storage (`required`, no default) + 2. `S3` access key (`required`, no default) + 3. `S3` secret key (`required`, no default) + 4. `S3` storage region (default: `aws-global`) === ":simple-yaml: `YAML`" ```yaml s3client: - aws: - addressStyle: "PATH" #(1)! - requestTimeout: "45s" #(2)! - checksumValidationEnabled: false #(3)! - chunkedEncodingEnabled: true #(4)! - upload: - bufferSize: "32MiB" #(5)! - partSize: "8MiB" #(6)! - - url: "http://localhost:9000" #(7)! - accessKey: "someKey" #(8)! - secretKey: "someSecret" #(9)! - region: "aws-global" #(10)! - telemetry: - logging: - enabled: false #(11)! - metrics: - enabled: true #(12)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! - tags: #(14)! - key1: value1 - key2: value2 - tracing: - enabled: true #(15)! - attributes: #(16)! - key1: value1 - key2: value2 - ``` - - 1. Which type of [file access to use](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), can have values `PATH` or `VIRTUAL_HOSTED` - 2. The maximum execution time of the operation - 3. Whether to check the checksum [MD5 of files before uploading and when retrieving](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) from AWS - 4. Whether to encode in chunks when [signing file data](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#chunkedEncodingEnabled(java.lang.Boolean)) when uploading to AWS - 5. Maximum buffer size when loading files (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 6. Maximum file chunk size when loading a file at a time (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 7. S3 storage URL - 8. S3 access key - 9. S3 access secret - 10. S3 storage region - 11. Enables module logging (default is `false`) - 12. Enables module metrics (default is `true`) - 13. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 14. Configures tags for metrics (optional) - 15. Enables module tracing (default is `true`) - 16. Configures attributes for tracing (optional) + url: "http://localhost:9000" #(1)! + accessKey: "someKey" #(2)! + secretKey: "someSecret" #(3)! + region: "aws-global" #(4)! + ``` + + 1. `URL` of the `S3` storage (`required`, no default) + 2. `S3` access key (`required`, no default) + 3. `S3` secret key (`required`, no default) + 4. `S3` storage region (default: `aws-global`) + +??? note "Full Configuration" + + Complete configuration described in the `AwsS3ClientConfig` and `S3Config` classes (example values or default values are specified): + + ===! ":material-code-json: `HOCON`" + + ```javascript + s3client { + aws { + addressStyle = "PATH" //(1)! + requestTimeout = "45s" //(2)! + checksumValidationEnabled = false //(3)! + chunkedEncodingEnabled = true //(4)! + upload { + bufferSize = "32MiB" //(5)! + partSize = "8MiB" //(6)! + } + } + + url = "http://localhost:9000" //(7)! + accessKey = "someKey" //(8)! + secretKey = "someSecret" //(9)! + region = "aws-global" //(10)! + telemetry { + logging { + enabled = false //(11)! + } + metrics { + enabled = true //(12)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(13)! + tags = { // (14)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(15)! + attributes = { // (16)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Object access style, can have values `PATH` or `VIRTUAL_HOSTED` (default: `PATH`) + 2. Maximum operation execution time (default: `45s`) + 3. Whether to check the [MD5 checksum before upload and on retrieval](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) from `AWS` (default: `false`) + 4. Whether to use chunked encoding when signing file data during upload to `AWS` (default: `true`) + 5. Maximum buffer size for file uploads (default: `32MiB`) + 6. Maximum file part size for a single file upload (default: `8MiB`) + 7. `S3` storage `URL` (`required`, default is not specified) + 8. `S3` access key (`required`, default is not specified) + 9. `S3` access secret (`required`, default is not specified) + 10. `S3` storage region (default: `aws-global`) + 11. Enables module logging (default: `false`) + 12. Enables module metrics (default: `true`) + 13. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Configures metric tags (default: `{}`) + 15. Enables module tracing (default: `true`) + 16. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + s3client: + aws: + addressStyle: "PATH" #(1)! + requestTimeout: "45s" #(2)! + checksumValidationEnabled: false #(3)! + chunkedEncodingEnabled: true #(4)! + upload: + bufferSize: "32MiB" #(5)! + partSize: "8MiB" #(6)! + + url: "http://localhost:9000" #(7)! + accessKey: "someKey" #(8)! + secretKey: "someSecret" #(9)! + region: "aws-global" #(10)! + telemetry: + logging: + enabled: false #(11)! + metrics: + enabled: true #(12)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! + tags: #(14)! + key1: value1 + key2: value2 + tracing: + enabled: true #(15)! + attributes: #(16)! + key1: value1 + key2: value2 + ``` + + 1. Object access style, can have values `PATH` or `VIRTUAL_HOSTED` (default: `PATH`) + 2. Maximum operation execution time (default: `45s`) + 3. Whether to check the [MD5 checksum before upload and on retrieval](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) from `AWS` (default: `false`) + 4. Whether to use chunked encoding when signing file data during upload to `AWS` (default: `true`) + 5. Maximum buffer size for file uploads (default: `32MiB`) + 6. Maximum file part size for a single file upload (default: `8MiB`) + 7. `S3` storage `URL` (`required`, default is not specified) + 8. `S3` access key (`required`, default is not specified) + 9. `S3` access secret (`required`, default is not specified) + 10. `S3` storage region (default: `aws-global`) + 11. Enables module logging (default: `false`) + 12. Enables module metrics (default: `true`) + 13. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Configures metric tags (default: `{}`) + 15. Enables module tracing (default: `true`) + 16. Configures tracing attributes (default: `{}`) Module metrics are described in the [Metrics Reference](metrics.md#s3-client) section. ### Response format { #response-format } -When using AWS module, it is possible to return special response formats specific only to AWS library: +When using the `AWS` module, it is possible to return special response formats specific to the `AWS` library: -| Операция | Формат ответа | +| Operation | Response format | |--------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Get file](#get-file) | [GetObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/GetObjectResponse.html) / [ResponseInputStream](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/ResponseInputStream.html) | | [Get file metadata](#metadata) | [HeadObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/HeadObjectResponse.html) | @@ -182,10 +219,16 @@ When using AWS module, it is possible to return special response formats specifi | [Add file](#add-file) | [PutObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/PutObjectResponse.html) | | [Delete file](#delete-file) | [DeleteObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/DeleteObjectResponse.html) / [DeleteObjectsResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/DeleteObjectsResponse.html) | +For `@S3.Get` operations that retrieve an object or metadata, absence of an object can be described in the response type. +`Java` supports `Optional`, `Optional`, `Optional`, +`Optional>` and `Optional`. +`Kotlin` uses nullable response types for this: `S3Object?`, `S3ObjectMeta?`, `GetObjectResponse?`, +`ResponseInputStream?` and `HeadObjectResponse?`. + ## Minio { #minio } -S3 client implementation based on [Minio](https://github.com/minio/minio-java) library. -Note that the implementation uses [OkHttp](https://github.com/square/okhttp) written in Kotlin and uses appropriate dependencies. +`S3` client implementation based on the [Minio](https://github.com/minio/minio-java) library. +Note that the implementation uses [OkHttp](https://github.com/square/okhttp), written in `Kotlin`, and its dependencies. Available components for injection: @@ -225,104 +268,139 @@ You can add [OkHttp module](http-client.md#okhttp) dependency or a standard HTTP ### Configuration { #configuration-2 } -Complete configuration described in the `MinioS3ClientConfig` and `S3Config` classes (example values or default values are specified): +Basic Minio S3 client configuration parameters: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript s3client { - minio { - addressStyle = "PATH" //(1)! - requestTimeout = "45s" //(2)! - upload { - partSize = "8MiB" //(3)! - } - } - - url = "http://localhost:9000" //(4)! - accessKey = "someKey" //(5)! - secretKey = "someSecret" //(6)! - region = "aws-global" //(7)! - telemetry { - logging { - enabled = false //(8)! - } - metrics { - enabled = true //(9)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(10)! - tags = { // (11)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(12)! - attributes = { // (13)! - "key1" = "value1" - "key2" = "value2" - } - } - } + url = "http://localhost:9000" //(1)! + accessKey = "someKey" //(2)! + secretKey = "someSecret" //(3)! + region = "aws-global" //(4)! } ``` - 1. Which type of [file access to use](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), can have values `PATH` or `VIRTUAL_HOSTED` - 2. Maximum execution time of the operation - 3. Maximum file chunk size for a single file upload (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 4. S3 storage URL - 5. S3 access key - 6. S3 access secret - 7. S3 storage region - 8. Enables module logging (default is `false`) - 9. Enables module metrics (default is `true`) - 10. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 11. Configures tags for metrics (optional) - 12. Enables module tracing (default is `true`) - 13. Configures attributes for tracing (optional) + 1. `URL` of the `S3` storage (`required`, no default) + 2. `S3` access key (`required`, no default) + 3. `S3` secret key (`required`, no default) + 4. `S3` storage region (default: `aws-global`) === ":simple-yaml: `YAML`" ```yaml s3client: - minio: - addressStyle: "PATH" #(1)! - requestTimeout: "45s" #(1)! - upload: - partSize: "8MiB" #(2)! - - url: "http://localhost:9000" #(3)! - accessKey: "someKey" #(4)! - secretKey: "someSecret" #(5)! - region: "aws-global" #(6)! - telemetry: - logging: - enabled: false #(7)! - metrics: - enabled: true #(8)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! - tags: #(10)! - key1: value1 - key2: value2 - tracing: - enabled: true #(11)! - attributes: #(12)! - key1: value1 - key2: value2 - ``` - - 1. Which type of [file access to use](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), can have values `PATH` or `VIRTUAL_HOSTED` - 2. Maximum execution time of the operation - 3. Maximum file chunk size for a single file upload (specified as a number in bytes / or as `4MiB` / `4MB` / `1000Kb` etc.) - 4. S3 storage URL - 5. S3 access key - 6. S3 access secret - 7. S3 storage region - 8. Enables module logging (default is `false`) - 9. Enables module metrics (default is `true`) - 10. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 11. Configures tags for metrics (optional) - 12. Enables module tracing (default is `true`) - 13. Configures attributes for tracing (optional) + url: "http://localhost:9000" #(1)! + accessKey: "someKey" #(2)! + secretKey: "someSecret" #(3)! + region: "aws-global" #(4)! + ``` + + 1. `URL` of the `S3` storage (`required`, no default) + 2. `S3` access key (`required`, no default) + 3. `S3` secret key (`required`, no default) + 4. `S3` storage region (default: `aws-global`) + +??? note "Full Configuration" + + Complete configuration described in the `MinioS3ClientConfig` and `S3Config` classes (example values or default values are specified): + + ===! ":material-code-json: `HOCON`" + + ```javascript + s3client { + minio { + addressStyle = "PATH" //(1)! + requestTimeout = "45s" //(2)! + upload { + partSize = "8MiB" //(3)! + } + } + + url = "http://localhost:9000" //(4)! + accessKey = "someKey" //(5)! + secretKey = "someSecret" //(6)! + region = "aws-global" //(7)! + telemetry { + logging { + enabled = false //(8)! + } + metrics { + enabled = true //(9)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(10)! + tags = { // (11)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(12)! + attributes = { // (13)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Object access style, can have values `PATH` or `VIRTUAL_HOSTED` (default: `PATH`) + 2. Maximum operation execution time (default: `45s`) + 3. Maximum file part size for a single file upload (default: `8MiB`) + 4. `S3` storage `URL` (`required`, default is not specified) + 5. `S3` access key (`required`, default is not specified) + 6. `S3` access secret (`required`, default is not specified) + 7. `S3` storage region (default: `aws-global`) + 8. Enables module logging (default: `false`) + 9. Enables module metrics (default: `true`) + 10. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 11. Configures metric tags (default: `{}`) + 12. Enables module tracing (default: `true`) + 13. Configures tracing attributes (default: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + s3client: + minio: + addressStyle: "PATH" #(1)! + requestTimeout: "45s" #(2)! + upload: + partSize: "8MiB" #(3)! + + url: "http://localhost:9000" #(4)! + accessKey: "someKey" #(5)! + secretKey: "someSecret" #(6)! + region: "aws-global" #(7)! + telemetry: + logging: + enabled: false #(8)! + metrics: + enabled: true #(9)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(10)! + tags: #(11)! + key1: value1 + key2: value2 + tracing: + enabled: true #(12)! + attributes: #(13)! + key1: value1 + key2: value2 + ``` + + 1. Object access style, can have values `PATH` or `VIRTUAL_HOSTED` (default: `PATH`) + 2. Maximum operation execution time (default: `45s`) + 3. Maximum file part size for a single file upload (default: `8MiB`) + 4. `S3` storage `URL` (`required`, default is not specified) + 5. `S3` access key (`required`, default is not specified) + 6. `S3` access secret (`required`, default is not specified) + 7. `S3` storage region (default: `aws-global`) + 8. Enables module logging (default: `false`) + 9. Enables module metrics (default: `true`) + 10. Configures [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 11. Configures metric tags (default: `{}`) + 12. Enables module tracing (default: `true`) + 13. Configures tracing attributes (default: `{}`) ## Client declarative { #client-declarative } @@ -386,9 +464,14 @@ Configuration of a particular implementation of `@S3.Client`: 1. Path to the configuration of this particular client +`@S3.Client` without arguments is equivalent to `@S3.Client("")`: the annotation `value` is empty, +and `S3ClientConfig` will be read from an empty path via `Config.get("")`. +In practice, it is usually better to specify an explicit path, for example `@S3.Client("s3client.someClient")`, +so that the `bucket` configuration is separated from other clients. + Configuration in the case of the `s3client.someClient` path described in the `S3ClientConfig` class: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript s3client { @@ -398,7 +481,7 @@ Configuration in the case of the `s3client.someClient` path described in the `S3 } ``` - 1. Bucket ([bucket](https://aws.amazon.com/ru/s3/faqs/)) where files will be stored + 1. Bucket ([bucket](https://aws.amazon.com/s3/faqs/)) where files will be stored (`required`, default is not specified) === ":simple-yaml: `YAML`" @@ -408,7 +491,7 @@ Configuration in the case of the `s3client.someClient` path described in the `S3 bucket: "someBucket" #(1)! ``` - 1. Bucket ([bucket](https://aws.amazon.com/ru/s3/faqs/)) where files will be stored + 1. Bucket ([bucket](https://aws.amazon.com/s3/faqs/)) where files will be stored (`required`, default is not specified) #### Get file { #get-file } @@ -499,7 +582,7 @@ all method arguments must be part of the compound key. } ``` - 1. template to build the key template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. + 1. Template used to build the key: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` 2. All method arguments must be part of the key template === ":simple-kotlin: `Kotlin`" @@ -513,13 +596,13 @@ all method arguments must be part of the compound key. } ``` - 1. template to build the key template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. + 1. Template used to build the key: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` 2. All method arguments must be part of the key template #### Multiple keys { #multiple-keys } -It is also possible to retrieve multiple files at once by key, either as a complete file along with the `S3Object` data, -or a lighter version as a set of metadata files without `S3ObjectMeta` data. +It is also possible to retrieve multiple files by keys, either as complete objects with data (`S3Object`) +or as lightweight metadata without object data (`S3ObjectMeta`). ===! ":fontawesome-brands-java: `Java`" @@ -532,8 +615,8 @@ or a lighter version as a set of metadata files without `S3ObjectMeta` data. } ``` - 1. Операция получения файла для множества ключей **не должна** содержать шаблон ключа - 2. Операция должна принимать список ключей и отдавать список `S3Object` либо `S3ObjectMeta` + 1. The get operation for multiple keys **must not** contain a key template + 2. The operation must accept a list of keys and return a list of `S3Object` or `S3ObjectMeta` === ":simple-kotlin: `Kotlin`" @@ -546,16 +629,53 @@ or a lighter version as a set of metadata files without `S3ObjectMeta` data. } ``` - 1. The get file operation for multiple keys **must** not** contain a key pattern - 2. the operation must accept a list of keys and give a list of `S3Object` or `S3ObjectMeta`. + 1. The get operation for multiple keys **must not** contain a key template + 2. The operation must accept a list of keys and return a list of `S3Object` or `S3ObjectMeta` + +#### Optional response { #optional-get } + +If absence of a file should not result in `S3NotFoundException`, the `@S3.Get` result can be made optional. +For standard `Kora` types, `Java` uses `Optional` and `Optional`; +the `AWS` module also supports `Optional`, +`Optional>` and `Optional`. +`Kotlin` uses nullable response types for the same cases. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @S3.Client("s3client.someClient") + public interface SomeClient { + + @S3.Get + Optional object(String key); + + @S3.Get + Optional meta(String key); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @S3.Client("s3client.someClient") + interface SomeClient { + + @S3.Get + fun object(key: String): S3Object? + + @S3.Get + fun meta(key: String): S3ObjectMeta? + } + ``` ### List files { #list-files } The section describes the operation to get a list of files/metadata using a declarative S3 client. It is suggested that the `@S3.List` annotation be used to specify the operation. -You can specify [key prefix](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html) to sample the desired keys matching the prefix, -you can also specify a file selection limit, the maximum number of files for the operation is `1000`: +You can specify a [key prefix](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html) to select keys matching that prefix, +and you can also set a file selection limit using the `limit` parameter of `@S3.List`. +The `limit` value must be in the `1..1000` range, and the default is `1000`. ===! ":fontawesome-brands-java: `Java`" @@ -576,7 +696,7 @@ you can also specify a file selection limit, the maximum number of files for the 1. prefix can be passed as a method argument if it is not specified in the annotation 2. prefix can be specified in the annotation - 3. you can specify the file selection limit for the enumeration operation, the maximum number of files for the `1000` operation: + 3. You can specify the file selection limit for the list operation via `limit`; the allowed range is `1..1000`, and the default is `1000` === ":simple-kotlin: `Kotlin`" @@ -597,7 +717,7 @@ you can also specify a file selection limit, the maximum number of files for the 1. prefix can be passed as a method argument if it is not specified in the annotation 2. prefix can be specified in the annotation - 3. you can specify the file selection limit for the enumeration operation, the maximum number of files for the `1000` operation: + 3. You can specify the file selection limit for the list operation via `limit`; the allowed range is `1..1000`, and the default is `1000` #### Metadata { #metadata-2 } @@ -647,7 +767,7 @@ all method arguments must be part of a compound key. } ``` - 1. template to build the prefix template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. + 1. Template used to build the prefix: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` === ":simple-kotlin: `Kotlin`" @@ -660,11 +780,11 @@ all method arguments must be part of a compound key. } ``` - 1. template to build the prefix template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. + 1. Template used to build the prefix: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` #### Separator { #separator } -You can specify a delimiter for [key prefix](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html), to exclude required files from the sample: +You can specify a delimiter for the [key prefix](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html) to filter the list result: ===! ":fontawesome-brands-java: `Java`" @@ -677,7 +797,7 @@ You can specify a delimiter for [key prefix](https://docs.aws.amazon.com/AmazonS } ``` - 1. Указывается разделитель по которому будет фильтроваться перечисления файлов + 1. Delimiter used to filter file listing === ":simple-kotlin: `Kotlin`" @@ -690,7 +810,7 @@ You can specify a delimiter for [key prefix](https://docs.aws.amazon.com/AmazonS } ``` - 1. Specifies the delimiter by which the file enumeration will be filtered + 1. Delimiter used to filter file listing ### Add file { #add-file } @@ -738,11 +858,152 @@ It is required to specify the key and body of the file to be added: #### File body { #file-body } -File body (`S3Body`) can be created from `byte[]` / `ByteBuffer` / `InputStream` / `Flow.Publisher` via the corresponding static constructor methods. +File body (`S3Body`) can be created from `byte[]`, `ByteBuffer`, `InputStream` or `Flow.Publisher` +using the corresponding static factory methods. Every factory has overloads that additionally accept the +`type` (`Content-Type`) and `encoding` (`Content-Encoding`) values: + +| Factory method | Source | Size | Description | +|-----------------------------------------------|------------------------------|------------|--------------------------------------------------------------------------------------------------------| +| `S3Body.ofBytes(byte[])` | `byte[]` | Known | Body from an in-memory byte array | +| `S3Body.ofBuffer(ByteBuffer)` | `ByteBuffer` | Known | Body from an in-memory buffer (uses `remaining()` as the size) | +| `S3Body.ofInputStream(InputStream, long)` | `InputStream` | Known | Streaming body whose exact length is passed explicitly as the `size` argument | +| `S3Body.ofInputStreamReadAll(InputStream)` | `InputStream` | Known | Reads the whole stream into memory **immediately**, then behaves like a byte array | +| `S3Body.ofInputStreamUnbound(InputStream)` | `InputStream` | Unknown | Streaming body of unknown length (`size()` returns `-1`) | +| `S3Body.ofPublisher(Flow.Publisher)` | `Flow.Publisher` | Unknown | Reactive streaming body of unknown length (`size()` returns `-1`) | +| `S3Body.ofPublisher(Flow.Publisher, long)` | `Flow.Publisher` | Known | Reactive streaming body whose length is passed explicitly as the `size` argument | + +The body itself exposes the following accessors: + +| Method | Description | +|---------------------------------------------|-----------------------------------------------------------------------------------| +| `byte[] asBytes()` | Reads the entire body into a byte array (drains the underlying stream) | +| `InputStream asInputStream()` | Returns the body as a blocking `InputStream` | +| `Flow.Publisher asPublisher()` | Returns the body as a reactive `Flow.Publisher` | +| `long size()` | Content length in bytes, or `-1` if unknown (unbound stream / publisher) | +| `String type()` | `Content-Type` of the body | +| `String encoding()` | `Content-Encoding` of the body | + +If the file is very large or its length is unknown and streaming is required, it is recommended to create the body using +`S3Body.ofPublisher(...)` or `S3Body.ofInputStreamUnbound(...)`. + +If no file type is specified, `application/octet-stream` will be used. +For `@S3.Put`, the body can also be passed directly as `byte[]` or `ByteBuffer`; in that case the client creates `S3Body` itself. +The `@S3.Put` annotation allows specifying `type` and `encoding`, which will be written as `Content-Type` and `Content-Encoding`. + +An `HTTP` server can stream a request body into `S3` without reading the whole file into memory first. +To do this, accept the request body as `Flow.Publisher` and pass it to `S3Body.ofPublisher(...)`. +If the body size is known, for example from the `Content-Length` header, it is better to pass that size to `S3Body`; +if the size is unknown, use an overload without size and the size will be considered unknown. -If the file is very large or its length is unknown and streaming is required, it is recommended to create the body using `S3Body.ofPublisher()` or `S3Body.ofInputStreamUnbound()`. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController + public final class UploadController { + + private final S3KoraClient s3; + + public UploadController(S3KoraClient s3) { + this.s3 = s3; + } + + @HttpRoute(method = HttpMethod.PUT, path = "/files/{key}") + public HttpServerResponse upload(@Path String key, + @Header("Content-Type") @Nullable String contentType, + @Header("Content-Length") @Nullable Long contentLength, + Flow.Publisher body) { + var type = contentType == null ? "application/octet-stream" : contentType; + var s3Body = contentLength == null + ? S3Body.ofPublisher(body, type) + : S3Body.ofPublisher(body, contentLength, type); + + this.s3.put("documents", key, s3Body); + return HttpServerResponse.of(201); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController + class UploadController( + private val s3: S3KoraClient + ) { + + @HttpRoute(method = HttpMethod.PUT, path = "/files/{key}") + fun upload( + @Path key: String, + @Header("Content-Type") contentType: String?, + @Header("Content-Length") contentLength: Long?, + body: Flow.Publisher + ): HttpServerResponse { + val type = contentType ?: "application/octet-stream" + val s3Body = if (contentLength == null) { + S3Body.ofPublisher(body, type) + } else { + S3Body.ofPublisher(body, contentLength, type) + } + + s3.put("documents", key, s3Body) + return HttpServerResponse.of(201) + } + } + ``` -If no file type is specified, it will be set as `application/octet-stream`. +In this variant, `Kora` obtains `Flow.Publisher` from the `HTTP` request body through the standard +`HttpServerRequestMapper`, and the `S3` client reads the same stream during upload. The handler does not need to call +`asBytes()`, `asInputStream().readAllBytes()` or `S3Body.ofInputStreamReadAll(...)` if the goal is not to keep the whole file in memory. + +#### Content type and encoding { #content-type } + +Instead of constructing an `S3Body` yourself, you can pass the body directly as `byte[]` or `ByteBuffer` and let the client +wrap it into an `S3Body`. In that case the `type` (`Content-Type`) and `encoding` (`Content-Encoding`) attributes of `@S3.Put` +are used to build the body: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @S3.Client("s3client.someClient") + public interface SomeClient { + + @S3.Put(value = "some-key", type = "image/jpeg", encoding = "gzip") //(1)! + void operation1(byte[] body); //(2)! + + @S3.Put("some-key") + void operation2(ByteBuffer body); //(3)! + } + ``` + + 1. `type` maps to `Content-Type` and `encoding` maps to `Content-Encoding` + 2. When the body is `byte[]` or `ByteBuffer`, the client builds the `S3Body` itself using the annotation's `type`/`encoding` + 3. If neither `type` nor `encoding` is set, `application/octet-stream` is used as the `Content-Type` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @S3.Client("s3client.someClient") + interface SomeClient { + + @S3.Put(value = "some-key", type = "image/jpeg", encoding = "gzip") //(1)! + fun operation1(body: ByteArray) //(2)! + + @S3.Put("some-key") + fun operation2(body: ByteBuffer) //(3)! + } + ``` + + 1. `type` maps to `Content-Type` and `encoding` maps to `Content-Encoding` + 2. When the body is `ByteArray` or `ByteBuffer`, the client builds the `S3Body` itself using the annotation's `type`/`encoding` + 3. If neither `type` nor `encoding` is set, `application/octet-stream` is used as the `Content-Type` + +!!! warning "Body type" + + The body of an `@S3.Put` operation must be `S3Body`, `byte[]` or `ByteBuffer`, otherwise a compilation error occurs. + The `type` and `encoding` attributes only apply to raw `byte[]`/`ByteBuffer` bodies; when you pass a ready `S3Body`, + its own `type()`/`encoding()` values are used and the annotation attributes are ignored. #### Key template { #key-template-2 } @@ -760,8 +1021,8 @@ all method arguments must be part of a compound key. } ``` - 1. template to build the key template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. - 2. All method arguments must be part of the key template either `S3Body` + 1. Template used to build the key: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` + 2. All method arguments must be part of the key template or be `S3Body` === ":simple-kotlin: `Kotlin`" @@ -774,8 +1035,8 @@ all method arguments must be part of a compound key. } ``` - 1. template to build the key template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. - 2. All method arguments must be part of the key template either `S3Body` + 1. Template used to build the key: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` + 2. All method arguments must be part of the key template or be `S3Body` ### Delete file { #delete-file } @@ -797,7 +1058,7 @@ It is suggested to use the `@S3.Delete` annotation for the operation. ``` 1. file deletion operation - 2. Receiving a file with data in response + 2. File key to delete 3. The key of the file can be specified in the annotation === ":simple-kotlin: `Kotlin`" @@ -815,7 +1076,7 @@ It is suggested to use the `@S3.Delete` annotation for the operation. ``` 1. file deletion operation - 2. Receiving a file with data in response + 2. File key to delete 3. The key of the file can be specified in the annotation #### Key template { #key-template-3 } @@ -834,7 +1095,7 @@ all method arguments must be part of the composite key. } ``` - 1. template to build the key template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. + 1. Template used to build the key: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` 2. All method arguments must be part of the key template === ":simple-kotlin: `Kotlin`" @@ -848,13 +1109,12 @@ all method arguments must be part of the composite key. } ``` - 1. template to build the key template, each template argument will be substituted via `toString()`, the arguments in the template are specified as method argument names in `{covens}`. + 1. Template used to build the key: each template argument is substituted via `toString()`, and template arguments are specified as method argument names in `{curly braces}` 2. All method arguments must be part of the key template #### Multiple keys { #multiple-keys-2 } -It is also possible to retrieve multiple files at once by key, either as a complete file along with the `S3Object` data, -or a lighter version as a set of metadata files without `S3ObjectMeta` data. +It is also possible to delete multiple files by keys. ===! ":fontawesome-brands-java: `Java`" @@ -867,8 +1127,8 @@ or a lighter version as a set of metadata files without `S3ObjectMeta` data. } ``` - 1. a get file operation for multiple keys **must** not** contain a key pattern - 2. the operation must accept a list of keys and return `void`. + 1. The delete operation for multiple keys **must not** contain a key template + 2. The operation must accept a list of keys and return `void` === ":simple-kotlin: `Kotlin`" @@ -881,12 +1141,12 @@ or a lighter version as a set of metadata files without `S3ObjectMeta` data. } ``` - 1. a get file operation for multiple keys **must** not** contain a key pattern - 2. the operation must accept a list of keys and return `void`. + 1. The delete operation for multiple keys **must not** contain a key template + 2. The operation must accept a list of keys and return `void` ### Signatures { #signatures } -Available signatures for repository methods out of the box: +Available signatures for declarative `S3` client methods out of the box: ===! ":fontawesome-brands-java: `Java`" @@ -894,6 +1154,7 @@ Available signatures for repository methods out of the box: - `T myMethod()` - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) + - `CompletableFuture myMethod()` [CompletableFuture](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html) - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" @@ -903,17 +1164,474 @@ Available signatures for repository methods out of the box: - `myMethod(): T` - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) +## Models { #models } + +Both declarative and imperative clients return the same set of model types (unless the `AWS` module's +[native response format](#response-format) is used). All models are read-only interfaces. + +### S3Object { #model-s3-object } + +Full object together with its data, returned by [get](#get-file) operations and available inside [S3ObjectList](#model-s3-object-list): + +| Method | Description | +|---------------------|-----------------------------------------------------------------| +| `String key()` | Object key | +| `Instant modified()`| Last modification time | +| `long size()` | Object size in bytes | +| `S3Body body()` | Object [body](#file-body) with the data | + +### S3ObjectMeta { #model-s3-object-meta } + +Lightweight metadata without the object data, returned by metadata [get](#metadata) operations and available inside +[S3ObjectMetaList](#model-s3-object-meta-list). Retrieving metadata is faster because the object body is not transferred: + +| Method | Description | +|----------------------|---------------------------------| +| `String key()` | Object key | +| `Instant modified()` | Last modification time | +| `long size()` | Object size in bytes | + +### S3ObjectList { #model-s3-object-list } + +List of full objects returned by [list](#list-files) operations. Extends `S3ObjectMetaList`, so it also exposes the prefix and metadata: + +| Method | Description | +|-------------------------------|---------------------------------------------------| +| `String prefix()` | Prefix used for the listing | +| `List objects()` | Objects that matched the prefix (with data) | +| `List metas()` | Metadata of the objects that matched the prefix | + +### S3ObjectMetaList { #model-s3-object-meta-list } + +List of metadata returned by metadata [list](#metadata-2) operations: + +| Method | Description | +|-------------------------------|-------------------------------------------------| +| `String prefix()` | Prefix used for the listing | +| `List metas()` | Metadata of the objects that matched the prefix | + +### S3ObjectUpload { #model-s3-object-upload } + +Result of an [add file](#add-file) operation: + +| Method | Description | +|-----------------------|-----------------------------------------------------------------------------| +| `String versionId()` | Version identifier of the uploaded object (if bucket versioning is enabled) | + ## Client imperative { #client-imperative } -It is possible to implement an imperative Kora client to work with S3, both synchronous and asynchronous clients are provided: +It is possible to inject an imperative `Kora` client to work with `S3`; both synchronous and asynchronous clients are provided: - `S3KoraClient` - client for synchronous operation -- `S3KoraAsyncClient` - client for asynchronous operation. +- `S3KoraAsyncClient` - client for asynchronous operation + +Both clients work with explicit `bucket` and `key` parameters and support retrieving objects or metadata, listing objects by prefix, +uploading `S3Body`, and deleting one or more objects. Unlike the declarative client, they are not tied to a single `bucket` from +configuration — the `bucket` is passed to each method explicitly. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final S3KoraClient s3; + + public SomeService(S3KoraClient s3) { + this.s3 = s3; + } + + public byte[] download(String bucket, String key) { + S3Object object = s3.get(bucket, key); //(1)! + return object.body().asBytes(); + } + } + ``` + + 1. Throws `S3NotFoundException` if the object is missing + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val s3: S3KoraClient + ) { + + fun download(bucket: String, key: String): ByteArray { + val obj = s3.get(bucket, key) //(1)! + return obj.body().asBytes() + } + } + ``` + + 1. Throws `S3NotFoundException` if the object is missing + +### Synchronous client { #client-imperative-sync } + +The `S3KoraClient` interface provides the following operations: + +| Method | Description | +|---------------------------------------------------------------------------------------|-------------------------------------------------------------------| +| `S3Object get(bucket, key)` | Get a single object with data | +| `S3ObjectMeta getMeta(bucket, key)` | Get metadata of a single object | +| `List get(bucket, Collection keys)` | Get multiple objects with data | +| `List getMeta(bucket, Collection keys)` | Get metadata of multiple objects | +| `S3ObjectList list(bucket[, prefix[, delimiter, limit]])` | List objects by prefix (with data) | +| `S3ObjectMetaList listMeta(bucket[, prefix[, delimiter, limit]])` | List object metadata by prefix | +| `List list(bucket, Collection prefixes[, delimiter, limit])` | List objects for several prefixes at once | +| `List listMeta(bucket, Collection prefixes[, delimiter, limit])` | List object metadata for several prefixes at once | +| `S3ObjectUpload put(bucket, key, S3Body body)` | Add an object and return the upload result | +| `void delete(bucket, key)` | Delete a single object | +| `void delete(bucket, Collection keys)` | Delete multiple objects (throws `S3DeleteException` on failure) | + +The `list`/`listMeta` overloads without `delimiter`/`limit` default `delimiter` to `null` and `limit` to `1000`. +The `limit` argument must be in the `1..1000` range. + +===! ":fontawesome-brands-java: `Java`" + + ```java + // get a single object and its metadata + S3Object object = s3.get("documents", "report.pdf"); + S3ObjectMeta meta = s3.getMeta("documents", "report.pdf"); + + // get several objects at once + List objects = s3.get("documents", List.of("a.pdf", "b.pdf")); + + // list by prefix with a delimiter and a limit + S3ObjectList list = s3.list("documents", "2024/", "/", 100); + for (S3Object o : list.objects()) { + // ... + } + + // list several prefixes at once + List perPrefix = s3.listMeta("documents", List.of("2023/", "2024/")); + + // add an object + S3ObjectUpload upload = s3.put("documents", "report.pdf", S3Body.ofBytes(bytes)); + String versionId = upload.versionId(); + + // delete a single object and a batch of objects + s3.delete("documents", "report.pdf"); + s3.delete("documents", List.of("a.pdf", "b.pdf")); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // get a single object and its metadata + val obj = s3.get("documents", "report.pdf") + val meta = s3.getMeta("documents", "report.pdf") + + // get several objects at once + val objects = s3.get("documents", listOf("a.pdf", "b.pdf")) + + // list by prefix with a delimiter and a limit + val list = s3.list("documents", "2024/", "/", 100) + for (o in list.objects()) { + // ... + } + + // list several prefixes at once + val perPrefix = s3.listMeta("documents", listOf("2023/", "2024/")) + + // add an object + val upload = s3.put("documents", "report.pdf", S3Body.ofBytes(bytes)) + val versionId = upload.versionId() + + // delete a single object and a batch of objects + s3.delete("documents", "report.pdf") + s3.delete("documents", listOf("a.pdf", "b.pdf")) + ``` + +### Asynchronous client { #client-imperative-async } + +The `S3KoraAsyncClient` interface mirrors `S3KoraClient` method-for-method, but every operation returns a +[CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) +(`CompletionStage` for delete operations): + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final S3KoraAsyncClient s3; + + public SomeService(S3KoraAsyncClient s3) { + this.s3 = s3; + } + + public CompletionStage download(String bucket, String key) { + return s3.get(bucket, key) + .thenApply(object -> object.body().asBytes()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val s3: S3KoraAsyncClient + ) { + + fun download(bucket: String, key: String): CompletionStage { + return s3.get(bucket, key) + .thenApply { it.body().asBytes() } + } + } + ``` + +## Native clients { #native-clients } + +Besides the declarative and imperative `Kora` clients, the underlying native `SDK` clients are also available for injection. +They are useful for advanced operations that are not covered by the declarative/imperative API (for example, bucket management, +object copying, presigned URLs, and so on). + +The [AWS module](#aws) provides: + +- `S3Client` — synchronous `AWS` client +- `S3AsyncClient` — asynchronous `AWS` client +- `S3AsyncClient` with `@Tag(MultipartUpload.class)` — asynchronous `AWS` client preconfigured for [multipart uploads](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/internal/multipart/MultipartS3AsyncClient.html) according to `upload.partSize` and `upload.bufferSize` + +The [Minio module](#minio) provides: + +- `MinioClient` — synchronous `Minio` client +- `MinioAsyncClient` — asynchronous `Minio` client + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class BucketService { + + private final S3Client s3Client; //(1)! + private final S3AsyncClient multipartClient; + + public BucketService(S3Client s3Client, + @Tag(MultipartUpload.class) S3AsyncClient multipartClient) { //(2)! + this.s3Client = s3Client; + this.multipartClient = multipartClient; + } + + public void ensureBucket(String bucket) { + s3Client.createBucket(b -> b.bucket(bucket)); + } + } + ``` + + 1. Native `AWS` `S3Client` injected directly + 2. Asynchronous client tagged with `@Tag(MultipartUpload.class)` for multipart uploads + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class BucketService( + private val s3Client: S3Client, //(1)! + @Tag(MultipartUpload::class) private val multipartClient: S3AsyncClient //(2)! + ) { + + fun ensureBucket(bucket: String) { + s3Client.createBucket { it.bucket(bucket) } + } + } + ``` + + 1. Native `AWS` `S3Client` injected directly + 2. Asynchronous client tagged with `@Tag(MultipartUpload::class)` for multipart uploads ## Exceptions { #exceptions } -Special errors will be thrown if a client operation error occurs: +If a client operation fails, one of the `S3` exceptions is thrown. All of them inherit from the base `S3Exception`, +which itself extends `RuntimeException`, so handling is optional and unchecked. + +**Exception hierarchy:** + +``` +RuntimeException +└── S3Exception + ├── S3NotFoundException + └── S3DeleteException +``` + +The base `S3Exception` exposes the error code and message reported by the storage: + +| Method | Description | +|-----------------------------|------------------------------------------------------| +| `String getErrorCode()` | Storage error code (for example, `NoSuchKey`) | +| `String getErrorMessage()` | Storage error message | -- `S3NotFoundException` - in case it does not find a file by the specified key -- `S3DeleteException` - in case of file deletion error -- `S3Exception` - in any other case +**Handling example:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final S3KoraClient s3; + + public SomeService(S3KoraClient s3) { + this.s3 = s3; + } + + public void call(String bucket) { + try { + s3.delete(bucket, List.of("a.pdf", "b.pdf")); + } catch (S3NotFoundException e) { + // Object or bucket is missing: getErrorCode() is NoSuchKey or NoSuchBucket + } catch (S3DeleteException e) { + // One or more objects were not deleted + for (S3DeleteException.Error error : e.getErrors()) { + // error.key(), error.bucket(), error.code(), error.message() + } + } catch (S3Exception e) { + // Any other storage error: getErrorCode(), getErrorMessage() + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val s3: S3KoraClient + ) { + + fun call(bucket: String) { + try { + s3.delete(bucket, listOf("a.pdf", "b.pdf")) + } catch (e: S3NotFoundException) { + // Object or bucket is missing: errorCode is NoSuchKey or NoSuchBucket + } catch (e: S3DeleteException) { + // One or more objects were not deleted + for (error in e.errors) { + // error.key(), error.bucket(), error.code(), error.message() + } + } catch (e: S3Exception) { + // Any other storage error: errorCode, errorMessage + } + } + } + ``` + +### S3NotFoundException { #not-found-exception } + +Thrown when a requested object or bucket does not exist. + +**Causes:** + +- Object key does not exist (`getErrorCode()` returns `NoSuchKey`) +- Bucket does not exist (`getErrorCode()` returns `NoSuchBucket`) + +**Recommendations:** + +- Make the `@S3.Get` result [optional](#optional-get) (`Optional`/nullable) if absence of an object is a normal outcome +- Verify the `bucket` from configuration and the requested `key` + +### S3DeleteException { #delete-exception } + +Thrown by batch `delete(bucket, keys)` operations when one or more objects could not be deleted. +It exposes the list of individual failures: + +| Method | Description | +|----------------------|----------------------------------------------------------------| +| `List getErrors()` | Per-object failures, each with `key()`, `bucket()`, `code()`, `message()` | + +**Recommendations:** + +- Inspect `getErrors()` to determine which objects failed and why +- Retry the failed keys separately if the failure is transient + +### S3Exception { #base-exception } + +Base exception thrown for any other storage or client error that is not a missing object or a batch-delete failure. + +**Recommendations:** + +- Log `getErrorCode()` and `getErrorMessage()` for diagnostics +- Enable client [logging](#configuration) at `DEBUG` level to inspect the underlying request/response + +## Testing { #testing } + +Declarative and imperative `S3` clients can be tested with [@KoraAppTest](junit5.md) together with a real +`S3`-compatible storage started in a [Testcontainers](https://java.testcontainers.org/) container (for example, `Minio`). +The storage connection parameters are supplied to the application config via system properties: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @TestcontainersMinio( + mode = ContainerMode.PER_RUN, + bucket = @Bucket(value = SomeClientTests.BUCKET, create = Bucket.Mode.PER_METHOD, drop = Bucket.Mode.PER_METHOD)) + @KoraAppTest(Application.class) + class SomeClientTests implements KoraAppTestConfigModifier { + + static final String BUCKET = "simple"; + + @ConnectionMinio + private MinioConnection minioConnection; + + @TestComponent + private SomeClient client; + + @Override + public KoraConfigModification config() { + return KoraConfigModification + .ofSystemProperty("S3_URL", minioConnection.params().uri().toString()) + .withSystemProperty("S3_ACCESS_KEY", minioConnection.params().accessKey()) + .withSystemProperty("S3_SECRET_KEY", minioConnection.params().secretKey()) + .withSystemProperty("S3_BUCKET", BUCKET); + } + + @Test + void putAndGet() { + var value = "value".getBytes(StandardCharsets.UTF_8); + client.putObject("k1", S3Body.ofBytes(value)); + + var found = client.getObject("k1"); + assertArrayEquals(value, found.body().asBytes()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @TestcontainersMinio( + mode = ContainerMode.PER_RUN, + bucket = Bucket(value = [BUCKET], create = Bucket.Mode.PER_METHOD, drop = Bucket.Mode.PER_METHOD)) + @KoraAppTest(Application::class) + class SomeClientTests : KoraAppTestConfigModifier { + + @ConnectionMinio + lateinit var minioConnection: MinioConnection + + @TestComponent + lateinit var client: SomeClient + + override fun config(): KoraConfigModification = KoraConfigModification + .ofSystemProperty("S3_URL", minioConnection.params().uri().toString()) + .withSystemProperty("S3_ACCESS_KEY", minioConnection.params().accessKey()) + .withSystemProperty("S3_SECRET_KEY", minioConnection.params().secretKey()) + .withSystemProperty("S3_BUCKET", BUCKET) + + @Test + fun putAndGet() { + val value = "value".toByteArray() + client.putObject("k1", S3Body.ofBytes(value)) + + val found = client.getObject("k1") + assertArrayEquals(value, found.body().asBytes()) + } + + companion object { + const val BUCKET = "simple" + } + } + ``` diff --git a/mkdocs/docs/en/documentation/scheduling.md b/mkdocs/docs/en/documentation/scheduling.md index 95031a6..1dd3ff8 100644 --- a/mkdocs/docs/en/documentation/scheduling.md +++ b/mkdocs/docs/en/documentation/scheduling.md @@ -4,24 +4,35 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora scheduling for native and Quartz schedulers, fixed rate, fixed delay, one-shot and cron jobs, triggers, shutdown, and concurrency controls; key triggers include @ScheduleAtFixedRate, @ScheduleWithFixedDelay, @ScheduleOnce, @ScheduleWithCron, @ScheduleWithTrigger, @DisallowConcurrentExecution, SchedulingModule, QuartzModule." --- -A module for creating declarative-style planners using annotations. +The Kora scheduling module allows application methods to run on a schedule in a declarative style through annotations. +At compile time, Kora generates task components and connects them to the selected scheduling mechanism. -===! ":fontawesome-brands-java: `Java`" +Two options are available: the native scheduler based on `ScheduledExecutorService` from the `JDK`, and the scheduler based on `Quartz`. +The native option is suitable for simple periodic tasks inside one application, while `Quartz` is useful for `cron` expressions, custom `Trigger` instances, and additional task execution rules. - When applying aspects, class must not be `final` +## Native Scheduler { #native } -=== ":simple-kotlin: `Kotlin`" +The native scheduler uses the standard [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) that comes with the `JDK`. + +Special annotations are used to create tasks through aspects, and they correspond to `ScheduledExecutorService` methods. +Annotation parameters match the parameters of the `scheduleAtFixedRate`, `scheduleWithFixedDelay`, and `schedule` methods. - When applying aspects, class must be `open` +All annotations have the `config` parameter. +If it is specified, parameter values are taken from the configuration at that path and have priority over annotation values. +The configuration of a specific task can also contain the `telemetry` section; its values override the common scheduler telemetry for that task. -## Native { #native } +Scheduled methods must satisfy the following requirements: -Creating a scheduler using the standard [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) that comes with the JVM. +- The enclosing class must be a component in the [dependency graph](container.md), for example annotated with `@Component`. +- The native scheduler method must have no arguments (the `Quartz` scheduler additionally allows an optional [JobExecutionContext](#job-context) argument). +- The method return value is ignored. +- In `Kotlin` the method must not be a `suspend` function. -In order to create a scheduler via aspects, special annotations are used that essentially duplicate the `ScheduledExecutorService` method signatures. -The parameters of the annotations correspond to the parameters of the methods `scheduleAtFixedRate`, `schedule`, `scheduleWithFixedDelay` respectively. +!!! warning "Interval is required" -Also all annotations have the `config` argument, if it is present, the parameter values will be taken from the configuration on the specified path. + `@ScheduleAtFixedRate` requires `period` and `@ScheduleWithFixedDelay` requires `delay`. + If neither the annotation attribute (its default is `0`) nor a `config` path providing the value is set, + compilation fails with `Either period() or config() annotation parameter must be provided`. ### Dependency { #dependency } @@ -41,7 +52,7 @@ Also all annotations have the `config` argument, if it is present, the parameter === ":simple-kotlin: `Kotlin`" [Dependency](general.md#dependencies) `build.gradle.kts`: - ```groovy + ```kotlin implementation("ru.tinkoff.kora:scheduling-jdk") ``` @@ -53,7 +64,7 @@ Also all annotations have the `config` argument, if it is present, the parameter ### Configuration { #configuration } -Example of the complete configuration described in the `ScheduledExecutorServiceConfig` class (default values are specified): +Complete configuration example described by the `ScheduledExecutorServiceConfig` class with default values: ===! ":material-code-json: `Hocon`" @@ -84,14 +95,14 @@ Example of the complete configuration described in the `ScheduledExecutorService } ``` - 1. Maximum number of threads in [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) - 2. Time to wait for jobs to complete before shutting down the scheduler in case of [graceful shutdown](container.md#graceful-shutdown) - 3. Enables module logging (default `false`) - 4. Enables module metrics (default `true`) - 5. Configuring [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 6. Configures tags for metrics (optional) - 7. Enables module tracing (default `true`) - 8. Configures attributes for tracing (optional) + 1. Maximum number of threads in [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) (default: `2`) + 2. Time to wait for tasks to complete before scheduler shutdown during [graceful shutdown](container.md#component-lifecycle) (default: `30s`) + 3. Enables module logging (default: `false`) + 4. Enables module metrics (default: `true`) + 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Configures metric tags (default: `{}`) + 7. Enables module tracing (default: `true`) + 8. Configures tracing attributes (default: `{}`) === ":simple-yaml: `YAML`" @@ -115,24 +126,66 @@ Example of the complete configuration described in the `ScheduledExecutorService key2: value2 ``` - 1. Maximum number of threads in [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) - 2. Time to wait for jobs to complete before shutting down the scheduler in case of [graceful shutdown](container.md#graceful-shutdown) - 3. Enables module logging (default `false`) - 4. Enables module metrics (default `true`) - 5. Configuring [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 6. Configures tags for metrics (optional) - 7. Enables module tracing (default `true`) - 8. Configures attributes for tracing (optional) + 1. Maximum number of threads in [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) (default: `2`) + 2. Time to wait for tasks to complete before scheduler shutdown during [graceful shutdown](container.md#component-lifecycle) (default: `30s`) + 3. Enables module logging (default: `false`) + 4. Enables module metrics (default: `true`) + 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Configures metric tags (default: `{}`) + 7. Enables module tracing (default: `true`) + 8. Configures tracing attributes (default: `{}`) Module metrics are described in the [Metrics Reference](metrics.md#scheduling) section. -### Fixed rate { #fixed-rate } +A specific task configuration may also contain its own `telemetry` section, which overrides the scheduler-wide `scheduling.telemetry` for that task only. +Unset values fall back to the common configuration, so it is enough to specify only what should differ: -Scheduling with tasks running at fixed time intervals, regardless of whether the previous task has completed or not -This can lead to simultaneous execution of several tasks. +===! ":material-code-json: `Hocon`" + + ```javascript + scheduling { + jobs { + fix-rate { + period = "50ms" + telemetry { + logging.enabled = true //(1)! + metrics.enabled = false //(2)! + } + } + } + } + ``` + + 1. Overrides `scheduling.telemetry.logging.enabled` for this task only + 2. Overrides `scheduling.telemetry.metrics.enabled` for this task only + +=== ":simple-yaml: `YAML`" + + ```yaml + scheduling: + jobs: + fix-rate: + period: "50ms" + telemetry: + logging: + enabled: true #(1)! + metrics: + enabled: false #(2)! + ``` -For example, if the period is set to 10 seconds, and each task execution takes 5 seconds, -next task execution will start 5 seconds after the previous one completed. + 1. Overrides `scheduling.telemetry.logging.enabled` for this task only + 2. Overrides `scheduling.telemetry.metrics.enabled` for this task only + +Observability of scheduled tasks can also be customized in code by registering a component that implements +`SchedulingLoggerFactory`, `SchedulingMetricsFactory`, `SchedulingTracerFactory`, or the whole `SchedulingTelemetryFactory`. + +### Fixed Rate { #fixed-rate } + +Scheduling with tasks started at a fixed time interval, regardless of whether the previous execution has completed. +This can lead to concurrent execution of several tasks. + +For example, if the period is 10 seconds and each task execution takes 5 seconds, +the next task starts 5 seconds after the previous one completes. ===! ":fontawesome-brands-java: `Java`" @@ -162,7 +215,9 @@ next task execution will start 5 seconds after the previous one completed. #### Configuration { #configuration-2 } -It is possible to transfer parameters via configuration, it has priority over the parameters specified in the annotation: +Parameters can be passed through configuration; the configuration has priority over annotation values. +The `config` path is arbitrary, but by convention it is nested under the `scheduling` section so that a task's +parameters and its `telemetry` live together (as in the [example project](https://github.com/kora-projects/kora-examples), `scheduling.jobs.fix-rate`): ===! ":fontawesome-brands-java: `Java`" @@ -170,7 +225,7 @@ It is possible to transfer parameters via configuration, it has priority over th @Component public class SomeService { - @ScheduleAtFixedRate(config = "job") + @ScheduleAtFixedRate(config = "scheduling.jobs.fix-rate") void schedule() { // do something } @@ -183,45 +238,51 @@ It is possible to transfer parameters via configuration, it has priority over th @Component class SomeService { - @ScheduleAtFixedRate(config = "job") + @ScheduleAtFixedRate(config = "scheduling.jobs.fix-rate") fun schedule() { // do something } } ``` -SomeService of configuration via a config file: +Configuration file example: ===! ":material-code-json: `Hocon`" ```javascript - job { - initialDelay = "50ms" //(1)! - period = "50ms" //(2)! + scheduling { + jobs { + fix-rate { + initialDelay = "50ms" //(1)! + period = "50ms" //(2)! + } + } } ``` - 1. Initial delay interval before the first task - 2. Intermittent interval between tasks + 1. Initial delay before the first task (default: `0ms`) + 2. Periodic interval between tasks (`required`, no default) === ":simple-yaml: `YAML`" ```yaml - job: - initialDelay: "50ms" #(1)! - period: "50ms" #(2)! + scheduling: + jobs: + fix-rate: + initialDelay: "50ms" #(1)! + period: "50ms" #(2)! ``` - 1. Initial delay interval before the first task - 2. Intermittent interval between tasks + 1. Initial delay before the first task (default: `0ms`) + 2. Periodic interval between tasks (`required`, no default) -### Fixed delay { #fixed-delay } +### Fixed Delay { #fixed-delay } -The scheduler waits for a fixed period of time from the end of the previous task execution. -Multiple tasks will not be executed simultaneously. +The scheduler waits for a fixed time interval from the end of the previous task execution. +Multiple executions of the same task will not happen concurrently. -It does not matter how long the current execution takes, -the next task will start after the previous task is finished and the specified waiting interval. +It does not matter how long the current execution takes: +the next task starts after the previous task completes and the configured delay passes. ===! ":fontawesome-brands-java: `Java`" @@ -251,7 +312,7 @@ the next task will start after the previous task is finished and the specified w #### Configuration { #configuration-3 } -It is possible to transfer parameters via configuration, it has priority over the parameters specified in the annotation: +Parameters can be passed through configuration; it has priority over annotation values: ===! ":fontawesome-brands-java: `Java`" @@ -259,7 +320,7 @@ It is possible to transfer parameters via configuration, it has priority over th @Component public class SomeService { - @ScheduleWithFixedDelay(config = "job") + @ScheduleWithFixedDelay(config = "scheduling.jobs.fix-delay") void schedule() { // do something } @@ -272,41 +333,47 @@ It is possible to transfer parameters via configuration, it has priority over th @Component class SomeService { - @ScheduleWithFixedDelay(config = "job") + @ScheduleWithFixedDelay(config = "scheduling.jobs.fix-delay") fun schedule() { // do something } } ``` -SomeService of configuration via a config file: +Configuration file example: ===! ":material-code-json: `Hocon`" ```javascript - job { - initialDelay = "50ms" //(1)! - delay = "50ms" //(2)! + scheduling { + jobs { + fix-delay { + initialDelay = "50ms" //(1)! + delay = "50ms" //(2)! + } + } } ``` - 1. Initial delay interval before the first task - 2. Intermittent delay interval between tasks + 1. Initial delay before the first task (default: `0ms`) + 2. Periodic delay between tasks (`required`, no default) === ":simple-yaml: `YAML`" ```yaml - job: - initialDelay: "50ms" #(1)! - delay: "50ms" #(2)! + scheduling: + jobs: + fix-delay: + initialDelay: "50ms" #(1)! + delay: "50ms" #(2)! ``` - 1. Initial delay interval before the first task - 2. Intermittent delay interval between tasks + 1. Initial delay before the first task (default: `0ms`) + 2. Periodic delay between tasks (`required`, no default) ### Once { #once } -Runs a single task at a certain fixed time interval. +Runs a task once after the configured time interval. ===! ":fontawesome-brands-java: `Java`" @@ -336,7 +403,7 @@ Runs a single task at a certain fixed time interval. #### Configuration { #configuration-4 } -It is possible to transfer parameters via configuration, it has priority over the parameters specified in the annotation: +Parameters can be passed through configuration; it has priority over annotation values: ===! ":fontawesome-brands-java: `Java`" @@ -344,7 +411,7 @@ It is possible to transfer parameters via configuration, it has priority over th @Component public class SomeService { - @ScheduleOnce(config = "job") + @ScheduleOnce(config = "scheduling.jobs.once") void schedule() { // do something } @@ -357,42 +424,87 @@ It is possible to transfer parameters via configuration, it has priority over th @Component class SomeService { - @ScheduleOnce(config = "job") + @ScheduleOnce(config = "scheduling.jobs.once") fun schedule() { // do something } } ``` -SomeService of configuration via a config file: +Configuration file example: ===! ":material-code-json: `Hocon`" ```javascript - job { - delay = "50ms" //(1)! + scheduling { + jobs { + once { + delay = "50ms" //(1)! + } + } } ``` - 1. Initial delay interval before the task + 1. Delay before the task (`required`, no default) === ":simple-yaml: `YAML`" ```yaml - job: - delay: "50ms" #(1)! + scheduling: + jobs: + once: + delay: "50ms" #(1)! ``` - 1. Initial delay interval before the task + 1. Delay before the task (`required`, no default) ### Graceful Shutdown { #graceful-shutdown } -If you want to pre-terminate processing on a scheduled service termination without waiting for the service to end, -you need to check [Thread.currentThread().isInterrupted()](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#isInterrupted--) status and terminate the service yourself. +During [graceful shutdown](container.md#component-lifecycle), the native scheduler waits for tasks to complete for `scheduling.shutdownWait`. +If a task needs to stop earlier, check [Thread.currentThread().isInterrupted()](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#isInterrupted--) and stop the work manually. + +### Programmatic Scheduling { #programmatic } + +For scheduling tasks in imperative style, the `JdkSchedulingExecutor` component can be injected. +It wraps the same `ScheduledExecutorService` as the annotations and exposes the `scheduleAtFixedRate`, `scheduleWithFixedDelay`, and `schedule` methods: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { + + private final JdkSchedulingExecutor executor; + + public SomeService(JdkSchedulingExecutor executor) { + this.executor = executor; + } + + public void start() { + executor.scheduleAtFixedRate(() -> { + // do something + }, 50, 50, TimeUnit.MILLISECONDS); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val executor: JdkSchedulingExecutor) { + + fun start() { + executor.scheduleAtFixedRate({ + // do something + }, 50, 50, TimeUnit.MILLISECONDS) + } + } + ``` ## Quartz { #quartz } -A library-based implementation of [Quartz](https://www.baeldung.com/quartz) as a scheduler for creating aspects. +The implementation based on the [Quartz](https://www.quartz-scheduler.org/) library is used for tasks with `cron` schedules, custom `Trigger` instances, and `Quartz` execution rules. ### Dependency { #dependency-2 } @@ -412,7 +524,7 @@ A library-based implementation of [Quartz](https://www.baeldung.com/quartz) as a === ":simple-kotlin: `Kotlin`" [Dependency](general.md#dependencies) `build.gradle.kts`: - ```groovy + ```kotlin implementation("ru.tinkoff.kora:scheduling-quartz") ``` @@ -424,7 +536,9 @@ A library-based implementation of [Quartz](https://www.baeldung.com/quartz) as a ### Configuration { #configuration-5 } -Configuration is specified as [Properties](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/configuration/) values in key and value format: +`Quartz` configuration is specified as [Properties](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/configuration/) values in key-value format. +Kora settings for graceful shutdown and telemetry are configured in the `scheduling` section. +The configuration of a specific `cron` task can also contain the `telemetry` section; its values override the common scheduler telemetry for that task. ===! ":material-code-json: `Hocon`" @@ -441,20 +555,30 @@ Configuration is specified as [Properties](https://www.quartz-scheduler.org/docu metrics { enabled = true //(4)! slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(5)! + tags = { // (6)! + "key1" = "value1" + "key2" = "value2" + } } tracing { - enabled = true //(6)! + enabled = true //(7)! + attributes = { // (8)! + "key1" = "value1" + "key2" = "value2" + } } } } ``` - 1. Quartz scheduler configuration parameters - 2. Whether to wait for jobs to complete before shutting down scheduler in case of [graceful shutdown](container.md#graceful-shutdown) - 3. Enables module logging (default `false`) - 4. Enables module metrics (default `true`) - 5. Configuring [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 6. Enables module tracing (default `true`) + 1. `Quartz` scheduler configuration parameters (by default, properties from `quartz.properties` below are used) + 2. Whether to wait for tasks to complete before scheduler shutdown during [graceful shutdown](container.md#component-lifecycle) (default: `true`) + 3. Enables module logging (default: `false`) + 4. Enables module metrics (default: `true`) + 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Configures metric tags (default: `{}`) + 7. Enables module tracing (default: `true`) + 8. Configures tracing attributes (default: `{}`) === ":simple-yaml: `YAML`" @@ -468,17 +592,25 @@ Configuration is specified as [Properties](https://www.quartz-scheduler.org/docu enabled: false #(3)! metrics: enabled: true #(4)! - slo: [ 3, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! - telemetry: - enabled: true #(6)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! + tags: #(6)! + key1: value1 + key2: value2 + tracing: + enabled: true #(7)! + attributes: #(8)! + key1: value1 + key2: value2 ``` - 1. Quartz scheduler configuration parameters - 2. Whether to wait for jobs to complete before shutting down scheduler in case of [graceful shutdown](container.md#graceful-shutdown) - 3. Enables module logging (default `false`) - 4. Enables module metrics (default `true`) - 5. Configuring [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 6. Enables module tracing (default `true`) + 1. `Quartz` scheduler configuration parameters (by default, properties from `quartz.properties` below are used) + 2. Whether to wait for tasks to complete before scheduler shutdown during [graceful shutdown](container.md#component-lifecycle) (default: `true`) + 3. Enables module logging (default: `false`) + 4. Enables module metrics (default: `true`) + 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for metrics (default: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Configures metric tags (default: `{}`) + 7. Enables module tracing (default: `true`) + 8. Configures tracing attributes (default: `{}`) Default settings are used from: @@ -502,9 +634,74 @@ Default settings are used from: ### Cron { #cron } -Usage [Cron](http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html) expressions to run scheduled tasks. +[`cron` expressions](http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html) are used to run scheduled tasks. + +A `Quartz` expression has six required fields and an optional seventh year field, separated by spaces: + +| Field | Allowed values | Required | +|--------------|---------------------|----------| +| Seconds | `0-59` | yes | +| Minutes | `0-59` | yes | +| Hours | `0-23` | yes | +| Day of month | `1-31` | yes | +| Month | `1-12` or `JAN-DEC` | yes | +| Day of week | `1-7` or `SUN-SAT` | yes | +| Year | empty, `1970-2099` | no | + +Besides plain numbers, ranges (`8-10`), lists (`6,19`), and steps (`0/30`), the following special characters are supported: + +| Character | Meaning | +|-----------|--------------------------------------------------------------------------------------------------| +| `*` | All values of the field (for example `*` in the minute field means "every minute") | +| `?` | No specific value, used in the day-of-month or day-of-week field when the other one is specified | +| `L` | Last (last day of the month, or last given weekday of the month) | +| `W` | Nearest weekday to the given day of month | +| `#` | The N-th given weekday of the month, for example `5#2` is the second Friday | + +Expression examples: + +| Expression | Meaning | +|---------------------|---------------------------------------------| +| `0 0 * * * ?` | The top of every hour of every day | +| `*/10 * * * * ?` | Every ten seconds | +| `0 0 8-10 * * ?` | 8, 9 and 10 o'clock of every day | +| `0 0/30 8-10 * * ?` | 8:00, 8:30, 9:00, 9:30, 10:00 and 10:30 | +| `0 0 0 L * ?` | Last day of the month at midnight | +| `0 0 0 1W * ?` | First weekday of the month at midnight | +| `0 0 0 ? * 5#2` | The second Friday of the month at midnight | -Starts a single task at a certain fixed time interval. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { + + @ScheduleWithCron("* * * ? * * *") //(1)! + void schedule() { + // do something + } + } + ``` + + 1. `cron` expression that runs the task every second + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService { + + @ScheduleWithCron("* * * ? * * *") //(1)! + fun schedule() { + // do something + } + } + ``` + + 1. `cron` expression that runs the task every second + +The `identity` attribute sets the [Quartz Trigger identity](https://www.quartz-scheduler.org/api/2.3.0/org/quartz/TriggerBuilder.html) +used to name the task, which is useful for identifying and replacing tasks, especially with clustered or persistent `JobStore` implementations: ===! ":fontawesome-brands-java: `Java`" @@ -512,14 +709,14 @@ Starts a single task at a certain fixed time interval. @Component public class SomeService { - @ScheduleWithCron("0 0 * * * * ?") //(1)! + @ScheduleWithCron(value = "0 0 * * * ?", identity = "my-hourly-job") //(1)! void schedule() { // do something } } ``` - 1. Cron expression saying to run a task every hour and every day + 1. `cron` expression that runs the task at the top of every hour, registered under the trigger identity `my-hourly-job` === ":simple-kotlin: `Kotlin`" @@ -527,18 +724,20 @@ Starts a single task at a certain fixed time interval. @Component class SomeService { - @ScheduleWithCron("0 0 * * * * ?") //(1)! + @ScheduleWithCron(value = "0 0 * * * ?", identity = "my-hourly-job") //(1)! fun schedule() { // do something } } ``` - 1. Cron expression saying to run a task every hour and every day + 1. `cron` expression that runs the task at the top of every hour, registered under the trigger identity `my-hourly-job` #### Configuration { #configuration-6 } -It is possible to transfer parameters via configuration, it has priority over the parameters specified in the annotation: +Parameters can be passed through configuration; the configuration has priority over annotation values. +As with the native scheduler, the `config` path is arbitrary and by convention is nested under the `scheduling` section +(as in the [example project](https://github.com/kora-projects/kora-examples), `scheduling.jobs.quartz`): ===! ":fontawesome-brands-java: `Java`" @@ -546,7 +745,7 @@ It is possible to transfer parameters via configuration, it has priority over th @Component public class SomeService { - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") void schedule() { // do something } @@ -559,7 +758,7 @@ It is possible to transfer parameters via configuration, it has priority over th @Component class SomeService { - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") fun schedule() { // do something } @@ -571,25 +770,31 @@ Configuration example: ===! ":material-code-json: `Hocon`" ```javascript - job { - cron = "0 0 * * * * ?" //(1)! + scheduling { + jobs { + quartz { + cron = "* * * ? * * *" //(1)! + } + } } ``` - 1. Cron expression saying to run a task every hour and every day + 1. `cron` expression that runs the task every second (`required`, no default) === ":simple-yaml: `YAML`" ```yaml - job: - cron: "0 0 * * * * ?" #(1)! + scheduling: + jobs: + quartz: + cron: "* * * ? * * *" #(1)! ``` - 1. Cron expression saying to run a task every hour and every day + 1. `cron` expression that runs the task every second (`required`, no default) ### Trigger { #trigger } -This involves creating your custom `trigger` based on the Quartz library and registering it in the application dependency container with a specific tag and then using it via annotation. +For a custom schedule, you can create a `Trigger` from the `Quartz` library, register it in the dependency graph with a tag, and then use this tag in the `@ScheduleWithTrigger` annotation. ===! ":fontawesome-brands-java: `Java`" @@ -619,8 +824,8 @@ This involves creating your custom `trigger` based on the Quartz library and reg } ``` - 1. Trigger tag - 2. Trigger tag + 1. Tag used to register the `Trigger` in the dependency graph. + 2. The same tag used by the task to receive the `Trigger`. === ":simple-kotlin: `Kotlin`" @@ -645,19 +850,20 @@ This involves creating your custom `trigger` based on the Quartz library and reg @Component class SomeService { - @ScheduleWithTrigger(@Tag(SomeService.class)) //(2)! + @ScheduleWithTrigger(@Tag(SomeService::class)) //(2)! fun schedule() { // do something } } ``` - 1. Trigger tag - 2. Trigger tag + 1. Tag used to register the `Trigger` in the dependency graph. + 2. The same tag used by the task to receive the `Trigger`. -### Non-concurrent execution { #non-concurrent-execution } +### Non-Concurrent Execution { #non-concurrent-execution } -Annotation that says that a method with an annotation should not be executed in parallel. +The `@DisallowConcurrentExecution` annotation prevents concurrent execution of the same method by the `Quartz` scheduler. +It is the `Kora` counterpart of `org.quartz.DisallowConcurrentExecution` and can be placed on any `@Schedule*`-annotated method. ===! ":fontawesome-brands-java: `Java`" @@ -666,7 +872,7 @@ Annotation that says that a method with an annotation should not be executed in public class SomeService { @DisallowConcurrentExecution - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") void schedule() { // do something } @@ -680,20 +886,56 @@ Annotation that says that a method with an annotation should not be executed in class SomeService { @DisallowConcurrentExecution - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") fun schedule() { // do something } } ``` -### Persistent execution { #persistent-execution } +### Job Context { #job-context } + +A `Quartz` scheduled method may optionally declare a single `org.quartz.JobExecutionContext` argument. +When it is present, `Kora` passes the current execution context to the method; when it is absent, the method is called with no arguments. +The context gives access to the task's `org.quartz.JobDataMap`, which is the way to read and write state associated with the task: -Annotation that says to forcefully update `org.quartz.JobDataMap` during execution and requires -the scheduler to resave `org.quartz.JobDataMap` upon completion of execution. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { -It is recommended to use this annotation in conjunction with the `@DisallowConcurrentExecution` annotation -to avoid conflicts when storing data during concurrent task execution. + @ScheduleWithCron(config = "scheduling.jobs.quartz") + void schedule(JobExecutionContext context) { + JobDataMap data = context.getJobDetail().getJobDataMap(); + int counter = data.containsKey("counter") ? data.getInt("counter") : 0; + data.put("counter", counter + 1); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService { + + @ScheduleWithCron(config = "scheduling.jobs.quartz") + fun schedule(context: JobExecutionContext) { + val data = context.jobDetail.jobDataMap + val counter = if (data.containsKey("counter")) data.getInt("counter") else 0 + data.put("counter", counter + 1) + } + } + ``` + +### Persisting Job Data { #persistent-execution } + +The `@PersistJobDataAfterExecution` annotation tells `Quartz` to store the updated `org.quartz.JobDataMap` back after task execution, +so that the changes made through the [JobExecutionContext](#job-context) are visible in the next execution. + +It is recommended to use it together with `@DisallowConcurrentExecution` +to avoid data storage conflicts during concurrent task execution. ===! ":fontawesome-brands-java: `Java`" @@ -701,24 +943,67 @@ to avoid conflicts when storing data during concurrent task execution. @Component public class SomeService { + @DisallowConcurrentExecution @PersistJobDataAfterExecution - @ScheduleWithCron(config = "job") - void schedule() { - // do something + @ScheduleWithCron(config = "scheduling.jobs.quartz") + void schedule(JobExecutionContext context) { + JobDataMap data = context.getJobDetail().getJobDataMap(); + int counter = data.containsKey("counter") ? data.getInt("counter") : 0; + data.put("counter", counter + 1); //(1)! } } ``` + 1. The updated value is persisted after execution and available in the next run + === ":simple-kotlin: `Kotlin`" ```kotlin @Component class SomeService { + @DisallowConcurrentExecution @PersistJobDataAfterExecution - @ScheduleWithCron(config = "job") - fun schedule() { - // do something + @ScheduleWithCron(config = "scheduling.jobs.quartz") + fun schedule(context: JobExecutionContext) { + val data = context.jobDetail.jobDataMap + val counter = if (data.containsKey("counter")) data.getInt("counter") else 0 + data.put("counter", counter + 1) //(1)! + } + } + ``` + + 1. The updated value is persisted after execution and available in the next run + +### Graceful Shutdown { #graceful-shutdown-quartz } + +During [graceful shutdown](container.md#component-lifecycle), the `scheduling.waitForJobComplete` option controls how the `Quartz` scheduler stops. +With `true` (default) it calls `scheduler.shutdown(true)` and blocks until running tasks finish; with `false` it stops without waiting. +As with the native scheduler, long-running tasks should still cooperatively check +[Thread.currentThread().isInterrupted()](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#isInterrupted--) and stop the work manually. + +### Scheduler { #scheduler } + +The underlying `org.quartz.Scheduler` is registered as a component and can be injected for advanced scenarios, +such as registering tasks programmatically or inspecting the scheduler state: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { + + private final Scheduler scheduler; + + public SomeService(Scheduler scheduler) { + this.scheduler = scheduler; } } ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val scheduler: Scheduler) + ``` diff --git a/mkdocs/docs/en/documentation/soap-client.md b/mkdocs/docs/en/documentation/soap-client.md index 1aba59c..363ae53 100644 --- a/mkdocs/docs/en/documentation/soap-client.md +++ b/mkdocs/docs/en/documentation/soap-client.md @@ -1,10 +1,14 @@ --- -description: "Explains Kora SOAP client setup, SOAP client configuration, usage patterns, generated clients, and wsdl2java Gradle plugin integration. Use when working with SoapClientModule, @SoapClient, wsdl2java, JAX-WS, SOAPAction, WebServiceClient." +description: "Explains Kora SOAP client setup, configuration, usage, generated clients, request customization and WS-Security, exception handling, testing, and the wsdl2java Gradle plugin. Use when working with SoapClientModule, wsdl2java, JAX-WS, SOAPAction, WebServiceClient, SoapFaultException, SoapEnvelopeProcessors." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora SOAP client setup, SOAP client configuration, usage patterns, generated clients, and wsdl2java Gradle plugin integration; key triggers include SoapClientModule, @SoapClient, wsdl2java, JAX-WS, SOAPAction, WebServiceClient." + use_when: "Use this file for Kora docs or implementation questions about Kora SOAP client setup, configuration, usage patterns, generated clients, envelope processors and WS-Security authorization, body-mapper logging, exception handling, testing, and wsdl2java Gradle plugin integration; key triggers include SoapClientModule, SoapServiceConfig, wsdl2java, JAX-WS, SOAPAction, WebServiceClient, SoapException, SoapFaultException, InvalidHttpResponseSoapException, SoapEnvelopeProcessors, wssAuth." --- -A module for creating and registering SOAP services by classes annotated `javax.jws.WebService`/`jakarta.jws.WebService`. +`SOAP` is a protocol for exchanging `XML` messages, often used for integration with external systems over `HTTP` and a `WSDL` contract. +The `soap-client` module creates client implementations for interfaces annotated with `javax.jws.WebService` or `jakarta.jws.WebService` and registers them in the application graph. + +Usually, such interfaces and related `JAXB` classes are generated from `WSDL`, for example with `wsdl2java`. +After generation, Kora creates the client implementation and connects it to an `HTTP client`, `XML` mapping, and telemetry. ## Dependency { #dependency } @@ -34,26 +38,83 @@ A module for creating and registering SOAP services by classes annotated `javax. interface Application : SoapClientModule ``` -**Requires** the [HTTP client](http-client.md) implementation to be connected. +**Requires** an [`HTTP client`](http-client.md) implementation (for example `http-client-jdk` or `http-client-ok`) +and a configuration module ([HOCON](config.md#hocon) or [YAML](config.md#yaml)) to be present in the application. + +When `SOAP` interfaces and `JAXB` classes are generated with the [`wsdl2java` plugin](#wsdl2java-plugin) in `jakarta` mode, +the required `jakarta.*` / `JAXB` runtime is already provided by the generated sources and the `JDK`. +In that case the transitive `jakarta` / `Glassfish` / `activation` dependencies of `soap-client` can be excluded to avoid version clashes: + +===! ":fontawesome-brands-java: `Java`" + + `build.gradle`: + ```groovy + implementation("ru.tinkoff.kora:soap-client") { + exclude group: "jakarta.xml" + exclude group: "jakarta.jws" + exclude group: "jakarta.xml.ws" + exclude group: "jakarta.xml.bind" + exclude group: "org.glassfish.jaxb" + exclude group: "com.sun.activation" + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + `build.gradle.kts`: + ```groovy + implementation("ru.tinkoff.kora:soap-client") { + exclude(group = "jakarta.xml") + exclude(group = "jakarta.jws") + exclude(group = "jakarta.xml.ws") + exclude(group = "jakarta.xml.bind") + exclude(group = "org.glassfish.jaxb") + exclude(group = "com.sun.activation") + } + ``` ## Description { #description } -It is understood that we have classes annotated with `javax.jws.WebService`/`jakarta.jws.WebService` that can be created by other means, -such as [Gradle Plugin](#wsdl2java-plugin). +The application is expected to already have interfaces annotated with `javax.jws.WebService` or `jakarta.jws.WebService` +(both annotation families are supported). They can be written manually, but are usually created from `WSDL` +by a separate tool, for example a [Gradle plugin](#wsdl2java-plugin). -Based on such classes, Kora is used to create SOAP client implementations with the Impl suffix in the same package and register them as a module with config. +Based on such interfaces, the annotation processor (bundled in the `annotation-processors` artifact) creates in the same package: -Then the configuration and the SOAP service itself become available for dependency injection automatically. +- A client implementation named `$_SoapClientImpl`, registered as a `@DefaultComponent` in the application graph. +- A module named `$_SoapClientModule` annotated with `@Module`, which registers the `SoapServiceConfig` + (tagged with `@Tag(.class)`) and the client itself. + +After that, the configuration and the `SOAP client` become available for dependency injection automatically. + +### How it works { #how-it-works } + +At runtime the generated client uses the connected `HttpClient` and behaves as follows: + +- Sends an `HTTP POST` request with `Content-Type: text/xml` to the address from the `url` configuration parameter. +- Adds the `SOAPAction` `HTTP` header only when `action` is set on the method's `@WebMethod` annotation. +- Applies the `timeout` configuration value as the request timeout. +- Treats `HTTP 200` as a successful response and unmarshals the body into the method's return type. +- Treats `HTTP 500` as a `SOAP Fault` and converts it either to a [typed WSDL fault exception](#exception-handling) or to `SoapFaultException`. +- Raises `InvalidHttpResponseSoapException` for any other `HTTP` status code. +- Parses `multipart` (`XOP` / `MTOM` attachment) responses automatically. +- For every `@WebMethod` it generates a synchronous method and a `Async` method returning `CompletionStage` for [non-blocking calls](#asynchronous). ## Configuration { #configuration } -All configurations for SOAP clients are created with the prefix `soapClient`, -and the bulk of the client configuration is under the client name from the WSDL annotation `@WebService`, -which corresponds often to the `` tag in the WSDL configuration. +All configurations for `SOAP clients` are created with the `soapClient` prefix. +The main part of the client configuration is placed under the service name from the `@WebService` annotation. -SOAP service named `SimpleService` will have a configuration with the path `soapClient.SimpleService`. +The section name is selected in this order: -Example of the complete configuration described in the `SoapServiceConfig` class (default or example values are specified): +1. `name` from `@WebService` +2. `serviceName` from `@WebService` +3. `portName` from `@WebService` +4. interface name + +A `SOAP client` named `SimpleService` will have the `soapClient.SimpleService` configuration path. + +Basic configuration parameters: ===! ":material-code-json: `Hocon`" @@ -62,38 +123,12 @@ Example of the complete configuration described in the `SoapServiceConfig` class SimpleService { url = "https://localhost:8090" //(1)! timeout = "60s" //(2)! - telemetry { - logging { - enabled = false //(3)! - } - metrics { - enabled = true //(4)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(5)! - tags = { // (6)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(7)! - attributes = { // (8)! - "key1" = "value1" - "key2" = "value2" - } - } - } } } ``` - 1. URL of the service where requests will be sent (**required**) - 2. Maximum request time - 3. Enables module logging (default `false`) - 4. Enables module metrics (default `true`) - 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 6. Configures tags for metrics (optional) - 7. Enables module tracing (default `true`) - 8. Configures attributes for tracing (optional) + 1. Service `URL` where requests will be sent (`required`, no default). + 2. Maximum request execution time (default not specified, optional). === ":simple-yaml: `YAML`" @@ -102,36 +137,100 @@ Example of the complete configuration described in the `SoapServiceConfig` class SimpleService: url: "https://localhost:8090" #(1)! timeout: "60s" #(2)! - telemetry: - logging: - enabled: false #(3)! - metrics: - enabled: true #(4)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! - tags: #(6)! - key1: value1 - key2: value2 - tracing: - enabled: true #(7)! - attributes: #(8)! - key1: value1 - key2: value2 - ``` - - 1. URL of the service where requests will be sent (**required**) - 2. Maximum request time - 3. Enables module logging (default `false`) - 4. Enables module metrics (default `true`) - 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metrics - 6. Configures tags for metrics (optional) - 7. Enables module tracing (default `true`) - 8. Configures attributes for tracing (optional) + ``` + + 1. Service `URL` where requests will be sent (`required`, no default). + 2. Maximum request execution time (default not specified, optional). + +??? note "Full Configuration" + + Example of the complete configuration described by the `SoapServiceConfig` class: + + ===! ":material-code-json: `Hocon`" + + ```javascript + soapClient { + SimpleService { + url = "https://localhost:8090" //(1)! + timeout = "60s" //(2)! + telemetry { + logging { + enabled = false //(3)! + } + metrics { + enabled = true //(4)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(5)! + tags = { // (6)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(7)! + attributes = { // (8)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + ``` + + 1. Service `URL` where requests will be sent (required, default: not specified). + 2. Maximum request execution time (default: `60s`). + 3. Enables module logging (default: `false`). + 4. Enables module metrics (default: `true`). + 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 6. Additional tags for metrics (default: `{}`). + 7. Enables module tracing (default: `true`). + 8. Additional attributes for tracing (default: `{}`). + + === ":simple-yaml: `YAML`" + + ```yaml + soapClient: + SimpleService: + url: "https://localhost:8090" #(1)! + timeout: "60s" #(2)! + telemetry: + logging: + enabled: false #(3)! + metrics: + enabled: true #(4)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! + tags: #(6)! + key1: value1 + key2: value2 + tracing: + enabled: true #(7)! + attributes: #(8)! + key1: value1 + key2: value2 + ``` + + 1. Service `URL` where requests will be sent (required, default: not specified). + 2. Maximum request execution time (default: `60s`). + 3. Enables module logging (default: `false`). + 4. Enables module metrics (default: `true`). + 5. Configures [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) for the [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) metric (default: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 6. Additional tags for metrics (default: `{}`). + 7. Enables module tracing (default: `true`). + 8. Additional attributes for tracing (default: `{}`). Module metrics are described in the [Metrics Reference](metrics.md#soap-client) section. +The configuration is described by the `SoapServiceConfig` interface. The `url` parameter is **required**: +if it is missing from the configuration, the application graph fails to build with a `ConfigValueExtractionException` +(missing value after parse). The `timeout` parameter defaults to `60s`. + +The configuration is registered in the graph under `@Tag(.class)`, so when a client is +[constructed manually](#request-customization) the `SoapServiceConfig` dependency must be resolved with that same tag. + ## Usage { #usage } -Once all components have been created the created SOAP service is available for deployment, an example for a `SimpleService` service is shown below: +After all components are created, the `SOAP client` becomes available for injection. +Below is an example for the `SimpleService` client: ===! ":fontawesome-brands-java: `Java`" @@ -156,10 +255,392 @@ Once all components have been created the created SOAP service is available for } ``` -## Wsdl2java plugin { #wsdl2java-plugin } +### Invocation { #invocation } + +A generated method accepts the request type and returns the typed response. +For the `SimpleService` client with a `test` operation: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final SimpleService service; + + public SomeService(SimpleService service) { + this.service = service; + } + + public String call() throws Exception { + var request = new TestRequest(); + request.setVal1("1"); + request.setVal2("2"); + + TestResponse response = service.test(request); + return response.getVal1(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val service: SimpleService) { -[Gradle Plugin](https://github.com/bjornvester/wsdl2java-gradle-plugin) can be used as one option to create classes annotated `javax.jws.WebService`/`jakarta.jws.WebService` -based on [WSDL](https://coderlessons.com/tutorials/xml-tekhnologii/uznaite-wsdl/wsdl-kratkoe-rukovodstvo). + fun call(): String? { + val request = TestRequest().apply { + val1 = "1" + val2 = "2" + } + + val response = service.test(request) + return response.val1 + } + } + ``` + +### Asynchronous { #asynchronous } + +For every `@WebMethod`, the generator also creates a `Async` method returning `CompletionStage` for non-blocking calls. +The async method is declared on the generated `$_SoapClientImpl` class rather than on the `WSDL` interface, +so to use it you cast the injected client to the generated implementation type: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final SimpleService service; + + public SomeService(SimpleService service) { + this.service = service; + } + + public CompletionStage callAsync() { + var request = new TestRequest(); + request.setVal1("1"); + request.setVal2("2"); + + return (($SimpleService_SoapClientImpl) service).testAsync(request) + .thenApply(TestResponse::getVal1); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val service: SimpleService) { + + fun callAsync(): CompletionStage { + val request = TestRequest().apply { + val1 = "1" + val2 = "2" + } + + return (service as `$SimpleService_SoapClientImpl`).testAsync(request) + .thenApply { it.val1 } + } + } + ``` + +## Request customization { #request-customization } + +`SOAP` clients do not use the `@InterceptWith` mechanism of [declarative HTTP clients](http-client.md#interceptors). +Instead, the generated `$_SoapClientImpl` provides a **secondary constructor** that accepts a +`Function` envelope processor. The processor is applied to the request `SOAP` envelope +before it is marshalled and sent — this is the extension point for adding `SOAP` headers (authorization, tracing, +custom elements) or otherwise transforming the outgoing envelope. + +The generated implementation has two constructors: + +- `(HttpClient, SoapClientTelemetryFactory, SoapServiceConfig)` — used by the generated `@DefaultComponent`; applies `Function.identity()` (no changes). +- `(HttpClient, SoapClientTelemetryFactory, SoapServiceConfig, Function)` — lets you supply a custom processor. + +To use a custom processor, register your own factory that returns the client **interface** type and constructs the +implementation with the processor. Because it provides the same interface type, your factory **overrides** the generated +`@DefaultComponent`. Resolve `SoapServiceConfig` with `@Tag(.class)` — the tag under which the generated module registers it: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface SoapModule { + + default SimpleService simpleService(HttpClient httpClient, + SoapClientTelemetryFactory telemetryFactory, + @Tag(SimpleService.class) SoapServiceConfig config) { + var processor = SoapEnvelopeProcessors.wssAuth("username", "password"); //(1)! + try { + return new $SimpleService_SoapClientImpl(httpClient, telemetryFactory, config, processor); + } catch (Exception e) { + throw new IllegalStateException(e); + } + } + } + ``` + + 1. Any `Function` can be used here; `SoapEnvelopeProcessors.wssAuth` is a built-in one. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface SoapModule { + + fun simpleService(httpClient: HttpClient, + telemetryFactory: SoapClientTelemetryFactory, + @Tag(SimpleService::class) config: SoapServiceConfig): SimpleService { + val processor = SoapEnvelopeProcessors.wssAuth("username", "password") //(1)! + return `$SimpleService_SoapClientImpl`(httpClient, telemetryFactory, config, processor) + } + } + ``` + + 1. Any `Function` can be used here; `SoapEnvelopeProcessors.wssAuth` is a built-in one. + +A custom processor can add arbitrary `SOAP` headers by appending to `envelope.getHeader().getAny()`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + Function processor = envelope -> { + envelope.getHeader().getAny().add(myHeaderElement); // org.w3c.dom.Element or a JAXB object + return envelope; + }; + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val processor = Function { envelope -> + envelope.header.any.add(myHeaderElement) // org.w3c.dom.Element or a JAXB object + envelope + } + ``` + +### Authorization { #authorization } + +`SoapEnvelopeProcessors.wssAuth(username, password)` is a built-in processor that adds a +[WS-Security](https://en.wikipedia.org/wiki/WS-Security) `UsernameToken` header (`Username` plus a plaintext `Password`) +to every request envelope. Wire it exactly as shown above by passing it as the envelope processor to the client constructor. + +## Logging { #logging } + +When `telemetry.logging.enabled` is `true`, the client logs the full request and response `SOAP` envelopes (the `XML` bodies). +To mask or transform logged payloads (for example, to hide sensitive data), override the `SoapClientLogger.SoapClientLoggerBodyMapper` +component. `SoapClientModule` provides it as a `@DefaultComponent`, so a user `@Component` implementation replaces it: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MaskingBodyMapper implements SoapClientLogger.SoapClientLoggerBodyMapper { + + @Override + public String mapRequest(String serviceName, String soapMethod, byte[] requestAsBytes) { + return ""; + } + + @Override + public String mapResponseSuccess(String serviceName, String soapMethod, byte[] responseAsBytes) { + return new String(responseAsBytes, StandardCharsets.UTF_8); + } + + @Override + public String mapResponseFailure(String serviceName, String soapMethod, byte[] responseAsBytes) { + return new String(responseAsBytes, StandardCharsets.UTF_8); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MaskingBodyMapper : SoapClientLogger.SoapClientLoggerBodyMapper { + + override fun mapRequest(serviceName: String, soapMethod: String, requestAsBytes: ByteArray): String { + return "" + } + + override fun mapResponseSuccess(serviceName: String, soapMethod: String, responseAsBytes: ByteArray): String { + return String(responseAsBytes, StandardCharsets.UTF_8) + } + + override fun mapResponseFailure(serviceName: String, soapMethod: String, responseAsBytes: ByteArray): String { + return String(responseAsBytes, StandardCharsets.UTF_8) + } + } + ``` + +## Exception handling { #exception-handling } + +All `SOAP` client failures are unchecked. Transport and `HTTP` errors extend the base `SoapException` +(a `RuntimeException`), so a single `catch (SoapException e)` handles those, or a specific subtype can be caught. +The `XML` marshalling/unmarshalling exceptions extend `RuntimeException` directly (not `SoapException`), so they must +be caught separately. + +Main exception types: + +- `SoapException` — base unchecked exception (extends `RuntimeException`) for transport and `HTTP` `SOAP` client failures. +- `SoapFaultException` (extends `SoapException`) — the server returned a `SOAP Fault` that does not match a typed `WSDL` fault. `getFault()` returns a `SoapFault` exposing `getFaultcode()` (`QName`), `getFaultstring()`, `getFaultactor()`, and `getDetail()`. +- `InvalidHttpResponseSoapException` (extends `SoapException`) — the server returned an unexpected `HTTP` status code (anything other than `200` or `500`). +- `SoapRequestMarshallingException` (extends `RuntimeException`, **not** `SoapException`) — the request envelope could not be marshalled to `XML`. +- `SoapResponseUnmarshallingException` (extends `RuntimeException`, **not** `SoapException`) — the response `XML` could not be unmarshalled. + +When a `WSDL` operation declares faults (``), the generator emits typed checked exceptions annotated with `@WebFault`, +and the method throws them directly when the returned fault `detail` matches one of them. If the fault does not match any +declared type, `SoapFaultException` is thrown instead. + +===! ":fontawesome-brands-java: `Java`" + + ```java + try { + var response = service.test(request); + // ... use the response + } catch (MyServiceFault e) { //(1)! + // handle a specific declared WSDL fault + } catch (SoapFaultException e) { //(2)! + SoapFault fault = e.getFault(); + var code = fault.getFaultcode(); + var message = fault.getFaultstring(); + } catch (InvalidHttpResponseSoapException e) { + // unexpected HTTP status code + } catch (SoapException e) { + // any other transport/HTTP SOAP failure + } catch (SoapRequestMarshallingException | SoapResponseUnmarshallingException e) { + // XML (un)marshalling failure — extends RuntimeException, not SoapException + } + ``` + + 1. Typed `@WebFault` exception generated from a ``; the concrete class name comes from the `WSDL`. + 2. Any `SOAP Fault` that does not match a declared typed fault. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + try { + val response = service.test(request) + // ... use the response + } catch (e: MyServiceFault) { //(1)! + // handle a specific declared WSDL fault + } catch (e: SoapFaultException) { //(2)! + val fault = e.fault + val code = fault.faultcode + val message = fault.faultstring + } catch (e: InvalidHttpResponseSoapException) { + // unexpected HTTP status code + } catch (e: SoapException) { + // any other transport/HTTP SOAP failure + } catch (e: SoapRequestMarshallingException) { + // request XML marshalling failure — extends RuntimeException, not SoapException + } catch (e: SoapResponseUnmarshallingException) { + // response XML unmarshalling failure — extends RuntimeException, not SoapException + } + ``` + + 1. Typed `@WebFault` exception generated from a ``; the concrete class name comes from the `WSDL`. + 2. Any `SOAP Fault` that does not match a declared typed fault. + +### Low-level result model { #result-model } + +Internally the request engine `SoapRequestExecutor` returns a `SoapResult`, a sealed interface with two records: +`SoapResult.Success(Object body)` and `SoapResult.Failure(SoapFault fault, String faultMessage)`. +The generated client maps `Success` to the typed response and `Failure` to a typed fault exception or `SoapFaultException`, +so you normally do not work with `SoapResult` directly. + +## Testing { #testing } + +The client can be tested with [`@KoraAppTest`](junit5.md) by injecting it as a `@TestComponent` and pointing `url` at a mock server. +The example below overrides `SOAP_CLIENT_URL` to the mock server address and invokes `service.test(request)`, checking the typed response: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SimpleServiceTests implements KoraAppTestConfigModifier { + + @TestComponent + private SimpleService service; + + @Override + public KoraConfigModification config() { + return KoraConfigModification.ofSystemProperty("SOAP_CLIENT_URL", "http://localhost:8080"); + } + + @Test + void testCall() throws Exception { + // the mock server responds with a TestResponse envelope for the request below + var request = new TestRequest(); + request.setVal1("1"); + request.setVal2("2"); + + var response = service.test(request); + assertEquals("1", response.getVal1()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SimpleServiceTests : KoraAppTestConfigModifier { + + @TestComponent + lateinit var service: SimpleService + + override fun config(): KoraConfigModification = + KoraConfigModification.ofSystemProperty("SOAP_CLIENT_URL", "http://localhost:8080") + + @Test + fun testCall() { + // the mock server responds with a TestResponse envelope for the request below + val request = TestRequest().apply { + val1 = "1" + val2 = "2" + } + + val response = service.test(request) + assertEquals("1", response.val1) + } + } + ``` + +The request envelope sent to the server and the response envelope it returns look like this on the wire: + +```xml + + + + + + 1 + 2 + + + + + + + + + + 1 + + + +``` + +## `wsdl2java` Plugin { #wsdl2java-plugin } + +A [Gradle plugin](https://github.com/bjornvester/wsdl2java-gradle-plugin) can be used as one option for creating interfaces annotated with `javax.jws.WebService` or `jakarta.jws.WebService`, +as well as `JAXB` classes based on `WSDL`. ### Dependency { #dependency-2 } @@ -183,7 +664,8 @@ based on [WSDL](https://coderlessons.com/tutorials/xml-tekhnologii/uznaite-wsdl/ ### Usage { #usage-2 } -Suppose we have a WSDL where `SimpleService` is declared, then configuring the plugin for `jakarta` annotation will look like this: +Suppose there is a `WSDL` where the `SimpleService` service is declared. +Then the plugin configuration for generation with `jakarta` annotations will look like this: ===! ":fontawesome-brands-java: `Java`" @@ -225,3 +707,6 @@ Suppose we have a WSDL where `SimpleService` is declared, then configuring the p ) } ``` + +The `useJakarta = true` option makes the plugin generate interfaces with `jakarta.jws` annotations. +Set it to `false` (or omit it) to generate `javax.jws` annotations instead — the annotation processor supports both. diff --git a/mkdocs/docs/en/documentation/tracing.md b/mkdocs/docs/en/documentation/tracing.md index 52b25b1..8849a38 100644 --- a/mkdocs/docs/en/documentation/tracing.md +++ b/mkdocs/docs/en/documentation/tracing.md @@ -1,26 +1,36 @@ --- -description: "Explains Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing. Use when working with TracingModule, OpenTelemetry, GrpcSender, OpentelemetryContext, Span, TraceContext, OTLP." +description: "Explains Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing. Use when working with OpentelemetryTracingModule, OpenTelemetry, OpentelemetryContext, Span, OTLP." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing; key triggers include TracingModule, OpenTelemetry, GrpcSender, OpentelemetryContext, Span, TraceContext, OTLP." + use_when: "Use this file for Kora docs or implementation questions about Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing; key triggers include OpentelemetryTracingModule, OpenTelemetry, OpentelemetryContext, Span, OTLP." --- -Module for collecting application trace according to [OpenTelemetry] standard(https://opentelemetry.io/docs/what-is-opentelemetry/) -and export trace by gRPC in OTLP format. +Tracing helps link separate application operations into a single execution chain and understand where a request spent time or failed. +Kora uses [`OpenTelemetry`](https://opentelemetry.io/docs/what-is-opentelemetry/) to create `Span`, store the current tracing context in `OpentelemetryContext`, and export data in the `OTLP` format. + +The current `Span` is stored in the Kora context, so it can be propagated between application components and used when manually creating nested `Span`. +When `OpentelemetryContext` is set, Kora also adds `traceId` and `spanId` to `MDC` so these identifiers appear in logs when the logging module is used. + +Most `Span` are created automatically: the module instruments the HTTP server and client, database, `Kafka` consumer and producer, gRPC server and client, and other subsystems out of the box, +and propagates the trace context between services over the [W3C traceparent](https://www.w3.org/TR/trace-context/) standard. + +Kora provides two mutually exclusive exporter modules, `OTLP/gRPC` and `OTLP/HTTP`; choose exactly one depending on the protocol your collector accepts. +Either exporter module transitively provides the core tracing wiring (`OpentelemetryTracingModule`) and the automatic instrumentation (`OpentelemetryModule`), so no other tracing dependency is required. For a step-by-step walkthrough before the reference details, see [Observability](../guides/observability.md). ## gRPC { #grpc } -Module allows trace collection using [gRPC protocol](https://github.com/open-telemetry/oteps/blob/main/text/0035-opentelemetry-protocol.md#protocol-details) by means of `GrpcSender`. +The module exports tracing data to `OpenTelemetry Collector` through `OTLP/gRPC`. +It builds an `OtlpGrpcSpanExporter` behind a `BatchSpanProcessor`, and the typical collector endpoint is `http://localhost:4317`. ===! ":fontawesome-brands-java: `Java`" - Зависимость `build.gradle`: + [Dependency](general.md#dependencies) in `build.gradle`: ```groovy implementation "ru.tinkoff.kora:opentelemetry-tracing-exporter-grpc" ``` - Модуль: + Module: ```java @KoraApp public interface Application extends OpentelemetryGrpcExporterModule { } @@ -28,12 +38,12 @@ Module allows trace collection using [gRPC protocol](https://github.com/open-tel === ":simple-kotlin: `Kotlin`" - Зависимость `build.gradle.kts`: + [Dependency](general.md#dependencies) in `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:opentelemetry-tracing-exporter-grpc") ``` - Модуль: + Module: ```kotlin @KoraApp interface Application : OpentelemetryGrpcExporterModule @@ -41,16 +51,17 @@ Module allows trace collection using [gRPC protocol](https://github.com/open-tel ## HTTP { #http } -Module allows to collect trace using [HTTP protocol](https://github.com/open-telemetry/oteps/blob/main/text/0099-otlp-http.md) by means of `HttpSender`. +The module exports tracing data to `OpenTelemetry Collector` through `OTLP/HTTP`. +It builds an `OtlpHttpSpanExporter` behind a `BatchSpanProcessor`, and the typical collector endpoint is `http://localhost:4318/v1/traces`. ===! ":fontawesome-brands-java: `Java`" - Зависимость `build.gradle`: + [Dependency](general.md#dependencies) in `build.gradle`: ```groovy implementation "ru.tinkoff.kora:opentelemetry-tracing-exporter-http" ``` - Модуль: + Module: ```java @KoraApp public interface Application extends OpentelemetryHttpExporterModule { } @@ -58,12 +69,12 @@ Module allows to collect trace using [HTTP protocol](https://github.com/open-tel === ":simple-kotlin: `Kotlin`" - Зависимость `build.gradle.kts`: + [Dependency](general.md#dependencies) in `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:opentelemetry-tracing-exporter-http") ``` - Модуль: + Module: ```kotlin @KoraApp interface Application : OpentelemetryHttpExporterModule @@ -71,9 +82,13 @@ Module allows to collect trace using [HTTP protocol](https://github.com/open-tel ## Configuration { #configuration } -`endpoint` is the only a required field, attributes from the `attributes` field will be sent with each span. +Export parameters under `tracing.exporter` are described by `OpentelemetryGrpcExporterConfig` (for `OTLP/gRPC`) and `OpentelemetryHttpExporterConfig` (for `OTLP/HTTP`); both classes share the same field set. +Resource attributes under `tracing.attributes` are described by `OpentelemetryResourceConfig`. +If `tracing.exporter.endpoint` is not specified, no exporter is created (the config resolves to the internal `Empty` value and a no-op `SpanExporter`/`SpanProcessor` is used), and the application starts without sending traces to an external collector. -Parameters described in the `OpentelemetryGrpcExporterConfig`/`OpentelemetryHttpExporterConfig` and `OpentelemetryResourceConfig` classes: +The `tracing.attributes` field defines `OpenTelemetry Resource` attributes that are attached to **every** exported `Span` of the whole service. +It usually contains the service name and namespace, for example `service.name` and `service.namespace`. +These service-wide `Resource` attributes are different from per-module span attributes configured under `.telemetry.tracing.attributes`, which are added only to the spans of a specific subsystem — see [Module tracing configuration](#module-config). ===! ":material-code-json: `Hocon`" @@ -89,7 +104,7 @@ Parameters described in the `OpentelemetryGrpcExporterConfig`/`OpentelemetryHttp batchExportTimeout = "30s" //(7)! compression = "gzip" //(8)! exportUnsampledSpans = false //(9)! - retry { + retryPolicy { maxAttempts = 5 //(10)! initialBackoff = "1s" //(11)! maxBackoff = "5s" //(12)! @@ -103,22 +118,21 @@ Parameters described in the `OpentelemetryGrpcExporterConfig`/`OpentelemetryHttp } ``` - 1. URL from [OpenTelemetry](https://opentelemetry.io/docs/collector/) service collector (**mandatory**) - 2. Time to wait for connection to exporter - 3. Maximum time to wait for telemetry processing by collector - 4. Time between exporting telemetry to the collector - 5. Maximum number of telemetry within one export - 6. Maximum queue size of unsent telemetry - 7. Maximum waiting time for export - 8. Telemetry compression mechanism when exporting - 9. Whether to export unsampled telemetry - 10. Maximum number of export attempts - 11. Initial value of waiting time before next export attempt - 12. Maximum wait value before next export attempt - 13. Waiting delay value multiplier - 14. Additional telemetry attributes - -Translated with DeepL.com (free version) + 1. `OpenTelemetry Collector` endpoint for exporting traces (default: not specified, optional). `gRPC` usually uses `http://localhost:4317`, and `HTTP` usually uses `http://localhost:4318/v1/traces`. + 2. Timeout for establishing a connection to the exporter (default: not specified, optional). + 3. Maximum time to wait while the exporter sends data (default: `3s`). + 4. Delay between sending accumulated `Span` to the collector (default: `2s`). + 5. Maximum number of `Span` in one export batch (default: `512`). + 6. Maximum queue size for `Span` waiting to be sent (default: `2048`). + 7. Maximum time the `BatchSpanProcessor` waits for one accumulated batch to be exported; this is distinct from `exportTimeout`, which bounds a single `OTLP` request (default: `30s`). + 8. Data compression used during export, `gzip` or `none` (default: `gzip`). + 9. Whether to export `Span` that were not selected by `Sampler` (default: `false`). + 10. Maximum number of retry attempts (default: `5`). + 11. Initial delay before a retry attempt (default: `1s`). + 12. Maximum delay before a retry attempt (default: `5s`). + 13. Delay multiplier between retry attempts (default: `1.5`). + 14. `OpenTelemetry Resource` attributes added to exported `Span` (default: `{}`). + === ":simple-yaml: `YAML`" ```yaml @@ -133,7 +147,7 @@ Translated with DeepL.com (free version) batchExportTimeout: 30s #(7)! compression: gzip #(8)! exportUnsampledSpans: false #(9)! - retry: + retryPolicy: maxAttempts: 5 #(10)! initialBackoff: 1s #(11)! maxBackoff: 5s #(12)! @@ -143,26 +157,224 @@ Translated with DeepL.com (free version) service.namespace: kora ``` - 1. URL from [OpenTelemetry](https://opentelemetry.io/docs/collector/) service collector (**mandatory**) - 2. Time to wait for connection to exporter - 3. Maximum time to wait for telemetry processing by collector - 4. Time between exporting telemetry to the collector - 5. Maximum number of telemetry within one export - 6. Maximum queue size of unsent telemetry - 7. Maximum waiting time for export - 8. Telemetry compression mechanism when exporting - 9. Whether to export unsampled telemetry - 10. Maximum number of export attempts - 11. Initial value of waiting time before next export attempt - 12. Maximum wait value before next export attempt - 13. Waiting delay value multiplier - 14. Additional telemetry attributes - -Trace collection configuration parameters are described in modules that include trace collection, e.g. [HTTP server](http-server.md), [HTTP client](http-client.md), etc. + 1. `OpenTelemetry Collector` endpoint for exporting traces (default: not specified, optional). `gRPC` usually uses `http://localhost:4317`, and `HTTP` usually uses `http://localhost:4318/v1/traces`. + 2. Timeout for establishing a connection to the exporter (default: not specified, optional). + 3. Maximum time to wait while the exporter sends data (default: `3s`). + 4. Delay between sending accumulated `Span` to the collector (default: `2s`). + 5. Maximum number of `Span` in one export batch (default: `512`). + 6. Maximum queue size for `Span` waiting to be sent (default: `2048`). + 7. Maximum time the `BatchSpanProcessor` waits for one accumulated batch to be exported; this is distinct from `exportTimeout`, which bounds a single `OTLP` request (default: `30s`). + 8. Data compression used during export, `gzip` or `none` (default: `gzip`). + 9. Whether to export `Span` that were not selected by `Sampler` (default: `false`). + 10. Maximum number of retry attempts (default: `5`). + 11. Initial delay before a retry attempt (default: `1s`). + 12. Maximum delay before a retry attempt (default: `5s`). + 13. Delay multiplier between retry attempts (default: `1.5`). + 14. `OpenTelemetry Resource` attributes added to exported `Span` (default: `{}`). + +The example project uses environment substitution for the endpoint and overrides a few export parameters: + +===! ":material-code-json: `Hocon`" + + ```javascript + tracing { + exporter { + endpoint = ${METRIC_COLLECTOR_ENDPOINT} //(1)! + exportTimeout = "250s" + scheduleDelay = "50ms" + maxExportBatchSize = 10000 + } + attributes { + "service.name" = "kora-java-telemetry" + "service.namespace" = "kora" + } + } + ``` + + 1. Resolved from the `METRIC_COLLECTOR_ENDPOINT` environment variable, see [environment substitution](config.md#environment-variables). + +=== ":simple-yaml: `YAML`" + + ```yaml + tracing: + exporter: + endpoint: ${METRIC_COLLECTOR_ENDPOINT} #(1)! + exportTimeout: "250s" + scheduleDelay: "50ms" + maxExportBatchSize: 10000 + attributes: + service.name: "kora-java-telemetry" + service.namespace: "kora" + ``` + + 1. Resolved from the `METRIC_COLLECTOR_ENDPOINT` environment variable, see [environment substitution](config.md#environment-variables). + +## Automatic tracing { #automatic } + +Once one exporter module is added, Kora instruments its subsystems automatically: for every incoming request, outgoing call, message, query, or scheduled run it creates a `Span`, attaches it to the current `OpentelemetryContext`, nests it under the currently active `Span`, and propagates the trace context across service boundaries. +No annotations or manual code are required for these `Span`. + +For example, the `GET /text` controller from the telemetry example produces a `SERVER` span named `GET /text` automatically: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController + public final class SimpleController { + + @HttpRoute(method = HttpMethod.GET, path = "/text") + public HttpServerResponse get() { + return HttpServerResponse.of(200, HttpBody.plaintext("Hello world")); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController + class SimpleController { + + @HttpRoute(method = HttpMethod.GET, path = "/text") + fun get(): HttpServerResponse { + return HttpServerResponse.of(200, HttpBody.plaintext("Hello world")) + } + } + ``` + +The table below lists the subsystems instrumented by `OpentelemetryModule`, the resulting `Span` name and [kind](https://opentelemetry.io/docs/specs/otel/trace/api/#spankind), and the main attributes. +Attribute names follow the [OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/). + +| Subsystem | Span name | Kind | Key attributes | +|------------------------|------------------------------------------------|------------|-------------------------------------------------------------------------------------------------------------------------| +| HTTP server | ` `, e.g. `GET /text` | `SERVER` | `http.request.method`, `url.scheme`, `url.path`, `http.route`, `server.address`, `http.response.status_code` | +| HTTP client | ` ` | `CLIENT` | `http.request.method`, `server.address`, `server.port`, `url.scheme`, `url.full`, `http.response.status_code` | +| Database | query operation name | `CLIENT` | `db.system`, `db.user`, `db.statement` | +| Kafka consumer | `kafka.poll`, ` receive`, ` process` | `CONSUMER` | `messaging.system` = `kafka`, `messaging.operation`, `messaging.destination.name`, `messaging.kafka.message.offset` | +| Kafka producer | ` send`, `producer transaction` | `PRODUCER` / `INTERNAL` | `messaging.system` = `kafka`, `messaging.operation` = `publish`, `messaging.destination.name` | +| gRPC server | `/` | `SERVER` | `rpc.system` = `grpc`, `rpc.service`, `rpc.method`, `network.peer.address` | +| gRPC client | `` | `CLIENT` | `rpc.system` = `grpc`, `rpc.service`, `rpc.method`, `server.address`, `server.port` | +| S3 client | `S3 ` | `CLIENT` | `client.name`, `http.request.method`, `aws.s3.bucket`, `aws.s3.key`, `http.response.status_code` | +| SOAP client | `SOAP ` | `CLIENT` | `rpc.service`, `rpc.method`, `rpc.system` | +| JMS consumer | ` receive` | `CONSUMER` | `messaging.system` = `jms`, `messaging.destination.name`, `messaging.message.id` | +| Scheduling | ` ` | `INTERNAL` | `code.function`, `code.filepath` | +| Cache | `cache.call` | `INTERNAL` | `operation`, `cache`, `origin` | + +`Camunda` (BPMN engine, REST) and `Zeebe` worker subsystems are also instrumented when their modules are present. + +On failure Kora sets the span status to `ERROR` and records the exception via `Span#recordException`; on success the status is set to `OK`. + +## Module tracing configuration { #module-config } + +Tracing of each instrumented subsystem is configured under that module's `telemetry.tracing` section, described by `ru.tinkoff.kora.telemetry.common.TelemetryConfig.TracingConfig`. +Two options are available for every subsystem: + +- `enabled` (default: `true`) — turns the subsystem's spans on or off. Set to `false` to stop creating spans for a specific module without removing the exporter. +- `attributes` (default: `{}`) — a map of key/value pairs added to every span produced **by that module only**. These per-span attributes differ from the service-wide `tracing.attributes` (`Resource` attributes) that apply to all spans. + +The `telemetry.tracing` section lives at the same path as the module's own configuration, for example `httpServer.telemetry.tracing`, `db.telemetry.tracing`, `grpcServer.telemetry.tracing`, or `kafka..telemetry.tracing`. + +===! ":material-code-json: `Hocon`" + + ```javascript + httpServer { + telemetry { + tracing { + enabled = true //(1)! + attributes { //(2)! + "component" = "gateway" + } + } + } + } + db { + telemetry { + tracing { + enabled = false //(3)! + } + } + } + ``` + + 1. Enables tracing for the HTTP server (default: `true`). + 2. Per-span attributes added only to HTTP server spans (default: `{}`). + 3. Disables tracing for database queries (default: `true`). + +=== ":simple-yaml: `YAML`" + + ```yaml + httpServer: + telemetry: + tracing: + enabled: true #(1)! + attributes: #(2)! + component: "gateway" + db: + telemetry: + tracing: + enabled: false #(3)! + ``` + + 1. Enables tracing for the HTTP server (default: `true`). + 2. Per-span attributes added only to HTTP server spans (default: `{}`). + 3. Disables tracing for database queries (default: `true`). + +Module-specific tracing parameters are also described in those modules' own documentation, for example [HTTP server](http-server.md), [HTTP client](http-client.md), [gRPC server](grpc-server.md), [gRPC client](grpc-client.md), and [Kafka](kafka.md). + +## Context propagation { #propagation } + +Kora stitches distributed traces together with the [W3C Trace Context](https://www.w3.org/TR/trace-context/) standard: every instrumented client injects the current `traceparent` into the outgoing carrier, and every instrumented server extracts it to establish the parent of the new `Span`. +This happens automatically and requires no configuration: + +- **HTTP** — `traceparent` is injected into request headers by the HTTP client and extracted from request headers by the HTTP server. +- **Kafka** — `traceparent` is injected into record headers by the producer and extracted from record headers by the consumer (the per-record `process` span also links back to the batch `receive` span). +- **gRPC** — `traceparent` is injected into call metadata by the client and extracted from metadata by the server. +- **JMS** — `traceparent` is extracted from message properties by the consumer. + +Because the current `Span` lives in the Kora `Context`, any span you create manually (see [Synchronous tracing](#tracing-sync)) is automatically picked up and propagated by the instrumented clients called within the same context — you do not need to pass headers yourself. + +## Sampling { #sampling } + +The core tracing components are provided by `OpentelemetryTracingModule` as `@DefaultComponent`, which means each of them can be overridden by declaring your own component of the same type: + +- `Sampler` — decides which `Span` are recorded. The default is `Sampler.parentBased(Sampler.alwaysOn())`, i.e. record every root `Span` and follow the parent's decision for child `Span`. +- `IdGenerator` — generates trace and span identifiers. The default is `IdGenerator.random()`. +- `Supplier` — limits on attributes, events, and links per `Span`. The default is `SpanLimits.getDefault()`. + +To apply head-based sampling, override the `Sampler` factory method in your application, for example to record roughly 10% of root traces: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends OpentelemetryGrpcExporterModule { + + @Override + default Sampler opentelemetryTracingSampler() { + return Sampler.parentBased(Sampler.traceIdRatioBased(0.1)); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : OpentelemetryGrpcExporterModule { + + override fun opentelemetryTracingSampler(): Sampler { + return Sampler.parentBased(Sampler.traceIdRatioBased(0.1)) + } + } + ``` + +The `exportUnsampledSpans` export option controls whether `Span` that were **not** selected by the `Sampler` are still sent to the collector; it is `false` by default, so only sampled `Span` are exported. ## Tracing context { #tracing-context } -Obtain the current tracing `Span`, you can use the `getSpan` method in `OpentelemetryContext`: +To get the current `Span`, use the `getSpan` method on `OpentelemetryContext`: ===! ":fontawesome-brands-java: `Java`" @@ -173,10 +385,10 @@ Obtain the current tracing `Span`, you can use the `getSpan` method in `Opentele === ":simple-kotlin: `Kotlin`" ```kotlin - val span = OpentelemetryContext.getSpan(); + val span = OpentelemetryContext.getSpan() ``` -Obtain the current trace ID, you can use the `getTraceId()` method in `OpentelemetryContext`: +To get the current trace identifier, use the `getTraceId()` method on `OpentelemetryContext`: ===! ":fontawesome-brands-java: `Java`" @@ -187,13 +399,30 @@ Obtain the current trace ID, you can use the `getTraceId()` method in `Opentelem === ":simple-kotlin: `Kotlin`" ```kotlin - val traceId = OpentelemetryContext.getTraceId(); + val traceId = OpentelemetryContext.getTraceId() ``` -## Tracing sync { #tracing-sync } +If there is no current `Span`, both methods return `null`. +If you need an invalid placeholder value from `OpenTelemetry`, use `getSpanOrInvalid()` and `getTraceIdOrInvalid()`, which return `Span.getInvalid()` and its all-zero trace identifier instead of `null`. -In addition to automatically created spans, you can use the `Tracer` object from the dependency container. -You can create a span with the current one in parent as follows: +For manual span management, `OpentelemetryContext` also exposes an instance API used together with the Kora `Context`: + +- `OpentelemetryContext.get(ctx)` — returns the `OpentelemetryContext` stored in the given Kora `Context` (creating an empty one if absent). +- `OpentelemetryContext.set(ctx, otctx)` — stores the `OpentelemetryContext` in the Kora `Context` and updates the `traceId`/`spanId` in `MDC`. +- `otctx.add(span)` — returns a new `OpentelemetryContext` with the given `Span` (or any `ImplicitContextKeyed`) added as the current one. +- `otctx.getContext()` — returns the underlying `io.opentelemetry.context.Context`, used as the parent when building a nested `Span`. + +## Log correlation { #mdc } + +When `OpentelemetryContext.set` is called (by the automatic instrumentation or by your manual tracing code), Kora writes the current `traceId` and `spanId` into the [MDC](logging-slf4j.md). +When there is no current `Span`, these keys are removed from the `MDC` again. + +As a result, if the [logging module](logging-slf4j.md) is used, every log line emitted within a traced operation carries the `traceId` and `spanId`, which lets you jump from a log entry to the corresponding trace in your observability backend and back. + +## Synchronous tracing { #tracing-sync } + +In addition to `Span` automatically created by the framework, you can use the `Tracer` object from the application graph and create custom nested `Span`. +When tracing manually, it is important to save the current `OpentelemetryContext`, set the new context for the duration of the operation, and restore the original context in `finally`. ===! ":fontawesome-brands-java: `Java`" @@ -211,8 +440,8 @@ You can create a span with the current one in parent as follows: var ctx = ru.tinkoff.kora.common.Context.current(); var otctx = OpentelemetryContext.get(ctx); var span = tracer.spanBuilder("myOperation") - .setParent(otctx.getContext()) - .startSpan(); + .setParent(otctx.getContext()) + .startSpan(); OpentelemetryContext.set(ctx, otctx.add(span)); try { @@ -263,20 +492,21 @@ You can create a span with the current one in parent as follows: } } - fun doWork(): String = // do some work + fun doWork(): String { + // do some work + } } ``` -## Асинхронная трассировка { #async-tracing } +## Asynchronous tracing { #async-tracing } -In addition to spans automatically created by the framework, you can use the `Tracer` object from the container to create your own traces. The main challenge lies in correctly propagating the context `Fork` to another execution thread to ensure the trace works properly. - -To create a trace for asynchronous code inherited from the current parent context, you can do the following: +When switching to another execution thread, pass not only `Span`, but also the Kora context. +Use `Context.fork()` for `CompletionStage` and `Context.Kotlin.asCoroutineContext(ctx)` for `suspend` code. ===! ":fontawesome-brands-java: `Java`" - Example is shown for the `CompletableStage` asynchronous approach: - + Example for asynchronous code with `CompletionStage`: + ```java @Component public final class MyService { @@ -291,23 +521,23 @@ To create a trace for asynchronous code inherited from the current parent contex var ctx = ru.tinkoff.kora.common.Context.current().fork(); var otctx = OpentelemetryContext.get(ctx); var span = tracer.spanBuilder("myOperation") - .setParent(otctx.getContext()) - .startSpan(); + .setParent(otctx.getContext()) + .startSpan(); return CompletableFuture.supplyAsync(() -> { - OpentelemetryContext.set(ctx, otctx.add(span)); - var result = doWork(); - return result; - }) - .whenComplete((r, e) -> { - if (e != null) { - span.recordException(e); - span.setStatus(StatusCode.ERROR, e.getMessage()); - } else { - span.setStatus(StatusCode.OK); - } - span.end(); - }); + OpentelemetryContext.set(ctx, otctx.add(span)); + return doWork(); + }) + .whenComplete((r, e) -> { + if (e != null) { + span.recordException(e); + span.setStatus(StatusCode.ERROR, e.getMessage()); + } else { + span.setStatus(StatusCode.OK); + } + span.end(); + OpentelemetryContext.set(ctx, otctx); + }); } public String doWork() { @@ -318,7 +548,7 @@ To create a trace for asynchronous code inherited from the current parent contex === ":simple-kotlin: `Kotlin`" - Example is shown for the `suspend` asynchronous approach: + Example for asynchronous `suspend` code: ```kotlin @Component @@ -348,6 +578,8 @@ To create a trace for asynchronous code inherited from the current parent contex } } - fun doWork(): String = // do some work + fun doWork(): String { + // do some work + } } ``` diff --git a/mkdocs/docs/en/documentation/validation.md b/mkdocs/docs/en/documentation/validation.md index 73d8668..9a7f8df 100644 --- a/mkdocs/docs/en/documentation/validation.md +++ b/mkdocs/docs/en/documentation/validation.md @@ -1,10 +1,14 @@ --- -description: "Explains Kora validation annotations, class and method validation, argument and result validation, custom validators, and supported validation signatures. Use when working with @Validate, @Valid, @NotNull, @NotEmpty, @Pattern, @Range, @Size, @Validator." +description: "Explains Kora validation annotations, class and method validation, argument and result validation, custom validators, mapping validation failures to HTTP 400, and supported validation signatures. Use when working with @Validate, @Valid, @NotBlank, @NotEmpty, @Pattern, @Range, @Size, @Validator, ValidatorModule, ValidationModule." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora validation annotations, class and method validation, argument and result validation, custom validators, and supported validation signatures; key triggers include @Validate, @Valid, @NotNull, @NotEmpty, @Pattern, @Range, @Size, @Validator, ValidationModule." + use_when: "Use this file for Kora docs or implementation questions about Kora validation annotations, class and method validation, argument and result validation, custom validators, mapping ViolationException to HTTP 400, and supported validation signatures; key triggers include @Validate, @Valid, @NotBlank, @NotEmpty, @Pattern, @Range, @Size, @ValidatedBy, Validator, ValidatorFactory, ViolationException, ValidationHttpServerInterceptor, ValidatorModule, ValidationModule." --- -Module for validating classes/records and methods using annotations. +The Kora validation module checks models, method arguments, and method results using annotations. +For models, Kora generates a `Validator` at compile time, and for methods it applies the `@Validate` aspect that calls the required checks before or after method execution. + +Validation works without using `Reflection` at application runtime: object structure, nested fields, method signatures, and available validators are checked by annotation processors during the build. +Validation errors are returned as a list of `Violation` or thrown as `ViolationException`. For a step-by-step walkthrough before the reference details, see [Validation](../guides/validation.md). @@ -26,7 +30,7 @@ For a step-by-step walkthrough before the reference details, see [Validation](.. === ":simple-kotlin: `Kotlin`" [Dependency](general.md#dependencies) `build.gradle.kts`: - ```groovy + ```kotlin implementation("ru.tinkoff.kora:validation-module") ``` @@ -36,39 +40,73 @@ For a step-by-step walkthrough before the reference details, see [Validation](.. interface Application : ValidationModule ``` -## Validation annotations { #validation-annotations } +The module ships two mixin interfaces, and you pick one depending on whether the application serves `HTTP`: + +| Module | Artifact | Provides | Use when | +|--------|----------|----------|----------| +| `ValidatorModule` | `validation-common` | Generated `Validator` beans, all built-in constraint factories, and element validators (`Validator>`, `Validator>`, `Validator>`) | Libraries and non-`HTTP` applications, or when you handle `ViolationException` yourself | +| `ValidationModule` | `validation-module` | Everything from `ValidatorModule` **plus** the `ValidationHttpServerInterceptor` that maps `ViolationException` to an [HTTP 400 response](#validation-response-http) | `HTTP` services that should return `400` to clients automatically | + +`ValidationModule` extends `ValidatorModule`, so wiring `ValidationModule` also gives you everything the base module provides. +The dependency shown above (`validation-module`) is the right choice for an `HTTP` service; a library that only needs to generate validators can depend on `validation-common` and wire `ValidatorModule` instead. + +## Validation Annotations { #validation-annotations } + +Validation annotations tell Kora what to check on a field, method argument, or method result. +They can be applied directly, or nested validation can be triggered through `@Valid` when the type has a generated or manually provided `Validator`. + +!!! warning "Kora validation is not Jakarta Bean Validation" -Special validation annotations are used by Kora to validate fields/arguments, they represent simple checks. + Kora validation is **not** [Jakarta Bean Validation (JSR-380)](https://jakarta.ee/specifications/bean-validation/). + All Kora constraint annotations live in the `ru.tinkoff.kora.validation.common.annotation` package and are processed at compile time. + In particular, Kora ships **no** `@NotNull` constraint annotation: a value is required by default, and to make it optional you mark it with any `@Nullable` annotation (see [Optional Fields](#optional-fields)). + Kora does recognize a standard `@Nonnull` / `@NotNull` marker (from `javax.annotation`, `jakarta.annotation`, and similar packages) as an explicit not-`null` requirement, which matters mainly for [`JsonNullable`](#json-nullable) fields. -Available validation annotations: +The structural annotations that drive validation: -- `@NotEmpty` - Checks that the string is not empty -- `@NotBlank` - Checks that the string does not consist of empty characters -- `@Pattern` - Checks if the string matches Regular Expression (RegEx) -- `@Range` - Checks that the number is in the specified range -- `@Size` - Checks that a collection (List, Set, Map) or `String` has a size in the specified range. +- `@Valid` - on a class or `record` generates a `Validator` for that type; on a field, argument, or method result triggers nested validation through the `Validator` of the corresponding type. Applicable to types, fields, parameters, and methods. +- `@Validate` - marks a method whose arguments and/or result should be validated by the aspect; the `failFast` parameter controls stopping on the first error (default: `false`). Applicable to methods only. +- `@ValidatedBy` - links a custom constraint annotation with a `ValidatorFactory` that builds its `Validator` (see [Custom Validation Annotations](#custom-validation-annotations)). Applicable to annotation types only. -## Class validation { #class-validation } +The built-in constraint annotations and their parameters: -It is suggested to use the `@Valid` annotation to mark a class that needs a validator from the Kora framework. +| Annotation | Supported types | Parameters (defaults) | Description | +|------------|-----------------|-----------------------|-------------| +| `@NotBlank` | `String`, `CharSequence` | — | Value is not `null` and contains at least one non-whitespace character. | +| `@NotEmpty` | `String`, `CharSequence`, `Iterable`, `Collection`, `List`, `Set`, `Map` | — | Value is not `null` and not empty. | +| `@Pattern` | `String`, `CharSequence` | `value` (required, no default), `flags` (default: `0`) | Value matches the `value` regular expression; `flags` maps to [`java.util.regex.Pattern`](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/regex/Pattern.html#field.summary) flags. | +| `@Range` | `Short`, `Integer`, `Long`, `Float`, `Double`, `BigInteger`, `BigDecimal` | `from` (required, no default), `to` (required, no default), `boundary` (default: `INCLUSIVE_INCLUSIVE`) | Number lies within `[from, to]`; `boundary` controls whether the bounds are inclusive. | +| `@Size` | `String`, `CharSequence`, `Collection`, `List`, `Set`, `Map` | `min` (default: `0`), `max` (required, no default) | Size (length) of the value is within `min` and `max`. | -An example of a labeled class for validation looks like this: +!!! note + + Watch the required parameters: `@Size.max` has **no default**, so omitting it is a compile error; `@Range.from` and `@Range.to` are both required and are declared as `double`. + The `@Range.boundary` value is a `Range.Boundary` enum with the variants `EXCLUSIVE_EXCLUSIVE`, `INCLUSIVE_EXCLUSIVE`, `EXCLUSIVE_INCLUSIVE`, and `INCLUSIVE_INCLUSIVE`. + +## Class Validation { #class-validation } + +The `@Valid` annotation on a class or `record` tells Kora to create a `Validator` for that type. +The generated validator becomes a regular dependency graph component and can be injected by the `Validator` signature. ===! ":fontawesome-brands-java: `Java`" ```java @Valid - public record Foo(String number) { } + public record User(@NotBlank String id, + @Size(min = 3, max = 6) String name, + @Nullable String status) { } ``` === ":simple-kotlin: `Kotlin`" ```kotlin @Valid - data class Foo(val number: String) + data class User(@field:NotBlank val id: String, + @field:Size(min = 3, max = 6) val name: String, + val status: String?) ``` -A validator of that class will then be available in the dependency container: +A validator for this class will then be available in the dependency container: ===! ":fontawesome-brands-java: `Java`" @@ -76,10 +114,10 @@ A validator of that class will then be available in the dependency container: @Component public final class Example { - private final Validator fooValidator; - - public Example(Validator fooValidator) { - this.fooValidator = fooValidator; + private final Validator userValidator; + + public Example(Validator userValidator) { + this.userValidator = userValidator; } } ``` @@ -88,20 +126,21 @@ A validator of that class will then be available in the dependency container: ```kotlin @Component - class Example(val fooValidator: Validator) + class Example(val userValidator: Validator) ``` -Created validators can be implemented as dependencies in any component, in the examples above the validator for the `Foo` class, -can be implemented by its signature `Validator` as a component dependency and used manually for validation. +Generated validators can be injected as dependencies into any component. +In the example above, the validator for `User` is injected by the `Validator` signature and can be used manually. -The validator returns a list of violations after validation, they can be used to manually compose the error either -you can use the `validateAndThrow` method which throws a `ViolationException` exception in case of a validation error. +The `validate(...)` method returns a list of `Violation`. +You can process this list yourself or call `validateAndThrow(...)`, which throws `ViolationException` if there are violations. +See [Manual Validation](#manual-validation) for the full imperative API. -### Field validation { #field-validation } +### Field Validation { #field-validation } -It is expected to use a special provided validation [annotation](#validation-annotations) validation set for field validation. +Field validation uses the set of [annotations](#validation-annotations) provided by the module. -An example of an object marked up for validation looks like this: +An object marked for validation looks like this: ===! ":fontawesome-brands-java: `Java`" @@ -110,51 +149,50 @@ An example of an object marked up for validation looks like this: public record Foo(@NotEmpty String number) { } ``` - For Record classes, the syntax for accessing fields via Record-like getter contracts is used, - in the case of `Foo` and the `code` field, *getter* `code()` will be used in the created `Validator`. + For a `record`, fields are accessed through the methods of the `record` itself. + For `Foo` and the `number` field, the generated `Validator` will use the `number()` method. - For a regular class it is expected that Java *Getters* syntax will be used, for example for the `id` field *getter* `getId()` will be used, - where *getter* should have at least *package-private* visibility. + For a regular class, the `JavaBeans` syntax is used: for example, the `getId()` method will be used for the `id` field. + This method must have at least `package-private` visibility. === ":simple-kotlin: `Kotlin`" ```kotlin @Valid - Data class Foo(@field:NotEmpty val number: String) + data class Foo(@field:NotEmpty val number: String) ``` -#### Required fields { #required-fields } +#### Required Fields { #required-fields } -All fields are required (`NotNull`) by default, so `NotNull` checks will be created for all of them in the `Validator`. +All fields are considered required by default, so `null` checks are created for them. -#### Optional fields { #optional-fields } +#### Optional Fields { #optional-fields } ===! ":fontawesome-brands-java: `Java`" - In order to specify a field as not required, you need to mark it with any `@Nullable` annotation, - **will not** create a *null* check for such a field: + To mark a field as optional, annotate it with any `@Nullable` annotation. + For such a field, a `null` check **will not** be created: ```java @Valid public record Foo(@Nullable String number) { } //(1)! ``` - 1. Any `@Nullable` annotation will do, such as `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / etc. + 1. Any `@Nullable` annotation is suitable, for example `javax.annotation.Nullable`, `jakarta.annotation.Nullable`, or `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - It is expected to use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark such a field as Nullable: + To mark a field as optional, use [`Kotlin Nullability`](https://kotlinlang.org/docs/null-safety.html) syntax and add `?` to the field type. + For such a field, a `null` check **will not** be created: ```kotlin @Valid data class Foo(val number: String?) ``` -#### Embedded fields { #embedded-fields } +#### Nested Fields { #embedded-fields } -In order to validate fields of complex objects for which validators are created (or provided independently), -or fields that are not supported by standard validation tools, -the `@Valid` annotation is supposed to be used: +Use `@Valid` to validate nested objects that have generated or manually provided validators. ===! ":fontawesome-brands-java: `Java`" @@ -176,71 +214,229 @@ the `@Valid` annotation is supposed to be used: data class Bar(val number: String) ``` -In the example above, a `Validator` validator would be created for `Bar` and a `Validator` would be created for `Foo`, -where when the `Validator` validator is called, the validator for `Validator` will be called internally. +In the example above, `Validator` will be created for `Bar`, and `Validator` will be created for `Foo`. +When `Validator` is called, it will call `Validator` internally. + +#### Collection Validation { #collection-validation } + +`@Valid` on a `List`, `Set`, or `Collection` field validates **every element** through the element's `Validator`. +The `ValidatorModule` provides these element validators out of the box (`Validator>`, `Validator>`, `Validator>`), so no extra wiring is needed. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Valid + public record Foo(@Valid List bars) { } + + @Valid + public record Bar(@NotBlank String number) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Valid + data class Foo(@field:Valid val bars: List) + + @Valid + data class Bar(@field:NotBlank val number: String) + ``` + +Each `Bar` in the list is validated, and the violation path is indexed by element position, for example `bars[0].number`. +Constraints such as [`@Size`](#validation-annotations) can be combined with `@Valid` on the same collection to check both the collection size and each element. + +#### `Sealed` Hierarchies { #sealed-validation } + +Kora can create a `Validator` for `sealed` hierarchies. +If `@Valid` is placed on a `sealed` type, the generated validator determines the actual subtype and calls the validator for the matching final implementation. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Valid + public sealed interface Command permits CreateCommand { + + @Valid + record CreateCommand(@NotBlank String name) implements Command { } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Valid + sealed interface Command { + + @Valid + data class CreateCommand(@field:NotBlank val name: String) : Command + } + ``` -### Validation options { #validation-options } +#### `JsonNullable` { #json-nullable } -There are two types of validation: +For `JsonNullable`, Kora validates the `T` value inside the container. +If `JsonNullable` is in the `undefined` state, regular value checks are not performed. +Use `@NotNull` or `@Nonnull` to disallow `undefined` or `null`. -- `Full` - all fields that are just marked up are checked, all possible validation errors are collected - and only then an exception is thrown. (**Default behavior**) -- `FailFast` - exception is thrown on the first validation error encountered. +#### Validation Options { #validation-options } -Example of FailFast validation: -```java -ValidatorContext context = ValidationContext.builder().failFast(true).build(); -List violations = fooValidator.validate(value,context); -``` +There are two validation modes, selected through the `ValidationContext` passed to `validate(...)`: -## Method validation { #method-validation } +- `Full` - all marked fields are checked, all possible validation errors are collected, and only then a list of violations is returned or an exception is thrown. This is the default behavior. +- `FailFast` - validation stops on the first found error. -It is expected to use a special provided set of [annotations](#validation-annotations) validation for validating method arguments and result. +A `ValidationContext` can be built in several equivalent ways: -### Argument validation { #argument-validation } +- `ValidationContext.builder().build()` - default `Full` context (same as calling `validate(value)` without a context). +- `ValidationContext.full()` - explicit `Full` context. +- `ValidationContext.failFast()` - `FailFast` context. +- `ValidationContext.builder().failFast(true).build()` - builder form of `FailFast`. -It is required to use the `@Validate` annotation over the method to validate method arguments: +Example of `FailFast` validation: + +===! ":fontawesome-brands-java: `Java`" + + ```java + ValidationContext context = ValidationContext.failFast(); + List violations = userValidator.validate(value, context); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val context = ValidationContext.failFast() + val violations = userValidator.validate(value, context) + ``` + +### Manual Validation { #manual-validation } + +A generated `Validator` is an ordinary component, so it can be injected and called directly — for example in a service that is not an `HTTP` controller, or when you want to inspect violations instead of throwing. ===! ":fontawesome-brands-java: `Java`" ```java @Component - public class SomeService { + public final class UserService { + + private final Validator validator; + + public UserService(Validator validator) { + this.validator = validator; + } + + public void process(User user) { + List violations = validator.validate(user); //(1)! + if (!violations.isEmpty()) { + Violation first = violations.get(0); + throw new IllegalStateException(first.path().full() + ": " + first.message()); //(2)! + } + } + } + ``` + + 1. `validate(value)` collects **all** violations; use `validate(value, context)` to pass validation options. + 2. Each `Violation` exposes `path()` and `message()`. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class UserService(private val validator: Validator) { + + fun process(user: User) { + val violations = validator.validate(user) //(1)! + if (violations.isNotEmpty()) { + val first = violations.first() + throw IllegalStateException("${first.path().full()}: ${first.message()}") //(2)! + } + } + } + ``` + + 1. `validate(value)` collects **all** violations; use `validate(value, context)` to pass validation options. + 2. Each `Violation` exposes `path()` and `message()`. + +The `Validator` contract offers the following methods: + +- `validate(value)` / `validate(value, context)` - return a `List` that is empty when the value is valid (a `null` value fails with a violation). +- `validateAndThrow(value)` / `validateAndThrow(value, context)` - throw `ViolationException` when any violation occurs, and do nothing otherwise. + +When a `ViolationException` is caught, `getViolations()` returns the aggregated `List`, and `getMessage()` returns a preformatted multi-line summary of every violation path and message. + +## Method Validation { #method-validation } + +Method argument and result validation uses the `@Validate` aspect and the set of [annotations](#validation-annotations) provided by the module. +Kora generates aspect code at compile time, so a class with such methods must support aspect application. + +### Argument Validation { #argument-validation } + +To validate method arguments, use the `@Validate` annotation on the method and annotate the arguments with the required [constraints](#validation-annotations). +Arguments can be validated by constraint annotations directly, or by `@Valid` when the argument type has its own `Validator`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class ArgumentValidator { + + @Valid + public record User(@NotBlank String id, + @Size(min = 3, max = 6) String name, + @Nullable String status) { } @Validate - public int validate(@NotEmpty String argument) { - return 1; + public int calculate(@Valid User user, //(1)! + @Range(from = 1, to = 900) int weight, //(2)! + @Pattern("ME\\d+") String code) { //(3)! + return Integer.parseInt(code.substring(2)); } } ``` + 1. Nested validation through `Validator`. + 2. Numeric range constraint applied directly to the argument. + 3. Regular expression constraint applied directly to the argument. + === ":simple-kotlin: `Kotlin`" ```kotlin @Component - open class SomeService { + open class ArgumentValidator { + + @Valid + data class User(@field:NotBlank val id: String, + @field:Size(min = 3, max = 6) val name: String, + val status: String?) @Validate - fun validate(@NotEmpty argument: String): Int { - return 1 + fun calculate(@Valid user: User, //(1)! + @Range(from = 1.0, to = 900.0) weight: Int, //(2)! + @Pattern("ME\\d+") code: String): Int { //(3)! + return code.substring(2).toInt() } } ``` -#### Required arguments { #required-arguments } + 1. Nested validation through `Validator`. + 2. Numeric range constraint applied directly to the argument. + 3. Regular expression constraint applied directly to the argument. + +If any argument fails validation, the aspect throws `ViolationException` **before** the method body runs. -All arguments are required (`NotNull`) by default, so `NotNull` checks will be created for all of them. +#### Required Arguments { #required-arguments } -#### Optional arguments { #optional-arguments } +All arguments are considered required by default, so `null` checks are created for them. + +#### Optional Arguments { #optional-arguments } ===! ":fontawesome-brands-java: `Java`" - In order to specify an argument as not required requires marking it with any `@Nullable` annotation, - **will not** create a *null* check for such an argument: + To mark an argument as optional, annotate it with any `@Nullable` annotation. + For such an argument, a `null` check **will not** be created: ```java @Component - Public class SomeService { + public class SomeService { @Validate public int validate(@Nullable String argument) { //(1)! @@ -249,11 +445,12 @@ All arguments are required (`NotNull`) by default, so `NotNull` checks will be c } ``` - 1. Any `@Nullable` annotation will do, such as `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / etc. + 1. Any `@Nullable` annotation is suitable, for example `javax.annotation.Nullable`, `jakarta.annotation.Nullable`, or `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - It is expected to use the [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) syntax and mark such an argument as Nullable: + To mark an argument as optional, use [`Kotlin Nullability`](https://kotlinlang.org/docs/null-safety.html) syntax and add `?` to the argument type. + For such an argument, a `null` check **will not** be created: ```kotlin @Component @@ -266,11 +463,9 @@ All arguments are required (`NotNull`) by default, so `NotNull` checks will be c } ``` -#### Embedded arguments { #embedded-arguments } +#### Nested Arguments { #embedded-arguments } -In order to validate fields of complex objects for which validators are created (or provided independently), -or fields that are not supported by standard validation tools, -`@Valid` annotation is supposed to be used: +Use `@Valid` to validate nested arguments that have generated or manually provided validators. ===! ":fontawesome-brands-java: `Java`" @@ -304,13 +499,64 @@ or fields that are not supported by standard validation tools, } ``` -In the example above, a `Validator` validator would be created for `Bar` and a `Validator` would be created for `Foo`, -where when the `Validator` validator is called, the validator for `Validator` will be called internally. +In the example above, `Validator` will be created for `Foo`. +When the method is called, the `@Validate` aspect will call this validator for the `argument` argument. -### Result validation { #result-validation } +### Result Validation { #result-validation } -In order to validate the result of a method, it is required to use the `@Validate` annotation over the method and mark it up with the appropriate [annotations](#validation-annotations). -In order to check that the value is not `null`, you need to use any `@NotNull/@Nonnull` annotation: +To validate a method result, use the `@Validate` annotation on the method and annotate the result with the corresponding [annotations](#validation-annotations). +Place `@Valid` on the method to run nested validation through the return type's `Validator`. +To require that the result is not `null`, use any `@Nonnull` or `@NotNull` annotation. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class ResultValidator { + + @Valid + public record User(@NotBlank String id, + @Size(min = 3, max = 6) String name, + @Nullable @Size(min = 1, max = 10) String status) { } //(1)! + + @Valid //(3)! + @Validate //(2)! + public User create(String name, String status) { + return new User(UUID.randomUUID().toString(), name, status); + } + } + ``` + + 1. Constraints can be stacked: `status` is optional (`@Nullable`), but when present its length must be within `@Size`. + 2. Indicates that the method requires validation. + 3. Indicates that the result should be validated through the `Validator` of the return type. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + open class ResultValidator { + + @Valid + data class User(@field:NotBlank val id: String, + @field:Size(min = 3, max = 6) val name: String, + @field:Size(min = 1, max = 10) val status: String?) //(1)! + + @Valid //(3)! + @Validate //(2)! + fun create(name: String, status: String): User { + return User(UUID.randomUUID().toString(), name, status) + } + } + ``` + + 1. Constraints can be stacked: `status` is optional (nullable), but when present its length must be within `@Size`. + 2. Indicates that the method requires validation. + 3. Indicates that the result should be validated through the `Validator` of the return type. + +The result validation runs **after** the method body, on its return value; if it fails, the aspect throws `ViolationException` instead of returning the value. + +Constraints can also be applied to the result container itself. For example, a collection result can be size-checked and its elements validated at the same time: ===! ":fontawesome-brands-java: `Java`" @@ -330,9 +576,9 @@ In order to check that the value is not `null`, you need to use any `@NotNull/@N } ``` - 1. Indicates that the method requires validation - 2. Indicates that the result requires validation with a validator from the return value type - 3. Standard validation annotation + 1. Indicates that the method requires validation. + 2. Indicates that the result should be validated through the `Validator` of the return type. + 3. Standard validation annotation. === ":simple-kotlin: `Kotlin`" @@ -349,19 +595,18 @@ In order to check that the value is not `null`, you need to use any `@NotNull/@N } ``` - 1. Indicates that the method requires validation - 2. Indicates that the result requires validation with a validator from the return value type - 3. Standard validation annotation + 1. Indicates that the method requires validation. + 2. Indicates that the result should be validated through the `Validator` of the return type. + 3. Standard validation annotation. -### Validation options { #validation-options-2 } +### Validation Options { #validation-options-2 } -There are two types of validation: +There are two validation modes: -- `Full` - all fields that are just marked up are validated, all possible validation errors are collected - and only then an exception is thrown. (**Default behavior**) -- `FailFast` - exception is thrown on the first validation error encountered. +- `Full` - all marked arguments and the result are checked, all possible validation errors are collected, and only then an exception is thrown. This is the default behavior. +- `FailFast` - an exception is thrown on the first found error. -Example of FailFast validation: +Example of `FailFast` validation: ===! ":fontawesome-brands-java: `Java`" @@ -380,18 +625,193 @@ Example of FailFast validation: ```kotlin @Component - class SomeService { + open class SomeService { @Validate(failFast = true) fun validate(@NotEmpty c2: String): Int = 1 } ``` -## Custom validation annotations { #custom-validation-annotations } +## Validation HTTP Response { #validation-response-http } + +When a Kora `HTTP` service uses the `ValidationModule` (from the `validation-module` artifact), a failed validation can be turned into an `HTTP` `400` response automatically instead of an uncaught error. + +This is handled by the `ValidationHttpServerInterceptor` — an [HTTP server interceptor](http-server.md#interceptors) that catches `ViolationException` thrown by the `@Validate` aspect (including an exception wrapped in `CompletionException` for asynchronous signatures) and produces the response. +By default it returns status `400` with the `ViolationException` [message](#manual-validation) as a plain-text body; a custom [response mapper](#validation-response-custom) can replace that. + +Global interceptors are collected by the `@Tag(HttpServerModule.class)` tag (see [Interceptors](http-server.md#interceptors)), so the interceptor must be provided **with that tag** to apply to every route: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends + ValidationModule, //(1)! + UndertowHttpServerModule, + JsonModule { + + @Tag(HttpServerModule.class) //(2)! + default ValidationHttpServerInterceptor validationHttpServerInterceptor(@Nullable ViolationExceptionHttpServerResponseMapper mapper) { + return new ValidationHttpServerInterceptor(mapper); //(3)! + } + } + ``` + + 1. `ValidationModule` extends `ValidatorModule` and provides the `ValidationHttpServerInterceptor` and `ViolationExceptionHttpServerResponseMapper` wiring. + 2. Registers the interceptor as a **global** HTTP server interceptor. + 3. Passing `null` as the mapper keeps the default `400` plain-text response. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : ValidationModule, //(1)! + UndertowHttpServerModule, + JsonModule { + + @Tag(HttpServerModule::class) //(2)! + fun validationInterceptor(mapper: ViolationExceptionHttpServerResponseMapper?): ValidationHttpServerInterceptor { + return ValidationHttpServerInterceptor(mapper) //(3)! + } + } + ``` + + 1. `ValidationModule` extends `ValidatorModule` and provides the `ValidationHttpServerInterceptor` and `ViolationExceptionHttpServerResponseMapper` wiring. + 2. Registers the interceptor as a **global** HTTP server interceptor. + 3. Passing `null` as the mapper keeps the default `400` plain-text response. + +A `@Validate`-annotated controller method then produces a `400` for the client whenever its arguments or result fail validation, with no per-controller wiring: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record UserRequest(@NotBlank @Size(min = 2, max = 100) String name, + @NotBlank @Pattern("^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$") String email) { } + + @Component + @HttpController + public final class UserController { + + @HttpRoute(method = HttpMethod.POST, path = "/users") + @Validate //(1)! + @Json + public UserResponse createUser(@Valid @Json UserRequest request) { //(2)! + // request is already validated here + } + } + ``` + + 1. Enables argument (and result) validation for this route. + 2. Nested validation of the request body; a violation yields `HTTP` `400` before the body runs. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class UserRequest(@field:NotBlank @field:Size(min = 2, max = 100) val name: String, + @field:NotBlank @field:Pattern("^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$") val email: String) + + @Component + @HttpController + class UserController { + + @HttpRoute(method = HttpMethod.POST, path = "/users") + @Validate //(1)! + @Json + fun createUser(@Valid @Json request: UserRequest): UserResponse { + // request is already validated here + } + } + ``` + + 1. Enables argument (and result) validation for this route. + 2. Nested validation of the request body; a violation yields `HTTP` `400` before the body runs. + +### Custom Response { #validation-response-custom } + +To control the status, headers, or body of the response — for example, to return a structured `JSON` error instead of the default plain text — provide a `ViolationExceptionHttpServerResponseMapper` component. +Its `apply(request, exception)` method returns the `HttpServerResponse` to send; returning `null` falls back to the default `400` plain-text response. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json //(1)! + public record ValidationErrorResponse(String code, String message, List errors) { } + + @Json + public record ValidationErrorDetails(String field, String message) { } + + @KoraApp + public interface Application extends + ValidationModule, + UndertowHttpServerModule, + JsonModule { + + default ViolationExceptionHttpServerResponseMapper violationExceptionMapper(JsonWriter writer) { + return (request, exception) -> { + var errors = exception.getViolations().stream() //(2)! + .map(v -> new ValidationErrorDetails(v.path().full(), v.message())) + .toList(); + var body = new ValidationErrorResponse("VALIDATION_ERROR", "Validation failed", errors); + return HttpServerResponse.of(400, HttpBody.json(writer.toByteArrayUnchecked(body))); //(3)! + }; + } + + @Tag(HttpServerModule.class) + default ValidationHttpServerInterceptor validationHttpServerInterceptor(ViolationExceptionHttpServerResponseMapper mapper) { + return new ValidationHttpServerInterceptor(mapper); + } + } + ``` + + 1. Serialized with the [JSON module](json.md). + 2. `ViolationException.getViolations()` returns every `Violation`; `path().full()` is the dotted path (e.g. `customer.address.city`). + 3. Any `HttpServerResponse` may be returned; returning `null` would fall back to the default `400`. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json //(1)! + data class ValidationErrorResponse(val code: String, val message: String, val errors: List) + + @Json + data class ValidationErrorDetails(val field: String, val message: String) + + @KoraApp + interface Application : ValidationModule, + UndertowHttpServerModule, + JsonModule { + + fun violationExceptionMapper(writer: JsonWriter): ViolationExceptionHttpServerResponseMapper { + return ViolationExceptionHttpServerResponseMapper { request, exception -> + val errors = exception.violations.map { //(2)! + ValidationErrorDetails(it.path().full(), it.message()) + } + val body = ValidationErrorResponse("VALIDATION_ERROR", "Validation failed", errors) + HttpServerResponse.of(400, HttpBody.json(writer.toByteArrayUnchecked(body))) //(3)! + } + } + + @Tag(HttpServerModule::class) + fun validationInterceptor(mapper: ViolationExceptionHttpServerResponseMapper): ValidationHttpServerInterceptor { + return ValidationHttpServerInterceptor(mapper) + } + } + ``` + + 1. Serialized with the [JSON module](json.md). + 2. `ViolationException.getViolations()` returns every `Violation`; `path().full()` is the dotted path (e.g. `customer.address.city`). + 3. Any `HttpServerResponse` may be returned; returning `null` would fall back to the default `400`. + +## Custom Validation Annotations { #custom-validation-annotations } + +A custom validation annotation is needed when the standard checks are not enough. +It connects an annotation with a `ValidatorFactory`, and the factory creates a `Validator` for a specific value type. -Creating your custom annotation requires: +To create a custom annotation: -1) Create an inheritor of `Validator`: +1. Create a `Validator` implementation: ===! ":fontawesome-brands-java: `Java`" @@ -417,7 +837,7 @@ Creating your custom annotation requires: ```kotlin class MyValidStringValidator : Validator { - fun validate(value: String?, context: ValidationContext): List { + override fun validate(value: String?, context: ValidationContext): List { if (value == null) { return listOf(context.violates("Should be not empty, but was null")) } else if (value.isEmpty()) { @@ -428,7 +848,7 @@ Creating your custom annotation requires: } ``` -2) Create `ValidatorFactory` implementation: +2. Create a `ValidatorFactory` subtype: ===! ":fontawesome-brands-java: `Java`" @@ -442,7 +862,7 @@ Creating your custom annotation requires: interface MyValidValidatorFactory : ValidatorFactory ``` -3) Register the inheritor of `ValidatorFactory` as a component: +3. Register the `ValidatorFactory` as a component: ===! ":fontawesome-brands-java: `Java`" @@ -473,7 +893,7 @@ Creating your custom annotation requires: ``` -4) Create a validation annotation and annotate it `@ValidatedBy` with the previously created `ValidatorFactory` inheritor: +4. Create a validation annotation and mark it with `@ValidatedBy` using the previously created `ValidatorFactory` subtype: ===! ":fontawesome-brands-java: `Java`" @@ -493,7 +913,7 @@ Creating your custom annotation requires: annotation class MyValid ``` -5) Annotate field/argument/result: +5. Mark a field, argument, or result with the new annotation: ===! ":fontawesome-brands-java: `Java`" @@ -509,27 +929,89 @@ Creating your custom annotation requires: data class Foo(@field:MyValid val number: String) ``` +### Parameterized Constraints { #parameterized-constraints } + +A custom constraint annotation may declare parameters. +When it does, its `ValidatorFactory` subtype must declare a `create(...)` method whose parameter list matches the annotation attributes (the **same number of parameters, in declaration order**). +Kora reads the annotation values (with defaults applied) at compile time and passes them into that `create(...)` method; if no matching `create(...)` overload exists, the build fails. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Retention(RetentionPolicy.CLASS) + @Target({ElementType.FIELD, ElementType.PARAMETER}) + @ValidatedBy(PrefixedValidatorFactory.class) + public @interface Prefixed { + + String value(); //(1)! + } + + public interface PrefixedValidatorFactory extends ValidatorFactory { + + @Override + default Validator create() { //(2)! + throw new UnsupportedOperationException("Prefix is required"); + } + + Validator create(String prefix); //(3)! + } + ``` + + 1. A single annotation attribute. + 2. The inherited no-argument factory method is not usable for this constraint. + 3. Matching single-parameter `create(...)`; Kora passes `value()` into `prefix`. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Retention(AnnotationRetention.RUNTIME) + @Target(AnnotationTarget.FIELD, AnnotationTarget.PROPERTY, AnnotationTarget.VALUE_PARAMETER) + @ValidatedBy(PrefixedValidatorFactory::class) + annotation class Prefixed(val value: String) //(1)! + + interface PrefixedValidatorFactory : ValidatorFactory { + + override fun create(): Validator = //(2)! + throw UnsupportedOperationException("Prefix is required") + + fun create(prefix: String): Validator //(3)! + } + ``` + + 1. A single annotation attribute. + 2. The inherited no-argument factory method is not usable for this constraint. + 3. Matching single-parameter `create(...)`; Kora passes `value` into `prefix`. + +The factory is registered as a component exactly like the parameterless case (step 3 above). +This is the same mechanism the built-in constraints use, and their public factory interfaces expose reusable overloads that a custom factory can delegate to: + +- `RangeValidatorFactory` - `create(double from, double to)` and `create(double from, double to, Range.Boundary boundary)`. +- `SizeValidatorFactory` - `create(int to)` and `create(int from, int to)`. +- `PatternValidatorFactory` - `create(String pattern)` and `create(String pattern, int flags)`. +- `NotEmptyValidatorFactory` and `NotBlankValidatorFactory` - the parameterless `create()`. + ## Signatures { #signatures } -Available signatures for repository methods out of the box: +Method signatures supported by the `@Validate` aspect out of the box: ===! ":fontawesome-brands-java: `Java`" - Class must be non `final` in order for aspects to work. + The class must not be `final` for aspects to work. - The `T` refers to the type of the return value. + `T` means the return value type. - `T myMethod()` - `Optional myMethod()` - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (require [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (requires [dependency](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - Class must be `open` in order for aspects to work. + The class must be `open` for aspects to work. - By `T` we mean the type of the return value, either `T?`, or `Unit`. + `T` means the return value type, `T?`, or `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (require [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) + - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (requires [dependency](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) as `implementation`) diff --git a/mkdocs/docs/en/guides/database-cassandra.md b/mkdocs/docs/en/guides/database-cassandra.md index ab46e73..90c11f2 100644 --- a/mkdocs/docs/en/guides/database-cassandra.md +++ b/mkdocs/docs/en/guides/database-cassandra.md @@ -244,7 +244,7 @@ Update your Application interface to include the Cassandra module. } ``` -## Database entity { #entity-db } +## Database entity { #view-db } Replace the old in-memory storage model with a Cassandra DAO model used by repository mappings. diff --git a/mkdocs/docs/en/guides/database-jdbc-advanced.md b/mkdocs/docs/en/guides/database-jdbc-advanced.md index 9da464d..891f84e 100644 --- a/mkdocs/docs/en/guides/database-jdbc-advanced.md +++ b/mkdocs/docs/en/guides/database-jdbc-advanced.md @@ -246,7 +246,7 @@ Make sure the application interface includes the JDBC and Flyway modules. The HT Repositories do not create database infrastructure themselves. They depend on the `JdbcDatabaseModule` graph components. Flyway is also part of the graph, so migrations run before the application starts serving requests. -## New Entity { #new-entity } +## View { #view-new } Start with the simplest database model: a task row as the application writes it. Do not add read projections yet. At this point we only need the columns that are supplied during insert. diff --git a/mkdocs/docs/en/guides/database-jdbc.md b/mkdocs/docs/en/guides/database-jdbc.md index c2fcb96..c4b956c 100644 --- a/mkdocs/docs/en/guides/database-jdbc.md +++ b/mkdocs/docs/en/guides/database-jdbc.md @@ -251,7 +251,7 @@ Update your Application interface to include JDBC and Flyway modules. } ``` -## Database entity { #entity-db } +## Database entity { #view-db } Replace the old in-memory `User` storage model with JDBC DAO model used by repository mappings. diff --git a/mkdocs/docs/en/guides/dependency-injection-introduction.md b/mkdocs/docs/en/guides/dependency-injection-introduction.md index 67fb18a..04a7850 100644 --- a/mkdocs/docs/en/guides/dependency-injection-introduction.md +++ b/mkdocs/docs/en/guides/dependency-injection-introduction.md @@ -1569,7 +1569,7 @@ Classes annotated with `@Component` are automatically registered if they meet th - Final class (unless AOP aspects applied) - Constructor parameters become dependencies -### Basic Factory Methods { #basic-factory-methods } +### Basic Factory Methods { #method-factory-basics } Default methods in `@KoraApp` or `@Module` interfaces that return components: diff --git a/mkdocs/docs/en/guides/dependency-injection.md b/mkdocs/docs/en/guides/dependency-injection.md index 9e0904f..299797c 100644 --- a/mkdocs/docs/en/guides/dependency-injection.md +++ b/mkdocs/docs/en/guides/dependency-injection.md @@ -1305,7 +1305,7 @@ intact. **Why we need it**: libraries should provide safe defaults, but applications must keep final control over business-facing behavior. This matches [Dependency Injection with Kora: Standard factory](dependency-injection-introduction.md#defaultcomponent-factory), [@DefaultComponent](dependency-injection-introduction.md#defaultcomponent) -and [Container documentation: Standard factory](../documentation/container.md#standard-factory). +and [Container documentation: Standard factory](../documentation/container.md#default-factory). **What we are emulating**: application-specific customization of a shared library notifier without forking or rewriting the entire module. diff --git a/mkdocs/docs/ru/documentation/cache.md b/mkdocs/docs/ru/documentation/cache.md index 51e90e0..d5df9fe 100644 --- a/mkdocs/docs/ru/documentation/cache.md +++ b/mkdocs/docs/ru/documentation/cache.md @@ -4,14 +4,16 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora cache module, cache annotations, Caffeine and Redis cache backends, cache key mapping, telemetry, invalidation, and async cache signatures; key triggers include @Cache, @Cacheable, @CachePut, @CacheInvalidate, CaffeineCacheModule, RedisCacheModule, CacheKeyMapper, LoadableCache." --- -Модуль для создания кешей на основе [Caffeine](https://github.com/ben-manes/caffeine) или [Redis](https://redis.io/docs/about/) -с помощью аннотаций в декларативном стиле, так и использование их императивном стиле. +Модуль предоставляет типизированные кэши для хранения результатов вычислений и повторно используемых данных, +чтобы дорогостоящие операции не приходилось выполнять при каждом обращении. Кэш можно использовать декларативно через аннотации над методами +или императивно через внедряемый интерфейс, а в качестве хранилищ доступны локальный `Caffeine` и внешний `Redis`. +Локальный `Caffeine` полезен для быстрого внутрипроцессного хранения, а `Redis` подходит для общего кэша, используемого несколькими экземплярами приложения. -Если нужен пошаговый разбор перед справочным описанием, смотрите [Кеширование](../guides/cache.md) и [Многоуровневое кеширование](../guides/cache-multi-level.md). +Если нужен пошаговый разбор перед справочным описанием, смотрите [Кэш](../guides/cache.md) и [Многоуровневый кэш](../guides/cache-multi-level.md). ## Caffeine { #caffeine } -Реализация на основе библиотеки [Caffeine](https://github.com/ben-manes/caffeine) для кэша внутри памяти приложения. +Реализация на основе библиотеки [Caffeine](https://github.com/ben-manes/caffeine) для кэша приложения в оперативной памяти. ### Подключение { #dependency } @@ -43,9 +45,9 @@ agent: ### Конфигурация { #configuration } -Пример полной конфигурации для `mycache.config` кэша, параметры описаны в классе `CaffeineCacheConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации кэша по пути `mycache.config`; параметры описаны в классе `CaffeineCacheConfig` (приведены примерные значения или значения по умолчанию): -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript mycache { @@ -58,10 +60,10 @@ agent: } ``` - 1. Время по истечении которого значение для ключа будет удалено, отчитывается после добавления значения (необязательно) - 2. Время по истечении которого значение для ключа будет удалено, отчитывается после операции чтения (необязательно) - 3. Начальный размер кэша (помогает избежать расширения кэша в случае активного набухания) (необязательно) - 4. Максимальный размер кэша (При достижении границы **или чуть ранее** будет исключать из кэша [наименее актуальные значения](https://blog.skillfactory.ru/glossary/lru/)) (по умолчанию `100000`) + 1. Время, по истечении которого значение удаляется из кэша; отсчитывается после записи значения (по умолчанию не указано, опционально) + 2. Время, по истечении которого значение удаляется из кэша; отсчитывается после чтения значения (по умолчанию не указано, опционально) + 3. Начальный размер кэша, помогает избежать изменения размера при быстром росте количества значений (по умолчанию не указано, опционально) + 4. Максимальный размер кэша; при достижении границы **или немного раньше** вытесняются [наименее актуальные значения](https://blog.skillfactory.ru/glossary/lru/) (по умолчанию: `100000`) === ":simple-yaml: `YAML`" @@ -74,14 +76,52 @@ agent: maximumSize: 100000 #(4)! ``` - 1. Время по истечении которого значение для ключа будет удалено, отчитывается после добавления значения (необязательно) - 2. Время по истечении которого значение для ключа будет удалено, отчитывается после операции чтения (необязательно) - 3. Начальный размер кэша (помогает избежать расширения кэша в случае активного набухания) (необязательно) - 4. Максимальный размер кэша (При достижении границы **или чуть ранее** будет исключать из кэша [наименее актуальные значения](https://blog.skillfactory.ru/glossary/lru/)) (по умолчанию `100000`) + 1. Время, по истечении которого значение удаляется из кэша; отсчитывается после записи значения (по умолчанию не указано, опционально) + 2. Время, по истечении которого значение удаляется из кэша; отсчитывается после чтения значения (по умолчанию не указано, опционально) + 3. Начальный размер кэша, помогает избежать изменения размера при быстром росте количества значений (по умолчанию не указано, опционально) + 4. Максимальный размер кэша; при достижении границы **или немного раньше** вытесняются [наименее актуальные значения](https://blog.skillfactory.ru/glossary/lru/) (по умолчанию: `100000`) + +Базовый кэш `Caffeine` создается фабрикой `CaffeineCacheFactory`, предоставляемой как `@DefaultComponent`. +Если требуется настройка сверх перечисленных выше опций конфигурации (например, собственное вытеснение, слабые ключи или собственный весовой оценщик), +зарегистрируйте собственный компонент `CaffeineCacheFactory`, чтобы переопределить фабрику по умолчанию и настроить построитель `Caffeine` напрямую. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MyCaffeineCacheFactory implements CaffeineCacheFactory { + + @Nonnull + @Override + public Cache build(@Nonnull String name, @Nonnull CaffeineCacheConfig config) { + var builder = Caffeine.newBuilder().weakKeys(); + if (config.expireAfterWrite() != null) { + builder.expireAfterWrite(config.expireAfterWrite()); + } + builder.maximumSize(config.maximumSize()); + return builder.build(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyCaffeineCacheFactory : CaffeineCacheFactory { + + override fun build(name: String, config: CaffeineCacheConfig): Cache { + val builder = Caffeine.newBuilder().weakKeys() + config.expireAfterWrite()?.let { builder.expireAfterWrite(it) } + builder.maximumSize(config.maximumSize()) + return builder.build() + } + } + ``` ## Redis { #redis } -Реализация на основе базы данных в памяти [Redis](https://redis.io/docs/about/) и драйвера подключения [Lettuce](https://github.com/lettuce-io/lettuce-core). +Реализация на основе базы данных в оперативной памяти [Redis](https://redis.io/docs/about/) и драйвера подключения [Lettuce](https://github.com/lettuce-io/lettuce-core). ### Подключение { #dependency-2 } @@ -113,121 +153,156 @@ agent: ### Конфигурация { #configuration-2 } -Требуется отдельно сконфигурировать Lettuce драйвер для подключения к Redis. -Используется одно подключение для всех кешей. +Драйвер `Lettuce` необходимо настроить отдельно для подключения к `Redis`. +Для всех кэшей `Redis` используется одно подключение. -Пример полной конфигурации для *lettuce* драйвера, параметры описаны в классе `LettuceConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации Lettuce: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript lettuce { - uri = "redis://locahost:6379" //(1)! - user = "admin" //(2)! - password = "12345" //(3)! - database = 0 //(4)! - protocol = "RESP3" //(5)! - socketTimeout = "10s" //(6)! - commandTimeout = "60s" //(7)! - forceClusterClient = "false" //(8)! - ssl { - ciphers = [ "TLS_CHACHA20_POLY1305_SHA256" ] //(9)! - handshakeTimeout = "10s" //(10)! - } - telemetry { - logging { - enabled = false //(11)! - } - metrics { - enabled = true //(12)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(13)! - tags = { // (14)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(15)! - attributes = { // (16)! - "key1" = "value1" - "key2" = "value2" - } - } - } + uri = "redis://localhost:6379" //(1)! + commandTimeout = "60s" //(2)! } ``` - 1. URI для подключения к Redis (**обязательный**) - Подключение для 1 сервера: `redis://locahost:6379`, - Подключение для N серверов: `redis://locahost:6379,locahost:6380`, - Подключение для c SSL: `rediss://locahost:6380` - Подключение для c TLS: `redis+tls://locahost:6380` - 2. Имя пользователя для подключения (необязательно) - 3. Пароль пользователя для подключения (необязательно) - 4. Номер базы для подключения (необязательно) - 5. Протокол для подключения (необязательно) - 6. Таймаут времени подключения (необязательно) - 7. Таймаут времени выполнения команды (необязательно) - 8. Форсировать кластерное подключение даже если указан 1 URI для подключения (необязательно) - 9. Алгоритмы шифрования, используемые для безопасного соединения между клиентом и сервером (необязательно) - 10. Таймаут времени установки безопасного соединения с сервером (необязательно) - 11. Включает логгирование модуля (по умолчанию `false`) - 12. Включает метрики модуля (по умолчанию `true`) - 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 14. Настройка тегов для метрик (опционально) - 15. Включает трассировку модуля (по умолчанию `true`) - 16. Настройка атрибутов для трассировки (опционально) + 1. `URI` для подключения к `Redis` (`обязательный`, по умолчанию не указан) + 2. Таймаут выполнения команд (по умолчанию: `60s`) === ":simple-yaml: `YAML`" ```yaml lettuce: - uri: "redis://locahost:6379" #(1)! - user: "admin" #(2)! - password: "12345" #(3)! - database: 0 #(4)! - protocol: "RESP3" #(5)! - socketTimeout: "10s" #(6)! - commandTimeout: "60s" #(7)! - forceClusterClient: false #(8)! - ssl: - ciphers: - - "TLS_CHACHA20_POLY1305_SHA256" #(9)! - handshakeTimeout: "10s" #(10)! - telemetry: - logging: - enabled: false #(11)! - metrics: - enabled: true #(12)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! - tracing: - enabled: true #(14)! - ``` - - 1. URI для подключения к Redis (**обязательный**) - Подключение для 1 сервера: `redis://locahost:6379`, - Подключение для N серверов: `redis://locahost:6379,locahost:6380`, - Подключение для c SSL: `rediss://locahost:6380` - Подключение для c TLS: `redis+tls://locahost:6380` - 2. Имя пользователя для подключения (необязательно) - 3. Пароль пользователя для подключения (необязательно) - 4. Номер базы для подключения (необязательно) - 5. Протокол для подключения (необязательно) - 6. Таймаут времени подключения (необязательно) - 7. Таймаут времени выполнения команды (необязательно) - 8. Форсировать кластерное подключение даже если указан 1 URI для подключения (необязательно) - 9. Алгоритмы шифрования, используемые для безопасного соединения между клиентом и сервером (необязательно) - 10. Таймаут времени установки безопасного соединения с сервером (необязательно) - 11. Включает логгирование модуля (по умолчанию `false`) - 12. Включает метрики модуля (по умолчанию `true`) - 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 14. Включает трассировку модуля (по умолчанию `true`) - -Конфигурации Redis кэша настраивает именно поведение конкретного кэша. - -Пример полной конфигурации для `mycache.config` кэша, параметры описаны в классе `RedisCacheConfig` (указаны примеры значений): - -===! ":material-code-json: `Hocon`" + uri: "redis://localhost:6379" #(1)! + commandTimeout: "60s" #(2)! + ``` + + 1. `URI` для подключения к `Redis` (`обязательный`, по умолчанию не указан) + 2. Таймаут выполнения команд (по умолчанию: `60s`) + +??? note "Полная конфигурация" + + Пример полной конфигурации драйвера `Lettuce`; параметры описаны в классе `LettuceClientConfig` (приведены примерные значения или значения по умолчанию): + + ===! ":material-code-json: `HOCON`" + + ```javascript + lettuce { + uri = "redis://localhost:6379" //(1)! + user = "admin" //(2)! + password = "12345" //(3)! + database = 0 //(4)! + protocol = "RESP3" //(5)! + socketTimeout = "10s" //(6)! + commandTimeout = "60s" //(7)! + forceClusterClient = false //(8)! + ssl { + ciphers = [ "TLS_CHACHA20_POLY1305_SHA256" ] //(9)! + handshakeTimeout = "10s" //(10)! + } + telemetry { + logging { + enabled = false //(11)! + } + metrics { + enabled = true //(12)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(13)! + tags = { // (14)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(15)! + attributes = { // (16)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. `URI` для подключения к `Redis` (`обязательный`, по умолчанию не указан). + Подключение к одному серверу: `redis://localhost:6379`. + Подключение к нескольким серверам: `redis://localhost:6379,localhost:6380`. + Подключение с `SSL`: `rediss://localhost:6380`. + Подключение с `TLS`: `redis+tls://localhost:6380`. + 2. Имя пользователя для подключения (по умолчанию не указано, опционально) + 3. Пароль пользователя для подключения (по умолчанию не указано, опционально) + 4. Номер базы данных для подключения (по умолчанию не указано, опционально) + 5. Протокол подключения, может быть `RESP2` или `RESP3` (по умолчанию: `RESP3`) + 6. Таймаут подключения сокета (по умолчанию: `10s`) + 7. Таймаут выполнения команды (по умолчанию: `60s`) + 8. Создавать кластерный клиент даже при одном `URI` подключения (по умолчанию: `false`) + 9. Алгоритмы шифрования для защищенного соединения между клиентом и сервером (по умолчанию: `[]`) + 10. Таймаут установки защищенного соединения с сервером (по умолчанию: `10s`) + 11. Включает логирование модуля (по умолчанию: `false`) + 12. Включает метрики модуля (по умолчанию: `true`) + 13. Настройка [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Настройка тегов для метрик (по умолчанию: `{}`) + 15. Включает трассировку модуля (по умолчанию: `true`) + 16. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + lettuce: + uri: "redis://localhost:6379" #(1)! + user: "admin" #(2)! + password: "12345" #(3)! + database: 0 #(4)! + protocol: "RESP3" #(5)! + socketTimeout: "10s" #(6)! + commandTimeout: "60s" #(7)! + forceClusterClient: false #(8)! + ssl: + ciphers: + - "TLS_CHACHA20_POLY1305_SHA256" #(9)! + handshakeTimeout: "10s" #(10)! + telemetry: + logging: + enabled: false #(11)! + metrics: + enabled: true #(12)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! + tags: #(14)! + key1: value1 + key2: value2 + tracing: + enabled: true #(15)! + attributes: #(16)! + key1: value1 + key2: value2 + ``` + + 1. `URI` для подключения к `Redis` (`обязательный`, по умолчанию не указан). + Подключение к одному серверу: `redis://localhost:6379`. + Подключение к нескольким серверам: `redis://localhost:6379,localhost:6380`. + Подключение с `SSL`: `rediss://localhost:6380`. + Подключение с `TLS`: `redis+tls://localhost:6380`. + 2. Имя пользователя для подключения (по умолчанию не указано, опционально) + 3. Пароль пользователя для подключения (по умолчанию не указано, опционально) + 4. Номер базы данных для подключения (по умолчанию не указано, опционально) + 5. Протокол подключения, может быть `RESP2` или `RESP3` (по умолчанию: `RESP3`) + 6. Таймаут подключения сокета (по умолчанию: `10s`) + 7. Таймаут выполнения команды (по умолчанию: `60s`) + 8. Создавать кластерный клиент даже при одном `URI` подключения (по умолчанию: `false`) + 9. Алгоритмы шифрования для защищенного соединения между клиентом и сервером (по умолчанию: `[]`) + 10. Таймаут установки защищенного соединения с сервером (по умолчанию: `10s`) + 11. Включает логирование модуля (по умолчанию: `false`) + 12. Включает метрики модуля (по умолчанию: `true`) + 13. Настройка [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Настройка тегов для метрик (по умолчанию: `{}`) + 15. Включает трассировку модуля (по умолчанию: `true`) + 16. Настройка атрибутов для трассировки (по умолчанию: `{}`) + +Конфигурация кэша `Redis` определяет поведение конкретного кэша. + +Пример полной конфигурации кэша по пути `mycache.config`; параметры описаны в классе `RedisCacheConfig` (приведены примерные значения): + +===! ":material-code-json: `HOCON`" ```javascript mycache { @@ -239,9 +314,9 @@ agent: } ``` - 1. При записи устанавливает время [expiration](https://redis.io/commands/psetex/) - 2. При чтении устанавливает время [expiration](https://redis.io/commands/getex/) - 3. Префикс ключа в определенном кеше для избежания коллизий ключе в рамках Redis базы данных, может быть пустой строкой тогда ключи будут без префикса (**обязательный**) + 1. Задает время [устаревания](https://redis.io/commands/psetex/) значения при записи (по умолчанию не указано, опционально) + 2. Задает время [устаревания](https://redis.io/commands/getex/) значения при чтении (по умолчанию не указано, опционально) + 3. Префикс ключа для конкретного кэша, используется во избежание коллизий ключей в одной базе данных `Redis`; может быть пустой строкой, тогда ключи будут без префикса (`обязательный`, по умолчанию не указан) === ":simple-yaml: `YAML`" @@ -250,18 +325,98 @@ agent: config: expireAfterWrite: "10s" #(1)! expireAfterAccess: "10s" #(2)! - keyPrefix: "mykey" //(3)! + keyPrefix: "mykey" #(3)! ``` - 1. При записи устанавливает время [expiration](https://redis.io/commands/psetex/) - 2. При чтении устанавливает время [expiration](https://redis.io/commands/getex/) - 3. Префикс ключа в определенном кеше для избежания коллизий ключе в рамках Redis базы данных, может быть пустой строкой тогда ключи будут без префикса (**обязательный**) + 1. Задает время [устаревания](https://redis.io/commands/psetex/) значения при записи (по умолчанию не указано, опционально) + 2. Задает время [устаревания](https://redis.io/commands/getex/) значения при чтении (по умолчанию не указано, опционально) + 3. Префикс ключа для конкретного кэша, используется во избежание коллизий ключей в одной базе данных `Redis`; может быть пустой строкой, тогда ключи будут без префикса (`обязательный`, по умолчанию не указан) + +Метрики модуля описаны в разделе [Справочник по метрикам](metrics.md#cache). +Собственную телеметрию кэша для обоих хранилищ можно подключить, зарегистрировав nullable-компоненты `CacheMetrics` и `CacheTracer`, +которые получают `CacheTelemetryOperation` с именем операции, именем кэша и источником. + +### Мапперы ключей и значений { #redis-mappers } + +`Redis` хранит ключи и значения как массивы байтов, поэтому `RedisCache` использует два вида мапперов: -Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#cache). +- `RedisCacheKeyMapper` преобразует ключ кэша в `byte[]`. +- `RedisCacheValueMapper` записывает значение кэша в `byte[]` и читает его обратно. -#### Донастройка { #configurator } +Обычные ключи строятся через `RedisCacheKeyMapper` для типа ключа. Встроенные мапперы доступны для `String`, `byte[]`, +чисел, `BigInteger`, `BigDecimal`, `UUID`, `Boolean`, `Character`, `Instant`, `LocalDateTime`, `LocalDate`, `ZonedDateTime`, +`Duration`, `Period`, `Enum` и `Collection`, когда для `T` также доступен маппер. +Для `Enum` используется `toString()`, поэтому его можно переопределить, когда требуется другой формат ключа. -Можно зарегистрировать `LettuceConfigurator` который позволит до настроить `Lettuce` клиент перед созданием. +Для значений встроенные реализации `RedisCacheValueMapper` доступны для тех же простых типов, типов даты/времени, `Enum` и `byte[]`. +Для остальных типов используется маппер на основе `JsonWriter` и `JsonReader`, когда для типа доступна сериализация в JSON. +Если требуется другое представление, зарегистрируйте собственный компонент `RedisCacheValueMapper` или `RedisCacheKeyMapper`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class UserIdRedisKeyMapper implements RedisCacheKeyMapper { + + @Nonnull + @Override + public byte[] apply(@Nullable UserId key) { + return key == null + ? "NUL".getBytes(StandardCharsets.UTF_8) + : key.value().getBytes(StandardCharsets.UTF_8); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class UserIdRedisKeyMapper : RedisCacheKeyMapper { + + override fun apply(key: UserId?): ByteArray { + return key?.value?.toByteArray(Charsets.UTF_8) + ?: "NUL".toByteArray(Charsets.UTF_8) + } + } + ``` + +Частый случай — хранение значения-объекта в виде `JSON`. Пометьте тип значения аннотацией `@Json`: `Kora` генерирует для него `JsonWriter` и `JsonReader`, +а `RedisCacheModule` автоматически предоставляет подходящий `RedisCacheValueMapper` (`jsonRedisValueMapper`), поэтому для типов, сериализуемых в `JSON`, ручной маппер не нужен. +Чтобы использовать для такого типа другое представление, зарегистрируйте собственный компонент `RedisCacheValueMapper`, который переопределяет маппер `JSON` по умолчанию. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record UserData(String id, String name) { } + + @Cache("mycache.config") + public interface MyCache extends RedisCache { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class UserData(val id: String, val name: String) + + @Cache("mycache.config") + interface MyCache : RedisCache + ``` + +Для составного ключа на основе `record` или `data class` Kora генерирует отдельный `RedisCacheKeyMapper` для всего ключа целиком. +Он получает маппер для каждого поля, преобразует каждое поле в `byte[]` и соединяет части с помощью `RedisCacheKeyMapper.DELIMITER`. +Порядок частей соответствует порядку компонентов `record` или свойств `data class`. + +Для одиночного ключа встроенные реализации `RedisCacheKeyMapper` могут кодировать `null` специальным байтовым значением. +В составном ключе результат маппинга каждого поля должен быть не `null`: если собственный `RedisCacheKeyMapper` для поля возвращает `null`, +создание ключа завершается ошибкой. Для опциональных полей в составном ключе собственный маппер должен явно кодировать `null` +стабильным байтовым значением. + +#### Конфигуратор { #configurator } + +Можно зарегистрировать `LettuceConfigurator`, чтобы настроить клиент `Lettuce` до его создания. ===! ":fontawesome-brands-java: `Java`" @@ -269,8 +424,8 @@ agent: @Component public final class MyLettuceConfigurator implements LettuceConfigurator { @Override - public DefaultClientResources.Builder configure(DefaultClientResources.Builder resouceBuilder) { - return resouceBuilder; + public DefaultClientResources.Builder configure(DefaultClientResources.Builder resourceBuilder) { + return resourceBuilder; } @Override @@ -290,8 +445,8 @@ agent: ```kotlin class MyLettuceConfigurator : LettuceConfigurator { - override fun configure(resouceBuilder: DefaultClientResources.Builder): DefaultClientResources.Builder { - return resouceBuilder + override fun configure(resourceBuilder: DefaultClientResources.Builder): DefaultClientResources.Builder { + return resourceBuilder } override fun configure(clusterBuilder: ClusterClientOptions.Builder): ClusterClientOptions.Builder { @@ -304,13 +459,18 @@ agent: } ``` +Для продвинутых сценариев за пределами типизированного кэша для внедрения доступен `RedisCacheClient` — низкоуровневый клиент, работающий с сырыми `byte[]` +(`scan`/`get`/`mget`/`getex`/`set`/`mset`/`psetex`/`del`/`flushAll`) поверх общего подключения `Lettuce`; именно на этом клиенте построен `RedisCache`. + ## Использование { #usage } -Для создания кеша потребуется зарегистрировать типизированный `@Cache` контракт. -Интерфейс контракта должен наследоваться только от предоставляемых Kora'ой реализаций: `CaffeineCache` / `RedisCache`. -Для такого `@Cache` будет создана и добавлена в граф реализация, ее можно будет использовать для внедрения зависимостей. +Создание кэша потребует регистрации типизированного контракта `@Cache`. +Интерфейс контракта должен наследовать одну из реализаций `Kora`: `CaffeineCache` или `RedisCache`. +Для такого `@Cache` генерируется реализация и добавляется в граф, поэтому его можно внедрять как зависимость. -Для регистрации `@Cache` и указания конфига требуется проаннотировать аннотацией `@Cache` где аргумент `value` означает полный путь к конфига. +Аргумент `value` в `@Cache` определяет полный путь к конфигурации конкретного кэша. +Он указывает на объект конфигурации этого кэша, поэтому ключи конфигурации могут располагаться под вложенным путем, таким как `mycache.config { ... }`, +либо плоско прямо под путем, таким как `my-cache { ... }`, как используется в примерах проектов. Обе формы допустимы; выберите одну и держите ключи конфигурации под ней. ===! ":fontawesome-brands-java: `Java`" @@ -326,25 +486,164 @@ agent: interface MyCache : CaffeineCache ``` +### Опциональные значения { #optional-values } + +Если метод `Java` возвращает `Optional`, аспект кэширования может работать с такой сигнатурой напрямую. +То же правило применяется к асинхронным обёрткам, например `CompletionStage>` и `Mono>`. +Сам тип значения кэша может быть либо `T`, либо `Optional`: + +- `CaffeineCache` и метод `Optional get(String key)`; +- `CaffeineCache>` и метод `String get(String key)`; +- `CaffeineCache>` и метод `Optional get(String key)`. + +Для `@Cacheable` это позволяет отличить отсутствующую запись в кэше от результата метода, который также означает отсутствие данных. +Для `@CachePut` результат `Optional` обрабатывается согласно типу значения кэша: если кэш хранит `Optional`, сохраняется сам `Optional`, +а если кэш хранит `T`, сохраняется только присутствующее значение. + ### Императивный подход { #imperative } -Кеши доступны для внедрения как зависимости по интерфейсу и могут использовать вкупе с декларативными операциями. +Кэши доступны для внедрения как зависимости по интерфейсу и могут использоваться совместно с декларативными операциями. + +`CaffeineCache` предоставляет контракт `Cache` для синхронных операций и дополнительный метод `getAll()`. +`RedisCache` предоставляет `Cache` и `AsyncCache`: его можно использовать синхронно и асинхронно через `CompletionStage`. + +`Cache` предоставляет `get(...)`, `put(...)`, `computeIfAbsent(...)`, `invalidate(...)`, `invalidateAll(...)`, +а также пакетные варианты для коллекции ключей или отображения значений. `AsyncCache` предоставляет те же операции с суффиксом `Async`. +Методы `computeIfAbsent(...)` сначала пытаются получить значение из кэша; при промахе они вызывают переданную функцию загрузки и сохраняют результат. + +#### Составной кэш с `Cache.Builder` { #builder-composite-cache } -Реализация `CaffeineCache` предоставляет базовые контракты интерфейса `Cache` для синхронных операций, -а `RedisCache` предоставляет как `Cache` так и `AsyncCache` для асинхронных операций с `CompletionStage` сигнатурами. +Если в императивном коде нужен составной кэш, его можно построить как фасад через `Cache.Builder`. +Порядок слоёв определяется порядком добавления: обычно быстрый локальный кэш, такой как `Caffeine`, добавляется первым, +а более общий кэш, такой как `Redis`, добавляется после него. + +- `get(key)` проверяет кэши по порядку и возвращает первое найденное значение. +- `put(...)`, `invalidate(...)` и `invalidateAll()` выполняются во всех кэшах. +- `computeIfAbsent(...)` проверяет кэши по порядку; если значение найдено на нижнем слое, оно записывается в предыдущие слои. +- Если значение отсутствует во всех слоях, вызывается функция загрузки, а результат записывается во все кэши. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Cache("mycache.caffeine.config") + public interface MyCaffeineCache extends CaffeineCache { } -Интерфейсы предоставляют операции получения, удаления, обновления, пакетных операций и тп. -Также реализации кешей могут предоставлять специфичные для себя контракты. + @Cache("mycache.redis.config") + public interface MyRedisCache extends RedisCache { } + + @KoraApp + public interface Application extends CaffeineCacheModule, RedisCacheModule { + + default Cache compositeCache(MyCaffeineCache caffeineCache, MyRedisCache redisCache) { + return Cache.builder(caffeineCache) + .addCache(redisCache) + .build(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Cache("mycache.caffeine.config") + interface MyCaffeineCache : CaffeineCache + + @Cache("mycache.redis.config") + interface MyRedisCache : RedisCache + + @KoraApp + interface Application : CaffeineCacheModule, RedisCacheModule { + + fun compositeCache( + caffeineCache: MyCaffeineCache, + redisCache: MyRedisCache, + ): Cache { + return Cache.builder(caffeineCache) + .addCache(redisCache) + .build() + } + } + ``` + +Для асинхронного фасада используйте `AsyncCache.builder(...)`; в него можно добавлять только экземпляры `AsyncCache`. +Это подходит, например, для нескольких экземпляров `RedisCache` или других асинхронных реализаций с одинаковыми типами ключа и значения. + +===! ":fontawesome-brands-java: `Java`" + + ```java + default AsyncCache compositeAsyncCache(MyRedisCache redisCache1, MyRedisCache redisCache2) { + return AsyncCache.builder(redisCache1) + .addCache(redisCache2) + .build(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun compositeAsyncCache(redisCache1: MyRedisCache, redisCache2: MyRedisCache): AsyncCache { + return AsyncCache.builder(redisCache1) + .addCache(redisCache2) + .build() + } + ``` + +Фасад, построенный через `Cache.Builder`, не поддерживает прямой `get(Collection)`, а фасад, построенный через `AsyncCache.Builder`, не поддерживает прямой `getAsync(Collection)`. +Для пакетной загрузки используйте `computeIfAbsent(Collection, ...)` или `computeIfAbsentAsync(Collection, ...)`. + +#### Ручное управление устареванием { #redis-expiration-override } + +Помимо общего набора методов `Cache`/`AsyncCache`, `RedisCache` добавляет методы для переопределения настроенного `expireAfterWrite` для отдельной записи. +`putExpireAfterWrite(key, value, Duration)` и его пакетная перегрузка с `Map` записывают синхронно, тогда как `putAsyncExpireAfterWrite(...)` +(одиночный и пакетный с `Map`) возвращают `CompletionStage`. Переданный `Duration` применяется к этой конкретной записи вместо значения из конфигурации. +Эти методы доступны только для `Redis`, поскольку `RedisCache` наследует `AsyncCache`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Cache("mycache.config") + public interface MyCache extends RedisCache { } + + @Component + public class SomeService { + + private final MyCache cache; + + public SomeService(MyCache cache) { + this.cache = cache; + } + + public void cacheFor(String key, String value) { + cache.putExpireAfterWrite(key, value, Duration.ofMinutes(5)); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Cache("mycache.config") + interface MyCache : RedisCache + + @Component + class SomeService(private val cache: MyCache) { + + fun cacheFor(key: String, value: String) { + cache.putExpireAfterWrite(key, value, Duration.ofMinutes(5)) + } + } + ``` ### Декларативный подход { #declarative } -Все примеры использования аспектов будут подразумевать реализацию кэша выше. +Все примеры аспектов ниже предполагают реализацию кэша, приведённую выше. #### Получение { #get } -Для кэширования и получения значения из кэша для метода *get()* следует проаннотировать его аннотацией `@Cacheable`. +Чтобы кэшировать и извлекать значение из кэша для метода `get()`, пометьте его аннотацией `@Cacheable`. +Если значение найдено в кэше, исходный метод не вызывается; если значения нет, метод выполняется, а результат сохраняется в кэше. -Ключ для кэша составляет из аргументов метода, порядок аргументов имеет значение, в данном случае он будет составляться из значения `arg1`. +Ключ кэша строится из аргументов метода, и порядок аргументов имеет значение. В данном случае он строится из `arg1`. ===! ":fontawesome-brands-java: `Java`" @@ -372,12 +671,12 @@ agent: } ``` -#### Сохранение { #put } +#### Запись { #put } -Для добавления значений в кэш через метод *put()* следует проаннотировать его аннотацией `@CachePut`. -Метод проаннотированный `@CachePut` будет вызван и его значение положено в кэш определенный в *value*. +Чтобы добавлять значения в кэш через метод `put()`, пометьте его аннотацией `@CachePut`. +Метод с `@CachePut` вызывается всегда, а его результат помещается в кэш, определённый в `value`. -Ключ для кэша составляет из аргументов метода, порядок аргументов имеет значение, в данном случае он будет составляться из значения `arg1`. +Ключ кэша строится из аргументов метода, и порядок аргументов имеет значение. В данном случае он строится из `arg1`. ===! ":fontawesome-brands-java: `Java`" @@ -407,10 +706,10 @@ agent: #### Удаление { #invalidate } -Для удаления значения по ключу из кэша через метод *evict()* следует проаннотировать его аннотацией `@CacheInvalidate`. -Метод проаннотированный `@CacheInvalidate` будет вызван и затем по ключу для кэша определенного в *value* будут удалены значения по ключу. +Чтобы удалить значение из кэша по ключу через метод `evict()`, пометьте его аннотацией `@CacheInvalidate`. +Метод с `@CacheInvalidate` вызывается, а затем значение удаляется по ключу из кэша, определённого в `value`. -Ключ для кэша составляет из аргументов метода, порядок аргументов имеет значение, в данном случае он будет составляться из значения `arg1`. +Ключ кэша строится из аргументов метода, и порядок аргументов имеет значение. В данном случае он строится из `arg1`. ===! ":fontawesome-brands-java: `Java`" @@ -438,11 +737,12 @@ agent: } ``` -#### Полное удаление { #invalidate-all } +#### Удаление всех { #invalidate-all } -Для удаления всех значений из кэша через метод *evictAll()* следует проаннотировать его аннотацией `@CacheInvalidate` и указать параметр *invalidateAll = true*. +Чтобы удалить все значения из кэша через метод `evictAll()`, пометьте его аннотацией `@CacheInvalidate` +и укажите параметр `invalidateAll = true`. -Метод проаннотированный `@CacheInvalidate` будет вызван и затем будут удалены все из кэша определенного в *value*. +Метод с `@CacheInvalidate` вызывается, а затем все значения удаляются из кэша, определённого в `value`. ===! ":fontawesome-brands-java: `Java`" @@ -470,9 +770,10 @@ agent: } ``` -#### Композитный кэш { #composite-cache } +#### Составной кэш { #composite-cache } -В случае если у вас есть несколько кешей то требуется подключить оба модуля и указать соответствующее количество аннотаций над методом. +Если необходимо использовать несколько кэшей, подключите нужные модули и укажите несколько аннотаций над методом. +Например, так можно объединить быстрый локальный слой на `Caffeine` и общий слой на `Redis`. ===! ":fontawesome-brands-java: `Java`" @@ -502,7 +803,7 @@ agent: } ``` -А сам проаннотированный класс так: +И сам аннотированный класс: ===! ":fontawesome-brands-java: `Java`" @@ -532,11 +833,16 @@ agent: } ``` -Порядок вызова аспектов соответствует порядку аннотаций над методом, сверху внизу. +Порядок вызова следует порядку аннотаций над методом сверху вниз. +Для `@Cacheable` это означает, что первым проверяется верхний кэш; при промахе проверяется следующий кэш, +а после загрузки значения результат записывается обратно в проверенные кэши. +Та же модель композиции работает для повторяемых `@CachePut` и `@CacheInvalidate`: метод вызывается один раз, +а затем результат записывается во все перечисленные кэши или удаление выполняется во всех перечисленных кэшах. +Также можно использовать контейнерные аннотации `@Cacheables`, `@CachePuts` и `@CacheInvalidates`, когда такая форма удобнее. ## Ключ { #key } -В случае если ключ кэша представляет собой 1 аргумент, то требуется зарегистрировать `Cache` с сигнатурой соответствующей типам ключа и значения. +Если ключ кэша состоит из одного аргумента, зарегистрируйте `Cache` с сигнатурой, соответствующей типам ключа и значения. ===! ":fontawesome-brands-java: `Java`" @@ -554,11 +860,11 @@ agent: ### Преобразование { #conversion } -В случае если аргумент не может быть преобразован в ключ кеша, то реализация кеша затребует соответствующий преобразователь -с интерфейсом `CacheKeyMapper`, в случае если аргументов для ключа будет 2 то потребуется `CacheKeyMapper2` и так далее. +Если аргумент нельзя использовать напрямую в качестве ключа кэша, реализации требуется маппер +с интерфейсом `CacheKeyMapper`. Если для ключа два аргумента, требуется `CacheKeyMapper2`; если три — `CacheKeyMapper3`, и так далее вплоть до `CacheKeyMapper9`. -Такой преобразователь можно также предоставить вручную с помощью аннотации `@Mapping`, -пример преобразования сложного объекта в простой ключ кеша: +Такой маппер можно указать вручную через `@Mapping`. +Пример преобразования сложного объекта в простой ключ кэша: ===! ":fontawesome-brands-java: `Java`" @@ -591,23 +897,32 @@ agent: @Component class SomeService { + data class UserContext(val userId: String, val traceId: String) + + class UserContextMapping : CacheKeyMapper { + override fun map(arg: UserContext): String { + return arg.userId + } + } + + @Mapping(UserContextMapping::class) @Cacheable(MyCache::class) - fun get(arg1: String, arg2: BigDecimal): String { + fun get(context: UserContext): String { // do something } } ``` -### Композитный ключ { #composite-key } +### Составной ключ { #composite-key } -В случае если ключ кэша представляет собой N аргументов, то требуется зарегистрировать `Cache` с использованием -собственного класса который бы описывал такой ключ. +Если ключ кэша состоит из нескольких аргументов, зарегистрируйте `Cache` с собственным классом, +описывающим этот ключ. -Пример для `Cache` где композитный ключ состоит из 2 элементов: +Пример для `Cache`, где составной ключ состоит из двух элементов: ===! ":fontawesome-brands-java: `Java`" - - Предполагается создавать собственный `record` класс который бы описывал композитный ключ. + + Создайте собственный `record`, описывающий составной ключ. ```java @Cache("mycache.config") @@ -619,7 +934,7 @@ agent: === ":simple-kotlin: `Kotlin`" - Предполагается создавать собственный `data` класс который бы описывал композитный ключ. + Создайте собственный `data class`, описывающий составной ключ. ```kotlin @Cache("mycache.config") @@ -629,14 +944,20 @@ agent: } ``` -Если используется `RedisCache` то подразумевается что по умолчанию все аргументы композитного ключа будут не `null`, -либо потребуется использовать собственный преобразователь ключа. +Если используется `RedisCache`, для составного ключа генерируется `RedisCacheKeyMapper`. +Он использует маппер для каждого поля ключа и ожидает, что результат маппинга для каждого поля будет не `null`. +Встроенные мапперы могут кодировать `null` специальным значением, тогда как собственные мапперы должны делать это явно. ### Порядок аргументов { #argument-ordering } -В случае если метод принимает аргументы которые хочется исключить из композитного ключа, -либо же порядок аргументов не соответствует порядку аргументов конструктора композитного ключа, -следует использовать атрибут аннотации `parameters` и определить какие именно аргументы метода использовать и в каком порядке. +Если метод принимает аргументы, которые должны быть исключены из составного ключа, или порядок аргументов не совпадает +с порядком аргументов конструктора составного ключа, используйте атрибут `parameters` и укажите, +какие аргументы метода использовать и в каком порядке. + +`parameters` определяет полный набор аргументов метода, используемых для построения ключа. Каждое имя должно совпадать с именем аргумента метода, +а порядок должен соответствовать типу ключа: для одиночного аргумента — типу ключа `Cache`; для составного ключа — +порядку аргументов конструктора `record` или `data class`. +Если имя отсутствует, тип не совпадает или порядок не подходит к ключу, генерация приложения завершается ошибкой. ===! ":fontawesome-brands-java: `Java`" @@ -664,9 +985,11 @@ agent: } ``` -## Подгружаемый кэш { #loadable-cache } +## Loadable Cache { #loadable-cache } -Библиотека предоставляет компонент для построения сущности, которая объединяет операции GET и PUT, без использования аспектов - `LoadableCache` +Библиотека предоставляет компонент `LoadableCache`, который объединяет операции `get` и `put` без использования аспектов. +Он полезен, когда загрузкой значения нужно управлять вручную, сохраняя при этом стандартную логику: сначала проверить кэш, +а при промахе загрузить данные и сохранить их. ===! ":fontawesome-brands-java: `Java`" @@ -675,7 +998,7 @@ agent: public interface MyCache extends CaffeineCache { } @KoraApp - public interface Application : CaffeineCacheModule { + public interface Application extends CaffeineCacheModule { default LoadableCache loadableCache(MyCache cache, SomeService someService) { return cache.asLoadable(someService::loadEntity); @@ -701,9 +1024,46 @@ agent: } ``` +Для асинхронного кэша используйте `AsyncLoadableCache`. Он создаётся через `asLoadableAsyncSimple(...)` +для загрузки одного ключа или через `asLoadableAsync(...)` для пакетной загрузки нескольких ключей. +Оба варианта возвращают `CompletionStage` и подходят для `RedisCache`, поскольку он реализует `AsyncCache`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Cache("mycache.config") + public interface MyCache extends RedisCache { } + + @KoraApp + public interface Application extends RedisCacheModule { + + default AsyncLoadableCache loadableCache(MyCache cache, SomeService someService) { + return cache.asLoadableAsync(someService::loadEntities); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Cache("mycache.config") + interface MyCache : RedisCache + + @KoraApp + interface Application : RedisCacheModule { + + fun loadableCache( + cache: MyCache, + someService: SomeService, + ): AsyncLoadableCache { + return cache.asLoadableAsync(someService::loadEntities) + } + } + ``` + ## Сигнатуры { #signatures } -Доступные сигнатуры для методов которые поддерживают аннотации из коробки: +Доступные сигнатуры для методов, поддерживаемых аннотациями: ===! ":fontawesome-brands-java: `Java`" @@ -714,7 +1074,10 @@ agent: - `T myMethod()` - `Optional myMethod()` - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (требуется [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + + `@Cacheable` и `@CachePut` требуют возвращаемого значения и не могут применяться к `void`, `Mono`, `CompletionStage`, `Flux` или `Publisher`. + `@CacheInvalidate` можно применять к методам без результата, но нельзя применять к `Flux` или `Publisher`. === ":simple-kotlin: `Kotlin`" @@ -723,4 +1086,7 @@ agent: Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требуется [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + + `@Cacheable` и `@CachePut` требуют возвращаемого значения и не могут применяться к `Unit`. + `@CacheInvalidate` можно применять к методам без результата. diff --git a/mkdocs/docs/ru/documentation/camunda7-bpmn.md b/mkdocs/docs/ru/documentation/camunda7-bpmn.md index fe0f783..eaffae3 100644 --- a/mkdocs/docs/ru/documentation/camunda7-bpmn.md +++ b/mkdocs/docs/ru/documentation/camunda7-bpmn.md @@ -1,15 +1,18 @@ --- -description: "Explains Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry. Use when working with CamundaEngineBpmnModule, CamundaEngineConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." +description: "Explains Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry. Use when working with CamundaEngineBpmnModule, CamundaEngineBpmnConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry; key triggers include CamundaEngineBpmnModule, CamundaEngineConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." + use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 7 BPMN embedded process engine integration, deployment, worker components, configuration, and telemetry; key triggers include CamundaEngineBpmnModule, CamundaEngineBpmnConfig, ProcessEngine, JavaDelegate, @Component, Metrics Reference." --- ??? warning "Экспериментальный модуль" - **Эксперементальный** модуль является полностью рабочим и протестированным, но требует дополнительной апробации и аналитики по использованию, - по этой причине API может потенциально притерпеть незначительные изменения перед полной готовностью. + **Экспериментальный** модуль является полностью рабочим и протестированным, но требует дополнительной апробации и аналитики по использованию. + Поэтому `API` может получить незначительные изменения до полной готовности. -Модуль для подключения оркестратора BPMN процессов на основе [Camunda 7](https://docs.camunda.org/manual/7.21/) +Модуль подключает встроенный движок [Camunda 7](https://docs.camunda.org/manual/7.21/) для выполнения `BPMN`-процессов внутри приложения Kora. +Он создает и настраивает `ProcessEngine`, связывает его с `JDBC`-источником данных, регистрирует исполнителей из графа приложения, загружает `BPMN` / `FORM` / `DMN`-ресурсы из `classpath` и добавляет телеметрию выполнения. + +Чтобы предоставить `Camunda 7 REST API` и веб-приложения `Cockpit` / `Admin` / `Tasklist` по HTTP, используйте вместе с этим модулем отдельный [модуль Camunda 7 REST](camunda7-rest.md). ## Подключение { #dependency } @@ -39,11 +42,12 @@ agent: interface Application : CamundaEngineBpmnModule ``` -Требует подключения [JDBC модуля](database-jdbc.md). +Модуль требует подключения [модуля JDBC](database-jdbc.md). +По умолчанию используется основной `DataSource` приложения, но при необходимости можно предоставить отдельный `DataSource` с тегом `@Tag(CamundaBpmn.class)`. ## Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `CamundaEngineBpmnConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации, описанной в классе `CamundaEngineBpmnConfig`: ===! ":material-code-json: `Hocon`" @@ -56,13 +60,13 @@ agent: maxPoolSize = 25 //(2)! queueSize = 25 //(3)! maxJobsPerAcquisition = 2 //(4)! - virtualThreadsEnabled = true //(5)! + virtualThreadsEnabled = false //(5)! } deployment { tenantId = "Camunda" //(6)! name = "KoraEngineAutoDeployment" //(7)! deployChangedOnly = true //(8)! - resources = "classpath:bpm" //(9)! + resources = ["classpath:bpm"] //(9)! delay = "1m" //(10)! } parallelInitialization { @@ -83,8 +87,8 @@ agent: } metrics { enabled = true //(20)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(21)! - tags = { // (22)! + slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(21)! + tags = { //(22)! "key1" = "value1" "key2" = "value2" } @@ -92,7 +96,7 @@ agent: engineTelemetryEnabled = false //(23)! tracing { enabled = true //(24)! - attributes = { // (25)! + attributes = { //(25)! "key1" = "value1" "key2" = "value2" } @@ -103,31 +107,31 @@ agent: } ``` - 1. Минимальное количество живых потоков в [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 2. Максимальное кличество потоков в [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 3. Размер очереди задачи перед тем как задачи будут выброшены из очереди выполнения [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 4. Максимальное количество задач в выполнении [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (по умолчанию равно кол-во ядер процессора умноженных на 2) - 5. Использовать ли [виртуальные потоки](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) в как основу JobExecutor, все предыдущие опции не имеют значения в случае включения виртуальных потоков - 6. Индетефикатор тенант [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов (по умолчанию отсутсвует) - 7. Имя [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов - 8. Флаг который говорит что следует загружать только измененные ресурсы - 9. Пути для поиска BPMN/FORM/DMN ресурсов которые будут загружены в оркестратор после запуска - 10. Задержда перед тем как начать загрузку новых ресурсов в оркестратор - 11. Включить ли параллельную загрузку которая слегка улучшает скорость запуска оркестратора - 12. Проверять ли не полные запросы настройки оркестратора - 13. Индетификатор администратора Camunda (необязательный) - 14. Пароль администратора Camunda (необязательный) - 15. Имя администратора Camunda (необязательный) - 16. Фамилия администратора Camunda (необязательный) - 17. Email администратора Camunda (необязательный) - 18. Включает логгирование модуля (по умолчанию `false`) - 19. Включает логгирование стека ошибки (по умолчанию `true`) - 20. Включает метрики модуля (по умолчанию `true`) - 21. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 22. Настройка тегов для метрик (опционально) - 23. Включает сбор метрик/телеметрии оркестратора (по умолчанию `false`) - 24. Включает трассировку модуля (по умолчанию `true`) - 25. Настройка атрибутов для трассировки (опционально) + 1. Минимальное количество постоянно живущих потоков в [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (по умолчанию: `5`). + 2. Максимальное количество потоков в [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (по умолчанию: `25`). + 3. Размер очереди задач `JobExecutor`, при превышении которого новые задачи отклоняются (по умолчанию: `25`). + 4. Максимальное количество задач, забираемых `JobExecutor` за один запрос (по умолчанию: `Runtime.getRuntime().availableProcessors() * 2`). + 5. Использовать [виртуальные потоки](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) в качестве основы `JobExecutor` (по умолчанию: `false`). При включении этой опции настройки размера пула и очереди не используются. + 6. Идентификатор `tenant` для [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов (по умолчанию не указан, опционально). + 7. Имя [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов (по умолчанию: `KoraEngineAutoDeployment`). + 8. Загружать только измененные ресурсы за счет фильтрации дубликатов в `Camunda` (по умолчанию: `true`). + 9. Список путей для поиска `BPMN` / `FORM` / `DMN`-ресурсов (`обязательный`, по умолчанию не указан). Поддерживаются только пути с префиксом `classpath:`. + 10. Задержка перед загрузкой ресурсов в движок (по умолчанию не указана, опционально). + 11. Включить параллельную инициализацию движка (по умолчанию: `true`). + 12. Проверять незавершенные выражения движка при параллельной инициализации (по умолчанию: `true`). + 13. Идентификатор администратора `Camunda` (`обязательный`, по умолчанию не указан). Вся секция `admin` является опциональной. + 14. Пароль администратора `Camunda` (`обязательный`, по умолчанию не указан). Вся секция `admin` является опциональной. + 15. Имя администратора `Camunda` (по умолчанию не указано, опционально). Если не указано, используется `id` в верхнем регистре. + 16. Фамилия администратора `Camunda` (по умолчанию не указана, опционально). Если не указана, используется `id` в верхнем регистре. + 17. Адрес электронной почты администратора `Camunda` (по умолчанию не указан, опционально). Если не указан, используется `@localhost`. + 18. Включает логирование модуля (по умолчанию: `false`). + 19. Включает логирование стек-трейса ошибок (по умолчанию: `true`). + 20. Включает метрики модуля (по умолчанию: `true`). + 21. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 22. Теги метрик (по умолчанию: `{}`). + 23. Включает сбор встроенной телеметрии движка `Camunda` (по умолчанию: `false`). + 24. Включает трассировку модуля (по умолчанию: `true`). + 25. Атрибуты трассировки (по умолчанию: `{}`). === ":simple-yaml: `YAML`" @@ -140,12 +144,13 @@ agent: maxPoolSize: 25 #(2)! queueSize: 25 #(3)! maxJobsPerAcquisition: 2 #(4)! - virtualThreadsEnabled: true #(5)! + virtualThreadsEnabled: false #(5)! deployment: tenantId: "Camunda" #(6)! name: "KoraEngineAutoDeployment" #(7)! deployChangedOnly: true #(8)! - resources: "classpath:bpm" #(9)! + resources: #(9)! + - "classpath:bpm" delay: "1m" #(10)! parallelInitialization: enabled: true #(11)! @@ -162,7 +167,7 @@ agent: stacktrace: true #(19)! metrics: enabled: true #(20)! - slo: [ 0, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(21)! + slo: [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] #(21)! tags: #(22)! key1: value1 key2: value2 @@ -174,48 +179,106 @@ agent: key2: value2 ``` - 1. Минимальное количество живых потоков в [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 2. Максимальное кличество потоков в [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 3. Размер очереди задачи перед тем как задачи будут выброшены из очереди выполнения [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) - 4. Максимальное количество задач в выполнении [JobExecutor](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (по умолчанию равно кол-во ядер процессора умноженных на 2) - 5. Использовать ли [виртуальные потоки](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) в как основу JobExecutor, все предыдущие опции не имеют значения в случае включения виртуальных потоков - 6. Индетефикатор тенант [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов (по умолчанию отсутсвует) - 7. Имя [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов - 8. Флаг который говорит что следует загружать только измененные ресурсы - 9. Пути для поиска BPMN/FORM/DMN ресурсов которые будут загружены в оркестратор после запуска - 10. Задержда перед тем как начать загрузку новых ресурсов в оркестратор - 11. Включить ли параллельную загрузку которая слегка улучшает скорость запуска оркестратора - 12. Проверять ли не полные запросы настройки оркестратора - 13. Индетификатор администратора Camunda (необязательный) - 14. Пароль администратора Camunda (необязательный) - 15. Имя администратора Camunda (необязательный) - 16. Фамилия администратора Camunda (необязательный) - 17. Email администратора Camunda (необязательный) - 18. Включает логгирование модуля (по умолчанию `false`) - 19. Включает логгирование стека ошибки (по умолчанию `true`) - 20. Включает метрики модуля (по умолчанию `true`) - 21. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 22. Настройка тегов для метрик (опционально) - 23. Включает сбор метрик/телеметрии оркестратора (по умолчанию `false`) - 24. Включает трассировку модуля (по умолчанию `true`) - 25. Настройка атрибутов для трассировки (опционально) - -Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#camunda-7-bpmn). + 1. Минимальное количество постоянно живущих потоков в [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (по умолчанию: `5`). + 2. Максимальное количество потоков в [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/) (по умолчанию: `25`). + 3. Размер очереди задач `JobExecutor`, при превышении которого новые задачи отклоняются (по умолчанию: `25`). + 4. Максимальное количество задач, забираемых `JobExecutor` за один запрос (по умолчанию: `Runtime.getRuntime().availableProcessors() * 2`). + 5. Использовать [виртуальные потоки](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html) в качестве основы `JobExecutor` (по умолчанию: `false`). При включении этой опции настройки размера пула и очереди не используются. + 6. Идентификатор `tenant` для [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов (по умолчанию не указан, опционально). + 7. Имя [загрузки](https://docs.camunda.org/javadoc/camunda-bpm-platform/7.21/org/camunda/bpm/engine/repository/DeploymentBuilder.html) ресурсов (по умолчанию: `KoraEngineAutoDeployment`). + 8. Загружать только измененные ресурсы за счет фильтрации дубликатов в `Camunda` (по умолчанию: `true`). + 9. Список путей для поиска `BPMN` / `FORM` / `DMN`-ресурсов (`обязательный`, по умолчанию не указан). Поддерживаются только пути с префиксом `classpath:`. + 10. Задержка перед загрузкой ресурсов в движок (по умолчанию не указана, опционально). + 11. Включить параллельную инициализацию движка (по умолчанию: `true`). + 12. Проверять незавершенные выражения движка при параллельной инициализации (по умолчанию: `true`). + 13. Идентификатор администратора `Camunda` (`обязательный`, по умолчанию не указан). Вся секция `admin` является опциональной. + 14. Пароль администратора `Camunda` (`обязательный`, по умолчанию не указан). Вся секция `admin` является опциональной. + 15. Имя администратора `Camunda` (по умолчанию не указано, опционально). Если не указано, используется `id` в верхнем регистре. + 16. Фамилия администратора `Camunda` (по умолчанию не указана, опционально). Если не указана, используется `id` в верхнем регистре. + 17. Адрес электронной почты администратора `Camunda` (по умолчанию не указан, опционально). Если не указан, используется `@localhost`. + 18. Включает логирование модуля (по умолчанию: `false`). + 19. Включает логирование стек-трейса ошибок (по умолчанию: `true`). + 20. Включает метрики модуля (по умолчанию: `true`). + 21. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 22. Теги метрик (по умолчанию: `{}`). + 23. Включает сбор встроенной телеметрии движка `Camunda` (по умолчанию: `false`). + 24. Включает трассировку модуля (по умолчанию: `true`). + 25. Атрибуты трассировки (по умолчанию: `{}`). + +Секция `deployment` является опциональной: если она не указана, модуль не выполняет автоматическую загрузку ресурсов. +Если секция указана, `resources` должна содержать хотя бы один путь. +Ресурсы ищутся рекурсивно в `classpath`; неподдерживаемые пути без префикса `classpath:` пропускаются. + +Метрики модуля описаны в разделе [Справочник метрик](metrics.md#camunda-7-bpmn). + +## Загрузка ресурсов { #deployment } + +Когда секция `deployment` присутствует, модуль автоматически загружает ресурсы процессов в движок после его создания. +Ресурсы размещаются в `classpath` (обычно в каталоге `src/main/resources`) и указываются в списке `resources`: + +===! ":material-code-json: `Hocon`" + + ```javascript + camunda.engine.bpmn { + deployment { + resources = ["classpath:bpm"] //(1)! + } + } + ``` + + 1. Когда секция `deployment` присутствует, требуется хотя бы один путь. Поддерживаются только пути с префиксом `classpath:`. + +=== ":simple-yaml: `YAML`" + + ```yaml + camunda: + engine: + bpmn: + deployment: + resources: #(1)! + - "classpath:bpm" + ``` + + 1. Когда секция `deployment` присутствует, требуется хотя бы один путь. Поддерживаются только пути с префиксом `classpath:`. + +При следующей структуре каталогов путь `classpath:bpm` сканируется рекурсивно, и каждый поддерживаемый ресурс внутри него загружается: + +``` +src/main/resources/bpm/ +├── approve.form +├── helloworld.bpmn +└── onboarding.bpmn +``` + +Правила загрузки, которые стоит учитывать: + +- Поддерживаемые типы ресурсов — модели процессов `BPMN`, формы `FORM` и таблицы решений `DMN`. +- Загружаются только пути с префиксом `classpath:`. Любой другой путь пропускается с предупреждением в логе. +- Пути сканируются **рекурсивно**, поэтому вложенные каталоги внутри указанного пути также включаются. +- При `deployChangedOnly = true` (по умолчанию) включается фильтрация дубликатов `Camunda`, поэтому повторно загружаются только ресурсы, изменившиеся с момента предыдущей загрузки. +- Опциональный `tenantId` привязывает загрузку к конкретному `tenant`, а `delay` откладывает загрузку на заданное время после старта. +- Загрузка регистрируется под именем `name` (по умолчанию `KoraEngineAutoDeployment`). +- Если вся секция `deployment` опущена, модуль не загружает никакие ресурсы — предполагается, что вы загружаете их самостоятельно через `RepositoryService`. ## Исполнители { #applications } -Регистрировать в Camunda можно как свои [JavaDelegate](https://docs.camunda.org/manual/7.21/user-guide/process-engine/delegation-code/), -которые будут зарегистрированы в контексте по своему полному имени класса (`canonicalName`) так и по упрощенному имени класса (`simpleName`): +`Camunda` может вызывать компоненты приложения в качестве исполнителей процесса. +Обычные экземпляры [`JavaDelegate`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/delegation-code/) регистрируются в контексте по полному имени класса (`canonicalName`) и по короткому имени класса (`simpleName`). +Внутри `execute(...)` вы читаете и записываете переменные процесса через `DelegateExecution`: ===! ":fontawesome-brands-java: `Java`" ```java @Component - public final class SimpleDelegate implements JavaDelegate { + public final class ScoreCustomerDelegate implements JavaDelegate { - @Override - public void execute(DelegateExecution delegateExecution) throws Exception { + private static final Logger logger = LoggerFactory.getLogger(ScoreCustomerDelegate.class); + @Override + public void execute(DelegateExecution execution) { + int scoring = ThreadLocalRandom.current().nextInt(1, 100); + logger.info("Scored {} with result {}.", execution.getBusinessKey(), scoring); + execution.setVariable("result", scoring); } } ``` @@ -224,15 +287,30 @@ agent: ```kotlin @Component - class SimpleKoraDelegate : JavaDelegate { + class ScoreCustomerDelegate : JavaDelegate { - fun execute(delegateExecution: DelegateExecution) { + private val logger = LoggerFactory.getLogger(ScoreCustomerDelegate::class.java) + override fun execute(execution: DelegateExecution) { + val scoring = ThreadLocalRandom.current().nextInt(1, 100) + logger.info("Scored {} with result {}.", execution.businessKey, scoring) + execution.setVariable("result", scoring) } } ``` -Так и специализированные `KoraDelegate`, которые позволяют помимо стандартных именований регистрировать исполнителя с помощью произвольного имени в контексте по средствам метода `key()`: +Поскольку `JavaDelegate` регистрируется по короткому имени класса, `serviceTask` в модели `BPMN` ссылается на него по `simpleName` через `camunda:delegateExpression`: + +```xml + + Flow_score_in + Flow_score_out + +``` + +Используйте `KoraDelegate` для произвольного имени исполнителя. +Метод `key()` по умолчанию возвращает `canonicalName`, но его можно переопределить, чтобы задать имя, используемое в выражениях `BPMN`: ===! ":fontawesome-brands-java: `Java`" @@ -240,6 +318,7 @@ agent: @Component public final class SimpleDelegate implements KoraDelegate { + @Override public String key() { return "myKey"; } @@ -257,17 +336,227 @@ agent: @Component class SimpleKoraDelegate : KoraDelegate { - fun key() = "myKey" + override fun key(): String = "myKey" + + override fun execute(delegateExecution: DelegateExecution) { + + } + } + ``` + +На объявленный таким образом исполнитель ссылаются как `${myKey}` в `camunda:delegateExpression`, поэтому имя, используемое в модели процесса, больше не зависит от имени класса. + +Каждый исполнитель перед вызовом оборачивается фабрикой `KoraDelegateWrapperFactory`: она ответвляет текущий `Context` Kora на время выполнения исполнителя и применяет телеметрию модуля вокруг `execute(...)`. +Вы можете предоставить собственную `KoraDelegateWrapperFactory` в виде `@Component`, чтобы изменить это поведение. + +## Сервисы движка { #engine-services } + +Модуль предоставляет стандартные сервисы `Camunda` в виде компонентов графа зависимостей: + +- `RuntimeService` +- `RepositoryService` +- `ManagementService` +- `AuthorizationService` +- `DecisionService` +- `ExternalTaskService` +- `FilterService` +- `FormService` +- `TaskService` +- `HistoryService` +- `IdentityService` + +Эти сервисы можно внедрять в ваши компоненты обычным образом. + +## Запуск процессов и взаимодействие с ними { #usage } + +Внедрите `ProcessEngine` (или любой из перечисленных выше сервисов движка) в свои компоненты, чтобы запускать процессы и управлять их экземплярами. +Процесс запускается по `id` процесса `BPMN` через `RuntimeService`, а определения процессов можно запрашивать через `RepositoryService`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController("/camunda") + public final class CamundaController { + + private final ProcessEngine processEngine; + + public CamundaController(ProcessEngine processEngine) { + this.processEngine = processEngine; + } - fun execute(delegateExecution: DelegateExecution) { + @HttpRoute(method = HttpMethod.GET, path = "/start/onboarding") + public String startOnboarding() { + String businessKey = UUID.randomUUID().toString(); + ProcessInstance instance = processEngine.getRuntimeService() + .startProcessInstanceByKey("Onboarding", businessKey); + return instance.getId(); + } + } + ``` +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController("/camunda") + class CamundaController(private val processEngine: ProcessEngine) { + + @HttpRoute(method = HttpMethod.GET, path = "/start/onboarding") + fun startOnboarding(): String { + val businessKey = UUID.randomUUID().toString() + val instance = processEngine.runtimeService + .startProcessInstanceByKey("Onboarding", businessKey) + return instance.id } } ``` -## Донастройка { #engine-configuration } +Запущенный процесс можно продвигать и извне движка: `RuntimeService.correlateMessage(...)` доставляет событие-сообщение `BPMN`, а `TaskService` / `FormService` завершают пользовательские задачи и отправляют формы: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController("/camunda/process/onboarding") + public final class OnboardingController { + + private final FormService formService; + private final TaskService taskService; + private final RuntimeService runtimeService; + + public OnboardingController(FormService formService, TaskService taskService, RuntimeService runtimeService) { + this.formService = formService; + this.taskService = taskService; + this.runtimeService = runtimeService; + } + + @HttpRoute(path = "/cancel/{businessKey}", method = HttpMethod.GET) + public String customerCancellation(@Path String businessKey) { + runtimeService.correlateMessage("MessageCustomerCancellation", businessKey); + return "Cancelled: " + businessKey; + } + + @HttpRoute(path = "/order/{businessKey}", method = HttpMethod.GET) + public String customerOrder(@Path String businessKey) { + Task task = taskService.createTaskQuery().processInstanceBusinessKey(businessKey).active().singleResult(); + formService.submitTaskForm(task.getId(), Map.of("approved", true)); + return "Approved: " + businessKey; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController("/camunda/process/onboarding") + class OnboardingController( + private val formService: FormService, + private val taskService: TaskService, + private val runtimeService: RuntimeService + ) { + + @HttpRoute(path = "/cancel/{businessKey}", method = HttpMethod.GET) + fun customerCancellation(@Path businessKey: String): String { + runtimeService.correlateMessage("MessageCustomerCancellation", businessKey) + return "Cancelled: $businessKey" + } + + @HttpRoute(path = "/order/{businessKey}", method = HttpMethod.GET) + fun customerOrder(@Path businessKey: String): String { + val task = taskService.createTaskQuery().processInstanceBusinessKey(businessKey).active().singleResult() + formService.submitTaskForm(task.id, mapOf("approved" to true)) + return "Approved: $businessKey" + } + } + ``` + +## Источник данных и транзакции { #datasource } + +Движок сохраняет свое состояние через `JDBC`-`DataSource`, поэтому [модуль JDBC](database-jdbc.md) обязателен. +По умолчанию модуль переиспользует основной `DataSource` приложения, предоставляемый движку под тегом `@Tag(CamundaBpmn.class)`. +Чтобы выделить движку отдельный источник данных, предоставьте собственный `DataSource` с этим тегом: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(CamundaBpmn.class) + @Component + public DataSource camundaDataSource(/* ... */) { + return dataSource; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(CamundaBpmn::class) + @Component + fun camundaDataSource(/* ... */): DataSource { + return dataSource + } + ``` + +Компонент `CamundaEngineDataSource` абстрагирует `DataSource` движка вместе с его `CamundaTransactionManager`. +Реализация по умолчанию выполняет `JDBC` через `DataSource` с тегом `@Tag(CamundaBpmn.class)`; вы можете переопределить `CamundaEngineDataSource` как `@Component`, чтобы полностью контролировать, как движок получает соединения и управляет транзакциями. -Можно регистрировать произвольные `ProcessEngineConfigurator` которые позволяют донастраивать [ProcessEngine](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-bootstrapping/): +Исполнитель, выполняющий собственную `JDBC`-работу, может проводить ее внутри транзакции движка через `CamundaTransactionManager`. +`inContinueTx(...)` переиспользует соединение текущей транзакции движка (открывая новую только если активной нет), тогда как `inNewTx(...)` всегда открывает новую транзакцию; `currentConnection()` возвращает дескриптор для `commit()` / `rollback()` текущей транзакции: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class AuditDelegate implements JavaDelegate { + + private final CamundaTransactionManager transactionManager; + + public AuditDelegate(CamundaTransactionManager transactionManager) { + this.transactionManager = transactionManager; + } + + @Override + public void execute(DelegateExecution execution) { + transactionManager.inContinueTx(() -> { + // JDBC work sharing the engine transaction + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class AuditDelegate(private val transactionManager: CamundaTransactionManager) : JavaDelegate { + + override fun execute(execution: DelegateExecution) { + transactionManager.inContinueTx(Runnable { + // JDBC work sharing the engine transaction + }) + } + } + ``` + +## Исполнитель задач и готовность { #job-executor } + +Движок выполняет асинхронные продолжения и таймеры через [`JobExecutor`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/the-job-executor/). +Реализация выбирается опцией `jobExecutor.virtualThreadsEnabled`: при `false` (по умолчанию) используется исполнитель на пуле потоков с размерами, задаваемыми `corePoolSize` / `maxPoolSize` / `queueSize` / `maxJobsPerAcquisition`; при `true` используется исполнитель на [виртуальных потоках](https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html), а размеры пула и очереди игнорируются (см. пояснения в разделе [Конфигурация](#configuration)). + +Модуль автоматически регистрирует [пробу готовности](probes.md), которая сообщает о приложении как `UP` только после того, как `JobExecutor` становится активным. +Пока исполнитель задач не активирован, проба падает с сообщением `Camunda BPMN Engine JobExecutor is not active`, что удерживает приложение вне ротации, пока движок еще запускается. + +## Пользователь-администратор и Cockpit { #admin } + +Когда секция `admin` присутствует, модуль создает пользователя-администратора `Camunda`, гарантирует существование группы `camunda-admin` с полными правами и добавляет в нее пользователя (см. пояснения `admin` в разделе [Конфигурация](#configuration)). +Именно эту учетную запись вы используете для входа в веб-приложения `Cockpit` / `Admin` / `Tasklist`, предоставляемые [модулем Camunda 7 REST](camunda7-rest.md). +Если секция `admin` опущена, пользователь не создается. + +## Конфигурация движка { #engine-configuration } + +Для дополнительной настройки зарегистрируйте компонент `ProcessEngineConfigurator`. +Метод `prepare(...)` вызывается до создания [ProcessEngine](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-bootstrapping/) и получает `ProcessEngineConfiguration`; метод `setup(...)` вызывается после создания движка: ===! ":fontawesome-brands-java: `Java`" @@ -276,7 +565,12 @@ agent: public final class SimpleProcessEngineConfigurator implements ProcessEngineConfigurator { @Override - public void setup(ProcessEngine engine) { + public void prepare(ProcessEngineConfiguration configuration) { + + } + + @Override + public void setup(ProcessEngine engine) throws Exception { } } @@ -288,12 +582,100 @@ agent: @Component class SimpleProcessEngineConfigurator : ProcessEngineConfigurator { - fun setup(engine: ProcessEngine) { - + override fun prepare(configuration: ProcessEngineConfiguration) { + + } + + override fun setup(engine: ProcessEngine) { + } } ``` ## Плагины { #plugins } -Можно регистрировать произвольные [Plugin](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-plugins/) предоставляя их как компоненты в контейнер зависимостей. +Вы можете зарегистрировать произвольные [`ProcessEnginePlugin`](https://docs.camunda.org/manual/7.21/user-guide/process-engine/process-engine-plugins/), предоставив их в качестве компонентов в контейнере зависимостей Kora. +Модуль собирает все такие компоненты и передает их в конфигурацию движка при создании `ProcessEngine`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SimpleProcessEnginePlugin implements ProcessEnginePlugin { + + @Override + public void preInit(ProcessEngineConfigurationImpl configuration) { + + } + + @Override + public void postInit(ProcessEngineConfigurationImpl configuration) { + + } + + @Override + public void postProcessEngineBuild(ProcessEngine engine) { + + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SimpleProcessEnginePlugin : ProcessEnginePlugin { + + override fun preInit(configuration: ProcessEngineConfigurationImpl) { + + } + + override fun postInit(configuration: ProcessEngineConfigurationImpl) { + + } + + override fun postProcessEngineBuild(engine: ProcessEngine) { + + } + } + ``` + +## Версия Camunda { #version } + +Определенная версия `Camunda` доступна как внедряемый компонент `CamundaVersion`. +Его `version()` возвращает строку версии, сообщаемую пакетом `Camunda`, а `isEnterprise()` возвращает `true`, когда в classpath присутствует enterprise-дистрибутив (`-ee`): + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class VersionPrinter { + + public VersionPrinter(CamundaVersion version) { + if (version.isEnterprise()) { + // enterprise-only behavior + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class VersionPrinter(version: CamundaVersion) { + + init { + if (version.isEnterprise()) { + // enterprise-only behavior + } + } + } + ``` + +## Телеметрия { #telemetry } + +Модуль сообщает собственные логирование, метрики и трассировку для выполнения исполнителей через секцию конфигурации `telemetry`. +Метрики описаны в разделе [Справочник метрик](metrics.md#camunda-7-bpmn), а ответвление `Context`, выполняемое `KoraDelegateWrapperFactory`, ограничивает эту телеметрию рамками каждого вызова исполнителя. + +Независимо от телеметрии модуля, `telemetry.engineTelemetryEnabled` переключает сбор собственной встроенной телеметрии `Camunda` (по умолчанию отключен). diff --git a/mkdocs/docs/ru/documentation/camunda7-rest.md b/mkdocs/docs/ru/documentation/camunda7-rest.md index 8f2944e..0311852 100644 --- a/mkdocs/docs/ru/documentation/camunda7-rest.md +++ b/mkdocs/docs/ru/documentation/camunda7-rest.md @@ -6,10 +6,14 @@ agent: ??? warning "Экспериментальный модуль" - **Эксперементальный** модуль является полностью рабочим и протестированным, но требует дополнительной апробации и аналитики по использованию, - по этой причине API может потенциально притерпеть незначительные изменения перед полной готовностью. + **Экспериментальный** модуль является полностью рабочим и протестированным, но требует дополнительной апробации и аналитики по использованию. + По этой причине `API` может претерпеть незначительные изменения перед полной готовностью. -Модуль для подключения [REST API](https://docs.camunda.org/manual/7.21/reference/rest/overview/) для [Camunda 7 BPMN модуля](camunda7-bpmn.md) +Модуль подключает [`Camunda 7 REST API`](https://docs.camunda.org/manual/7.21/reference/rest/overview/) к приложению Kora и публикует стандартные ресурсы `CamundaRestResources` через отдельный `Undertow` HTTP-сервер. +Он используется вместе с [модулем `Camunda 7 BPMN`](camunda7-bpmn.md): `BPMN`-движок выполняет процессы, а REST-модуль открывает HTTP-доступ к операциям `Camunda 7`. + +Дополнительно модуль может отдавать `OpenAPI`-описание `REST API`, а также страницы `Swagger UI` и `RapiDoc`. +Для запросов к `REST API` доступны отдельные настройки `CORS`, логирования, метрик, трассировки и штатного завершения сервера. ## Подключение { #dependency } @@ -39,11 +43,24 @@ agent: interface Application : CamundaRestUndertowModule ``` -Требует подключения [Camunda BPMN модуля](camunda7-bpmn.md). +Требует подключения [модуля `Camunda 7 BPMN`](camunda7-bpmn.md). + +## HTTP-сервер { #http-server } + +Модуль запускает **отдельный** независимый `Undertow` HTTP-сервер, выделенный под `Camunda 7 REST API`. +Он слушает собственный `port` (по умолчанию: `8081`) и полностью изолирован от основного модуля [HTTP-сервера](http-server.md): +у него собственный фильтр [CORS](#cors), собственная [телеметрия](#telemetry) и собственное [штатное завершение](container.md#component-lifecycle). +Таким образом, `Camunda REST API` и собственные контроллеры приложения работают на разных портах и не разделяют обработку запросов или конфигурацию. + +`ProcessEngine`, обслуживающий эти запросы, предоставляется [модулем `Camunda 7 BPMN`](camunda7-bpmn.md); +данный модуль лишь открывает к нему HTTP-доступ по настроенному `path` (по умолчанию: `/engine-rest`). + +При завершении работы сервер перестает принимать новые запросы и ждет до `shutdownWait` (по умолчанию: `30s`) +завершения уже обрабатываемых запросов, прежде чем остановиться. ## Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `CamundaRestConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации, описанной в классе `CamundaRestConfig`: ===! ":material-code-json: `Hocon`" @@ -82,7 +99,7 @@ agent: stacktrace = true //(20)! mask = "***" //(21)! maskQueries = [ ] //(22)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(23)! + maskHeaders = [ "authorization" ] //(23)! pathTemplate = true //(24)! } metrics { @@ -105,37 +122,35 @@ agent: } ``` - 1. Включить/выключить REST API - 2. Путь префикс до REST API - 3. Порт на котором будет запускаться REST API сервер - 4. Максимальное время ожидания [штатного завершения](container.md#component-lifecycle) - 5. Относительный путь до OpenAPI файлов в `resources` директории, по умолчанию указан файл `openapi.json` OpenAPI из [зависимости Camunda](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi) - 6. Вкл/Выкл контроллера который отдает OpenAPI - 7. Путь по которому будет доступен OpenAPI - 5. Если указан один OpenAPI файл, является целиком путем по которому доступен файл - 6. Если указаны несколько OpenAPI файлов, является префиксом к пути перед именем файла `/openapi/{fileName}`, берется указанный путь и к нему добавляется имя файла без диреторий и его расширения, в случае файла `someDirectory/my-openapi-1.yaml` путь к файлу будет `/openapi/my-openapi-1` - 8. Вкл/Выкл контроллера который отдает SwaggerUI - 9. Путь по которому будет доступен SwaggerUI - 10. Вкл/Выкл контроллера который отдает Rapidoc - 11. Путь по которому будет доступен Rapidoc - 12. Включает CORS фильтр (по умолчанию `false`) - 13. Разрешенные источники для CORS (по умолчанию `null`) - 14. Разрешенные заголовки для CORS запросов (по умолчанию `["*"]`) - 15. Разрешенные HTTP методы для CORS запросов (по умолчанию `["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD"]`) - 16. Разрешает передачу учетных данных в CORS запросах (по умолчанию `true`) - 17. Заголовки, которые могут быть доступны клиенту в CORS ответе (по умолчанию `["*"]`) - 18. Максимальное время кэширования preflight запросов CORS (по умолчанию `1 час`) - 19. Включает логгирование модуля (по умолчанию `false`) - 20. Включает логгирование стэка вызовов в случае исключения - 21. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 22. Список параметров запроса которые следует скрывать - 23. Список заголовков запроса/ответа которые следует скрывать - 24. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 25. Включает метрики модуля (по умолчанию `true`) - 26. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 27. Настройка тегов для метрик (опционально) - 28. Включает трассировку модуля (по умолчанию `true`) - 29. Настройка атрибутов для трассировки (опционально) + 1. Включает `Camunda 7 REST API` (по умолчанию: `false`). + 2. Префикс пути для `Camunda 7 REST API` (по умолчанию: `/engine-rest`). + 3. Порт отдельного `Undertow` HTTP-сервера для `REST API` (по умолчанию: `8081`). + 4. Максимальное время ожидания [штатного завершения](container.md#component-lifecycle) HTTP-сервера (по умолчанию: `30s`). + 5. Путь к `OpenAPI`-файлу в `resources` (по умолчанию: `[ "openapi.json" ]`). По умолчанию используется файл из [зависимости `camunda-engine-rest-openapi`](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi). + 6. Включает контроллер, который отдает `OpenAPI`-файл (по умолчанию: `false`). + 7. Путь, по которому будет доступен `OpenAPI`-файл (по умолчанию: `/openapi`). + 8. Включает контроллер, который отдает `Swagger UI` (по умолчанию: `false`). + 9. Путь, по которому будет доступен `Swagger UI` (по умолчанию: `/swagger-ui`). + 10. Включает контроллер, который отдает `RapiDoc` (по умолчанию: `false`). + 11. Путь, по которому будет доступен `RapiDoc` (по умолчанию: `/rapidoc`). + 12. Включает фильтр `CORS` (по умолчанию: `false`). + 13. Разрешенный источник для `CORS` (по умолчанию не указано, необязательно). Если значение не указано, фильтр использует заголовок `Origin` из запроса, а если его нет, возвращает `*`. + 14. Разрешенные заголовки для `CORS`-запросов (по умолчанию: `[ "*" ]`). + 15. Разрешенные HTTP-методы для `CORS`-запросов (по умолчанию: `[ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ]`). + 16. Разрешает передачу учетных данных в `CORS`-запросах (по умолчанию: `true`). + 17. Заголовки, которые могут быть доступны клиенту в `CORS`-ответе (по умолчанию: `[ "*" ]`). + 18. Максимальное время кеширования предварительных `CORS`-запросов (по умолчанию: `1h`). + 19. Включает логирование модуля (по умолчанию: `false`). + 20. Включает логирование стека вызовов при исключении (по умолчанию: `true`). + 21. Маска для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`). + 22. Список параметров запроса, которые нужно скрывать в логах (по умолчанию: `[ ]`). + 23. Список заголовков запроса или ответа, которые нужно скрывать в логах (по умолчанию: `[ "authorization" ]`). + 24. Определяет, использовать ли шаблон пути при логировании (по умолчанию не указано, необязательно). Если не указано, полный путь используется только на уровне логирования `TRACE`; если `true`, используется шаблон пути; если `false`, используется полный путь. + 25. Включает метрики модуля (по умолчанию: `true`). + 26. Настраивает [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 27. Дополнительные теги для метрик (по умолчанию: `{}`). + 28. Включает трассировку модуля (по умолчанию: `true`). + 29. Дополнительные атрибуты для трассировки (по умолчанию: `{}`). === ":simple-yaml: `YAML`" @@ -156,21 +171,21 @@ agent: rapidoc: enabled: false #(10)! endpoint: "/rapidoc" #(11)! - cors: - enabled: false #(12)! - allowOrigin: "*" #(13)! - allowHeaders: [ "*" ] #(14)! - allowMethods: [ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ] #(15)! - allowCredentials: true #(16)! - exposeHeaders: [ "*" ] #(17)! - maxAge: "1h" #(18)! + cors: + enabled: false #(12)! + allowOrigin: "*" #(13)! + allowHeaders: [ "*" ] #(14)! + allowMethods: [ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ] #(15)! + allowCredentials: true #(16)! + exposeHeaders: [ "*" ] #(17)! + maxAge: "1h" #(18)! telemetry: logging: enabled: false #(19)! stacktrace: true #(20)! mask: "***" #(21)! maskQueries: [ ] #(22)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(23)! + maskHeaders: [ "authorization" ] #(23)! pathTemplate: true #(24)! metrics: enabled: true #(25)! @@ -178,45 +193,176 @@ agent: tags: #(27)! key1: value1 key2: value2 - tracing: - enabled: true #(28)! - attributes: #(29)! - key1: value1 - key2: value2 + tracing: + enabled: true #(28)! + attributes: #(29)! + key1: value1 + key2: value2 + ``` + + 1. Включает `Camunda 7 REST API` (по умолчанию: `false`). + 2. Префикс пути для `Camunda 7 REST API` (по умолчанию: `/engine-rest`). + 3. Порт отдельного `Undertow` HTTP-сервера для `REST API` (по умолчанию: `8081`). + 4. Максимальное время ожидания [штатного завершения](container.md#component-lifecycle) HTTP-сервера (по умолчанию: `30s`). + 5. Путь к `OpenAPI`-файлу в `resources` (по умолчанию: `[ "openapi.json" ]`). По умолчанию используется файл из [зависимости `camunda-engine-rest-openapi`](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi). + 6. Включает контроллер, который отдает `OpenAPI`-файл (по умолчанию: `false`). + 7. Путь, по которому будет доступен `OpenAPI`-файл (по умолчанию: `/openapi`). + 8. Включает контроллер, который отдает `Swagger UI` (по умолчанию: `false`). + 9. Путь, по которому будет доступен `Swagger UI` (по умолчанию: `/swagger-ui`). + 10. Включает контроллер, который отдает `RapiDoc` (по умолчанию: `false`). + 11. Путь, по которому будет доступен `RapiDoc` (по умолчанию: `/rapidoc`). + 12. Включает фильтр `CORS` (по умолчанию: `false`). + 13. Разрешенный источник для `CORS` (по умолчанию не указано, необязательно). Если значение не указано, фильтр использует заголовок `Origin` из запроса, а если его нет, возвращает `*`. + 14. Разрешенные заголовки для `CORS`-запросов (по умолчанию: `[ "*" ]`). + 15. Разрешенные HTTP-методы для `CORS`-запросов (по умолчанию: `[ "GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD" ]`). + 16. Разрешает передачу учетных данных в `CORS`-запросах (по умолчанию: `true`). + 17. Заголовки, которые могут быть доступны клиенту в `CORS`-ответе (по умолчанию: `[ "*" ]`). + 18. Максимальное время кеширования предварительных `CORS`-запросов (по умолчанию: `1h`). + 19. Включает логирование модуля (по умолчанию: `false`). + 20. Включает логирование стека вызовов при исключении (по умолчанию: `true`). + 21. Маска для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`). + 22. Список параметров запроса, которые нужно скрывать в логах (по умолчанию: `[ ]`). + 23. Список заголовков запроса или ответа, которые нужно скрывать в логах (по умолчанию: `[ "authorization" ]`). + 24. Определяет, использовать ли шаблон пути при логировании (по умолчанию не указано, необязательно). Если не указано, полный путь используется только на уровне логирования `TRACE`; если `true`, используется шаблон пути; если `false`, используется полный путь. + 25. Включает метрики модуля (по умолчанию: `true`). + 26. Настраивает [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 27. Дополнительные теги для метрик (по умолчанию: `{}`). + 28. Включает трассировку модуля (по умолчанию: `true`). + 29. Дополнительные атрибуты для трассировки (по умолчанию: `{}`). + +В листинге выше показаны все доступные параметры; на практике включают только то, что нужно. +Типовая настройка публикует `REST API` на произвольном `port` вместе с `OpenAPI`-описанием, `Swagger UI` и логированием запросов: + +===! ":material-code-json: `Hocon`" + + ```javascript + camunda { + rest { + enabled = true + port = 8090 + openapi { + enabled = true + swaggerui.enabled = true + } + telemetry.logging.enabled = true + } + } ``` - 1. Включить/выключить REST API - 2. Путь префикс до REST API - 3. Порт на котором будет запускаться REST API сервер - 4. Максимальное время ожидания [штатного завершения](container.md#component-lifecycle) - 5. Относительный путь до OpenAPI файлов в `resources` директории, по умолчанию указан файл `openapi.json` OpenAPI из [зависимости Camunda](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi) - 6. Вкл/Выкл контроллера который отдает OpenAPI - 7. Путь по которому будет доступен OpenAPI - 5. Если указан один OpenAPI файл, является целиком путем по которому доступен файл - 6. Если указаны несколько OpenAPI файлов, является префиксом к пути перед именем файла `/openapi/{fileName}`, берется указанный путь и к нему добавляется имя файла без диреторий и его расширения, в случае файла `someDirectory/my-openapi-1.yaml` путь к файлу будет `/openapi/my-openapi-1` - 8. Вкл/Выкл контроллера который отдает SwaggerUI - 9. Путь по которому будет доступен SwaggerUI - 10. Вкл/Выкл контроллера который отдает Rapidoc - 11. Путь по которому будет доступен Rapidoc - 12. Включает CORS фильтр (по умолчанию `false`) - 13. Разрешенные источники для CORS (по умолчанию `null`) - 14. Разрешенные заголовки для CORS запросов (по умолчанию `["*"]`) - 15. Разрешенные HTTP методы для CORS запросов (по умолчанию `["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD"]`) - 16. Разрешает передачу учетных данных в CORS запросах (по умолчанию `true`) - 17. Заголовки, которые могут быть доступны клиенту в CORS ответе (по умолчанию `["*"]`) - 18. Максимальное время кэширования preflight запросов CORS (по умолчанию `1 час`) - 19. Включает логгирование модуля (по умолчанию `false`) - 20. Включает логгирование стэка вызовов в случае исключения - 21. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 22. Список параметров запроса которые следует скрывать - 23. Список заголовков запроса/ответа которые следует скрывать - 24. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 25. Включает метрики модуля (по умолчанию `true`) - 26. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 27. Настройка тегов для метрик (опционально) - 28. Включает трассировку модуля (по умолчанию `true`) - 29. Настройка атрибутов для трассировки (опционально) +=== ":simple-yaml: `YAML`" + + ```yaml + camunda: + rest: + enabled: true + port: 8090 + openapi: + enabled: true + swaggerui: + enabled: true + telemetry: + logging: + enabled: true + ``` + +## OpenAPI { #openapi } + +Помимо самого `REST API`, отдельный сервер может отдавать `OpenAPI`-описание API вместе со +страницами [Swagger UI](https://swagger.io/tools/swagger-ui/) и [RapiDoc](https://rapidocweb.com/). +Все три по умолчанию отключены и включаются независимо друг от друга через секцию конфигурации `openapi`. + +При включении страницы доступны на `port` `REST`-сервера по настроенным путям: + +| Страница | Флаг конфигурации | Путь по умолчанию | +|----------------------|-----------------------------|-------------------| +| Спецификация OpenAPI | `openapi.enabled` | `/openapi` | +| Swagger UI | `openapi.swaggerui.enabled` | `/swagger-ui` | +| RapiDoc | `openapi.rapidoc.enabled` | `/rapidoc` | + +Например, при `port = 8090` и `openapi.enabled = true` спецификация отдается по адресу `http://localhost:8090/openapi`, +а `Swagger UI` (если включен) — по адресу `http://localhost:8090/swagger-ui`. + +По умолчанию модуль отдает `OpenAPI`-спецификацию, поставляемую в составе зависимости +[`camunda-engine-rest-openapi`](https://mvnrepository.com/artifact/org.camunda.bpm/camunda-engine-rest-openapi). +Когда используется эта встроенная спецификация, модуль подставляет в нее настроенные `port` и `path`, +поэтому отдаваемый `OpenAPI` всегда соответствует актуальному адресу `REST API`, даже если заданы значения, отличные от `8081` или `/engine-rest`. + +Чтобы вместо этого отдавать собственную спецификацию, укажите в `openapi.file` один или несколько файлов в `resources`: + +===! ":material-code-json: `Hocon`" + + ```javascript + camunda.rest.openapi { + enabled = true + file = [ "my-openapi.json" ] + } + ``` + +=== ":simple-yaml: `YAML`" + + ```yaml + camunda: + rest: + openapi: + enabled: true + file: [ "my-openapi.json" ] + ``` + +## CORS { #cors } + +У `REST`-сервера есть собственный фильтр [CORS](https://developer.mozilla.org/ru/docs/Web/HTTP/CORS), отключенный по умолчанию и включаемый через `cors.enabled`. +Если `cors.allowOrigin` не задан, фильтр возвращает в ответе заголовок `Origin` из запроса, +а при отсутствии заголовка `Origin` в запросе использует `*`. +Остальные параметры `cors.*` управляют разрешенными заголовками и методами, разрешена ли передача учетных данных, +заголовками, доступными клиенту, и временем кеширования предварительных запросов. + +## Телеметрия { #telemetry } + +Запросы, обрабатываемые `REST`-сервером, охвачены стандартными сигналами телеметрии Kora — [логированием](logging-slf4j.md), +[метриками](metrics.md) и [трассировкой](tracing.md) — которые настраиваются в секции `telemetry`. +Логирование по умолчанию отключено (`telemetry.logging.enabled`), а метрики и трассировка по умолчанию включены. + +Параметр `telemetry.logging.pathTemplate` управляет тем, как путь запроса отображается в логах: если он не задан, +используется шаблон пути, кроме уровня `TRACE`, где логируется полный путь; +`true` всегда использует шаблон пути, а `false` всегда использует полный путь. + +Метрики модуля описаны в разделе [Справочник метрик](metrics.md#camunda-rest). + +Стандартную телеметрию можно переопределить, зарегистрировав собственный компонент `CamundaRestLoggerFactory`, `CamundaRestMetricsFactory` +или `CamundaRestTracerFactory`, который заменяет соответствующий стандартный компонент, предоставленный через `@DefaultComponent`. ## Приложения { #applications } -Можно регистрировать произвольные `jakarta.ws.rs.core.Application` с ресурсами для API (например для других [webapp](https://docs.camunda.org/manual/7.21/webapps/)) предоставляя их как компоненты в контейнер зависимостей. +Модуль уже регистрирует стандартный `@Tag(CamundaRest.class)` `jakarta.ws.rs.core.Application`, который публикует стандартные +ресурсы `Camunda 7 REST API` (`CamundaRestResources`) вместе с `ResteasyJackson2Provider` для сериализации в `JSON`. + +Чтобы добавить собственные ресурсы `JAX-RS`, зарегистрируйте свой компонент `jakarta.ws.rs.core.Application`, помеченный тегом `@Tag(CamundaRest.class)`. +Все такие приложения собираются и объединяются со стандартным — их `getClasses()` и `getSingletons()` комбинируются — +поэтому пользовательские ресурсы отдаются на том же `REST`-сервере вместе со стандартными endpoint'ами Camunda. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(CamundaRest.class) + @Component + public final class CustomCamundaApplication extends Application { + + @Override + public Set> getClasses() { + return Set.of(CustomResource.class); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(CamundaRest::class) + @Component + class CustomCamundaApplication : Application() { + + override fun getClasses(): Set> { + return setOf(CustomResource::class.java) + } + } + ``` diff --git a/mkdocs/docs/ru/documentation/camunda8-worker.md b/mkdocs/docs/ru/documentation/camunda8-worker.md index c5fbe88..9ca25f5 100644 --- a/mkdocs/docs/ru/documentation/camunda8-worker.md +++ b/mkdocs/docs/ru/documentation/camunda8-worker.md @@ -1,15 +1,18 @@ --- -description: "Explains Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures. Use when working with @JobWorker, ZeebeClient, ActivatedJob, JobClient, Camunda8WorkerModule, Camunda8WorkerConfig." +description: "Explains Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures. Use when working with @JobWorker, @JobVariable, @JobVariables, ZeebeClient, JobContext, KoraJobWorker, JobWorkerException, ZeebeWorkerModule, ZeebeClientConfig, ZeebeWorkerConfig." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures; key triggers include @JobWorker, ZeebeClient, ActivatedJob, JobClient, Camunda8WorkerModule, Camunda8WorkerConfig." + use_when: "Use this file for Kora docs or implementation questions about Kora Camunda 8 Zeebe worker integration, worker configuration, job handling, variables, telemetry, and supported handler signatures; key triggers include @JobWorker, @JobVariable, @JobVariables, ZeebeClient, JobContext, KoraJobWorker, JobWorkerException, ZeebeWorkerModule, ZeebeClientConfig, ZeebeWorkerConfig." --- ??? warning "Экспериментальный модуль" - **Эксперементальный** модуль является полностью рабочим и протестированным, но требует дополнительной апробации и аналитики по использованию, - по этой причине API может потенциально притерпеть незначительные изменения перед полной готовностью. + **Экспериментальный** модуль является полностью рабочим и протестированным, но требует дополнительной апробации и аналитики по использованию, + по этой причине его `API` может потенциально претерпеть незначительные изменения перед полной готовностью. -Модуль для подключения клиента и создания исполнителей для внешнего оркестратора процессов [Camunda 8 (Zeebe)](https://docs.camunda.io/docs/components/concepts/job-workers/) +Модуль подключает клиент [Camunda 8 (Zeebe)](https://docs.camunda.io/docs/components/concepts/job-workers/) и создает +исполнителей заданий для внешнего оркестратора процессов. В `Kora` такой исполнитель объявляется обычным компонентом: +метод с аннотацией `@JobWorker` получает переменные процесса, выполняет работу и возвращает результат, который будет +передан обратно в `Zeebe`. ## Подключение { #dependency } @@ -41,9 +44,9 @@ agent: ## Конфигурация { #configuration } -Пример полной конфигурации клиента описанной в классе `ZeebeClientConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации клиента, описанной в классе `ZeebeClientConfig` (указаны примеры значений или значения по умолчанию): -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript zeebe { @@ -62,10 +65,10 @@ agent: attempts = 5 //(10)! delay = "100ms" //(11)! delayMax = "5s" //(12)! - stepFactor = 3.0 //(13)! + step = 3.0 //(13)! } } - http { + rest { url = "http://localhost:8080" //(14)! } deployment { @@ -96,28 +99,28 @@ agent: } ``` - 1. Максимальное количество потоков для исполнителей задач, по умолчанию равен кол-во ядер процессора либо минимум `2` - 2. Время соединения без активности чтения перед отправкой `KeepAlive` проверки - 3. Использовать ли TLS при подключении при соединении - 4. [Файловый путь](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) до файла сертификата который использовать при подключении, либо используется по умолчанию системный сертификат - 5. Максимальное время ожидания инициализации запуска исполнителей при старте сервиса (по умолчанию отсутвует) - 6. URL для подключения по gRPC - 7. Время сколько сообщение должно буферизироваться на брокере по gRPC соединению - 8. Максимальный размер сообщения по gRPC соединению - 9. Включена ли политика повтора исполнения в случае ошибки соединения - 10. Количество попыток - 11. Задержка между попытками - 12. Максимальная длительность повторов - 13. Шаг коэфициент увеличения времени задержки между попытками - 14. URL для подключения по HTTP - 15. Пути для поиска ресурсов которые будут загружены в оркесратор после запуска - 16. Максимальное время ожидания загрузки ресурсов - 17. Включает логгирование модуля (по умолчанию `false`) - 18. Включает метрики модуля (по умолчанию `true`) - 19. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 20. Настройка тегов для метрик (опционально) - 21. Включает трассировку модуля (по умолчанию `true`) - 22. Настройка атрибутов для трассировки (опционально) + 1. Максимальное количество потоков для исполнителей заданий (по умолчанию: количество ядер процессора, но не меньше `2`) + 2. Время без активности чтения перед отправкой проверки `KeepAlive` (по умолчанию: `45s`) + 3. Использовать ли `TLS` при подключении (по умолчанию: `true`) + 4. [Файловый путь](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) до сертификата для подключения; если не указан, используется системный сертификат (по умолчанию не указано, необязательно) + 5. Максимальное время ожидания проверки доступности топологии при запуске клиента (по умолчанию не указано, необязательно) + 6. `URL` для подключения по `gRPC` (`обязательная`, по умолчанию не указано) + 7. Время, в течение которого сообщение должно храниться на брокере при отправке по `gRPC` (по умолчанию: `1h`) + 8. Максимальный размер входящего сообщения по `gRPC` (по умолчанию: `4Mib`) + 9. Включена ли политика повторов для `gRPC`-соединения (по умолчанию: `true`) + 10. Количество попыток (по умолчанию: `5`) + 11. Начальная задержка между попытками (по умолчанию: `100ms`) + 12. Максимальная задержка между попытками (по умолчанию: `5s`) + 13. Коэффициент увеличения задержки между попытками (по умолчанию: `3.0`) + 14. `URL` для подключения к `REST`-адресу `Zeebe`; если указан, клиент предпочитает `REST` вместо `gRPC` для поддерживаемых операций (`обязательная` внутри необязательной секции `rest`, по умолчанию не указано) + 15. Пути для поиска ресурсов, которые будут загружены в оркестратор после запуска (по умолчанию: `[]`) + 16. Максимальное время ожидания загрузки ресурсов (по умолчанию: `45s`) + 17. Включает логирование модуля (по умолчанию: `false`) + 18. Включает метрики модуля (по умолчанию: `true`) + 19. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 20. Настройка тегов для метрик (по умолчанию: `{}`) + 21. Включает трассировку модуля (по умолчанию: `true`) + 22. Настройка атрибутов для трассировки (по умолчанию: `{}`) === ":simple-yaml: `YAML`" @@ -130,7 +133,7 @@ agent: certificatePath: "/file/path/to/cert.crt" #(4)! initializationFailTimeout: "15s" #(5)! grpc: - url: "grpc:#localhost:8090" //(6)! + url: "grpc://localhost:8090" #(6)! ttl: "1h" #(7)! maxMessageSize: "4Mib" #(8)! retryPolicy: @@ -138,9 +141,9 @@ agent: attempts: 5 #(10)! delay: "100ms" #(11)! delayMax: "5s" #(12)! - stepFactor: 3.0 #(13)! - http: - url: "http:#localhost:8080" //(14)! + step: 3.0 #(13)! + rest: + url: "http://localhost:8080" #(14)! deployment: resources: "classpath:bpm" #(15)! timeout: "45s" #(16)! @@ -150,48 +153,200 @@ agent: metrics: enabled: true #(18)! slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(19)! + tags: #(20)! + key1: value1 + key2: value2 tracing: - enabled: true #(20)! - ``` - - 1. Максимальное количество потоков для исполнителей задач - 2. Время соединения без активности чтения перед отправкой `KeepAlive` проверки - 3. Использовать ли TLS при подключении при соединении - 4. [Файловый путь](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) до файла сертификата который использовать при подключении, либо используется по умолчанию системный сертификат - 5. Максимальное время ожидания инициализации запуска исполнителей при старте сервиса (по умолчанию отсутвует) - 6. URL для подключения по gRPC - 7. Время сколько сообщение должно буферизироваться на брокере по gRPC соединению - 8. Максимальный размер сообщения по gRPC соединению - 9. Включена ли политика повтора исполнения в случае ошибки соединения - 10. Количество попыток - 11. Задержка между попытками - 12. Максимальная длительность повторов - 13. Шаг коэфициент увеличения времени задержки между попытками - 14. URL для подключения по HTTP - 15. Пути для поиска ресурсов которые будут загружены в оркесратор после запуска - 16. Максимальное время ожидания загрузки ресурсов - 17. Включает логгирование модуля (по умолчанию `false`) - 18. Включает метрики модуля (по умолчанию `true`) - 19. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 20. Включает трассировку модуля (по умолчанию `true`) + enabled: true #(21)! + attributes: #(22)! + key1: value1 + key2: value2 + ``` + + 1. Максимальное количество потоков для исполнителей заданий (по умолчанию: количество ядер процессора, но не меньше `2`) + 2. Время без активности чтения перед отправкой проверки `KeepAlive` (по умолчанию: `45s`) + 3. Использовать ли `TLS` при подключении (по умолчанию: `true`) + 4. [Файловый путь](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/FileInputStream.html) до сертификата для подключения; если не указан, используется системный сертификат (по умолчанию не указано, необязательно) + 5. Максимальное время ожидания проверки доступности топологии при запуске клиента (по умолчанию не указано, необязательно) + 6. `URL` для подключения по `gRPC` (`обязательная`, по умолчанию не указано) + 7. Время, в течение которого сообщение должно храниться на брокере при отправке по `gRPC` (по умолчанию: `1h`) + 8. Максимальный размер входящего сообщения по `gRPC` (по умолчанию: `4Mib`) + 9. Включена ли политика повторов для `gRPC`-соединения (по умолчанию: `true`) + 10. Количество попыток (по умолчанию: `5`) + 11. Начальная задержка между попытками (по умолчанию: `100ms`) + 12. Максимальная задержка между попытками (по умолчанию: `5s`) + 13. Коэффициент увеличения задержки между попытками (по умолчанию: `3.0`) + 14. `URL` для подключения к `REST`-адресу `Zeebe`; если указан, клиент предпочитает `REST` вместо `gRPC` для поддерживаемых операций (`обязательная` внутри необязательной секции `rest`, по умолчанию не указано) + 15. Пути для поиска ресурсов, которые будут загружены в оркестратор после запуска (по умолчанию: `[]`) + 16. Максимальное время ожидания загрузки ресурсов (по умолчанию: `45s`) + 17. Включает логирование модуля (по умолчанию: `false`) + 18. Включает метрики модуля (по умолчанию: `true`) + 19. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 20. Настройка тегов для метрик (по умолчанию: `{}`) + 21. Включает трассировку модуля (по умолчанию: `true`) + 22. Настройка атрибутов для трассировки (по умолчанию: `{}`) Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#camunda-8-worker). +### Развертывание ресурсов { #resource-deployment } + +Если в `deployment.resources` указаны пути, модуль во время запуска находит ресурсы в `classpath` и развертывает их в +`Zeebe` через компонент `ZeebeResourceDeployment`. Развертываются как `BPMN`-процессы, так и `DMN`-решения, найденные в +настроенных расположениях. Поддерживаются только пути с префиксом `classpath:`, например `classpath:bpm`; другие +расположения логируются и пропускаются. + +Разместите развертываемые ресурсы в соответствующей директории classpath: + +```text +src/main/resources/ +└── bpm/ + └── demo.bpmn +``` + +===! ":material-code-json: `HOCON`" + + ```javascript + zeebe { + client { + deployment { + resources = "classpath:bpm" //(1)! + } + } + } + ``` + + 1. Одно или несколько расположений в classpath для поиска `BPMN` / `DMN`-ресурсов (одно значение или список) + +=== ":simple-yaml: `YAML`" + + ```yaml + zeebe: + client: + deployment: + resources: "classpath:bpm" #(1)! + ``` + + 1. Одно или несколько расположений в classpath для поиска `BPMN` / `DMN`-ресурсов (одно значение или список) + +### Клиент { #client } + +Модуль создает компонент `ZeebeClient`, который можно внедрять в собственные сервисы, если нужно вручную запускать +процессы, публиковать сообщения или выполнять другие команды `Zeebe`. + +Например, чтобы запустить новый экземпляр процесса: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ProcessStarter { + + private final ZeebeClient client; + + public ProcessStarter(ZeebeClient client) { + this.client = client; + } + + public void start() { + ProcessInstanceEvent event = client.newCreateInstanceCommand() + .bpmnProcessId("demo") //(1)! + .latestVersion() //(2)! + .variables("{\"startId\":\"42\"}") //(3)! + .send() + .join(); //(4)! + } + } + ``` + + 1. Идентификатор `BPMN`-процесса, который нужно запустить + 2. Запуск последней развернутой версии процесса + 3. Начальные переменные процесса в виде `JSON`-строки (также принимаются `Map` или `@Json`-объект) + 4. Отправить команду и заблокироваться до подтверждения от `Zeebe` (используйте возвращаемый `CompletionStage` для неблокирующего вызова) + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ProcessStarter(private val client: ZeebeClient) { + + fun start() { + val event = client.newCreateInstanceCommand() + .bpmnProcessId("demo") //(1)! + .latestVersion() //(2)! + .variables("""{"startId":"42"}""") //(3)! + .send() + .join() //(4)! + } + } + ``` + + 1. Идентификатор `BPMN`-процесса, который нужно запустить + 2. Запуск последней развернутой версии процесса + 3. Начальные переменные процесса в виде `JSON`-строки (также принимаются `Map` или `@Json`-объект) + 4. Отправить команду и заблокироваться до подтверждения от `Zeebe` (используйте возвращаемый `CompletionStage` для неблокирующего вызова) + +Тот же клиент публикует сообщения (`client.newPublishMessageCommand()`) и выполняет любые другие команды `Zeebe`. + +#### Настройка клиента { #client-customization } + +`ZeebeClient` можно донастроить необязательными компонентами графа, которые модуль подхватывает автоматически: + +* `CredentialsProvider` — авторизация для `Zeebe` (`Camunda 8 SaaS` или self-managed с `OAuth`); +* `JsonMapper` — пользовательский `JSON`-маппер, используемый `ZeebeClient` для (де)сериализации переменных; +* `ScheduledExecutorService` — пул потоков, используемый исполнителями заданий; +* `ClientInterceptor` — `gRPC`-перехватчик, применяемый к каналу `Zeebe` (собираются все зарегистрированные перехватчики). + +Например, чтобы аутентифицироваться в `Camunda 8` через `OAuth`, предоставьте компонент `CredentialsProvider`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface ZeebeAuthModule { + + default CredentialsProvider zeebeCredentialsProvider() { + return CredentialsProvider.newCredentialsProviderBuilder() + .clientId("client-id") + .clientSecret("client-secret") + .audience("zeebe.camunda.io") + .authorizationServerUrl("https://login.cloud.camunda.io/oauth/token") + .build(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface ZeebeAuthModule { + + fun zeebeCredentialsProvider(): CredentialsProvider = + CredentialsProvider.newCredentialsProviderBuilder() + .clientId("client-id") + .clientSecret("client-secret") + .audience("zeebe.camunda.io") + .authorizationServerUrl("https://login.cloud.camunda.io/oauth/token") + .build() + } + ``` + ## Исполнители { #worker } -Исполнитель - это обработчик, способный выполнять определенное задание в процессе. -Каждый раз, когда необходимо выполнить такое задание, оно представляется в виде задачи исполнителю. +Исполнитель — это обработчик, способный выполнять определенное задание в процессе. +Когда в процессе появляется задание нужного типа, `Zeebe` активирует его и передает одному из исполнителей. ### Конфигурация { #configuration-2 } -Существует конфигурация по умолчанию, которая применяется ко всем исполнителям при создании -и затем применяются именованные настройки конкретного исполнителя ([по типу исполнителя `Type`](https://docs.camunda.io/docs/components/concepts/job-workers/)) -для переопределения настроек по умолчанию. -Можно изменить настройки по умолчанию для всех прерывателей одновременно изменив конфигурацию по умолчанию (`default`). +Существует конфигурация по умолчанию, которая применяется ко всем исполнителям при создании, а затем поверх нее +применяются именованные настройки конкретного исполнителя по [типу исполнителя (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/). +Чтобы изменить настройки сразу для всех исполнителей, переопределите секцию `default`. +Чтобы изменить настройки только одного исполнителя, добавьте секцию с именем его типа, указанного в `@JobWorker`. +Если секция `zeebe.worker.job` не указана, используется встроенная конфигурация по умолчанию. -Пример полной конфигурации исполнителя описан в классе `ZeebeWorkerConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации исполнителя, описанной в классе `ZeebeWorkerConfig` (указаны примеры значений или значения по умолчанию): -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript zeebe { @@ -210,7 +365,7 @@ agent: minDelay = "100ms" //(11)! maxDelay = "500ms" //(12)! factor = 1.0 //(10)! - jitter = 1.3 //(13)! + jitter = 1.1 //(13)! } } } @@ -218,20 +373,19 @@ agent: } ``` - 1. [Тип обработчика (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) или имя настроек по умолчанию (`default`) - 2. Включить ли исполнителя - 3. Максимальное время выполнения одной задачи исполнителем - 4. Максимальное количество задач, которые будут одновременно активированы только для этого исполнителя. Это используется для управления скорость работы производителя данных для согласования со скоростью работы исполнителя (`backpressure`) - 5. Ограничение времени запроса используемого для опроса нового задания исполнителем - 6. Максимальный интервал между опросами новых задач. Рабочий автоматически пытается всегда активировать новые задания после завершения работы. Если ни одно задание не может быть активировано после завершения, исполнитель будет периодически опрашивать новые задания - 7. Указывает индетификаторы тенантов, которые могут владеть любыми сущностями (например, определением процесса, экземплярами процесса и т. д.), полученными в результате выполнения этой команды - 8. Если установлено значение «включено», рабочий будет использовать сочетание потоковой передачи и опроса для активации заданий - 9. Если потоковая передача включена, устанавливает максимальное время жизни для данного потока - 10. Устанавливает минимальную задержку повтора. Обратите внимание, что из-за `jitter` задержка повтора может оказаться ниже этого минимума - 11. Устанавливает максимальную задержку повтора. Обратите внимание, что `jitter` может превысить эту максимальную задержку - 12. Устанавливает коэффициент умножения задержки. Предыдущая задержка умножается на этот коэффициент - 13. Устанавливает коэффициент джиттера. Следующая задержка изменяется случайным образом в диапазоне +/- этого коэффициента. - Например, если следующая задержка рассчитывается как 1 с, а `jitter` равен 0,1, то фактическая следующая задержка может быть где-то между 0,9 и 1,1 с + 1. [Тип исполнителя (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) или имя настроек по умолчанию `default` + 2. Включен ли исполнитель (по умолчанию: `true`) + 3. Максимальное время выполнения одного задания исполнителем (по умолчанию: `15m`) + 4. Максимальное количество заданий, которые будут одновременно активированы для этого исполнителя; используется для согласования скорости получения заданий со скоростью их обработки (`backpressure`) (по умолчанию: `32`) + 5. Ограничение времени запроса, который используется для опроса нового задания исполнителем (по умолчанию: `15s`) + 6. Максимальный интервал между опросами новых заданий; если после завершения работы новые задания не активированы, исполнитель периодически опрашивает брокер (по умолчанию: `100ms`) + 7. Идентификаторы `tenant`, для которых исполнитель может получать задания (по умолчанию: `[]`) + 8. Использовать ли потоковую передачу вместе с опросом для активации заданий (по умолчанию: `false`) + 9. Максимальное время жизни потока, если потоковая передача включена (по умолчанию: `15s`) + 10. Минимальная задержка повтора; из-за `jitter` фактическая задержка может оказаться ниже этого минимума (по умолчанию: `100ms`) + 11. Максимальная задержка повтора; из-за `jitter` фактическая задержка может превысить это значение (по умолчанию: `500ms`) + 12. Коэффициент умножения задержки: предыдущая задержка умножается на это значение (по умолчанию: `1.0`) + 13. Коэффициент `jitter`: следующая задержка случайно изменяется в диапазоне `+/-` этого коэффициента (по умолчанию: `1.1`) === ":simple-yaml: `YAML`" @@ -252,29 +406,75 @@ agent: minDelay: "100ms" #(11)! maxDelay: "500ms" #(12)! factor: 1.0 #(10)! - jitter: 1.3 #(13)! - ``` - - 1. [Тип обработчика (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) или имя настроек по умолчанию (`default`) - 2. Включить ли исполнителя - 3. Максимальное время выполнения одной задачи исполнителем - 4. Максимальное количество задач, которые будут одновременно активированы только для этого исполнителя. Это используется для управления скорость работы производителя данных для согласования со скоростью работы исполнителя (`backpressure`) - 5. Ограничение времени запроса используемого для опроса нового задания исполнителем - 6. Максимальный интервал между опросами новых задач. Рабочий автоматически пытается всегда активировать новые задания после завершения работы. Если ни одно задание не может быть активировано после завершения, исполнитель будет периодически опрашивать новые задания - 7. Указывает индетификаторы тенантов, которые могут владеть любыми сущностями (например, определением процесса, экземплярами процесса и т. д.), полученными в результате выполнения этой команды - 8. Если установлено значение «включено», рабочий будет использовать сочетание потоковой передачи и опроса для активации заданий - 9. Если потоковая передача включена, устанавливает максимальное время жизни для данного потока - 10. Устанавливает минимальную задержку повтора. Обратите внимание, что из-за `jitter` задержка повтора может оказаться ниже этого минимума - 11. Устанавливает максимальную задержку повтора. Обратите внимание, что `jitter` может превысить эту максимальную задержку - 12. Устанавливает коэффициент умножения задержки. Предыдущая задержка умножается на этот коэффициент - 13. Устанавливает коэффициент джиттера. Следующая задержка изменяется случайным образом в диапазоне +/- этого коэффициента. - Например, если следующая задержка рассчитывается как 1 с, а `jitter` равен 0,1, то фактическая следующая задержка может быть где-то между 0,9 и 1,1 с + jitter: 1.1 #(13)! + ``` + + 1. [Тип исполнителя (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) или имя настроек по умолчанию `default` + 2. Включен ли исполнитель (по умолчанию: `true`) + 3. Максимальное время выполнения одного задания исполнителем (по умолчанию: `15m`) + 4. Максимальное количество заданий, которые будут одновременно активированы для этого исполнителя; используется для согласования скорости получения заданий со скоростью их обработки (`backpressure`) (по умолчанию: `32`) + 5. Ограничение времени запроса, который используется для опроса нового задания исполнителем (по умолчанию: `15s`) + 6. Максимальный интервал между опросами новых заданий; если после завершения работы новые задания не активированы, исполнитель периодически опрашивает брокер (по умолчанию: `100ms`) + 7. Идентификаторы `tenant`, для которых исполнитель может получать задания (по умолчанию: `[]`) + 8. Использовать ли потоковую передачу вместе с опросом для активации заданий (по умолчанию: `false`) + 9. Максимальное время жизни потока, если потоковая передача включена (по умолчанию: `15s`) + 10. Минимальная задержка повтора; из-за `jitter` фактическая задержка может оказаться ниже этого минимума (по умолчанию: `100ms`) + 11. Максимальная задержка повтора; из-за `jitter` фактическая задержка может превысить это значение (по умолчанию: `500ms`) + 12. Коэффициент умножения задержки: предыдущая задержка умножается на это значение (по умолчанию: `1.0`) + 13. Коэффициент `jitter`: следующая задержка случайно изменяется в диапазоне `+/-` этого коэффициента (по умолчанию: `1.1`) + +Чтобы переопределить настройки для одного исполнителя, добавьте секцию с ключом по [типу исполнителя (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/), +указанному в `@JobWorker`. Именованная секция накладывается поверх `default`, который, в свою очередь, накладывается +поверх встроенных значений по умолчанию, поэтому в именованной секции достаточно перечислить только изменяемые ключи. +Установка `enabled = false` для именованного типа отключает только этого одного исполнителя. + +===! ":material-code-json: `HOCON`" + + ```javascript + zeebe { + worker { + job { + foo { //(1)! + timeout = "30s" + maxJobsActive = 8 + } + bar { //(2)! + enabled = false + } + } + } + } + ``` + + 1. Переопределяет только `timeout` и `maxJobsActive` для исполнителя `@JobWorker("foo")`; все остальные настройки берутся из `default` + 2. Отключает исполнителя `@JobWorker("bar")`, не затрагивая остальную конфигурацию + +=== ":simple-yaml: `YAML`" + + ```yaml + zeebe: + worker: + job: + foo: #(1)! + timeout: "30s" + maxJobsActive: 8 + bar: #(2)! + enabled: false + ``` + + 1. Переопределяет только `timeout` и `maxJobsActive` для исполнителя `@JobWorker("foo")`; все остальные настройки берутся из `default` + 2. Отключает исполнителя `@JobWorker("bar")`, не затрагивая остальную конфигурацию ### Декларативные { #declarative } -Можно создавать декларативно [исполнителей](https://docs.camunda.io/docs/components/concepts/job-workers/) которые будут выполнять работу в рамках Zeebe оркестратора. +Можно декларативно создавать [исполнителей](https://docs.camunda.io/docs/components/concepts/job-workers/), которые будут +выполнять работу в рамках оркестратора `Zeebe`. + +В аннотации `@JobWorker` указывается [тип исполнителя (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) +из процесса. По этому значению `Zeebe` связывает задание из `BPMN`-процесса с обработчиком в приложении. -В аннотации `JobWorker` указывается значение [типа исполнителя (`Type`)](https://docs.camunda.io/docs/components/concepts/job-workers/) в рамках процесса. +Метод исполнителя может объявлять только параметры `@JobVariable`, `@JobVariables` и `JobContext` — любой другой тип +параметра отклоняется на этапе компиляции. Сырые `JobClient` и `ActivatedJob` доступны только в [императивном](#imperative) исполнителе. ===! ":fontawesome-brands-java: `Java`" @@ -304,8 +504,8 @@ agent: #### Параметр контекст { #parameter-context } -Можно внедрять контекст исполнения как аргумент метода, -контекст исполнения имеет метаданные задачи, исполнителя и процесса доступные для текущей задачи, которая на исполнении. +Можно внедрить контекст задания как аргумент метода. +`JobContext` содержит метаданные текущего задания, исполнителя и процесса. ===! ":fontawesome-brands-java: `Java`" @@ -333,12 +533,62 @@ agent: } ``` +`JobContext` предоставляет следующие методы только для чтения: + +| Метод | Описание | +|------------------------------|--------------------------------------------------------------------------------------| +| `jobKey()` | Уникальный ключ активированного задания | +| `jobName()` | Имя/тип исполнителя, под которым зарегистрирован этот обработчик (значение `@JobWorker`) | +| `jobType()` | Тип активированного задания, как определено в `BPMN`-процессе | +| `jobWorker()` | Имя исполнителя, активировавшего задание на стороне брокера | +| `tenantId()` | Идентификатор `tenant`, которому принадлежит задание | +| `processId()` | Идентификатор `BPMN`-процесса | +| `processInstanceKey()` | Ключ экземпляра процесса, которому принадлежит задание | +| `processDefinitionVersion()` | Версия развернутого определения процесса | +| `processDefinitionKey()` | Ключ развернутого определения процесса | +| `elementId()` | Идентификатор `BPMN`-элемента, для которого создано задание | +| `elementInstanceKey()` | Ключ экземпляра `BPMN`-элемента | +| `headers()` | Пользовательские заголовки, заданные для задания в `BPMN`-модели | +| `retryCount()` | Количество оставшихся повторов для задания | +| `deadline()` | Момент (`Instant`), до которого задание эксклюзивно закреплено за исполнителем | +| `deadlineAsMillis()` | Тот же крайний срок, выраженный в миллисекундах эпохи | +| `variablesAsString()` | Сырые переменные задания в виде `JSON`-строки | + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeJob { + + @JobWorker("someJobType") + public void process(JobContext context) { + logger.info("Job {} of process {} at element {} with deadline {}", + context.jobType(), context.processInstanceKey(), context.elementId(), context.deadline()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeJob { + + @JobWorker("someJobType") + fun process(context: JobContext) { + logger.info("Job {} of process {} at element {} with deadline {}", + context.jobType(), context.processInstanceKey(), context.elementId(), context.deadline()) + } + } + ``` + #### Параметр переменная { #parameter-variable } -Можно внедрять [переменные процесса](https://docs.camunda.io/docs/components/concepts/variables/) как аргументы метода, -переменная процесса является частью состояния процесса и может быть установлена на старте или как часть результата исполнителя. +Можно внедрять [переменные процесса](https://docs.camunda.io/docs/components/concepts/variables/) как аргументы метода. +Переменная процесса является частью состояния процесса и может быть установлена при старте процесса или как часть результата исполнителя. -Важно, если указана хоть одна именованная переменная, то только эти переменные будут переданы на получение из оркестратора. +Если указана хотя бы одна переменная через `@JobVariable`, сгенерированный исполнитель будет запрашивать у `Zeebe` +только такие переменные. Если `@JobVariable` не используется, исполнитель запрашивает все переменные задания. ===! ":fontawesome-brands-java: `Java`" @@ -366,10 +616,10 @@ agent: } ``` -Можно указать имя переменной из контекста, либо будет использовано имя аргумента метода по умолчанию. +Можно указать имя переменной явно в `@JobVariable`, либо будет использовано имя аргумента метода по умолчанию. -Так как все переменные процесса обязаны быть JSON объектами, -то аргумент метода может представлять собой также любое отображение JSON объекта. +Так как переменные процесса передаются как `JSON`, аргумент метода может быть пользовательским типом, для которого +доступны `JsonReader` и `JsonWriter`. ===! ":fontawesome-brands-java: `Java`" @@ -404,8 +654,8 @@ agent: #### Параметр переменные { #parameter-variables } -Можно внедрить сразу несколько [переменных процесса](https://docs.camunda.io/docs/components/concepts/variables/) как аргумент метода, -как один объект, который представляет собой JSON объекты в состоянии процесса. +Можно внедрить сразу несколько [переменных процесса](https://docs.camunda.io/docs/components/concepts/variables/) одним +аргументом метода через `@JobVariables`. Такой аргумент представляет все переменные задания как один `JSON`-объект. ===! ":fontawesome-brands-java: `Java`" @@ -445,9 +695,9 @@ agent: #### Результат { #result } -Можно не просто выполнять работу, но и возвращать результат выполнения работы как переменную в контекст процесса. +Можно не только выполнять работу, но и возвращать результат как переменные в контекст процесса. -Результат можно возвращать как `Map` описывающую структуру JSON ответа. +Результат можно возвращать как `Map`, который описывает структуру `JSON`-ответа. ===! ":fontawesome-brands-java: `Java`" @@ -475,8 +725,8 @@ agent: } ``` -Так и возвращать сразу именованный результат как одну переменную, -что будет аналогом одного ключа и значения в `Map` объекте. +Также можно возвращать именованный результат как одну переменную. Это аналог одного ключа и значения в объекте +`Map`. В таком случае обязательно требуется указать имя переменной в аннотации `@JobVariable`: @@ -515,8 +765,20 @@ agent: #### Ошибки { #errors } -В случае если требуется завершить исполнение ошибкой, можно бросить исключение `JobWorkerException` где можно указать, -как код ошибки, так и сообщение и переменные процесса если того требуется. +Если требуется завершить исполнение ошибкой процесса, бросьте `JobWorkerException`. +В исключении можно указать код ошибки, сообщение и переменные процесса, если они нужны. +Это исключение преобразуется в команду `throwError` для `Zeebe`: `getCode()`, сообщение и `getVariables()` исключения +передаются как код ошибки, сообщение об ошибке и переменные команды. + +Если обработчик выбрасывает любое другое исключение, модуль оборачивает его в `JobWorkerException` с одним из следующих +встроенных кодов: + +| Код | Когда используется | +|-------------------|-----------------------------------------------------------------------------| +| `DESERIALIZATION` | Переменную задания не удалось прочитать/десериализовать в аргумент метода | +| `SERIALIZATION` | Результат исполнителя не удалось записать/сериализовать в переменные | +| `UNEXPECTED` | Из синхронного обработчика было выброшено неожиданное исключение | +| `INTERNAL` | Резервный код для любой другой ошибки, не охваченной выше | ===! ":fontawesome-brands-java: `Java`" @@ -526,11 +788,13 @@ agent: @JobWorker("someJobType") public User process() { - throw new JobWorkerException("DOESNT_WORK"); + throw new JobWorkerException("DOESNT_WORK"); //(1)! } } ``` + 1. Дополнительные перегрузки принимают сообщение/причину и `Map` переменных для добавления в команду `throwError` + === ":simple-kotlin: `Kotlin`" ```kotlin @@ -539,14 +803,17 @@ agent: @JobWorker("someJobType") fun process(): User { - throw JobWorkerException("DOESNT_WORK") + throw JobWorkerException("DOESNT_WORK") //(1)! } } ``` + 1. Дополнительные перегрузки принимают сообщение/причину и `Map` переменных для добавления в команду `throwError` + ### Императивные { #imperative } -Можно также создавать более низкоуровневые исполнители и напрямую работать с контрактами `ZeebeClient` и его интерфейсом. +Можно также создавать более низкоуровневые исполнители и напрямую работать с контрактами `ZeebeClient`. +Для этого компонент должен реализовать интерфейс `KoraJobWorker`. ===! ":fontawesome-brands-java: `Java`" @@ -559,34 +826,51 @@ agent: return "someJobType"; } + @Override + public List fetchVariables() { + return List.of("startId"); //(1)! + } + @Override public CompletionStage> handle(JobClient client, ActivatedJob job) { - return client.newCompleteCommand(job); + return CompletableFuture.completedFuture(client.newCompleteCommand(job)); } } ``` + 1. Из `Zeebe` запрашиваются только эти переменные; верните пустой список (по умолчанию), чтобы запросить **все** переменные + === ":simple-kotlin: `Kotlin`" ```kotlin @Component class SomeJob : KoraJobWorker { - fun type(): String = "someJobType" + override fun type(): String = "someJobType" + + override fun fetchVariables(): List = listOf("startId") //(1)! - fun handle(client: JobClient, job: ActivatedJob): CompletionStage> { - return client.newCompleteCommand(job) + override fun handle(client: JobClient, job: ActivatedJob): CompletionStage> { + return CompletableFuture.completedFuture(client.newCompleteCommand(job)) } } ``` + 1. Из `Zeebe` запрашиваются только эти переменные; верните пустой список (по умолчанию), чтобы запросить **все** переменные + +Метод `fetchVariables()` — императивный аналог `@JobVariable`: он определяет, какие переменные процесса `Zeebe` +отправляет вместе с заданием. По умолчанию он возвращает пустой список, что запрашивает все переменные; непустой список +ограничивает передаваемые данные только этими переменными. В отличие от декларативных исполнителей, `handle` получает +сырые `JobClient` и `ActivatedJob` и отвечает за завершение задания (например, через `client.newCompleteCommand(job)`). + ## Сигнатуры { #signatures } -Доступные сигнатуры для методов репозитория из коробки: +Доступные сигнатуры для методов исполнителя из коробки: ===! ":fontawesome-brands-java: `Java`" Под `T` подразумевается тип возвращаемого значения, либо `Void`. + Если результат равен `null` или `Optional.empty()`, задание будет завершено без добавления переменных. - `T myMethod()` - `Optional myMethod()` @@ -596,6 +880,7 @@ agent: === ":simple-kotlin: `Kotlin`" Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `Unit`. + Если результат равен `null`, задание будет завершено без добавления переменных. - `myMethod(): T` - `myMethod(): Deferred` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) diff --git a/mkdocs/docs/ru/documentation/config.md b/mkdocs/docs/ru/documentation/config.md index 736ac2a..99a39ab 100644 --- a/mkdocs/docs/ru/documentation/config.md +++ b/mkdocs/docs/ru/documentation/config.md @@ -4,15 +4,22 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora configuration system for HOCON and YAML, typed config extraction, config injection, config sources, watchers, and supported value types; key triggers include @ConfigSource, @ConfigValueExtractor, @Environment, @SystemProperties, Config, HoconConfigModule, YamlConfigModule." --- -Модуль конфигурации отвечает за отображение значений из файлов конфигурации на классы в Kora -и их последующее использование для настройки приложения. +Модуль конфигурации читает настройки приложения из файлов `HOCON` или `YAML`, переменных окружения, системных свойств +`Java` и отображает их на типизированные классы в `Kora`. Полученные объекты конфигурации становятся обычными +компонентами графа зависимостей и могут внедряться в сервисы, клиенты, серверы и другие интеграции. -Если нужен пошаговый разбор перед справочным описанием, смотрите [Конфигурация HOCON](../guides/config-hocon.md) и [Конфигурация YAML](../guides/config-yaml.md). +В `Kora` конфигурация приложения обычно описывается интерфейсом с аннотацией `@ConfigSource`: путь в файле указывает на +читаемую секцию, а методы интерфейса описывают обязательные значения, необязательные значения и значения по умолчанию. +Библиотеки и переиспользуемые формы конфигурации используют `@ConfigValueExtractor`, который создает только правило +извлечения, тогда как конкретный путь выбирается в модуле библиотеки. + +Для пошагового разбора перед справочным описанием смотрите [Конфигурация HOCON](../guides/config-hocon.md) и [Конфигурация YAML](../guides/config-yaml.md). ## HOCON { #hocon } Поддержка [HOCON](https://github.com/lightbend/config/blob/master/HOCON.md) реализована с помощью [Typesafe Config](https://github.com/lightbend/config). -HOCON — это формат конфиг-файлов, основанный на JSON. Формат менее строгий нежели JSON и обладает слегка другим синтаксисом. +`HOCON` — это формат конфигурационных файлов на основе `JSON`. Он менее строгий, чем `JSON`, и поддерживает подстановки, +значения по умолчанию и удобный синтаксис для вложенных объектов. ```javascript services { @@ -48,19 +55,24 @@ services { } ``` -1. Cтроковое значение конфигурации -2. Числовое значение конфигурации -3. Обязательное значение конфигурации которое подставляется из переменной окружения `REQUIRED_ENV_VALUE` -4. Необязательное значение конфигурации которое подставляется из переменной окружения `OPTIONAL_ENV_VALUE`, если таковой переменной не найдено то значение конфигурации будет опущено -5. Значение конфигурации со значением по умолчанию, значение по умолчанию указывается в `propDefault = 10` и если будет найдена переменная окружения `NON_DEFAULT_ENV_VALUE` то ее значение заменит значение по умолчанию -6. Значение конфигурации собранное из подстановок других частей конфигурации и значением `Other` между -7. Значение конфигурации списка строк, значение задается как массив строк либо также можно задать как строку, где значения разделены запятыми -8. Значение конфигурации списка строк, значение задается как строка, где значения разделены запятыми либо также можно задать как массив строк -9. Значение конфигурации в виде словаря ключ и значение -10. Значение конфигурации в виде отображенного класса -11. Значение конфигурации в виде списка отображенных классов - -Отображение конфигурации в коде: +1. Строковое значение конфигурации +2. Числовое значение конфигурации +3. Обязательное значение конфигурации, подставляемое из переменной окружения `REQUIRED_ENV_VALUE` +4. Необязательное значение конфигурации, подставляемое из переменной окружения `OPTIONAL_ENV_VALUE`; если переменная не найдена, значение конфигурации опускается +5. Значение конфигурации со значением по умолчанию: значение по умолчанию задается как `propDefault = 10`, а `NON_DEFAULT_ENV_VALUE`, если найдено, заменяет его +6. Значение конфигурации, собранное из подстановок других частей конфигурации со значением `Other` между ними +7. Значение конфигурации в виде списка строк; значение можно задать как массив строк или как строку с разделителем-запятой +8. Значение конфигурации в виде списка строк; значение можно задать как строку с разделителем-запятой или как массив строк +9. Значение конфигурации в виде словаря ключ-значение +10. Значение конфигурации в виде отображаемого класса +11. Значение конфигурации в виде списка отображаемых классов + +Значения также могут ссылаться на другие ключи конфигурации (само-ссылка / перекрестная ссылка) через `${path}`, +а на переменные окружения — через `${VAR}` (обязательные), `${?VAR}` (необязательные) или через резервный вариант по +умолчанию. Все подстановки разрешаются после слияния каждого слоя, поэтому ссылка может указывать на ключ, определенный +в другом файле или в другом слое конфигурации. + +Представление конфигурации в коде: ===! ":fontawesome-brands-java: `Java`" @@ -89,7 +101,7 @@ services { @ConfigValueExtractor public interface ObjectConfig { - + String p1(); String p2(); @@ -127,7 +139,7 @@ services { @ConfigValueExtractor interface ObjectConfig { - + fun p1(): String fun p2(): String @@ -169,34 +181,41 @@ services { ### Файл { #file } -По умолчанию ожидаются файлы конфигурации [reference.conf и application.conf](https://github.com/lightbend/config#note-about-resolving-substitutions-in-referenceconf-and-applicationconf) +По умолчанию ожидаются конфигурационные файлы [`reference.conf` и `application.conf`](https://github.com/lightbend/config#note-about-resolving-substitutions-in-referenceconf-and-applicationconf). -Во-первых, все файлы `reference.conf` объединяются, во-вторых, файл `application.conf` накладывается на неразрешенный -файл `reference.conf`, результат вычисляется и проверяется что все значения переменных доступны. +Сначала объединяются все файлы `reference.conf` из classpath, затем поверх неразрешенного `reference.conf` накладывается +`application.conf`, после чего результат разрешается и проверяются обязательные подстановки. -Предполагается что конфигурация приложения находится в файле `application.conf`, а конфигурации библиотек в `reference.conf`. +Ожидается, что конфигурация приложения находится в `application.conf`, а конфигурация библиотек — в `reference.conf`. -Приоритет считывания `application.conf` файла конфигурации: +`HOCON` также поддерживает директиву [`include`](https://github.com/lightbend/config/blob/master/HOCON.md#includes): +файлы, подключенные через `include`, участвуют в том же слиянии и разрешении подстановок, что и основной файл, +и отслеживаются [наблюдателем за конфигурацией](#config-watcher), поэтому изменения во включенном файле также обновляют граф. -- Использовать файл из `config.resource` если указан (файл из `resources` директории) -- Использовать файл из `config.file` если указан (файл из файловой системы) -- Использовать файл `application.conf` если имеется (файл из `resources` директории) -- Используется пустой файл конфигурации если все указанное выше отсутствует +Приоритет выбора файла приложения для `HOCON`: + +- Использовать файл из `config.resource`, если он указан (файл из каталога `resources`) +- Использовать файл из `config.file`, если он указан (файл из файловой системы) +- Использовать `application.conf`, если он присутствует (файл из каталога `resources`) +- Использовать пустую конфигурацию, если ничего из вышеперечисленного нет + +Одновременно можно указать только одно свойство: `config.resource` или `config.file`. Если указаны оба свойства, +приложение не запустится. ===! ":fontawesome-brands-java: `java`" - Пример указания конфига при запуске в терминале через `java`: + Пример указания конфигурации при запуске через `java`: ```shell java -Dconfig.file=path/to/configFile application ``` === ":simple-kotlin: `gradle`" - Пример указания конфига в `build.gradle`: + Пример указания конфигурации в `build.gradle`: ```groovy run { jvmArgs += [ - "-Dconfig.file=path/to/configFile", + "-Dconfig.file=path/to/configFile" ] } ``` @@ -207,41 +226,41 @@ services { ```yaml services: - foo: - bar: "SomeValue" #(1)! - baz: 10 #(2)! - propRequired: ${REQUIRED_ENV_VALUE} #(3)! - propOptional: ${?OPTIONAL_ENV_VALUE} #(4)! - propDefault: ${?NON_DEFAULT_ENV_VALUE:10} #(5)! - propReference: ${services.foo.bar}Other${services.foo.baz} #(6)! - propArray: ["v1", "v2"] #(7)! - propArrayAsString: "v1, v2" #(8)! - propMap: #(9)! - k1: "v1" - k2: "v2" - propObject: #(10)! - p1: "v1" - p2: "v2" - propObjects: #(11)! - - p1: "v1" - p2: "v2" - - p1: "v1" - p2: "v2" + foo: + bar: "SomeValue" #(1)! + baz: 10 #(2)! + propRequired: ${REQUIRED_ENV_VALUE} #(3)! + propOptional: ${?OPTIONAL_ENV_VALUE} #(4)! + propDefault: ${?NON_DEFAULT_ENV_VALUE:10} #(5)! + propReference: ${services.foo.bar}Other${services.foo.baz} #(6)! + propArray: ["v1", "v2"] #(7)! + propArrayAsString: "v1, v2" #(8)! + propMap: #(9)! + k1: "v1" + k2: "v2" + propObject: #(10)! + p1: "v1" + p2: "v2" + propObjects: #(11)! + - p1: "v1" + p2: "v2" + - p1: "v1" + p2: "v2" ``` -1. Cтроковое значение конфигурации -2. Числовое значение конфигурации -3. Обязательное значение конфигурации которое подставляется из переменной окружения `REQUIRED_ENV_VALUE` -4. Необязательное значение конфигурации которое подставляется из переменной окружения `OPTIONAL_ENV_VALUE`, если таковой переменной не найдено то значение конфигурации будет опущено -5. Значение конфигурации со значением по умолчанию, значение по умолчанию равно `10` и если будет найдена переменная окружения `NON_DEFAULT_ENV_VALUE` то ее значение заменит значение по умолчанию -6. Значение конфигурации собранное из подстановок других частей конфигурации и значением `Other` между -7. Значение конфигурации списка строк, значение задается как массив строк либо также можно задать как строку, где значения разделены запятыми -8. Значение конфигурации списка строк, значение задается как строка, где значения разделены запятыми либо также можно задать как массив строк -9. Значение конфигурации в виде словаря ключ и значение -10. Значение конфигурации в виде отображенного класса -11. Значение конфигурации в виде списка отображенных классов +1. Строковое значение конфигурации +2. Числовое значение конфигурации +3. Обязательное значение конфигурации, подставляемое из переменной окружения `REQUIRED_ENV_VALUE` +4. Необязательное значение конфигурации, подставляемое из переменной окружения `OPTIONAL_ENV_VALUE`; если переменная не найдена, значение конфигурации опускается +5. Значение конфигурации со значением по умолчанию: значение по умолчанию — `10`, а `NON_DEFAULT_ENV_VALUE`, если найдено, заменяет его +6. Значение конфигурации, собранное из подстановок других частей конфигурации со значением `Other` между ними +7. Значение конфигурации в виде списка строк; значение можно задать как массив строк или как строку с разделителем-запятой +8. Значение конфигурации в виде списка строк; значение можно задать как строку с разделителем-запятой или как массив строк +9. Значение конфигурации в виде словаря ключ-значение +10. Значение конфигурации в виде отображаемого класса +11. Значение конфигурации в виде списка отображаемых классов -Отображение конфигурации в коде: +Представление конфигурации в коде: ===! ":fontawesome-brands-java: `Java`" @@ -270,7 +289,7 @@ services: @ConfigValueExtractor public interface ObjectConfig { - + String p1(); String p2(); @@ -308,7 +327,7 @@ services: @ConfigValueExtractor interface ObjectConfig { - + fun p1(): String fun p2(): String @@ -350,46 +369,51 @@ services: ### Файл { #file-2 } -По умолчанию ожидаются файлы конфигурации `reference.yaml` и `application.yaml`. +По умолчанию ожидаются конфигурационные файлы `reference.yaml` и `application.yaml`. + +Сначала объединяются все файлы `reference.yaml` из classpath, затем поверх `reference.yaml` накладывается +`application.yaml`, после чего результат разрешается и проверяются обязательные подстановки. -Во-первых, все файлы `reference.yaml` объединяются, во-вторых, файл `application.yaml` накладывается на неразрешенный -файл `reference.yaml`, результат вычисляется и проверяется что все значения переменных доступны. +Ожидается, что конфигурация приложения находится в `application.yaml`, а конфигурация библиотек — в `reference.yaml`. -Предполагается что конфигурация приложения находится в файле `application.yaml`, а конфигурации библиотек в `reference.yaml`. +Приоритет выбора файла приложения для `YAML`: -Приоритет считывания `application.yaml` файла конфигурации: +- Использовать файл из `config.resource`, если он указан (файл из каталога `resources`) +- Использовать файл из `config.file`, если он указан (файл из файловой системы) +- Использовать `application.yaml`, если он присутствует (файл из каталога `resources`) +- Использовать пустую конфигурацию, если ничего из вышеперечисленного нет -- Использовать файл из `config.resource` если указан (файл из `resources` директории) -- Использовать файл из `config.file` если указан (файл из файловой системы) -- Использовать файл `application.yaml` если имеется (файл из `resources` директории) -- Используется пустой файл конфигурации если все указанное выше отсутствует +Одновременно можно указать только одно свойство: `config.resource` или `config.file`. Если указаны оба свойства, +приложение не запустится. ===! ":fontawesome-brands-java: `java`" - Пример указания конфига при запуске в терминале через `java`: + Пример указания конфигурации при запуске через `java`: ```shell java -Dconfig.file=path/to/configFile application ``` === ":simple-kotlin: `gradle`" - Пример указания конфига в `build.gradle`: + Пример указания конфигурации в `build.gradle`: ```groovy run { jvmArgs += [ - "-Dconfig.file=path/to/configFile", + "-Dconfig.file=path/to/configFile" ] } ``` -## Пользовательские конфигурации { #custom-configuration } +## Пользовательская конфигурация { #custom-configuration } -Пользовательская конфигурация предоставляет собой отображение файла конфигурации на пользовательский интерфейс. -Такой пользовательский интерфейс в последствии может быть внедрен как зависимость наравне с другими компонентами. +Пользовательская конфигурация отображает секцию конфигурационного файла на пользовательский тип. +Затем этот тип можно внедрять как зависимость точно так же, как любой другой компонент. -### В приложении { #application-config } +### Конфигурация приложения { #application-config } -Для создания пользовательских конфигураций следует использовать аннотацию `@ConfigSource`: +Для создания пользовательских конфигураций в приложении используйте аннотацию `@ConfigSource`. +Она генерирует `ConfigValueExtractor` для интерфейса и модуль, который добавляет готовый объект конфигурации в граф +зависимостей. Значение аннотации указывает на путь секции внутри итоговой конфигурации: ===! ":fontawesome-brands-java: `Java`" @@ -398,7 +422,7 @@ services: public interface FooServiceConfig { String bar(); - + int baz(); } ``` @@ -415,7 +439,7 @@ services: } ``` -Этот пример кода добавит в контейнер экземпляр класса `FooServiceConfig`, который при создании будет ожидать конфигурацию следующего вида: +Этот пример кода добавит экземпляр класса `FooServiceConfig` в контейнер зависимостей, который при создании будет ожидать конфигурацию следующего вида: ===! ":material-code-json: `Hocon`" @@ -460,12 +484,19 @@ services: class FooService(val config: FooServiceConfig) ``` -### В библиотеке { #library-config } +### Конфигурация библиотеки { #library-config } -Для создания пользовательских конфигураций в рамках пользовательских библиотеках следует использовать аннотацию `@ConfigValueExtractor` -которая создаст правила обработки файла конфигурации в экземпляр класса конфигурации. +Для создания пользовательских конфигураций в библиотеках используйте аннотацию `@ConfigValueExtractor`. +Она создает правило извлечения значения из `ConfigValue`, но не привязывает его к конкретному пути конфигурации. +Путь выбирается в фабричном методе модуля библиотеки, поэтому одну и ту же форму конфигурации можно переиспользовать для разных секций. +`@ConfigValueExtractor` можно использовать на интерфейсе, `record` или классе `Java`, а также на интерфейсе или `data class` `Kotlin`. -Рассмотрим пример когда есть такой класс конфигурации: +У аннотации есть параметр `mapNullAsEmptyObject` (по умолчанию: `true`). Когда он включен, отсутствующая секция +трактуется как пустой объект: обязательные поля по-прежнему приводят к ошибке, а необязательные поля и значения по +умолчанию ведут себя так, как будто присутствовала пустая секция. +Если `mapNullAsEmptyObject = false`, отсутствующая секция трактуется как `null` для всего объекта конфигурации. + +Рассмотрим такой класс конфигурации: ===! ":fontawesome-brands-java: `Java`" @@ -474,7 +505,7 @@ services: public interface FooLibraryConfig { String bar(); - + int baz(); } ``` @@ -491,7 +522,7 @@ services: } ``` -Для того чтобы библиотека предоставляла конфигурацию, требуется реализовать фабрику в модуле: +Чтобы библиотека предоставляла конфигурацию, реализуйте фабрику в модуле: ===! ":fontawesome-brands-java: `Java`" @@ -537,15 +568,17 @@ services: baz: 10 ``` -Затем подключив модуль `FooLibraryModule` в приложении, конфиг `FooServiceConfig` можно использовать как зависимость в других классах. +Затем, после подключения `FooLibraryModule` в приложении, `FooLibraryConfig` можно использовать как зависимость в других классах. ### Обязательные значения { #required-values } -По умолчанию все значения объявленные в конфиге считаются **обязательными** (*NotNull*) и должны присутствовать в файле конфигурации. +По умолчанию все объявленные в конфигурации значения считаются **обязательными** (`NotNull`) и должны присутствовать в +итоговой конфигурации. Если обязательное значение отсутствует или имеет значение `null`, приложение завершится с ошибкой +при создании объекта конфигурации. ### Необязательные значения { #optional-values } -Если есть необходимость указать значение из файла конфигурации как необязательное, то можно воспользоваться таким форматом: +Если требуется указать значение из конфигурационного файла как необязательное, можно использовать такой формат: ===! ":fontawesome-brands-java: `Java`" @@ -562,11 +595,11 @@ services: } ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдет любая аннотация `@Nullable`, например `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - Предполагается использовать [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис и помечать такой параметр как Nullable: + Используйте синтаксис [null-safety `Kotlin`](https://kotlinlang.org/docs/null-safety.html) и пометьте параметр как nullable: ```kotlin @ConfigSource("services.foo") @@ -578,9 +611,12 @@ services: } ``` +Также поддерживается тип возвращаемого значения `Optional` (отсутствующее значение отображается на `Optional.empty()`), +но значение `@Nullable` (или nullable-тип `Kotlin`) является рекомендуемым стилем. + ### Значения по умолчанию { #default-values } -Если есть необходимость использовать задать в отображении значение по умолчанию, то можно воспользоваться `default` модификатором: +Если требуется задать значение по умолчанию при отображении конфигурации, используйте `default`-метод: ===! ":fontawesome-brands-java: `Java`" @@ -610,19 +646,157 @@ services: } ``` +### Гибкие имена ключей { #relaxed-key-names } + +Ключи конфигурации сопоставляются с гибким именованием. Имя метода сравнивается с ключом в файле не только в его точной +форме, но и в вариантах `kebab-case` и `snake_case`. Это означает, что метод `someBarString()` одинаково разрешается из +`someBarString`, `some-bar-string` или `some_bar_string` в конфигурационном файле, поэтому команды, предпочитающие ключи +в стиле kebab-case или snake_case, могут сохранять свой стиль без переименования методов. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @ConfigValueExtractor + public interface BarConfig { + + String someBarString(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @ConfigValueExtractor + interface BarConfig { + + fun someBarString(): String + } + ``` + +Все три написания ключа ниже читаются в `someBarString()`: + +===! ":material-code-json: `Hocon`" + + ```javascript + bar { + someBarString = "value" //(1)! + # some-bar-string = "value" //(2)! + # some_bar_string = "value" //(3)! + } + ``` + + 1. Точное написание имени метода в `camelCase` + 2. Гибкое написание в `kebab-case` + 3. Гибкое написание в `snake_case` + +=== ":simple-yaml: `YAML`" + + ```yaml + bar: + someBarString: "value" #(1)! + # some-bar-string: "value" #(2)! + # some_bar_string: "value" #(3)! + ``` + + 1. Точное написание имени метода в `camelCase` + 2. Гибкое написание в `kebab-case` + 3. Гибкое написание в `snake_case` + +### Рекомендуемый стиль { #recommended-configuration-style } + +Обычно удобнее описывать конфигурацию как отдельный тип для конкретной интеграции или подсистемы: +HTTP-клиента, подключения к внешнему сервису, обработчика очереди и так далее. Такой тип должен четко разделять +обязательные значения, необязательные значения и значения, приходящие из переменных окружения. + +В примере ниже: + +1. `baseUrl` — обязательное значение из конфигурационного файла +2. `clientName` — необязательное значение из переменной окружения `ORDERS_CLIENT_NAME` +3. `token` — обязательное значение из переменной окружения `ORDERS_API_TOKEN` +4. `requestTimeout` имеет значение по умолчанию `2s` и может быть переопределено необязательной переменной окружения `ORDERS_REQUEST_TIMEOUT` + +===! ":fontawesome-brands-java: `Java`" + + ```java + import java.time.Duration; + import javax.annotation.Nullable; + + @ConfigSource("clients.orders") + public interface OrdersClientConfig { + + String baseUrl(); + + @Nullable + String clientName(); + + String token(); + + Duration requestTimeout(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import java.time.Duration + + @ConfigSource("clients.orders") + interface OrdersClientConfig { + + fun baseUrl(): String + + fun clientName(): String? + + fun token(): String + + fun requestTimeout(): Duration + } + ``` + +===! "`HOCON`" + + ```javascript + clients { + orders { + baseUrl = "https://orders.example.com" + clientName = ${?ORDERS_CLIENT_NAME} + token = ${ORDERS_API_TOKEN} + requestTimeout = 2s + requestTimeout = ${?ORDERS_REQUEST_TIMEOUT} + } + } + ``` + +=== "`YAML`" + + ```yaml + clients: + orders: + baseUrl: "https://orders.example.com" + clientName: ${?ORDERS_CLIENT_NAME} + token: ${ORDERS_API_TOKEN} + requestTimeout: ${?ORDERS_REQUEST_TIMEOUT:2s} + ``` + +Это сохраняет структуру конфигурации читаемой: обязательные настройки видны в типе конфигурации, секреты можно передавать +через переменные окружения, а безопасные значения по умолчанию остаются прямо в конфигурационном файле. + ## Внедрение конфигурации { #injecting-configuration } -Можно внедрять базовый класс `ru.tinkoff.kora.config.common.Config` который предоставляет из себя общую абстракцию над -отображением файла конфигурации. Результирующие отображение конфигурации состоит из нескольких слоев которые представляют из себя: +Можно внедрить базовый класс `ru.tinkoff.kora.config.common.Config`, который представляет дерево конфигурации и дает +доступ к значениям через метод `get(...)`. Итоговая конфигурация состоит из нескольких слоев: - Переменные окружения -- Системные переменные -- Файл конфигурации +- Системные свойства `Java` +- Конфигурационный файл + +Слои объединяются в таком порядке: переменные окружения, затем системные свойства, затем конфигурационный файл +приложения. Каждый следующий слой накладывается на предыдущий. ### Переменные окружения { #environment-variables } -В случае если требуется внедрить конфигурацию **только** [переменных окружения](https://ru.hexlet.io/courses/cli-basics/lessons/environment-variables/theory_unit), -то для этого можно использовать аннотацию `@Environment` как тег для класса конфигурации: +Если требуется внедрить конфигурацию, содержащую **только** [переменные окружения](https://en.wikipedia.org/wiki/Environment_variable), +используйте аннотацию `@Environment` как тег для класса конфигурации: ===! ":fontawesome-brands-java: `Java`" @@ -645,10 +819,10 @@ services: class FooService(@Environment val config: Config) ``` -### Системные переменные { #system-variables } +### Системные свойства { #system-variables } -В случае если требуется внедрить конфигурацию **только** [системных переменных](https://www.baeldung.com/java-system-get-property-vs-system-getenv), -то для этого можно использовать аннотацию `@SystemProperties` как тег для класса конфигурации: +Если требуется внедрить конфигурацию, содержащую **только** [системные свойства `Java`](https://www.baeldung.com/java-system-get-property-vs-system-getenv), +используйте аннотацию `@SystemProperties` как тег для класса конфигурации: ===! ":fontawesome-brands-java: `Java`" @@ -671,10 +845,10 @@ services: class FooService(@SystemProperties val config: Config) ``` -### Файл конфигурации { #configuration-file } +### Конфигурационный файл { #configuration-file } -В случае если требуется внедрить полную конфигурацию приложения которая состоит **только** из файла конфигурации, -то для этого можно использовать аннотацию `@ApplicationConfig` как тег для класса конфигурации: +Если требуется внедрить конфигурацию приложения, состоящую **только** из конфигурационного файла, +используйте аннотацию `@ApplicationConfig` как тег для класса конфигурации: ===! ":fontawesome-brands-java: `Java`" @@ -697,10 +871,10 @@ services: class FooService(@ApplicationConfig val config: Config) ``` -### Результирующая конфигурация { #resulting-configuration } +### Итоговая конфигурация { #resulting-configuration } -В случае если требуется внедрить полную конфигурацию приложения которая состоит из файла конфигурации, -переменных окружения и системных переменных, то для этого требуется просто внедрить класс конфигурации без тега: +Если требуется внедрить полную итоговую конфигурацию приложения, которая состоит из конфигурационного файла, +переменных окружения и системных свойств, просто внедрите класс конфигурации без тега: ===! ":fontawesome-brands-java: `Java`" @@ -723,28 +897,74 @@ services: class FooService(val config: Config) ``` -### Совет { #recommendations } +### Чтение сырого Config { #reading-raw-config-values } + +Когда внедряется сырой `Config`, значения читаются через метод `get(...)`, который возвращает узел `ConfigValue` +для запрошенного пути. `ConfigValue` — это sealed-тип с типизированными аксессорами: `asString()`, `asNumber()`, +`asBoolean()`, `asObject()`, `asArray()` и `isNull()`. Если значение имеет неожиданный тип, аксессор выбрасывает +`ConfigValueExtractionException`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class FooService { + + public FooService(Config config) { + ConfigValue value = config.get("services.foo.bar"); + if (!value.isNull()) { + String bar = value.asString(); + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class FooService(config: Config) { + + init { + val value = config["services.foo.bar"] + if (!value.isNull) { + val bar = value.asString() + } + } + } + ``` + +Как отмечено в разделе [Рекомендации](#recommendations), предпочитайте типизированные [пользовательские конфигурации](#custom-configuration) +чтению сырого `Config`. +Используйте API сырого чтения только если ни как не обойтись для динамического или обобщенного доступа через `ValueOf` чтобы не было обновления компонент. + +???+ warning "Внимание" + + **Мы настоятельно не рекомендуем** использовать `ru.tinkoff.kora.config.common.Config` напрямую как зависимость в компонентах, + потому что при обновлении конфигурации будут обновлены и все компоненты графа, которые ее используют. + Мы рекомендуем всегда создавать [пользовательские конфигурации](#custom-configuration). -???+ warning "Совет" +## Наблюдатель за конфигурацией { #config-watcher } - **Мы не советуем** использовать напрямую `ru.tinkoff.kora.config.common.Config` как зависимость в компонентах, - так как при обновлении конфигурации это повлечет обновление всех компонент графа которые используют его у себя, - рекомендуется всегда создавать [пользовательские конфигурации](#custom-configuration). +По умолчанию в `Kora` есть наблюдатель за конфигурационным файлом, который проверяет файл приложения на изменения и +запускает обновление графа зависимостей при изменении файла. Проверка выполняется каждые `1000` миллисекунд. -## Наблюдатель { #config-watcher } +Для `HOCON` наблюдатель также отслеживает файлы, подключенные через `include` внутри основного конфигурационного файла. +Если такой включенный файл изменяется, конфигурация перечитывается, и граф зависимостей также обновляется. -По умолчанию в Kora работает наблюдатель за файлом конфигурации который обновляет его содержимое, -что влечет в случае изменения файла конфигурации обновление графа зависимостей для компонент которые затронули изменения. +Наблюдатель работает только для файловой конфигурации, имеющей отслеживаемый источник. Если конфигурация пришла из +ресурса внутри архива или была собрана без файла приложения, на диске нечего обновлять. -Можно отключить наблюдатель с помощь: +Наблюдатель можно отключить с помощью: -1. Переменной окружения `KORA_CONFIG_WATCHER_ENABLED` -2. Системного свойства `kora.config.watcher.enabled` +1. Переменной окружения `KORA_CONFIG_WATCHER_ENABLED` (по умолчанию: `true`) +2. Системного свойства `kora.config.watcher.enabled` (по умолчанию: `true`) ## Поддерживаемые типы { #supported-types } -Экстракторы конфигурации предоставляют обширный список поддерживаемых типов, который охватывает большинство из того, -что вам может понадобиться для указания в пользовательских конфигурациях, либо вы можете расширить поведение собственным `ConfigValueExtractor` компонентом. +Экстракторы конфигурации предоставляют обширный список поддерживаемых типов, который покрывает большинство значений, +которые могут понадобиться в пользовательских конфигурациях. Если стандартного преобразования недостаточно, поведение +можно расширить пользовательским компонентом `ConfigValueExtractor`. ??? abstract "Список поддерживаемых типов" @@ -764,27 +984,188 @@ services: * Properties * Pattern * UUID - * Properties * LocalDate * LocalTime * LocalDateTime * OffsetTime * OffsetDateTime - * Enum (любой пользовательский ENUM тип) (Переопределить соответствие можно через переопределение `toString()`) - * `List` (где `T` любой из выше перечисленных типов) - * `Set` (где `T` любой из выше перечисленных типов) - * `Map` (где `K` или `V` любой из выше перечисленных типов) - * `Either` (где `A` и `B` любой из выше перечисленных типов) + * ConfigValue.ObjectValue + * Enum (любой пользовательский `enum`; отображение можно переопределить через `toString()`) + * `Optional` (где `T` — любой поддерживаемый тип) + * `List` (где `T` — любой поддерживаемый тип) + * `Set` (где `T` — любой поддерживаемый тип) + * `Map` или `Map` (где `K` и `V` поддерживаются соответствующими экстракторами) + * `Either` (где `A` и `B` — любые поддерживаемые типы) + +### Пользовательский экстрактор { #custom-extractor } + +Если для типа нет стандартного преобразования или требуется специальная логика разбора, добавьте пользовательский +компонент `ConfigValueExtractor`. Метод `extract(...)` получает значение конфигурации как `ConfigValue` +и должен вернуть готовое значение требуемого типа. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public final class TokenConfigValueExtractor implements ConfigValueExtractor { + + @Override + public Token extract(ConfigValue value) { + if (value instanceof ConfigValue.NullValue) { + return null; + } + return new Token(value.asString()); + } + } + ``` -### Размер { #size } +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + class TokenConfigValueExtractor : ConfigValueExtractor { -`Size` - специальный тип который позволяет задавать размер байт в удобной человеку системе исчислений по стандарту [IEEE 1541—2002](https://ru.ruwiki.ru/wiki/IEEE_1541-2002) (двоичный), так и в стандарте [СИ](https://ru.ruwiki.ru/wiki/%D0%95%D0%B4%D0%B8%D0%BD%D0%B8%D1%86%D1%8B_%D0%B8%D0%B7%D0%BC%D0%B5%D1%80%D0%B5%D0%BD%D0%B8%D1%8F_%D1%91%D0%BC%D0%BA%D0%BE%D1%81%D1%82%D0%B8_%D0%BD%D0%BE%D1%81%D0%B8%D1%82%D0%B5%D0%BB%D0%B5%D0%B9_%D0%B8_%D0%BE%D0%B1%D1%8A%D1%91%D0%BC%D0%B0_%D0%B8%D0%BD%D1%84%D0%BE%D1%80%D0%BC%D0%B0%D1%86%D0%B8%D0%B8#%D0%91%D0%B0%D0%B9%D1%82) (десятичный). + override fun extract(value: ConfigValue<*>): Token? { + if (value is ConfigValue.NullValue) { + return null + } + return Token(value.asString()) + } + } + ``` + +Если конкретный экстрактор должен использоваться только для одного поля, укажите его через `@Mapping`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @ConfigValueExtractor + public interface ApiConfig { + + @Mapping(TokenConfigValueExtractor.class) + Token token(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @ConfigValueExtractor + interface ApiConfig { + + @Mapping(TokenConfigValueExtractor::class) + fun token(): Token + } + ``` + +### Duration { #duration } + +`Duration` можно задать как число или строку. +Если указано число, оно трактуется как миллисекунды. +Если указана строка, поддерживается формат `java.time.Duration`, например `PT10S`, а также стиль `HOCON`: + +- `500ms` +- `10 seconds` +- `2 minutes` +- `1h` +- `1d` + +### Period { #period } + +`Period` можно задать как число или строку. +Если указано число, оно трактуется как дни. +Если указана строка, поддерживаются такие единицы: + +- `d` / `days` +- `w` / `weeks` +- `m` / `mo` / `months` +- `y` / `years` + +Например, `7d`, `2 weeks`, `3mo` или `1 year`. + +### Size { #size } + +`Size` — это специальный тип, который позволяет указывать размеры в байтах в удобной для человека нотации: согласно +стандарту [IEEE 1541-2002](https://en.wikipedia.org/wiki/IEEE_1541-2002) (двоичный) или стандарту +[SI](https://en.wikipedia.org/wiki/Binary_prefix) (десятичный). Примеры значений: -- `1Mb` - 1 мегабайт (`1.000.000` байт) -- `1Mib` - 1 мегабит (`1.048.576` байт) -- `1024b` - 1024 байт -- `1024` - 1024 байт +- `1Mb` — 1 мегабайт (`1.000.000` байт) +- `1Mib` — 1 мебибайт (`1.048.576` байт) +- `1024b` — 1024 байта +- `1024` — 1024 байта + +Если указано просто число без суффикса, считается, что указаны байты. + +### Either { #either } + +`Either` позволяет одному полю принимать две альтернативные формы. Экстрактор сначала пробует левый тип `A`, и если +извлечение завершается любым исключением, откатывается к правому типу `B`. Это полезно, когда значение может быть либо +простым скаляром, либо структурированным объектом. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.common.util.Either; + + @ConfigValueExtractor + public interface EndpointConfig { + + String host(); + + int port(); + } + + @ConfigSource("services.foo") + public interface FooServiceConfig { + + Either endpoint(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.util.Either + + @ConfigValueExtractor + interface EndpointConfig { + + fun host(): String + + fun port(): Int + } + + @ConfigSource("services.foo") + interface FooServiceConfig { + + fun endpoint(): Either + } + ``` + +Обе эти формы допустимы для поля `endpoint`: + +===! ":material-code-json: `Hocon`" + + ```javascript + services { + foo { + endpoint = "https://example.com" //(1)! + } + } + ``` + + 1. Разрешается как левый тип (`String`) + +=== ":simple-yaml: `YAML`" + + ```yaml + services: + foo: + endpoint: #(1)! + host: "example.com" + port: 8080 + ``` + + 1. Разрешается как правый тип (`EndpointConfig`) -Если указано просто число без суффикса, то считается что указаны байты. +Используйте `isLeft()` / `isRight()`, чтобы проверить, какая сторона была разрешена, и `left()` / `right()`, чтобы прочитать значение. diff --git a/mkdocs/docs/ru/documentation/container.md b/mkdocs/docs/ru/documentation/container.md index 3cb8c30..188a7c6 100644 --- a/mkdocs/docs/ru/documentation/container.md +++ b/mkdocs/docs/ru/documentation/container.md @@ -4,25 +4,32 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora compile-time dependency injection container, components, modules, factories, tags, lifecycle, graph resolution, and dependency wrappers; key triggers include @KoraApp, @Component, @Module, @KoraSubmodule, @Root, @Tag, @DefaultComponent, ValueOf, All, PromiseOf." --- -Контейнер зависимостей является ядром фреймворка Kora и отвечает за построение контейнера зависимостей, их валидацию, -их внедрение и их последующую инициализацию. -Kora имеет свой собственный самобытный контейнер зависимостей. +Контейнер зависимостей — это ядро фреймворка `Kora`. Он строит граф зависимостей, проверяет его, +внедряет компоненты, инициализирует их и позднее освобождает. +В отличие от контейнеров, которые собирают приложение при запуске путем сканирования classpath, `Kora` строит большую часть графа +во время компиляции и генерирует обычный `Java`-код для запуска приложения. -Работа контейнера в Kora разделена на две части: то что выполняется во время компиляции и то что выполняется во время исполнения. +Работа контейнера в `Kora` разделена на две части: время компиляции и время выполнения. +Во время компиляции `Kora` проверяет, что все зависимости могут быть найдены и связаны. Во время выполнения контейнер создает +компоненты, управляет их жизненным циклом и обновляет затронутые части графа при возникновении изменений. -Если нужен пошаговый разбор перед справочным описанием, смотрите [Введение во внедрение зависимостей](../guides/dependency-injection-introduction.md) и [Внедрение зависимостей](../guides/dependency-injection.md). +Пошаговый разбор перед справочным описанием смотрите в разделах [Введение во внедрение зависимостей](../guides/dependency-injection-introduction.md) и [Внедрение зависимостей](../guides/dependency-injection.md). -## Во время компиляции { #compile-time } +## Время компиляции { #compile-time } -На этап компиляции производится поиск компонентов для построения контейнера зависимостей всего приложения. -Это позволяет проводить валидацию контейнера зависимостей на этапе компиляции, до фактического старта приложения. +Во время компиляции компоненты обнаруживаются, чтобы построить контейнер зависимостей для всего приложения. +Это позволяет проверить контейнер зависимостей до фактического запуска приложения. ### Контейнер { #container } -За ядро контейнера зависимостей отвечает интерфейс помеченный аннотацией `@KoraApp`. -Этой аннотацией необходимо помечать интерфейс, внутри которого лежат фабричные методы для создания компонентов -и подключены [внешние модули](#external-module-factory) зависимостей. -Такой интерфейс может быть только один в рамках приложения. +Ядром контейнера зависимостей является интерфейс, помеченный аннотацией `@KoraApp`. +Эту аннотацию следует использовать на интерфейсе, который содержит фабричные методы для создания компонентов +и подключает [внешние модули](#external-module-factory). +В приложении может быть только один такой интерфейс. + +Обработчики аннотаций `Kora` анализируют исходный код в модуле компиляции, где объявлен `@KoraApp`, +а также в модулях, где объявлен [`@KoraSubmodule`](#submodule-factory). Обычные модули проекта без +`@KoraApp` или `@KoraSubmodule` не становятся областями обнаружения компонентов автоматически. ===! ":fontawesome-brands-java: `Java`" @@ -40,22 +47,22 @@ Kora имеет свой собственный самобытный конте ### Компоненты { #components } -Компонентом называется зависимость в контейнере зависимостей. -Все компоненты в Kora создаются в единственном экземпляре (`Singleton`). -Компоненты внедряются только если являются [самостоятельным компонентом](#root-component), либо если требуются в других компонентах как зависимости. +Компонент — это зависимость в контейнере зависимостей. +Все компоненты в `Kora` создаются в единственном экземпляре (`Singleton`). +Компоненты внедряются только если они являются [корневыми компонентами](#root-component) или если другие компоненты нуждаются в них как в зависимостях. -Компоненты неудовлетворяющие этим требованиям не попадают в контейнер зависимостей. +Компоненты, которые не удовлетворяют этим требованиям, не включаются в контейнер зависимостей. -#### Авто фабрика { #auto-factory } +#### Автоматическая фабрика { #auto-factory } -Аннотация `@Component` помечает класс, как доступный через контейнер. При этом к классу предъявляются следующие требования: +Аннотация `@Component` помечает класс как доступный через контейнер. К классу предъявляются следующие требования: ===! ":fontawesome-brands-java: `Java`" * Класс не должен быть абстрактным * У класса должен быть только один публичный конструктор - * Класс должен быть `final` (только если он не имеет аспектов) - + * Класс должен быть `final` (только если у него нет аспектов) + ```java @Component public final class SomeService { @@ -72,20 +79,20 @@ Kora имеет свой собственный самобытный конте * Класс не должен быть абстрактным * У класса должен быть только один публичный конструктор - * Класс не должен быть `open` (только если он не имеет аспектов) + * Класс не должен быть `open` (только если у него нет аспектов) ```kotlin @Component class SomeService(val otherService: OtherService) { } ``` -#### Основная фабрика { #basic-factory } +#### Фабрика метод { #method-factory } -Фабричный метод представляет собой метод с модификатором `default` который возвращает компонент, метод может принимать -как аргументы другие компоненты зависимости. +Фабричный метод — это метод с модификатором `default`, который возвращает компонент. +Метод может принимать другие компоненты-зависимости в качестве аргументов. -Контейнер ниже описывает две фабрики, где фабрика `otherService` требует компонент, создаваемый фабрикой `someService`. -Это самый базовый способ, как в контейнере могут регистрироваться компоненты: +Контейнер зависимостей ниже описывает две фабрики, где фабрика `otherService` требует компонент, созданный фабрикой `someService`. +Это самый базовый способ, которым компоненты могут быть зарегистрированы в контейнере: ===! ":fontawesome-brands-java: `Java`" @@ -117,14 +124,14 @@ Kora имеет свой собственный самобытный конте } ``` -Фабричный метод **не должен предоставлять** значение `null` как компонент. +Фабричный метод **не должен предоставлять** значение `null` в качестве компонента. -#### Модуль фабрика { #module-factory } +#### Фабрика модуль { #module-factory } -Компоненты для контейнера также могут находиться в модулях в рамках проекта приложения. -Под модулем понимается интерфейс в котором находятся фабричные методы. -Аннотация `@Module` помечает интерфейс как модуль, который нужно внедрить в наш контейнер на этапе компиляции. -Модуль должен находиться в рамках одной директории исходного кода с классом помеченным `@KoraApp`. +Компоненты для контейнера зависимостей также могут располагаться в модулях внутри проекта приложения. +Модуль — это интерфейс, который содержит фабричные методы. +Аннотация `@Module` помечает интерфейс как модуль, который должен быть внедрен в контейнер приложения во время компиляции. +Модуль должен находиться в той же директории исходного кода, что и класс, помеченный `@KoraApp`. Все фабричные методы внутри модуля становятся доступны контейнеру зависимостей: @@ -150,15 +157,18 @@ Kora имеет свой собственный самобытный конте } ``` -#### Внешняя модуль фабрика { #external-module-factory } +#### Фабрика внешний модуль { #external-module-factory } + +Компоненты для контейнера зависимостей также могут быть найдены во внешних модулях из сторонних зависимостей. +Модуль — это интерфейс, который содержит фабричные методы. +`Kora` не выполняет автоматический поиск модулей из внешних зависимостей, как это делают некоторые другие DI-решения. +Это позволяет разработчику точно контролировать, какие зависимости используются в приложении, и избегать +инициализации ненужных компонентов. -Компоненты для контейнера также могут искаться во внешних модулях из сторонних зависимостей. -Под модулем понимается интерфейс в котором находятся фабричные методы. -Kora не делает автоматический поиск модулей из внешних зависимостей, как это делают некоторые другие решения внедрения зависимостей. -Это позволяет разработчику точно контролировать и осознавать какие зависимости используются в его приложении и не допускать -инициализации множества лишних зависимостей и ухудшать тем самым работу приложения. +Все необходимые внешние модули из зависимостей должны быть подключены явно в интерфейсе, помеченном аннотацией `@KoraApp`, через наследование: -Все необходимые внешние модули из зависимостей должны быть подключены явно в интерфейс помеченный аннотацией `@KoraApp` через наследование: +Такой модуль может быть объявлен в любом интерфейсе: в сторонней библиотеке, в отдельном модуле проекта или рядом с самим `@KoraApp`. +Важно то, что интерфейс `@KoraApp` явно подключает его через наследование. ===! ":fontawesome-brands-java: `Java`" @@ -174,17 +184,20 @@ Kora не делает автоматический поиск модулей и interface Application : LogbackModule, JsonModule ``` -#### Межмодульная фабрика { #submodule-factory } +#### Фабрика подмодуль { #submodule-factory } -Аннотация `@KoraSubmodule` помечает интерфейс, для которого нужно собрать модуль для текущего модуля компиляции, -в него будут помещены все компоненты, помеченные аннотациями `@Module` и `@Component`. -Эта аннотация полезна, если вы разбиваете свой проект на [много-модульное приложение](https://docs.gradle.org/current/userguide/multi_project_builds.html) -с точки зрения инструмента сборки `Gradle.`, где -каждый из которых отвечает за какую-то часть функциональности, а само приложение с `@KoraApp` собирается в отдельном модуле компиляции. +Аннотация `@KoraSubmodule` помечает интерфейс, для которого должен быть построен модуль в рамках текущего модуля компиляции. +Он будет содержать все компоненты, помеченные аннотациями `@Module` и `@Component`. +Эта аннотация полезна, когда вы разбиваете проект на [многомодульное приложение](https://docs.gradle.org/current/userguide/multi_project_builds.html) +с точки зрения инструмента сборки `Gradle`, где каждый модуль отвечает за свою часть функциональности, +а приложение с `@KoraApp` собирается в отдельном модуле компиляции. +Такой подход помогает структурировать большой проект по доменным областям и улучшить время сборки: +изменения в одном модуле проекта не заставляют обработчик аннотаций заново анализировать весь код приложения. -Для интерфейса будет создан интерфейс наследник, в котором будут унаследованы все интерфейсы помеченные `@Module` и созданы методы фабрики для классов, помеченных как `@Component`. +Для интерфейса будет сгенерирован интерфейс-наследник. Он унаследует все интерфейсы, помеченные `@Module`, +и создаст фабричные методы для классов, помеченных как `@Component`. -Например, у вас есть отдельный модуль-приложения который содержит такой межмодуль: +Например, у вас есть отдельный модуль приложения, который содержит такой `@KoraSubmodule`: ===! ":fontawesome-brands-java: `Java`" @@ -224,7 +237,7 @@ Kora не делает автоматический поиск модулей и } ``` -И есть основной модуль-приложение с точкой сборки всего приложения: +И есть основной модуль приложения с точкой сборки для всего приложения: ===! ":fontawesome-brands-java: `Java`" @@ -240,12 +253,41 @@ Kora не делает автоматический поиск модулей и interface Application : SomeSubModule ``` -При этом в контейнер итогового приложения будет подключен модуль `SomeModule` подключенный в рамках `SomeSubModule`. +Это подключит модуль `SomeModule`, найденный через `SomeSubModule`, к итоговому контейнеру приложения. + +Распространенный практический сценарий использования `@KoraSubmodule` — это отдельный модуль `Gradle`, который владеет одной доменной областью и просто +агрегирует нужные ему [внешние модули](#external-module-factory) (базы данных, кэши и так далее) путем их наследования, +вместе со своими собственными классами `@Component` и интерфейсами `@Module`. Модуль приложения затем подключает +сгенерированные подмодули так же, как подключает любой другой модуль: + +===! ":fontawesome-brands-java: `Java`" + + ```java + // in the "pet" Gradle module + @KoraSubmodule + public interface PetModule extends JdbcDatabaseModule, CaffeineCacheModule { } + + // in the application Gradle module + @KoraApp + public interface Application extends PetModule, VetModule { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // in the "pet" Gradle module + @KoraSubmodule + interface PetModule : JdbcDatabaseModule, CaffeineCacheModule + + // in the application Gradle module + @KoraApp + interface Application : PetModule, VetModule + ``` -#### Дженерик фабрика { #generic-factory } +#### Фабрика обобщенная { #generic-factory } -Если в контейнере не удалось найти фабрику для конкретного типа, то Kora контейнер во время компиляции может попробовать поискать -методы с Дженерик параметрами, и при помощи этого метода создать экземпляр нужного класса. +Если контейнер зависимостей не смог найти фабрику для конкретного типа, контейнер `Kora` может попытаться найти +методы с обобщенными параметрами во время компиляции и использовать такой метод для создания экземпляра требуемого класса. ===! ":fontawesome-brands-java: `Java`" @@ -271,25 +313,57 @@ Kora не делает автоматический поиск модулей и } ``` -Теперь если какому-то компоненту понадобится GenericValidator как зависимость, то эта фабрика будет использована для его создания. +Теперь, если какому-либо компоненту нужен `GenericValidator` в качестве зависимости, для его создания будет использована эта фабрика. + +##### Информация об обобщенном типе { #type-ref } + +Если фабричному методу нужно знать точный обобщенный тип, запрашиваемый контейнером в данный момент, он может внедрить `TypeRef`. +Это полезно для инфраструктурных компонентов, которые создают зависимость по форме типа, а не только по «сырому» классу. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface SomeModule { + + default Validator> listValidator(Validator validator, TypeRef typeRef) { + return new ListValidator<>(validator, typeRef); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface SomeModule { + + fun listValidator(validator: Validator, typeRef: TypeRef): Validator> { + return ListValidator(validator, typeRef) + } + } + ``` + +`TypeRef` переносит информацию об обобщенном типе через стирание типов в `Java`. Большинству компонентов приложения он не нужен, +но он полезен для универсальных фабрик, мапперов и расширений контейнера. #### Механизм расширений { #extension-mechanism } -В случае, если ни одна из фабрик не смогла предоставить компонент, Kora может попробовать создать эту зависимость во время компиляции сама. -Для этого предусмотрен механизм расширений. Каждое расширение умеет сказать, может ли оно создать компонент нужного типа. -Если расширение может это сделать, то оно делает нужную кодогенерацию и сообщает, каким образом можно получить этот компонент. +Если ни одна из фабрик не смогла предоставить компонент, `Kora` может попытаться создать эту зависимость самостоятельно во время компиляции. +Для этого предоставляется механизм расширений. Каждое расширение способно сообщить, может ли оно создать компонент требуемого типа. +Если расширение может это сделать, оно выполняет необходимую генерацию кода и сообщает, как получить этот компонент. -Например, есть расширения, которые умеют создавать оптимальные Json читатели и писатели, репозитории и другие компоненты. -Поиск доступных расширений происходит благодаря механизму `ServiceLocator` из всех зависимостей предоставленных в области видимости обработчика аннотаций. +Например, существуют расширения, которые умеют создавать оптимальные компоненты `JsonReader` и `JsonWriter`, репозитории и другие компоненты. +Доступные расширения обнаруживаются через механизм `ServiceLocator` из всех зависимостей, предоставленных в области видимости обработчика аннотаций. -Механизм является скорее системным и используется зачастую внутренними модулями Kora. +Этот механизм является системным и чаще всего используется внутренними модулями `Kora`. -#### Стандартная фабрика { #standard-factory } +#### Фабрика по умолчанию { #default-factory } -Чтобы предоставлять фабричными методами компоненты по умолчанию, которые подразумевается что пользователь может у себя переопределить, +Чтобы предоставить компоненты по умолчанию через фабричные методы, которые, как предполагается, пользователь может переопределить, требуется использовать аннотацию `@DefaultComponent`. -В случае если будет контейнером зависимостей на этапе компиляции будет найден любой компонент который не использует эту аннотацию, -то предпочтение будет отдано ему при внедрении. +Если контейнер зависимостей находит во время компиляции любой компонент того же типа и с теми же тегами, но без `@DefaultComponent`, +то при внедрении будет предпочтен пользовательский компонент. ===! ":fontawesome-brands-java: `Java`" @@ -315,16 +389,16 @@ Kora не делает автоматический поиск модулей и } ``` -#### Авто создание { #auto-creation } +#### Автоматическое создание { #auto-creation } -Если же ни один из способов выше не смог предоставить компонент, -то Kora может попробовать самостоятельно создать компонент если он удовлетворяет требованиям аналогичным [авто фабрики](#components): +Если ни один из вышеперечисленных методов не смог предоставить компонент, +то `Kora` может попытаться создать компонент самостоятельно, если он удовлетворяет требованиям, аналогичным [автоматической фабрике](#auto-factory): ===! ":fontawesome-brands-java: `Java`" * Класс не должен быть абстрактным * У класса должен быть только один публичный конструктор - * Класс должен быть `final` (только если он не имеет аспектов) + * Класс должен быть `final` (только если у него нет аспектов) ```java public final class SomeService { @@ -353,7 +427,7 @@ Kora не делает автоматический поиск модулей и * Класс не должен быть абстрактным * У класса должен быть только один публичный конструктор - * Класс не должен быть `open` (только если он не имеет аспектов) + * Класс не должен быть `open` (только если у него нет аспектов) ```kotlin class SomeService(val otherService: OtherService) { } @@ -369,20 +443,20 @@ Kora не делает автоматический поиск модулей и } ``` -### Переопределение компонент { #component-override } +### Переопределение компонента { #component-override } -В случае если компонент предоставляется библиотекой как зависимость по умолчанию, -то можно создать фабрику в приложении без аннотации `@DefaultComponent` и такая зависимость переопределит ее. +В случае, если компонент предоставляется библиотекой как зависимость по умолчанию, +можно создать фабрику в приложении без аннотации `@DefaultComponent`, и такая зависимость переопределит его. -Так как все внешние модули подключаются как интерфейсы в ядро контейнера `@KoraApp` и их фабрики доступны, -то их можно просто переопределить как метод и предоставить свою реализацию. +Поскольку все внешние модули подключаются как интерфейсы к ядру контейнера `@KoraApp` и их фабрики доступны, +вы можете просто переопределить их как метод и предоставить свою собственную реализацию. -### Самостоятельный компонент { #root-component } +### Корневой компонент { #root-component } -Когда компонент требуется всегда инициализировать с запуском приложения, даже если он не является зависимостью других компонент, -предполагается использовать аннотацию `@Root` над фабричным методом или классом аннотированным `@Component`. +Когда требуется, чтобы компонент всегда инициализировался при запуске приложения, даже если он не является зависимостью других компонентов, +предполагается использовать аннотацию `@Root` над фабричным методом или классом, помеченным `@Component`. -Примером такого компонента может быть HTTP сервер, Kafka потребитель, компонент прогрева кешей. +Примером такого компонента может быть `HTTP`-сервер, потребитель `Kafka`, компонент прогрева кэша или обработчик выполняемой фоновой задачи. ===! ":fontawesome-brands-java: `Java`" @@ -412,9 +486,9 @@ Kora не делает автоматический поиск модулей и ===! ":fontawesome-brands-java: `Java`" - Если хочется внедрить необязательную зависимость, которая может отсутствовать то - предполагается пометить такой компонент любой `@Nullable` аннотацией, - тогда контейнер зависимостей не упадет на этапе компиляции из-за отсутвия компонента: + Если вы хотите ввести необязательную зависимость, которой может не существовать, то + предполагается пометить такой компонент любой аннотацией `@Nullable`, + тогда контейнер зависимостей не завершится с ошибкой во время компиляции из-за отсутствия компонента: ```java @Component @@ -428,23 +502,26 @@ Kora не делает автоматический поиск модулей и } ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдет любая аннотация `@Nullable`, например `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - Если хочется внедрить необязательную зависимость, которая может отсутствовать то - предполагается использовать [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис и помечать такой компонент как Nullable, - тогда контейнер зависимостей не упадет на этапе компиляции из-за отсутвия компонента: + Если вы хотите внедрить необязательную зависимость, которая может отсутствовать, используйте [синтаксис null-безопасности `Kotlin`](https://kotlinlang.org/docs/null-safety.html) + и пометьте этот компонент как допускающий `null`, + тогда контейнер зависимостей не завершится с ошибкой во время компиляции из-за отсутствия компонента: ```kotlin @Component class SomeService(val otherService: OtherService?) { } ``` -### Список компонент { #list-of-components } +Необязательность можно комбинировать с обертками контейнера: `ValueOf>`, `Optional>`, +`PromiseOf>` и `Optional>`. Это полезно, когда зависимость может отсутствовать, +но компоненту при этом все равно нужен отложенный доступ или возможность обновить ее через контейнер. -В контейнере может быть много экземпляров одного и того же типа, и если их все нужно собрать в одном месте, -то следует использовать специальный тип `All`. +### Список компонентов { #list-of-components } + +В контейнере может быть много экземпляров одного и того же типа, и если вы хотите собрать их все в одном месте, следует использовать специальный тип `All`. ===! ":fontawesome-brands-java: `Java`" @@ -480,28 +557,29 @@ Kora не делает автоматический поиск модулей и } ``` -Например, у нас есть некоторая сущность `Handler` и его имплементируют несколько разных типов в контейнере. -`SomeProcessor` при этом потребляет все возможные реализации этого типа. -**Важно** что пример выше возьмет все экземпляры `Handler` без тегов. +Например, у нас есть некоторая сущность `Handler`, и она внедряется N различными типами в контейнере. +`SomeProcessor` при этом потребляет все возможные реализации этого типа. +**Важно**: пример выше берет все экземпляры `Handler` без тегов. Сам тип `All` имеет следующий контракт: ```java -public interface All extends List {} +public sealed interface All extends List permits AllImpl {} ``` -Это маркерный тип, расширяющий `List` и его можно отдавать в конструкторах, которые ожидают `List`. +Это токен-тип, который расширяет `List` и может быть передан в конструкторы, ожидающие `List`. +Если вам нужно собрать ссылки на компоненты вместо самих компонентов, контейнер также поддерживает +`All>` и `All>`. ### Теги { #tags } -Иногда есть потребность предоставить разные экземпляры одного и того же типа в разные компоненты. Для этого их можно разграничить по тегам -посредством аннотации `@Tag`, которая принимает на вход класс тега. -Ожидается связка, где компонент зарегистрирован с определенным тегом и в точке внедрения он объявлен с точно таким же тегом. +Иногда возникает необходимость предоставить разные экземпляры одного и того же типа разным компонентам. Для этой цели их можно различать по тегам. +Для этого существует аннотация `@Tag`, которая принимает на вход класс тега. +Ожидается сопоставление, при котором компонент регистрируется с определенным тегом, а в точке внедрения объявляется с точно таким же тегом. -Используется именно класс, а не строковый литерал, потому что это проще для навигации по коду стандартными средствами разработки -и позволяет задать строгую типизацию тегам. +Используется именно класс, а не строковый литерал, потому что так проще ориентироваться в коде и проще выполнять рефакторинг. -Например, вот так можно внедрить разные экземпляры класса с общим интерфейсом по разным точкам внедрения: +Вот как можно внедрять разные экземпляры класса с общим интерфейсом в разные точки внедрения: ===! ":fontawesome-brands-java: `Java`" @@ -537,7 +615,7 @@ public interface All extends List {} fun someService1(): SomeService = SomeService1() @Tag(MyTag2::class) - fun someService1(): SomeService = SomeService2() + fun someService2(): SomeService = SomeService2() fun serviceA(@Tag(MyTag1::class) service: SomeService): ServiceA { return ServiceA(service) @@ -549,8 +627,8 @@ public interface All extends List {} } ``` -Теги над методом говорят какой с каким тегом надо зарегистрировать компонент, а теги в точках внедрения говорят с каким тегом ожидается компонент. -Также теги работают на параметрах конструктора, в связке с `@Component` или финальными классами. +Теги над методом сообщают, с каким тегом зарегистрировать компонент, а теги в точках внедрения сообщают, какой помеченный тегом компонент ожидать. +Теги также работают на параметрах конструктора, в сочетании с `@Component` или финальными классами. ===! ":fontawesome-brands-java: `Java`" @@ -600,9 +678,9 @@ public interface All extends List {} class ServiceB(private val service: @Tag(MyTag2::class) SomeService) ``` -#### Тег собственный { #tag-custom } +#### Собственный тег { #tag-custom } -Можно также вводить собственный аннотации теги и работать уже с ними, таким примером может служить [аннотация @Json](json.md) +Вы также можете создавать свои собственные аннотации-теги и работать с ними. Одним из примеров является [аннотация `@Json`](json.md). ===! ":fontawesome-brands-java: `Java`" @@ -630,7 +708,7 @@ public interface All extends List {} annotation class MyTag interface SomeModule { - + @MyTag fun someService(): SomeService = SomeService() @@ -640,9 +718,9 @@ public interface All extends List {} } ``` -#### Тег список { #tag-all } +#### Все по тегу { #tag-all } -Можно использовать тег также для получения списка всех компонент по определенному тегу: +Вы также можете использовать тег, чтобы получить список всех компонентов по определенному тегу: ===! ":fontawesome-brands-java: `Java`" @@ -682,9 +760,9 @@ public interface All extends List {} } ``` -#### Тег всех { #tag-any } +#### Любой тег { #tag-any } -Для получения списка вообще всех компонент с тегом и без, требуется использовать специальный тип тега `@Tag.Any`: +Чтобы получить список всех компонентов с тегом и без него, нужно использовать специальный тип тега `@Tag.Any`: ===! ":fontawesome-brands-java: `Java`" @@ -716,32 +794,115 @@ public interface All extends List {} fun handlerB(): HandlerB = HandlerB() - fun someProcessor(@Tag(Tag.Any::class) handlers: All): SomeProcessor { + fun someProcessor(@Tag(Tag.Any::class) handlers: All): SomeProcessor { return SomeProcessor(handlers) } } ``` -## Во время исполнения { #runtime } +### Циклические зависимости { #circular-dependencies } + +Поскольку `Kora` строит и проверяет весь граф зависимостей во время компиляции, цикл зависимостей +(компоненту `A` нужен `B`, компоненту `B` нужен `A`, возможно, через несколько компонентов между ними) обнаруживается во время компиляции, +а не приводит к сбою во время выполнения. То, как обрабатывается такой цикл, зависит от того, как объявлена зависимость внутри цикла. + +**Прямая зависимость от `final`-класса (или любого не-интерфейсного типа).** +Такой цикл не может быть разрешен, и компиляция завершается ошибкой. Ошибка указывает на тип, который замыкает цикл, и перечисляет +кандидатов цикла: + +``` +Encountered circular dependency in graph for source type: ru.tinkoff.kora.example.ServiceA (no tags) + Cycle dependency candidates: + - ru.tinkoff.kora.example.ServiceA + - ru.tinkoff.kora.example.ServiceB +Please check that you are not using cycle dependency in ru.tinkoff.kora.application.graph.Lifecycle, this is forbidden. +``` + +**Зависимость, объявленная через интерфейс (или не-`final`-класс).** +`Kora` разрывает цикл автоматически: для зависимости с типом-интерфейсом она генерирует ленивый прокси, реализующий +`ru.tinkoff.kora.common.PromisedProxy`, и внедряет прокси вместо реального компонента. Прокси разрешает +фактический компонент из графа при первом обращении (и повторно разрешает его после обновления графа), так что оба компонента могут быть +сконструированы. От разработчика никаких действий не требуется, но имейте в виду, что проксируемая сторона становится пригодной к использованию только после +того, как граф полностью связан, поэтому ее нельзя вызывать из конструктора. + +В примере ниже `ServiceAImpl` и `ServiceBImpl` ссылаются друг на друга через интерфейсы, поэтому цикл разрывается +автоматически сгенерированным `PromisedProxy`, и граф разрешается успешно: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public interface ServiceA { } + + public interface ServiceB { } + + @Component + public final class ServiceAImpl implements ServiceA { + + public ServiceAImpl(ServiceB serviceB) { } + } + + @Component + public final class ServiceBImpl implements ServiceB { + + public ServiceBImpl(ServiceA serviceA) { } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + interface ServiceA + + interface ServiceB + + @Component + class ServiceAImpl(serviceB: ServiceB) : ServiceA + + @Component + class ServiceBImpl(serviceA: ServiceA) : ServiceB + ``` + +Надежный способ намеренно разорвать цикл — внедрить одну из сторон через [`ValueOf`](#indirect-dependency) +или [`PromiseOf`](#updating-components) вместо прямой зависимости. Это отвязывает потребителя от жизненного цикла другого +компонента, поэтому контейнер больше не рассматривает эти два компонента как жесткий цикл: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ServiceAImpl implements ServiceA { + + public ServiceAImpl(ValueOf serviceB) { } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ServiceAImpl(serviceB: ValueOf) : ServiceA + ``` + +## Время выполнения { #runtime } -Контейнер зависимостей инициализируется максимально параллельно насколько это возможно в рамках того контейнера зависимостей, который был построен. +Контейнер зависимостей использует максимально возможный параллелизм в рамках построенного графа. -На этапе исполнения приложения выполняются следующие вещи: +Во время выполнения приложения контейнер делает следующее: * Инициализирует все компоненты в контейнере зависимостей * Отслеживает изменения в контейнере зависимостей -* Атомарно обновляет контейнер зависимостей при изменениях -* Осуществляет [штатное завершение (Graceful Shutdown)](#component-lifecycle) при получении сигнала SIGTERM +* Атомарно обновляет контейнер зависимостей при внесении изменений +* Выполняет [штатное завершение](#graceful-shutdown) при получении сигнала `SIGTERM` -Все компоненты используют превентивную инициализацию, то есть инициализируются сразу с запуском приложения. +Все компоненты используют энергичную инициализацию, что означает, что они инициализируются сразу при запуске приложения. ### Точка входа { #entrypoint } -Точка входа в приложение должна вызывать запуск `KoraApplication.run` с использованием созданного в процессе компиляции контейнера зависимостей. +Точка входа приложения должна вызывать `KoraApplication.run`, используя контейнер зависимостей, созданный во время компиляции. -В случае если интерфейс, помеченный `@KoraApp`, называется `Application`, в процессе компиляции в том же пакете -будет создан класс `ApplicationGraph`, который будет представлять реализацию контейнера зависимостей, и тогда точка входа -в рамках того же пакета будет выглядеть так: +Если интерфейс, помеченный `@KoraApp`, называется `Application`, то во время компиляции в том же пакете будет сгенерирован класс с именем `ApplicationGraph`. +Он представляет собой реализацию контейнера зависимостей, а точка входа +в том же пакете будет выглядеть так: ===! ":fontawesome-brands-java: `Java`" @@ -766,20 +927,24 @@ public interface All extends List {} } ``` +`KoraApplication.run` запускает контейнер и возвращает `RefreshableGraph` (`Graph` в сочетании с [`Lifecycle`](#component-lifecycle)). +Для работающего приложения вы обычно не взаимодействуете с ним напрямую, но он полезен в тестах и продвинутых сценариях, где +нужно найти компонент в графе или вручную инициировать обновление. + ### Жизненный цикл контейнера { #container-lifecycle } -Контейнер умеет инициализировать все компоненты в правильном порядке, при этом он это делает максимально параллельно, чтобы достигнуть максимально быстрого времени запуска. +Контейнер зависимостей умеет инициализировать все компоненты в правильном порядке и делает это максимально параллельно, чтобы достичь максимально быстрого времени запуска. -Когда контейнер больше не нужен, запускается механизм освобождения компонентов в обратном порядке. +Когда контейнер больше не нужен, он запускает механизм освобождения компонентов в обратном порядке. -В середине жизненного цикла может произойти обновление какого-либо компонента, и тогда контейнер обновляет все компоненты, -зависящие от изменённого. Это происходит атомарно: вначале процесса открывается транзакция, -которая закрывается только при условии успешной инициализации всех компонентов и откатывается, если произошла хотя бы одна ошибка. +В середине жизненного цикла компонент может быть обновлен, и тогда контейнер обновляет все компоненты, +которые зависят от измененного компонента. Это происходит атомарно: в начале процесса открывается транзакция, +которая закрывается только если все компоненты успешно инициализированы, и откатывается, если возникает хотя бы одна ошибка. -### Жизненный цикл компонент { #component-lifecycle } +### Жизненный цикл компонента { #component-lifecycle } -По умолчанию все компоненты создаются единственным экземпляром, они просто создаются через конструктор на этапе инициализации. -Если нужно сделать какие-то действия перед инициализацией компонента или перед его освобождением, то необходимо реализовать интерфейс `Lifecycle`: +По умолчанию все компоненты создаются как синглтоны через конструктор или фабричный метод во время инициализации. +Если вам нужно выполнить какие-либо действия при инициализации компонента или перед его освобождением, вы должны реализовать интерфейс `Lifecycle`: ```java public interface Lifecycle { @@ -790,9 +955,13 @@ public interface Lifecycle { } ``` -В контейнере все компоненты инициализируются асинхронно и параллельно настолько, насколько это возможно. +В контейнере зависимостей все компоненты инициализируются асинхронно и максимально параллельно. + +Если вам нужно предоставить компонент с жизненным циклом из фабричного метода, вы можете использовать класс `LifecycleWrapper`. +Он реализует сразу два контракта: -Если требуется предоставить компонент в методе фабрике с жизненным циклом можно воспользоваться классом `LifecycleWrapper`: +* `Lifecycle` — контейнер вызовет `init()` при запуске и `release()` при освобождении компонента +* `Wrapped` — контейнер внедрит значение `T`, возвращаемое методом `value()` ===! ":fontawesome-brands-java: `Java`" @@ -802,10 +971,10 @@ public interface Lifecycle { default Wrapped someService() { return new LifecycleWrapper<>(new SomeService(), (component) -> { - // логика инициализации + // initialize logic }, (component) -> { - // логика освобождения + // release logic }); } } @@ -820,25 +989,34 @@ public interface Lifecycle { fun someService(): Wrapped { return LifecycleWrapper(SomeService(), { component -> - // логика инициализации + // initialize logic }, { component -> - // логика освобождения + // release logic } ) } } ``` +Если вам нужно вернуть собственную обертку, она должна реализовывать `Wrapped`: + +```java +public interface Wrapped { + + T value(); +} +``` + ### Штатное завершение { #graceful-shutdown } -Все интеграции, которые предоставляет Kora такие, как [HTTP-сервер](http-server.md), [Kafka-потребитель](kafka.md) -и т.п., поддерживают [штатное завершение](https://maxilect.ru/blog/pochemu-vazhen-graceful-shutdown-v-oblachnoy-srede-na-pr/) из коробки посредством -[жизненного цикла компонент](#container-lifecycle). +Все интеграции, которые предоставляет `Kora`, такие как [HTTP-сервер](http-server.md) и [потребитель Kafka](kafka.md), +поддерживают [штатное завершение](https://www.techtarget.com/whatis/definition/graceful-shutdown-and-hard-shutdown) из коробки, используя +[жизненный цикл компонента](#component-lifecycle). -Также все компоненты, которые реализуют интерфейс `AutoClosable`, будут автоматически завершены штатно контейнером зависимостей перед освобождением. +Все компоненты, которые реализуют `AutoCloseable`, также будут автоматически закрыты контейнером зависимостей перед освобождением. -### Непрямые зависимости { #indirect-dependency } +### Косвенная зависимость { #indirect-dependency } Рассмотрим следующий пример: @@ -875,14 +1053,14 @@ public interface Lifecycle { } ``` -У нас два сервиса, и третий сервис, который зависит от них. Но есть разница в жизненном цикле. -Если мы принимаем тип как зависимость напрямую, то мы говорим контейнеру, что при обновлении компонента `ServiceA`, нужно точно также обновить компонент `ServiceC`. -Но когда мы используем тип обёртку `ValueOf`, то мы сообщаем контейнеру, -что `ServiceC` никак не связан с жизненным циклом `ServiceB` и в случае изменения `ServiceB` нам не нужно обновлять `ServiceC`. +У нас есть два сервиса и третий сервис, который зависит от них. Но есть разница в жизненном цикле. +Если мы берем тип как зависимость напрямую, то мы сообщаем контейнеру, что при обновлении компонента `ServiceA` нам нужно точно так же обновить компонент `ServiceC`. +Но когда мы используем обертку типа `ValueOf`, мы сообщаем контейнеру, +что `ServiceC` не связан с жизненным циклом `ServiceB`, и если `ServiceB` изменяется, то `ServiceC` обновлять не нужно. -#### Обновление компонент { #updating-components } +#### Обновление компонентов { #updating-components } -Обновление компонент возможно в случае если внедрения зависимостей используется обертка `ValueOf`: +Обновление компонента возможно, если для внедрения зависимости используется обертка `ValueOf`: ```java public interface ValueOf { @@ -893,20 +1071,73 @@ public interface ValueOf { } ``` -Мы можем получать актуальное состояние компонента в контейнере при помощи метода `get`. -Этот механизм используется в таких компонентах, которые нельзя перезагружать во время исполнения приложения. -Например, это касается различных серверов, которые слушают сокеты (http, grpc) — для них через `ValueOf` поставляются обработчики запросов, которые могут быть подвержены изменениям. +Метод `get()` возвращает текущее состояние компонента в контейнере. +Этот механизм используется в компонентах, которые не могут быть перезагружены во время работы приложения. +Например, это касается различных серверов, которые слушают сокеты (`HTTP`, `gRPC`): обработчики запросов, которые могут изменяться, +поставляются им через `ValueOf`. + +С помощью метода `refresh()` вы можете инициировать обновление компонента. Этот механизм используется, например, компонентом, +который отслеживает изменения файла конфигурации на диске. +Когда содержимое файла изменяется, он инициирует обновление компонента конфигурации, и затем все изменения распространяются +по цепочке компонентов, связанных прямыми зависимостями. + +`ValueOf` также имеет дополнительные методы для удобной работы с обернутым значением: + +* `map(...)` — преобразует значение внутри `ValueOf` без изменения связи с исходным компонентом +* `optional()` — преобразует `ValueOf` в `ValueOf>` + +Если компоненту нужна отложенная ссылка, он может использовать `PromiseOf`. +Метод `get()` возвращает `Optional`: до связывания графа он пуст, а после связывания получает текущий компонент из контейнера. + +```java +public interface PromiseOf { + + Optional get(); +} +``` + +Как и `ValueOf`, `PromiseOf` поддерживает `map(...)` и `optional()`. +Большинству бизнес-кода нужна только прямая зависимость или `ValueOf`; `PromiseOf` предназначен для более низкоуровневых сценариев, +где компоненту нужен отложенный доступ к другой части графа. -При помощи функции `refresh` мы можем инициировать обновление компонента. Этот механизм например используется в компоненте отслеживающем изменения файла конфигурации на диске. -При изменении контента файла, он инициирует обновления компонента конфигурации, и дальше все изменения распространяются по цепочке компонентов, связанных прямой связью. +Если компонент, полученный через `ValueOf>`, нужно передать дальше как обычный `ValueOf`, +вы можете использовать `Wrapped.UnwrappedValue.unwrap(...)`. Это полезно для оберток, которые добавляют жизненный цикл или другое поведение, +но должны предоставлять наружу обычное значение. -### Инспекция компонент { #component-inspection } +#### Слушатели обновлений { #refresh-listener } -Есть ситуации, когда есть некоторый компонент в контейнере, который нужно дополнительно изменить или инициализировать, -но при этом нужно, чтобы никто не начал работать с этим компонентом до того, как мы сделаем эти действия. -Для этого случая предусмотрен механизм перехвата компонентов. Вам нужно положить в контейнер объект реализующий интерфейс `GraphInterceptor`. +Если компоненту нужно знать, что граф был успешно обновлен, он может реализовать `RefreshListener`: + +```java +public interface RefreshListener { + + void graphRefreshed() throws Exception; +} +``` + +Контейнер вызывает `graphRefreshed()` после успешного обновления графа. Если компонент одновременно является оберткой значения и слушателем обновлений, +он может реализовать комбинированный интерфейс `WrappedRefreshListener`. + +`RefreshListener` нужен только для получения уведомления после завершения обновления. Он не требуется для того, чтобы контейнер +пересоздал компонент. Если обновление затрагивает компонент или его зависимости, а другие компоненты внедрили его напрямую, +без `ValueOf` или `PromiseOf`, то эти зависимые компоненты также будут обновлены автоматически. + +### Инспекция компонента { #component-inspection } + +Бывают ситуации, когда компонент в контейнере нужно дополнительно изменить или инициализировать, +но никто не должен начинать работать с этим компонентом до завершения этих действий. +Для этого случая существует механизм перехвата компонентов. Поместите в контейнер объект, реализующий интерфейс `GraphInterceptor`. + +```java +public interface GraphInterceptor { + + T init(T value); + + T release(T value); +} +``` -Например, этот механизм может использоваться для прогрева кэша на базе `JdbcDatabase`: +Например, этот механизм можно использовать для прогрева кэша на основе `JdbcDatabase`: ===! ":fontawesome-brands-java: `Java`" @@ -942,6 +1173,7 @@ public interface ValueOf { } ``` -Интерфейс `GraphInterceptor` практически повторяет контракт `Lifecycle`, за исключением возвращаемого типа. -Тут мы ожидаем, что метод может вернуть изменённый или вообще другой экземпляр объекта данного типа, -и уже этот объект будет использован как зависимость другими компонентами. +Интерфейс `GraphInterceptor` почти такой же, как контракт `Lifecycle`, за исключением возвращаемого типа. +Метод `init(T value)` получает уже полностью инициализированный компонент. Метод может вернуть измененный или совершенно другой +экземпляр данного типа, и этот объект будет использован как зависимость другими компонентами. +Метод `release(T value)` получает компонент перед освобождением, то есть это все еще работающий и еще не очищенный экземпляр. diff --git a/mkdocs/docs/ru/documentation/database-cassandra.md b/mkdocs/docs/ru/documentation/database-cassandra.md index f592da2..0ca8d1d 100644 --- a/mkdocs/docs/ru/documentation/database-cassandra.md +++ b/mkdocs/docs/ru/documentation/database-cassandra.md @@ -4,9 +4,14 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora Cassandra repositories, Cassandra driver configuration, profiles, entity and UDT mapping, async access, and repository signatures; key triggers include @Repository, @Query, @EntityCassandra, @Table, @Id, @Column, @UDT, CassandraModule, CassandraRepository." --- -Модуль предоставляет реализацию репозиториев для базы данных [Cassandra](https://cassandra.apache.org/_/cassandra-basics.html) с использованием драйвера [DataStax](https://docs.datastax.com/en/developer/java-driver/4.17/). +Модуль предоставляет реализацию репозитория для базы данных [Cassandra](https://cassandra.apache.org/_/cassandra-basics.html) с использованием драйвера [DataStax](https://docs.datastax.com/en/developer/java-driver/4.17/). +`Cassandra` — это распределенная колоночная база данных, где запросы пишутся на `CQL`, а модель данных обычно проектируется под конкретные сценарии чтения. +В Kora модуль Cassandra предоставляет декларативные репозитории поверх `CqlSession`: приложение пишет `CQL`-запросы в `@Query`, а Kora на этапе компиляции генерирует код подготовки запроса, связывания параметров и отображения результата. -Если нужен пошаговый разбор перед справочным описанием, смотрите [База данных Cassandra](../guides/database-cassandra.md). +Общие правила для отображений, `@Repository`, `@Query`, макросов, пакетных запросов и аннотаций `@Table`, `@Column`, `@Id`, `@Embedded` описаны в разделе [общих правил работы с базами данных](database-common.md). +Этот документ охватывает специфичные для Cassandra части: подключение драйвера, конфигурацию `CqlSession`, профили выполнения, `UDT`, отображатели и поддерживаемые сигнатуры методов. + +Пошаговый разбор перед справочным описанием смотрите в разделе [База данных Cassandra](../guides/database-cassandra.md). ## Подключение { #dependency } @@ -38,7 +43,10 @@ agent: ## Конфигурация { #configuration } -Пример простой конфигурация, описанной в классе `CassandraConfig` (указаны примеры значений): +Конфигурация читается из секции `cassandra` и описывается интерфейсом `CassandraConfig`. +Как минимум необходимо указать `basic.contactPoints`. Остальные параметры необязательны или передаются драйверу только при явной настройке. + +Пример простой конфигурации: ===! ":material-code-json: `Hocon`" @@ -59,12 +67,12 @@ agent: } ``` - 1. Адреса нод Cassandra для подключения к базе данных (**обязательный**) - 2. Имя датацентра Cassandra (по умолчанию отсутвует) - 3. Имя keyspace для подключения (по умолчанию отсутвует) - 4. Ограничение время выполнения запросов в рамках подключения (по умолчанию отсутвует) - 5. Имя пользователя для подключения (по умолчанию отсутвует) - 6. Пароль пользователя для подключения (по умолчанию отсутвует) + 1. Адреса узлов `Cassandra` для подключения к базе данных (`обязательно`, без значения по умолчанию) + 2. Имя датацентра `Cassandra` (по умолчанию не указано, необязательно) + 3. Имя `keyspace` для подключения (по умолчанию не указано, необязательно) + 4. Таймаут выполнения запроса для подключения (по умолчанию не указано, необязательно) + 5. Имя пользователя для подключения (по умолчанию не указано, необязательно) + 6. Пароль для подключения (по умолчанию не указано, необязательно) === ":simple-yaml: `YAML`" @@ -81,208 +89,218 @@ agent: password: "password" #(6)! ``` - 1. Адреса нод Cassandra для подключения к базе данных (**обязательный**) - 2. Имя датацентра Cassandra (по умолчанию отсутвует) - 3. Имя keyspace для подключения (по умолчанию отсутвует) - 4. Ограничение время выполнения запросов в рамках подключения (по умолчанию отсутвует) - 5. Имя пользователя для подключения (по умолчанию отсутвует) - 6. Пароль пользователя для подключения (по умолчанию отсутвует) + 1. Адреса узлов `Cassandra` для подключения к базе данных (`обязательно`, без значения по умолчанию) + 2. Имя датацентра `Cassandra` (по умолчанию не указано, необязательно) + 3. Имя `keyspace` для подключения (по умолчанию не указано, необязательно) + 4. Таймаут выполнения запроса для подключения (по умолчанию не указано, необязательно) + 5. Имя пользователя для подключения (по умолчанию не указано, необязательно) + 6. Пароль для подключения (по умолчанию не указано, необязательно) ??? abstract "Пример полной конфигурации" - Пример полной конфигурации с примерами значений которые могут быть описаны (конфигурация описана в классе `CassandraConfig`): + Полная конфигурация с примерами значений. Описания параметров являются общими для примеров `HOCON` и `YAML`. ===! ":material-code-json: `Hocon`" ```javascript cassandra { auth { - login = "username" - password = "password" + login = "username" //(1)! + password = "password" //(2)! } basic { - contactPoints = [ "127.0.0.1:9042", "127.0.0.2:9042" ] // хосты нод кассандры - sessionName = "some-session-name" // имя сессии - dc = "datacenter1" // Имя датацентра - sessionKeyspace = "test-db" // Название keyspace для этой сессии - - loadBalancingPolicy.slowReplicaAvoidance = true // Флаг включения механизма избегания медленных реплик - cloud.secureConnectBundle = "/location/of/secure/connect/bundle" // Расположения бандла для подключения к Datastax Apache Cassandra. Путь должен быть валидным URL'ом. По умолчанию, если не указан протокол, будет считаться что это file:// - request { // Настройки запросов - timeout = "5s" // таймаут запроса - consistency = "LOCAL_ONE" // уровень консистентности, допустимые значения: ANY, ONE, TWO, THREE, QUORUM, ALL, LOCAL_QUORUM, EACH_QUORUM, SERIAL, LOCAL_SERIAL, LOCAL_ONE - pageSize = 5000 // Ограничение размера страницы (определяет, сколько строк может быть возвращено за один запрос) - serialConsistency = "LOCAL_SERIAL" // Уровень консистентности для легковесных транзакций(LWT). Допустимые значения SERIAL и LOCAL_SERIAL. - defaultIdempotence = false // Настройки значения идемпотентности для запросов + contactPoints = [ "127.0.0.1:9042", "127.0.0.2:9042" ] //(3)! + sessionName = "some-session-name" //(4)! + dc = "datacenter1" //(5)! + sessionKeyspace = "test-db" //(6)! + + loadBalancingPolicy.slowReplicaAvoidance = true //(7)! + cloud.secureConnectBundle = "/location/of/secure/connect/bundle" //(8)! + request { + timeout = "5s" //(9)! + consistency = "LOCAL_ONE" //(10)! + pageSize = 5000 //(11)! + serialConsistency = "LOCAL_SERIAL" //(12)! + defaultIdempotence = false //(13)! } } - advanced { // Расширенные настройки - sessionLeak.threshold = 4 // Максимальное количество активных сессий + + advanced { + sessionLeak.threshold = 4 //(14)! connection { - connectTimeout = "10s" // Таймаут подключения - initQueryTimeout = "10s" // Таймаут инициализации запроса - setKeyspaceTimeout = "10s" // Таймаут установки keyspace - maxRequestsPerConnection = 1024 // Ограничение запросов на одно подключение - maxOrphanRequests = 256 // Максимальное количество "осиротевших" запросов, т.е. тех, ответ на которые по тем или иным причинам прекратили ожидать. - warnOnInitError = true // Выводить ошибки при инициализации в лог - pool { // Настройки пула. - localSize = 10 - remoteSize = 10 + connectTimeout = "10s" //(15)! + initQueryTimeout = "10s" //(16)! + setKeyspaceTimeout = "10s" //(17)! + maxRequestsPerConnection = 1024 //(18)! + maxOrphanRequests = 256 //(19)! + warnOnInitError = true //(20)! + pool { + localSize = 10 //(21)! + remoteSize = 10 //(22)! } } - reconnectOnInit = false // Повторять попытку инициализации, если при первой попытке все ноды, указанные в contactpoints, не ответили - reconnectionPolicy { // Политика переподключения - базовая и максимальная задержка. По умолчанию, при неудачно попытке используется первое значение, затем при каждой следующей - удваивается, пока не достигнет максимального значения - baseDelay = "1s" - maxDelay = "60s" + reconnectOnInit = false //(23)! + reconnectionPolicy { + baseDelay = "1s" //(24)! + maxDelay = "60s" //(25)! + } + loadBalancingPolicy.dcFailover { + maxNodesPerRemoveDc = 1 //(26)! + allowForLocalConsistencyLevels = false //(27)! } - sslEngineFactory { - cipherSuites = [ "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_CBC_SHA" ] - hostnameValidation = true // Валидация имени хоста - keystorePath = "/path/to/client.keystore" // Путь к хранилищу ключей - keystorePassword = "password" // Пароль от хранилища ключей - truststorePath = "/path/to/client.truststore" // Путь к доверенному хранилищу - truststorePassword = "password" // Пароль от доверенного хранилища + cipherSuites = [ "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_CBC_SHA" ] //(28)! + hostnameValidation = true //(29)! + keystorePath = "/path/to/client.keystore" //(30)! + keystorePassword = "password" //(31)! + truststorePath = "/path/to/client.truststore" //(32)! + truststorePassword = "password" //(33)! } - - timestampGenerator { // Генератор, добавляющий timestamp к каждому запросу. По умолчанию используется AtomicTimestampGenerator - forceJavaClock = false // Принудительно использовать Java system clock - driftWarning.threshold = "1s" // Указывает, насколько далеко в будущее могут "убегать" таймстэмпы при высокой нагрузке - driftWarning.interval = "10s" // Интервал логирования предупреждений, есди таймстэмпы продолжают "убегать" вперёд. + timestampGenerator { + forceJavaClock = false //(34)! + driftWarning.threshold = "1s" //(35)! + driftWarning.interval = "10s" //(36)! } - protocol { - version = "V4" // Версия протокола Cassandra - compression = "lz4" // Сжатие - maxFrameLength = 268435456 // Максимальная длина фрейма в байтах + version = "V4" //(37)! + compression = "lz4" //(38)! + maxFrameLength = 268435456 //(39)! } request { - warnIfSetKeyspace = true // Логировать предупреждение о том, что в запросе выполняется установка keyspace - trace { // Настройки встроенного механизма трейсинга запросов - attempts = 5 // Количество попыток - interval = "1ms" // Интервал между попытками - consistency = "ONE" // Уровень консистентности + warnIfSetKeyspace = true //(40)! + trace { + attempts = 5 //(41)! + interval = "1ms" //(42)! + consistency = "ONE" //(43)! } - logWarnings = true + logWarnings = true //(44)! } - metrics { // session-level метрики, по умолчанию выключены все - node.enabled = [] // Список включенных метрик. Включаемые: bytes-sent, connected-nodes, cql-requests, cql-client-timeouts, cql-prepared-cache-size, throttling.delay, throttling.errors, continuous-cql-requests - session.enabled = [] - publishPercentileHistogram = false // публиковать ли персентили в метриках в рамках мин/макс вместе с SLO - node.cqlMessages { // Дополнительные настройки для метрик, если нужны: - lowestLatency = "1ms" - highestLatency = "90s" - significantDigits = 1 - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] + metrics { + idGenerator { + name = "TaggingMetricIdGenerator" //(45)! + prefix = "my-app" //(46)! } - session.cqlRequests { - lowestLatency = "1ms" - highestLatency = "90s" - significantDigits = 1 - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] + node { + enabled = [ "bytes-sent", "bytes-received", "open-connections" ] //(47)! + cqlMessages { + lowestLatency = "1ms" //(48)! + highestLatency = "90s" //(49)! + significantDigits = 1 //(50)! + refreshInterval = "10s" //(51)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(52)! + } } - session.throttlingDelay { - lowestLatency = "1ms" - highestLatency = "90s" - significantDigits = 1 - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] + session { + enabled = [ "connected-nodes", "cql-requests", "cql-client-timeouts" ] //(53)! + cqlRequests { + lowestLatency = "1ms" //(54)! + highestLatency = "90s" //(55)! + significantDigits = 1 //(56)! + refreshInterval = "10s" //(57)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(58)! + } + throttlingDelay { + lowestLatency = "1ms" //(59)! + highestLatency = "90s" //(60)! + significantDigits = 1 //(61)! + refreshInterval = "10s" //(62)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(63)! + } } + publishPercentileHistogram = false //(64)! } socket { - tcpNoDelay = true // Флаг для отключения Nagle алгоритма, по умолчанию true(выключен), т.к. драйвер имеет собственный message coalescing algorithm - keepAlive = false - reuseAddress = true // Позволять переиспользовать адрес - lingerInterval = 0 - receiveBufferSize = 65535 - sendBufferSize = 65535 + tcpNoDelay = true //(65)! + keepAlive = false //(66)! + reuseAddress = true //(67)! + lingerInterval = 0 //(68)! + receiveBufferSize = 65535 //(69)! + sendBufferSize = 65535 //(70)! } heartbeat { - interval = "30s" - timeout = "2m" + interval = "30s" //(71)! + timeout = "2m" //(72)! } - metadata { // Настройки, отвечающие за schema metadata + metadata { schema { - enabled = true - requestTimeout = "20s" - requestPageSize = 20 - refreshedKeyspaces = [ "ks1", "ks2" ] - debouncer.window = "1s" // Время, которое драйвер ждёт перед применением обновления - debouncer.maxEvents = 20 // Максимальное количество обновлений, которое может быть накоплено + enabled = true //(73)! + requestTimeout = "20s" //(74)! + requestPageSize = 20 //(75)! + refreshedKeyspaces = [ "ks1", "ks2" ] //(76)! + debouncer.window = "1s" //(77)! + debouncer.maxEvents = 20 //(78)! } - topologyEventDebouncer.window = "1s" // Окно для отправки события. - topologyEventDebouncer.maxEvents = 20 // Максимальное количество событий в пачке - tokenMapEnabled = true + topologyEventDebouncer.window = "1s" //(79)! + topologyEventDebouncer.maxEvents = 20 //(80)! + tokenMapEnabled = true //(81)! } controlConnection { - timeout = "10s" + timeout = "10s" //(82)! schemaAgreement { - interval = 200ms - timeout = "10s" - warnOnFailure = true + interval = "200ms" //(83)! + timeout = "10s" //(84)! + warnOnFailure = true //(85)! } } preparedStatements { - prepareOnAllNodes = true // Выполнять подготовку запроса на всех нодах после её успешного выполнения на одной ноде. + prepareOnAllNodes = true //(86)! reprepareOnUp { - enabled = true // Подготавливать запросы для новых нод - checkSystemTable = false // Проверять наличие prepare statement в system.prepared_statements ноды перед подготовкой - maxStatements = 0 // Максимальной количество запросов, которые можно переподготовить - maxParallelism = 100 // Максимальное количество конкурентных запросов - timeout = 20s + enabled = true //(87)! + checkSystemTable = false //(88)! + maxStatements = 0 //(89)! + maxParallelism = 100 //(90)! + timeout = "20s" //(91)! } - preparedCache.weakValues = false + preparedCache.weakValues = false //(92)! } - netty { // Настройки Netty event loop, используемой в драйвере - ioGroup.size = 0 // Количество тредов - ioGroup.shutdown { // Настройки штатного завершения - quietPeriod = 2 - timeout = 15 - unit = "SECONDS" + netty { + ioGroup.size = 0 //(93)! + ioGroup.shutdown { + quietPeriod = 2 //(94)! + timeout = 15 //(95)! + unit = "SECONDS" //(96)! } - adminGroup.size = 2 // Event loop группа, используемая только для админских задач, не связанных с IO + adminGroup.size = 2 //(97)! adminGroup.shutdown { - quietPeriod = 2 - timeout = 15 - unit = "SECONDS" + quietPeriod = 2 //(98)! + timeout = 15 //(99)! + unit = "SECONDS" //(100)! } - timer.tickDuration = "100ms" // Настройки того, как часто таймер должен пробуждаться для проверки просроченных задач - timer.ticksPerWheel = 2048 - daemon = false + timer.tickDuration = "100ms" //(101)! + timer.ticksPerWheel = 2048 //(102)! + daemon = false //(103)! + } + coalescer.rescheduleInterval = "10ms" //(104)! + resolveContactPoints = false //(105)! + throttler { + throttlerClass = "ConcurrencyLimitingRequestThrottler" //(106)! + maxConcurrentRequests = 1024 //(107)! + maxRequestsPerSecond = 10000 //(108)! + maxQueueSize = 10000 //(109)! + drainInterval = "1ms" //(110)! } - coalescer.rescheduleInterval = "10ms" - resolveContactPoints = false } - profiles { // Настройки, переопределяемые в профиле + + profiles { someProfile { - basic { - // basic.request.timeout - // basic.request.consistency - } - advanced { - // advanced.request.trace.consistency - // advanced.request.trace.attempts - } + basic.request.timeout = "10s" //(111)! + basic.request.consistency = "LOCAL_QUORUM" //(112)! + advanced.request.trace.attempts = 3 //(113)! + advanced.request.trace.consistency = "ONE" //(114)! } - } + } + telemetry { - logging { - enabled = false - } + logging.enabled = false //(115)! metrics { - enabled = true - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - tags = { - "key1" = "value1" - "key2" = "value2" - } + enabled = true //(116)! + slo = [ 1, 10, 50, 100, 200, 500, 1000 ] //(117)! + tags = { "key1" = "value1", "key2" = "value2" } //(118)! } tracing { - enabled = true - attributes = { - "key1" = "value1" - "key2" = "value2" - } + enabled = true //(119)! + attributes = { "key1" = "value1", "key2" = "value2" } //(120)! } } } @@ -292,177 +310,309 @@ agent: ```yaml cassandra: - advanced: # Расширенные настройки - coalescer: - rescheduleInterval: "10ms" + auth: + login: "username" #(1)! + password: "password" #(2)! + basic: + contactPoints: [ "127.0.0.1:9042", "127.0.0.2:9042" ] #(3)! + sessionName: "some-session-name" #(4)! + dc: "datacenter1" #(5)! + sessionKeyspace: "test-db" #(6)! + loadBalancingPolicy: + slowReplicaAvoidance: true #(7)! + cloud: + secureConnectBundle: "/location/of/secure/connect/bundle" #(8)! + request: + timeout: "5s" #(9)! + consistency: "LOCAL_ONE" #(10)! + pageSize: 5000 #(11)! + serialConsistency: "LOCAL_SERIAL" #(12)! + defaultIdempotence: false #(13)! + advanced: + sessionLeak: + threshold: 4 #(14)! connection: - connectTimeout: "10s" # Таймаут подключения - initQueryTimeout: "10s" # Таймаут инициализации запроса - setKeyspaceTimeout: "10s" # Таймаут установки keyspace - maxOrphanRequests: 256 # Максимальное количество "осиротевших" запросов, т.е. тех, ответ на которые по тем или иным причинам прекратили ожидать. - maxRequestsPerConnection: 1024 # Ограничение запросов на одно подключение - pool: # Настройки пула. - localSize: 10 - remoteSize: 10 - warnOnInitError: true # Выводить ошибки при инициализации в лог - controlConnection: - schemaAgreement: - interval: "200ms" - timeout: "10s" - warnOnFailure: true - timeout: "10s" + connectTimeout: "10s" #(15)! + initQueryTimeout: "10s" #(16)! + setKeyspaceTimeout: "10s" #(17)! + maxRequestsPerConnection: 1024 #(18)! + maxOrphanRequests: 256 #(19)! + warnOnInitError: true #(20)! + pool: + localSize: 10 #(21)! + remoteSize: 10 #(22)! + reconnectOnInit: false #(23)! + reconnectionPolicy: + baseDelay: "1s" #(24)! + maxDelay: "60s" #(25)! + loadBalancingPolicy: + dcFailover: + maxNodesPerRemoveDc: 1 #(26)! + allowForLocalConsistencyLevels: false #(27)! + sslEngineFactory: + cipherSuites: [ "TLS_RSA_WITH_AES_128_CBC_SHA", "TLS_RSA_WITH_AES_256_CBC_SHA" ] #(28)! + hostnameValidation: true #(29)! + keystorePath: "/path/to/client.keystore" #(30)! + keystorePassword: "password" #(31)! + truststorePath: "/path/to/client.truststore" #(32)! + truststorePassword: "password" #(33)! + timestampGenerator: + forceJavaClock: false #(34)! + driftWarning: + threshold: "1s" #(35)! + interval: "10s" #(36)! + protocol: + version: "V4" #(37)! + compression: "lz4" #(38)! + maxFrameLength: 268435456 #(39)! + request: + warnIfSetKeyspace: true #(40)! + trace: + attempts: 5 #(41)! + interval: "1ms" #(42)! + consistency: "ONE" #(43)! + logWarnings: true #(44)! + metrics: + idGenerator: + name: "TaggingMetricIdGenerator" #(45)! + prefix: "my-app" #(46)! + node: + enabled: [ "bytes-sent", "bytes-received", "open-connections" ] #(47)! + cqlMessages: + lowestLatency: "1ms" #(48)! + highestLatency: "90s" #(49)! + significantDigits: 1 #(50)! + refreshInterval: "10s" #(51)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(52)! + session: + enabled: [ "connected-nodes", "cql-requests", "cql-client-timeouts" ] #(53)! + cqlRequests: + lowestLatency: "1ms" #(54)! + highestLatency: "90s" #(55)! + significantDigits: 1 #(56)! + refreshInterval: "10s" #(57)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(58)! + throttlingDelay: + lowestLatency: "1ms" #(59)! + highestLatency: "90s" #(60)! + significantDigits: 1 #(61)! + refreshInterval: "10s" #(62)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(63)! + publishPercentileHistogram: false #(64)! + socket: + tcpNoDelay: true #(65)! + keepAlive: false #(66)! + reuseAddress: true #(67)! + lingerInterval: 0 #(68)! + receiveBufferSize: 65535 #(69)! + sendBufferSize: 65535 #(70)! heartbeat: - interval: "30s" - timeout: "2m" - metadata: # Настройки, отвечающие за schema metadata + interval: "30s" #(71)! + timeout: "2m" #(72)! + metadata: schema: + enabled: true #(73)! + requestTimeout: "20s" #(74)! + requestPageSize: 20 #(75)! + refreshedKeyspaces: [ "ks1", "ks2" ] #(76)! debouncer: - maxEvents: 20 # Максимальное количество обновлений, которое может быть накоплено - window: "1s" # Время, которое драйвер ждёт перед применением обновления - enabled: true - refreshedKeyspaces: - - ks1 - - ks2 - requestPageSize: 10 - requestTimeout: "20s" - tokenMapEnabled: true - topologyEventDebouncer: - maxEvents: 20 # Максимальное количество событий в пачке - window: "1s" # Окно для отправки события. - metrics: - publishPercentileHistogram: false # публиковать ли персентили в метриках в рамках мин/макс вместе с SLO - node: - enabled: [] # Список включенных метрик. Включаемые: bytes-sent, connected-nodes, cql-requests, cql-client-timeouts, cql-prepared-cache-size, throttling.delay, throttling.errors, continuous-cql-requests - cqlMessages: # Дополнительные настройки для метрик, если нужны: - lowestLatency: "1ms" - highestLatency: "90s" - significantDigits: 1 - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - session: # session-level метрики, по умолчанию выключены все - enabled: [] # Список включенных метрик. Включаемые: bytes-sent, connected-nodes, cql-requests, cql-client-timeouts, cql-prepared-cache-size, throttling.delay, throttling.errors, continuous-cql-requests - cqlRequests: - lowestLatency: "1ms" - highestLatency: "90s" - significantDigits: 1 - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - throttlingDelay: - lowestLatency: "1ms" - highestLatency: "90s" - significantDigits: 1 - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - netty: # Настройки Netty event loop, используемой в драйвере - adminGroup: # Event loop группа, используемая только для админских задач, не связанных с IO - shutdown: - quietPeriod: 2 - timeout: 15 - unit: SECONDS - size: 2 - daemon: false - ioGroup: - shutdown: # Настройки штатного завершения - quietPeriod: 2 - timeout: 15 - unit: SECONDS - size: 0 # Количество тредов - timer: - tickDuration: "100ms" # Настройки того, как часто таймер должен пробуждаться для проверки просроченных задач - ticksPerWheel: 2048 - preparedStatements: - prepareOnAllNodes: true # Выполнять подготовку запроса на всех нодах после её успешного выполнения на одной ноде. - preparedCache: - weakValues: false - reprepareOnUp: - enabled: true # Подготавливать запросы для новых нод - checkSystemTable: false # Проверять наличие prepare statement в system.prepared_statements ноды перед подготовкой - maxParallelism: 100 # Максимальное количество конкурентных запросов - maxStatements: 0 # Максимальной количество запросов, которые можно переподготовить - timeout: "20s" - protocol: - compression: "lz4" # Сжатие - maxFrameLength: 268435456 # Максимальная длина фрейма в байтах - version: "V4" # Версия протокола Cassandra - reconnectOnInit: false # Повторять попытку инициализации, если при первой попытке все ноды, указанные в contactpoints, не ответили - reconnectionPolicy: # Политика переподключения - базовая и максимальная задержка. По умолчанию, при неудачно попытке используется первое значение, затем при каждой следующей - удваивается, пока не достигнет максимального значения - baseDelay: "1s" - maxDelay: "60s" - request: - logWarnings: true - trace: - attempts: 5 # Количество попыток - consistency: ONE # Уровень консистентности - interval: "1ms" # Интервал между попытками - warnIfSetKeyspace: true # Логировать предупреждение о том, что в запросе выполняется установка keyspace - resolveContactPoints: false - sessionLeak: - threshold: 4 - socket: - keepAlive: false - lingerInterval: 0 - receiveBufferSize: 65535 - reuseAddress: true # Позволять переиспользовать адрес - sendBufferSize: 65535 - tcpNoDelay: true # Флаг для отключения Nagle алгоритма, по умолчанию true(выключен), т.к. драйвер имеет собственный message coalescing algorithm - sslEngineFactory: - cipherSuites: - - TLS_RSA_WITH_AES_128_CBC_SHA - - TLS_RSA_WITH_AES_256_CBC_SHA - hostnameValidation: true # Валидация имени хоста - keystorePassword: "password" # Пароль от хранилища ключей - keystorePath: "/path/to/client.keystore" # Путь к хранилищу ключей - truststorePassword: "password" # Пароль от доверенного хранилища - truststorePath: "/path/to/client.truststore" # Путь к доверенному хранилищу - timestampGenerator: # Генератор, добавляющий timestamp к каждому запросу. По умолчанию используется AtomicTimestampGenerator - driftWarning: - interval: "10s" # Интервал логирования предупреждений, есди таймстэмпы продолжают "убегать" вперёд. - threshold: "1s" # Указывает, насколько далеко в будущее могут "убегать" таймстэмпы при высокой нагрузке - forceJavaClock: false # Принудительно использовать Java system clock - auth: - login: "username" - password: "password" - basic: - cloud: - secureConnectBundle: "/location/of/secure/connect/bundle" - contactPoints: - - "127.0.0.1:9042" - - "127.0.0.2:9042" - dc: "datacenter1" - loadBalancingPolicy: - slowReplicaAvoidance: true - request: - consistency: LOCAL_ONE - defaultIdempotence: false - pageSize: 5000 - serialConsistency: LOCAL_SERIAL - timeout: "5s" - sessionKeyspace: "test-db" - sessionName: "some-session-name" - profiles: # Настройки, переопределяемые в профиле - someProfile: - advanced: - #advanced.request.trace.consistency - #advanced.request.trace.attempts - basic: - #basic.request.timeout - #basic.request.consistency + window: "1s" #(77)! + maxEvents: 20 #(78)! + topologyEventDebouncer: + window: "1s" #(79)! + maxEvents: 20 #(80)! + tokenMapEnabled: true #(81)! + controlConnection: + timeout: "10s" #(82)! + schemaAgreement: + interval: "200ms" #(83)! + timeout: "10s" #(84)! + warnOnFailure: true #(85)! + preparedStatements: + prepareOnAllNodes: true #(86)! + reprepareOnUp: + enabled: true #(87)! + checkSystemTable: false #(88)! + maxStatements: 0 #(89)! + maxParallelism: 100 #(90)! + timeout: "20s" #(91)! + preparedCache: + weakValues: false #(92)! + netty: + ioGroup: + size: 0 #(93)! + shutdown: + quietPeriod: 2 #(94)! + timeout: 15 #(95)! + unit: "SECONDS" #(96)! + adminGroup: + size: 2 #(97)! + shutdown: + quietPeriod: 2 #(98)! + timeout: 15 #(99)! + unit: "SECONDS" #(100)! + timer: + tickDuration: "100ms" #(101)! + ticksPerWheel: 2048 #(102)! + daemon: false #(103)! + coalescer: + rescheduleInterval: "10ms" #(104)! + resolveContactPoints: false #(105)! + throttler: + throttlerClass: "ConcurrencyLimitingRequestThrottler" #(106)! + maxConcurrentRequests: 1024 #(107)! + maxRequestsPerSecond: 10000 #(108)! + maxQueueSize: 10000 #(109)! + drainInterval: "1ms" #(110)! + profiles: + someProfile: + basic: + request: + timeout: "10s" #(111)! + consistency: "LOCAL_QUORUM" #(112)! + advanced: + request: + trace: + attempts: 3 #(113)! + consistency: "ONE" #(114)! telemetry: logging: - enabled: false + enabled: false #(115)! metrics: - enabled: true - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] - tags: - key1: value1 - key2: value2 + enabled: true #(116)! + slo: [ 1, 10, 50, 100, 200, 500, 1000 ] #(117)! + tags: { key1: "value1", key2: "value2" } #(118)! tracing: - enabled: true - attributes: - key1: value1 - key2: value2 + enabled: true #(119)! + attributes: { key1: "value1", key2: "value2" } #(120)! ``` -### Ручная конфигурация { #code-configuration } - -Возможно конфигурировать драйвер вручную в коде, используя `CassandraConfigurer` для модификации построителя `CqlSession`: + 1. Имя пользователя для аутентификации в `Cassandra` (по умолчанию не указано, необязательно). + 2. Пароль для аутентификации в `Cassandra` (по умолчанию не указано, необязательно). + 3. Адреса узлов `Cassandra` в формате `host:port` (`обязательно`, без значения по умолчанию). + 4. Имя сессии драйвера, используемое в логах, метриках и диагностике (по умолчанию не указано, необязательно). + 5. Локальный датацентр для политики балансировки нагрузки (по умолчанию не указано, необязательно). + 6. `keyspace`, который будет установлен для сессии после подключения (по умолчанию не указано, необязательно). + 7. Включает избегание медленных реплик в стандартной политике балансировки нагрузки (по умолчанию не указано, необязательно). + 8. Путь или `URL` к `Secure Connect Bundle` для подключения к `DataStax Astra` / облачной Cassandra (по умолчанию не указано, необязательно). + 9. Обычный таймаут запроса (по умолчанию не указано, необязательно). + 10. Уровень согласованности обычного запроса, например `ONE`, `LOCAL_ONE`, `LOCAL_QUORUM`, `QUORUM`, `ALL` (по умолчанию не указано, необязательно). + 11. Размер страницы результата, то есть максимальное количество строк, запрашиваемых за один сетевой обмен (по умолчанию не указано, необязательно). + 12. Уровень последовательной согласованности для облегченных транзакций `LWT`: `SERIAL` или `LOCAL_SERIAL` (по умолчанию не указано, необязательно). + 13. Значение идемпотентности запроса по умолчанию; влияет на то, можно ли безопасно применять повторные попытки и спекулятивное выполнение (по умолчанию не указано, необязательно). + 14. Порог предупреждения об утечке сессии драйвера (по умолчанию не указано, необязательно). + 15. Таймаут открытия сетевого соединения с узлом (по умолчанию не указано, необязательно). + 16. Таймаут запросов, которые драйвер выполняет при инициализации соединения (по умолчанию не указано, необязательно). + 17. Таймаут установки `keyspace` на соединении (по умолчанию не указано, необязательно). + 18. Максимальное количество одновременных запросов на одно соединение (по умолчанию не указано, необязательно). + 19. Максимальное количество запросов, ответ на которые уже не ожидается, но которые все еще могут завершиться внутри драйвера (по умолчанию не указано, необязательно). + 20. Логирует предупреждение при неудачной инициализации соединения для отдельного узла (по умолчанию не указано, необязательно). + 21. Размер пула соединений для узлов локального датацентра (по умолчанию не указано, необязательно). + 22. Размер пула соединений для удаленных узлов (по умолчанию не указано, необязательно). + 23. Разрешает повторную попытку инициализации, когда во время запуска все `contactPoints` не отвечают (по умолчанию не указано, необязательно). + 24. Начальная задержка политики переподключения (по умолчанию не указано, необязательно). + 25. Максимальная задержка политики переподключения (по умолчанию не указано, необязательно). + 26. Максимальное количество узлов удаленного датацентра, которые могут использоваться для отказоустойчивости (по умолчанию не указано, необязательно). + 27. Разрешает переключение на удаленный датацентр для локальных уровней согласованности (по умолчанию не указано, необязательно). + 28. Разрешенные наборы шифров для `SSL/TLS` (по умолчанию не указано, необязательно). + 29. Проверяет, что имя хоста узла соответствует сертификату `SSL/TLS` (по умолчанию не указано, необязательно). + 30. Путь к клиентскому keystore (по умолчанию не указано, необязательно). + 31. Пароль клиентского keystore (по умолчанию не указано, необязательно). + 32. Путь к truststore (по умолчанию не указано, необязательно). + 33. Пароль truststore (по умолчанию не указано, необязательно). + 34. Принудительно использует системные часы Java для генерации временных меток запросов (по умолчанию не указано, необязательно). + 35. Порог предупреждения о смещении временной метки в будущее (по умолчанию не указано, необязательно). + 36. Минимальный интервал между предупреждениями о смещении временной метки (по умолчанию не указано, необязательно). + 37. Версия бинарного протокола Cassandra, например `V4` (по умолчанию не указано, необязательно). + 38. Алгоритм сжатия протокола, например `lz4` или `snappy` (по умолчанию не указано, необязательно). + 39. Максимальный размер кадра протокола в байтах (по умолчанию не указано, необязательно). + 40. Логирует предупреждение, когда запрос явно меняет `keyspace` (по умолчанию не указано, необязательно). + 41. Количество попыток получить информацию трассировки запроса из Cassandra (по умолчанию не указано, необязательно). + 42. Интервал между попытками получить информацию трассировки запроса (по умолчанию не указано, необязательно). + 43. Уровень согласованности для запросов к таблицам трассировки (по умолчанию не указано, необязательно). + 44. Логирует предупреждения, возвращаемые Cassandra вместе с ответом на запрос (по умолчанию не указано, необязательно). + 45. Имя генератора идентификаторов метрик драйвера (по умолчанию: `TaggingMetricIdGenerator`). + 46. Префикс имен метрик драйвера (по умолчанию не указано, необязательно). + 47. Включенные метрики уровня узла (по умолчанию: `open-connections`, `in-flight`, `bytes-received`, `bytes-sent`, `write-timeouts`, `read-timeouts`, `aborted-requests`). + 48. Наименьшая ожидаемая задержка для гистограммы метрики `node.cqlMessages` (по умолчанию: `1ms`). + 49. Наибольшая ожидаемая задержка для гистограммы метрики `node.cqlMessages` (по умолчанию: `90s`). + 50. Количество значащих цифр для гистограммы метрики `node.cqlMessages` (по умолчанию не указано, необязательно). + 51. Интервал обновления снимка для гистограммы метрики `node.cqlMessages` (по умолчанию не указано, необязательно). + 52. Границы `SLO` для метрики `node.cqlMessages` (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 53. Включенные метрики уровня сессии (по умолчанию: `connected-nodes`, `cql-requests`, `cql-client-timeouts`, `cql-prepared-cache-size`, `throttling.delay`, `throttling.queue-size`). + 54. Наименьшая ожидаемая задержка для гистограммы метрики `session.cqlRequests` (по умолчанию: `1ms`). + 55. Наибольшая ожидаемая задержка для гистограммы метрики `session.cqlRequests` (по умолчанию: `90s`). + 56. Количество значащих цифр для гистограммы метрики `session.cqlRequests` (по умолчанию не указано, необязательно). + 57. Интервал обновления снимка для гистограммы метрики `session.cqlRequests` (по умолчанию не указано, необязательно). + 58. Границы `SLO` для метрики `session.cqlRequests` (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 59. Наименьшая ожидаемая задержка для гистограммы метрики `session.throttlingDelay` (по умолчанию: `1ms`). + 60. Наибольшая ожидаемая задержка для гистограммы метрики `session.throttlingDelay` (по умолчанию: `90s`). + 61. Количество значащих цифр для гистограммы метрики `session.throttlingDelay` (по умолчанию не указано, необязательно). + 62. Интервал обновления снимка для гистограммы метрики `session.throttlingDelay` (по умолчанию не указано, необязательно). + 63. Границы `SLO` для метрики `session.throttlingDelay` (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 64. Публикует процентильные гистограммы для метрик драйвера (по умолчанию: `false`). + 65. Включает `TCP_NODELAY`, что отключает алгоритм Нейгла (по умолчанию не указано, необязательно). + 66. Включает `SO_KEEPALIVE` для TCP-сокетов (по умолчанию не указано, необязательно). + 67. Включает `SO_REUSEADDR` для TCP-сокетов (по умолчанию не указано, необязательно). + 68. Значение `SO_LINGER` для TCP-сокетов (по умолчанию не указано, необязательно). + 69. Размер буфера приема TCP-сокета в байтах (по умолчанию не указано, необязательно). + 70. Размер буфера отправки TCP-сокета в байтах (по умолчанию не указано, необязательно). + 71. Интервал отправки `heartbeat` по простаивающему соединению (по умолчанию не указано, необязательно). + 72. Таймаут ожидания ответа на `heartbeat` (по умолчанию не указано, необязательно). + 73. Включает загрузку и обновление метаданных схемы (по умолчанию не указано, необязательно). + 74. Таймаут запросов метаданных схемы (по умолчанию не указано, необязательно). + 75. Размер страницы запросов метаданных схемы (по умолчанию не указано, необязательно). + 76. Список имен `keyspace`, метаданные схемы которых обновляются драйвером (по умолчанию не указано, необязательно). + 77. Окно для объединения событий обновления схемы перед обработкой (по умолчанию не указано, необязательно). + 78. Максимальное количество событий обновления схемы, которое может накопиться в окне (по умолчанию не указано, необязательно). + 79. Окно для объединения событий изменения топологии кластера (по умолчанию не указано, необязательно). + 80. Максимальное количество событий изменения топологии, которое может накопиться в окне (по умолчанию не указано, необязательно). + 81. Включает карту токенов для маршрутизации запросов к владельцам данных (по умолчанию не указано, необязательно). + 82. Таймаут служебного `control connection` (по умолчанию не указано, необязательно). + 83. Интервал проверки `schema agreement` между узлами (по умолчанию не указано, необязательно). + 84. Максимальное время ожидания `schema agreement` (по умолчанию не указано, необязательно). + 85. Логирует предупреждение, если `schema agreement` не достигнута вовремя (по умолчанию не указано, необязательно). + 86. Подготавливает запрос на всех узлах после того, как он успешно подготовлен на одном узле (по умолчанию не указано, необязательно). + 87. Повторно подготавливает запросы на узле, который снова стал доступен (по умолчанию не указано, необязательно). + 88. Проверяет системную таблицу `system.prepared_statements` перед повторной подготовкой запроса (по умолчанию не указано, необязательно). + 89. Максимальное количество запросов для повторной подготовки; `0` означает отсутствие ограничения на стороне драйвера (по умолчанию не указано, необязательно). + 90. Максимальное количество параллельных запросов повторной подготовки (по умолчанию не указано, необязательно). + 91. Таймаут повторной подготовки запросов на одном узле (по умолчанию не указано, необязательно). + 92. Хранит значения кэша подготовленных запросов через слабые ссылки (по умолчанию не указано, необязательно). + 93. Количество потоков `Netty` для сетевого ввода-вывода; `0` позволяет драйверу выбрать автоматически (по умолчанию не указано, необязательно). + 94. Период затишья для плавной остановки `ioGroup` (по умолчанию не указано, необязательно). + 95. Максимальное время ожидания остановки `ioGroup` (по умолчанию не указано, необязательно). + 96. Единица измерения для параметров остановки `ioGroup` (по умолчанию не указано, необязательно). + 97. Количество потоков `Netty` для административных задач драйвера (по умолчанию не указано, необязательно). + 98. Период затишья для плавной остановки `adminGroup` (по умолчанию не указано, необязательно). + 99. Максимальное время ожидания остановки `adminGroup` (по умолчанию не указано, необязательно). + 100. Единица измерения для параметров остановки `adminGroup` (по умолчанию не указано, необязательно). + 101. Длительность одного тика таймера `Netty` для отложенных задач драйвера (по умолчанию не указано, необязательно). + 102. Количество тиков в колесе таймера `Netty` (по умолчанию не указано, необязательно). + 103. Делает потоки `Netty` демон-потоками (по умолчанию не указано, необязательно). + 104. Интервал перепланирования для объединения сообщений перед отправкой (по умолчанию не указано, необязательно). + 105. Разрешает драйверу разрешать `contactPoints` через DNS во время запуска (по умолчанию не указано, необязательно). + 106. Класс ограничителя запросов драйвера (по умолчанию не указано, необязательно). + 107. Максимальное количество одновременных запросов для ограничителя (по умолчанию не указано, необязательно). + 108. Максимальное количество запросов в секунду для ограничителя (по умолчанию не указано, необязательно). + 109. Максимальный размер очереди запросов ограничителя (по умолчанию не указано, необязательно). + 110. Интервал, с которым ограничитель освобождает запросы из очереди (по умолчанию не указано, необязательно). + 111. Переопределение `basic.request.timeout` для профиля `someProfile` (по умолчанию не указано, необязательно). + 112. Переопределение `basic.request.consistency` для профиля `someProfile` (по умолчанию не указано, необязательно). + 113. Переопределение `advanced.request.trace.attempts` для профиля `someProfile` (по умолчанию не указано, необязательно). + 114. Переопределение `advanced.request.trace.consistency` для профиля `someProfile` (по умолчанию не указано, необязательно). + 115. Включает логирование запросов Kora (по умолчанию: `false`). + 116. Включает метрики запросов Kora (по умолчанию: `true`). + 117. Границы `SLO` метрик Kora (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 118. Дополнительные теги метрик Kora (по умолчанию: `{}`). + 119. Включает трассировку запросов Kora (по умолчанию: `true`). + 120. Дополнительные атрибуты трассировки Kora (по умолчанию: `{}`). + +### Конфигурация в коде { #code-configuration } + +Драйвер можно настроить вручную в коде, зарегистрировав компонент `CassandraConfigurer`. +Метод `configure` получает `CqlSessionBuilder` и `ProgrammaticDriverConfigLoaderBuilder`, +поэтому вы можете настроить построитель сессии и переопределить низкоуровневые параметры драйвера, которые не доступны через секцию конфигурации `cassandra`: ===! ":fontawesome-brands-java: `Java`" @@ -471,7 +621,7 @@ agent: public final class MyCassandraConfigurer implements CassandraConfigurer { @Override - public CqlSessionBuilder configure(CqlSessionBuilder builder) { + public CqlSessionBuilder configure(CqlSessionBuilder builder, ProgrammaticDriverConfigLoaderBuilder loaderBuilder) { return builder.withClientId(UUID.randomUUID()); } } @@ -482,7 +632,8 @@ agent: ```kotlin @Component class MyCassandraConfigurer : CassandraConfigurer { - override fun configure(builder: CqlSessionBuilder): CqlSessionBuilder { + + override fun configure(builder: CqlSessionBuilder, loaderBuilder: ProgrammaticDriverConfigLoaderBuilder): CqlSessionBuilder { return builder.withClientId(UUID.randomUUID()) } } @@ -490,23 +641,108 @@ agent: ## Использование { #usage } +Чтобы создать репозиторий, объявите интерфейс с `@Repository` и унаследуйте `CassandraRepository`. +Такой репозиторий получает доступ к `CqlSession` через сгенерированный код и использует `@Query` для выполнения `CQL`-запросов. +Параметры запроса связываются по имени: `:id`, `:entity.field`, `:filter.value`. + +Отображения описываются с помощью [общих аннотаций баз данных](database-common.md) и помечаются `@EntityCassandra`, +чтобы `Kora` сгенерировала отображатель на этапе компиляции (см. [Отображение](#view)): + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends CassandraRepository { } + public interface EntityRepository extends CassandraRepository { + + @EntityCassandra + @Table("entities") + record Entity(@Id String id, + @Column("value1") int field1, + String value2, + @Nullable String value3) {} + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + @Nullable + Entity findById(String id); + + @Query("SELECT id, value1, value2, value3 FROM entities") //(2)! + List findAll(); + + @Query("INSERT INTO %{entity#inserts}") //(3)! + void insert(Entity entity); + } ``` + 1. Использует макрос `%{return#selects}` и `%{return#table}`. Разворачивается в запрос: + ```sql + SELECT id, value1, value2, value3 + FROM entities + WHERE id = :id + ``` + Метод использует макросы для `SELECT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + 2. Поля перечислены вручную без использования макросов — это допустимо, но требует поддержки при изменении отображения. + 3. Использует макрос `%{entity#inserts}`. Разворачивается в запрос: + ```sql + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + ``` + Метод использует макросы для `INSERT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : CassandraRepository + interface EntityRepository : CassandraRepository { + + @EntityCassandra + @Table("entities") + data class Entity( + @field:Id val id: String, + @field:Column("value1") val field1: Int, + val value2: String, + val value3: String? + ) + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + fun findById(id: String): Entity? + + @Query("INSERT INTO %{entity#inserts}") //(3)! + fun insert(entity: Entity) + } ``` + 1. Использует макрос `%{return#selects}` и `%{return#table}`. Разворачивается в запрос: + ```sql + SELECT id, value1, value2, value3 + FROM entities + WHERE id = :id + ``` + Метод использует макросы для `SELECT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + 2. Поля перечислены вручную без использования макросов — это допустимо, но требует поддержки при изменении отображения. + 3. Использует макрос `%{entity#inserts}`. Разворачивается в запрос: + ```sql + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + ``` + Метод использует макросы для `INSERT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + +`CQL` остается под контролем разработчика: вы сами пишете текст запроса, тогда как `Kora` берет на себя только связывание параметров, +выполнение запроса и отображение результата. +Общие правила для отображений, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch` и макросов описаны в разделе +[Общие правила работы с базами данных](database-common.md#macros). + +**Связывание параметров:** Kora выполняет типизированное внедрение аргументов в CQL-запрос на этапе компиляции. +Параметры запроса (например, `:id`, `:entity.field1`) заменяются в сгенерированном коде на соответствующие вызовы драйвера Cassandra. +Например, для параметра `String id` будет сгенерировано что-то вроде `statement.setString(1, id)`, где индекс соответствует порядку параметра в запросе. +Это обеспечивает безопасность (защита от CQL-инъекций) и производительность (использование подготовленных запросов драйвером). + +В отличие от реляционных баз данных, в `Cassandra` нет транзакций. +Когда нужно, чтобы несколько операторов применились атомарно, используйте `@Batch`-метод (`CQL` `BATCH`), как показано выше; +его семантика и макросы описаны в разделе [общих правил работы с базами данных](database-common.md). + ### Профиль { #profile } -Можно переопределять общие настройки, частными настройками из профиля, предположим есть такая конфигурация профиля `someProfile`: +Можно переопределить общие настройки частными настройками из профиля. Предположим, есть такая конфигурация профиля `someProfile`: ===! ":material-code-json: `Hocon`" @@ -531,7 +767,7 @@ agent: timeout: "10s" ``` -Применить настройки из профиля `someProfile`, достаточно сделать следующее: +Чтобы применить настройки из профиля `someProfile`, достаточно сделать следующее: ===! ":fontawesome-brands-java: `Java`" @@ -558,24 +794,55 @@ agent: } ``` -Настройки, указанные в профиле, будут применяться к каждому запросу, конкретно в этом случае — будет установлен таймаут в 10с. +Настройки, указанные в профиле, будут применяться к каждому запросу, в частности в данном случае будет установлен таймаут 10s. +Профиль применяется только к методу, помеченному `@CassandraProfile`; остальные методы репозитория продолжают использовать базовую конфигурацию. + +## Отображение { #mapping } + +Можно переопределить отображение различных частей [отображения](database-common.md) и параметров запроса — `Kora` предоставляет для этого специальные интерфейсы. +Из коробки `CassandraModule` предоставляет отображатели для распространенных типов: `String`, числовых типов, `Boolean`, `BigDecimal`, `BigInteger`, `UUID`, `ByteBuffer`, `LocalDate`, `LocalTime`, `LocalDateTime`, `ZonedDateTime` и `Instant`. +Если тип не входит в этот набор или ему нужно особое представление в `CQL`, добавьте собственный отображатель через `@Mapping`. + +### Отображение { #view } + +Используйте аннотацию `@EntityCassandra` для оптимального отображения. +Аннотация позволяет обработчику аннотаций сгенерировать все необходимые отображатели за **один раунд** аннотационной обработки. +Без этой аннотации отображатели генерируются по требованию, что может потребовать **множества раундов** обработки и значительно увеличить время компиляции. +Это рекомендуемый способ отображения каждого типа, возвращаемого из репозитория или связываемого в нем. -## Конвертация { #mapping } +Ожидается, что все вложенные отображения и типы [UDT](#udt) также используют эту аннотацию. -Возможно переопределять преобразование различных частей [сущности](database-common.md) и параметров запроса, для этого Kora предоставляет специальные интерфейсы. +===! ":fontawesome-brands-java: `Java`" + + ```java + @EntityCassandra + public record Entity(String id, String name) {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @EntityCassandra + data class Entity(val id: String, val name: String) + ``` ### Результат { #result } -Если требуется преобразовать результат вручную, предлагается использовать `CassandraResultSetMapper`: +Если нужно вручную преобразовать весь результат синхронного запроса, используйте `CassandraResultSetMapper`. +Он получает `ResultSet` и возвращает значение метода репозитория: одиночный объект, список, `Optional` или другой поддерживаемый тип. ===! ":fontawesome-brands-java: `Java`" ```java - final class ResultMapper implements CassandraResultSetMapper { + final class ResultMapper implements CassandraResultSetMapper> { @Override - public UUID apply(ResultSet rows) { - // код преобразования + public List apply(ResultSet rows) { + var result = new ArrayList(); + for (var row : rows) { + result.add(row.getUuid("id")); + } + return result; } } @@ -590,12 +857,12 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. + В Kotlin отображатели нужно писать только для типов `T?`, поэтому в интерфейсах тип указывается как `@Nullable`. ```kotlin - class ResultMapper : CassandraResultSetMapper { - override fun apply(rows: ResultSet): UUID { - // код преобразования + class ResultMapper : CassandraResultSetMapper> { + override fun apply(rows: ResultSet): List { + return rows.map { it.getUuid("id") } } } @@ -608,9 +875,17 @@ agent: } ``` +Каждый интерфейс отображателя результата также предоставляет статические фабричные методы, которые строят полный отображатель результата из `CassandraRowMapper`, +так что один отображатель строки можно переиспользовать в разных сигнатурах: + +- `CassandraResultSetMapper` — `singleResultSetMapper`, `optionalResultSetMapper`, `listResultSetMapper`; +- `CassandraAsyncResultSetMapper` — `one`, `list` (автоматически проходит по страницам результата); +- `CassandraReactiveResultSetMapper` — `flux`, `mono`, `monoVoid`, `monoList`. + ### Строка { #row } -Если требуется преобразовать строку вручную, предлагается использовать `CassandraRowMapper`: +Если нужно вручную преобразовать одну строку результата, используйте `CassandraRowMapper`. +Этот отображатель применяется к каждой строке и подходит для возвращаемых значений вида `T`, `Optional`, `List`, `Flux` и `Flow`. ===! ":fontawesome-brands-java: `Java`" @@ -619,7 +894,7 @@ agent: @Override public UUID apply(Row row) { - return UUID.fromString(rs.getString(0)); + return UUID.fromString(row.getString(0)); } } @@ -634,13 +909,13 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. + В Kotlin отображатели нужно писать только для типов `T?`, поэтому в интерфейсах тип указывается как `@Nullable`. ```kotlin class RowMapper : CassandraRowMapper { override fun apply(row: Row): UUID { - return UUID.fromString(rs.getString(0)) + return UUID.fromString(row.getString(0)) } } @@ -653,9 +928,9 @@ agent: } ``` -### Колонка { #column } +### Столбец { #column } -Если требуется преобразовать значение колонки вручную, предлагается использовать `CassandraRowColumnMapper`: +Если нужно вручную преобразовать значение столбца, предлагается использовать `CassandraRowColumnMapper`: ===! ":fontawesome-brands-java: `Java`" @@ -681,7 +956,7 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. + В Kotlin отображатели нужно писать только для типов `T?`, поэтому в интерфейсах тип указывается как `@Nullable`. ```kotlin class ColumnMapper : CassandraRowColumnMapper { @@ -707,7 +982,8 @@ agent: ### Параметр { #parameter } -Если требуется преобразовать значение параметра запроса вручную, предлагается использовать `CassandraParameterColumnMapper`: +Если нужно вручную преобразовать значение параметра запроса, используйте `CassandraParameterColumnMapper`. +Он получает `SettableByName`, индекс параметра и значение из метода репозитория. ===! ":fontawesome-brands-java: `Java`" @@ -715,7 +991,7 @@ agent: public final class ParameterMapper implements CassandraParameterColumnMapper { @Override - public void set(SettableByName stmt, int index, @Nullable UUID value) { + public void apply(SettableByName stmt, int index, @Nullable UUID value) { if (value != null) { stmt.setString(index, value.toString()); } @@ -732,12 +1008,12 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. + В Kotlin отображатели нужно писать только для типов `T?`, поэтому в интерфейсах тип указывается как `@Nullable`. ```kotlin class ParameterMapper : CassandraParameterColumnMapper { - override fun set(stmt: SettableByName<*>, index: Int, value: UUID?) { + override fun apply(stmt: SettableByName<*>, index: Int, value: UUID?) { if (value != null) { stmt.setString(index, value.toString()) } @@ -752,17 +1028,19 @@ agent: } ``` -### Асинхронно { #async } +### Асинхронный { #async } -Из-за особенностей вспомогательного класса для извлечения данных из `AsyncResultSet` для асинхронных запросов (Mono или Suspend), можно использовать только `CassandraReactiveResultSetMapper`: +Для `CompletionStage` и `CompletableFuture` используйте `CassandraAsyncResultSetMapper`, который получает `AsyncResultSet` и возвращает `CompletionStage`. +Его метод `list` автоматически запрашивает последующие страницы результата, поэтому результат `List` собирает все страницы перед завершением. +Для реактивных типов `Mono` / `Flux` используйте `CassandraReactiveResultSetMapper`, который получает `ReactiveResultSet` и возвращает нужный `Publisher`. ===! ":fontawesome-brands-java: `Java`" ```java - final class AsyncResultMapper implements CassandraReactiveResultSetMapper> { + final class ReactiveResultMapper implements CassandraReactiveResultSetMapper> { @Override - public UUID apply(ResultSet rows) { + public Flux apply(ReactiveResultSet rows) { return Flux.from(rows).map(r -> UUID.fromString(r.getString(0))); } } @@ -770,7 +1048,7 @@ agent: @Repository public interface EntityRepository extends CassandraRepository { - @Mapping(AsyncResultMapper.class) + @Mapping(ReactiveResultMapper.class) @Query("SELECT id FROM entities") Flux getIds(); } @@ -779,7 +1057,7 @@ agent: === ":simple-kotlin: `Kotlin`" ```kotlin - class AsyncResultMapper : CassandraReactiveResultSetMapper> { + class ReactiveResultMapper : CassandraReactiveResultSetMapper> { override fun apply(rows: ReactiveResultSet): Flux { return Flux.from(rows).map { r -> UUID.fromString(r.getString(0)) } } @@ -788,58 +1066,216 @@ agent: @Repository interface EntityRepository : CassandraRepository { - @Mapping(AsyncResultMapper::class) + @Mapping(ReactiveResultMapper::class) @Query("SELECT id FROM entities") fun getIds(): Flux } ``` +## Ручной запрос { #query } + +Если запрос сложно выразить одной статической `@Query`, вы можете объявить обычный метод с реализацией и построить `CQL` вручную. +Репозиторий предоставляет `getCassandraConnectionFactory()`, а `CassandraConnectionFactory#query` выполняет такой запрос: +он подготавливает оператор через текущую `CqlSession`, оборачивает выполнение в телеметрию `Kora` и возвращает значение, полученное из колбэка. +Метод доступа `currentSession()` возвращает активную `CqlSession`, а `telemetry()` возвращает `DataBaseTelemetry`, используемую для отчетности. + +`QueryContext` содержит идентификатор запроса и итоговый `CQL`. +Идентификатор передается в телеметрию, поэтому используйте стабильное имя, например `Repository.method`. +Связывайте значения через `BoundStatement`, полученный из подготовленного оператора; никогда не конкатенируйте значения напрямую в строку запроса. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends CassandraRepository { + + default List findByFilter(@Nullable String value2) { + var sql = new StringBuilder("SELECT id, value1, value2, value3 FROM entities"); + if (value2 != null) { + sql.append(" WHERE value2 = ? ALLOW FILTERING"); + } + + var connectionFactory = getCassandraConnectionFactory(); + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return connectionFactory.query(queryContext, statement -> { + var boundStatement = (value2 != null) + ? statement.bind(value2) + : statement.bind(); + var resultSet = connectionFactory.currentSession().execute(boundStatement); + + var result = new ArrayList(); + for (var row : resultSet) { + result.add(new Entity( + row.getString("id"), + row.getInt("value1"), + row.getString("value2"), + row.getString("value3"))); + } + return result; + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : CassandraRepository { + + fun findByFilter(value2: String?): List { + val sql = StringBuilder("SELECT id, value1, value2, value3 FROM entities") + if (value2 != null) { + sql.append(" WHERE value2 = ? ALLOW FILTERING") + } + + val connectionFactory = cassandraConnectionFactory + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return connectionFactory.query(queryContext) { statement -> + val boundStatement = if (value2 != null) statement.bind(value2) else statement.bind() + val resultSet = connectionFactory.currentSession().execute(boundStatement) + + resultSet.map { row -> + Entity( + row.getString("id"), + row.getInt("value1"), + row.getString("value2"), + row.getString("value3") + ) + } + } + } + } + ``` + +Поскольку в `Cassandra` нет транзакций, `query` просто выполняется на текущей сессии с телеметрией; здесь нет фиксации или отката, которыми нужно управлять. + ## UDT { #udt } -Есть поддержка [UDT](https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useCreateUDT.html) -типов с помощью `@UDT` аннотации: +Поддерживаются типы [UDT](https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useCreateUDT.html) через аннотацию `@UDT`. +`UDT` описывает пользовательский тип Cassandra и может использоваться как поле обычного отображения. +Тип `@UDT` отображается как любое другое отображение, поэтому охватывающее отображение помечается `@EntityCassandra`. + +Для следующей схемы, где `username` — пользовательский тип, хранящийся в столбце `FROZEN`: + +```cql +CREATE TYPE IF NOT EXISTS username(first text, last text); + +CREATE TABLE IF NOT EXISTS entities_udt +( + id VARCHAR, + name FROZEN, + PRIMARY KEY (id) +); +``` + +отображение и репозиторий выглядят так: ===! ":fontawesome-brands-java: `Java`" ```java - @Table("entities") - public record Entity(String id, Name name) { + @Repository + public interface EntityRepository extends CassandraRepository { + + @EntityCassandra + record Entity(String id, Name name) { + + @UDT + record Name(String first, String last) {} + } - @UDT - public record Name(String first, String middle, String last) { } + @Query("SELECT * FROM entities_udt WHERE id = :id") + @Nullable + Entity findById(String id); + + @Query(""" + INSERT INTO entities_udt(id, name) + VALUES (:entity.id, :entity.name) + """) + void insert(Entity entity); } ``` === ":simple-kotlin: `Kotlin`" ```kotlin - @Table("entities") - data class Entity(val id: String, val name: Name) { + @Repository + interface EntityRepository : CassandraRepository { + + @EntityCassandra + data class Entity(val id: String, val name: Name) { + + @UDT + data class Name(val first: String, val last: String) + } - @UDT - data class Name(val first: String, val middle: String, val last: String) + @Query("SELECT * FROM entities_udt WHERE id = :id") + fun findById(id: String): Entity? + + @Query(""" + INSERT INTO entities_udt(id, name) + VALUES (:entity.id, :entity.name) + """) + fun insert(entity: Entity) } ``` +Если тип `UDT` используется не через охватывающее отображение, а как самостоятельный тип Cassandra, генерацию отображателя можно включить явно с помощью `@EntityCassandra`. +Это полезно, когда отображатель нужен как отдельный компонент графа. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @EntityCassandra + public record Name(String first, String last) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @EntityCassandra + data class Name(val first: String, val last: String) + ``` + +### Макросы { #macros } + +Для упрощения написания `CQL`-запросов используйте макросы — они разворачиваются в `CQL`-конструкции на этапе компиляции. +Примеры использования показаны выше в секции [Использование](#usage) (методы `findById` и `insert`). + +**Подробная документация:** [Общие правила работы с базами данных — Макросы](database-common.md#macros) + ## Сигнатуры { #signatures } -Доступные сигнатуры для методов репозитория из коробки: +Доступные из коробки сигнатуры методов репозитория: ===! ":fontawesome-brands-java: `Java`" - Под `T` подразумевается тип возвращаемого значения, либо `List`, либо `Void`. + `T` означает тип возвращаемого значения, либо `List`, либо `Void`. - `T myMethod()` - `@Nullable T myMethod()` - `Optional myMethod()` - - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) (надо предоставить `Executor`) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) + - `CompletableFuture myMethod()` [CompletableFuture](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (требует [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (требует [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + + Обертки `CompletionStage`, `CompletableFuture` и `Mono` также могут оборачивать `List`, + например `CompletionStage>` или `Mono>`. + + Параметры метода могут включать обычные значения, DTO, `@Batch List` для пакетного выполнения и `CqlSession`, когда методу нужен доступ к текущей сессии драйвера. === ":simple-kotlin: `Kotlin`" - Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `List`, либо `Unit`. + `T` означает тип возвращаемого значения, либо `T?`, либо `List`, либо `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требует [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требует [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + + Параметры метода могут включать обычные значения, DTO, `@Batch List` для пакетного выполнения и `CqlSession`, когда методу нужен доступ к текущей сессии драйвера. + +## Телеметрия { #telemetry } + +Логирование, метрики и трассировка настраиваются через блок `telemetry` в [конфигурации](#configuration) и описаны в разделе [Справочник метрик](metrics.md#database). +Чтобы переопределить телеметрию полностью, можно предоставить собственные SPI-фабрики, подробнее в [Общей документации по Базам данных](database-common.md#telemetry). diff --git a/mkdocs/docs/ru/documentation/database-common.md b/mkdocs/docs/ru/documentation/database-common.md index 02eacb0..5e8f0af 100644 --- a/mkdocs/docs/ru/documentation/database-common.md +++ b/mkdocs/docs/ru/documentation/database-common.md @@ -4,30 +4,130 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Common Kora database model and repository conventions: entities, identifiers, naming, embedded fields, query macros, batch queries, and repository inheritance; key triggers include @Table, @Column, @Id, @Embedded, @Repository, @Query, @Batch, @Mapping, Entity, Repository." --- -Основные принципы и механизмы работы модулей баз данных в Kora. +Базовые принципы и механизмы работы модулей баз данных в Kora. +В этом разделе описана общая модель для `JDBC`, `Cassandra`, `R2DBC` и `Vertx`: отображения, репозитории, параметры запросов, пакетные запросы, количество затронутых строк и макросы. +Конфигурация подключения, транзакции, поддерживаемые сигнатуры и специфичные для драйвера отображатели описаны в документации для каждой реализации базы данных. -Мы придерживаемся концепции, что самый лучший способ общения с базой данных SQL, это общение на ее родном языке SQL. -Другие инструменты зачастую имеют ограничения на использования специфичных функций определенной базы данных, -либо сложный программный язык построения запросов который требует дополнительное и значительное время на изучение и освоение, -несет в себе кучу не явностей и потенциальных ошибок со стороны разработчика, а также порой имеет низкую производительность. +Этот раздел намеренно не описывает специфичные для драйвера детали. +Конфигурацию подключения, транзакции, типы возвращаемых значений, генерируемые базой данных идентификаторы, служебные параметры методов +и точные интерфейсы отображателей смотрите в документации для нужной реализации: +[`JDBC`](database-jdbc.md), [`Cassandra`](database-cassandra.md), [`R2DBC`](database-r2dbc.md) или [`Vertx`](database-vertx.md). -Если нужен пошаговый разбор перед справочным описанием, смотрите [База данных JDBC](../guides/database-jdbc.md) и [База данных JDBC продвинутая](../guides/database-jdbc-advanced.md). +Мы считаем, что лучший способ общения с базой данных SQL — это общение на её родном языке SQL. +Другие инструменты часто имеют ограничения на использование специфичных функций конкретной базы данных +или сложный программный язык для построения запросов, который требует дополнительного и значительного времени на изучение и освоение, +несёт много неочевидности и потенциальных ошибок со стороны разработчика, а также порой обладает низкой производительностью. -## Сущность { #entity } +Если нужен пошаговый разбор перед справочным описанием, смотрите [База данных JDBC](../guides/database-jdbc.md) и [Продвинутая база данных JDBC](../guides/database-jdbc-advanced.md). -Сущность - представления данных из базы данных в виде класса с полями. +## Использование { #usage } -Сущности используемые в качестве возвращаемого значения, должны содержать один публичный -конструктор. Это может быть как конструктор по умолчанию, так и конструктор с параметрами. -Если Kora найдет конструктор с параметрами, то на его основе будет создаваться объект сущности. -В случае же с пустым конструктором поля будут заполняться [через сеттеры](https://docs.oracle.com/cd/E19316-01/819-3669/bnais/index.html). +Использование показано на примере [`JDBC`](database-jdbc.md) модуля, сначала репозиторий объявляется как интерфейс, помеченный аннотацией `@Repository`, и должен наследовать `JdbcRepository`. +Каждый метод, помеченный `@Query`, содержит обычный `SQL`-запрос. Параметры метода связываются по имени с помощью +синтаксиса `:parameter`, а к полям объекта можно обращаться как `:entity.field`. + +Отображения описываются с помощью общих аннотаций отображений и помечаются `@EntityJdbc`, +чтобы `Kora` сгенерировала отображатель на этапе компиляции оптимальнее: ===! ":fontawesome-brands-java: `Java`" ```java - public record Entity(String id, String name) {} + @Repository + public interface EntityRepository extends JdbcRepository { + + @EntityJdbc + @Table("entities") + record Entity(@Id long id, + String name, + @Nullable String description) {} + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + @Nullable + Entity findById(long id); + + @Query("SELECT id, name, description FROM entities") //(2)! + List findAll(); + + @Query("INSERT INTO %{entity#inserts}") //(3)! + UpdateCount insert(Entity entity); + } ``` + 1. Использует макрос `%{return#selects}` и `%{return#table}`. Разворачивается в запрос: + ```sql + SELECT id, name, description + FROM entities + WHERE id = :id + ``` + Метод использует макросы для `SELECT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + 2. Поля перечислены вручную без использования макросов — это допустимо, но требует поддержки при изменении отображения. + 3. Использует макрос `%{entity#inserts}`. Разворачивается в запрос: + ```sql + INSERT INTO entities(id, name, description) + VALUES(:entity.id, :entity.name, :entity.description) + ``` + Метод использует макросы для `INSERT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + @EntityJdbc + @Table("entities") + data class Entity( + @field:Id val id: Long, + val name: String, + val description: String? + ) + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + fun findById(id: Long): Entity? + + @Query("INSERT INTO %{entity#inserts}") //(3)! + fun insert(entity: Entity): UpdateCount + } + ``` + + 1. Использует макрос `%{return#selects}` и `%{return#table}`. Разворачивается в запрос: + ```sql + SELECT id, name, description + FROM entities + WHERE id = :id + ``` + Метод использует макросы для `SELECT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + 3. Использует макрос `%{entity#inserts}`. Разворачивается в запрос: + ```sql + INSERT INTO entities(id, name, description) + VALUES(:entity.id, :entity.name, :entity.description) + ``` + Метод использует макросы для `INSERT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + +`SQL` остается под контролем разработчика: вы можете использовать специфичные для базы данных возможности, тогда как `Kora` +берет на себя только безопасное связывание параметров, выполнение запроса и отображение результата. +Общие правила для отображений, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch` или макросов описаны в разделе +[макросы](#macros). + +**Связывание параметров:** Kora выполняет типизированное внедрение аргументов в SQL-запрос на этапе компиляции. +Параметры запроса (например, `:id`, `:entity.name`) заменяются в сгенерированном коде на соответствующие вызовы `PreparedStatement`. +Например, для параметра `String name` будет сгенерировано что-то вроде `statement.setString(1, name)`, где индекс соответствует порядку параметра в запросе. +Это обеспечивает безопасность (защита от SQL-инъекций) и производительность (использование подготовленных запросов). + +## Отображение { #view } + +Отображение — это представление данных из базы данных в виде класса с полями. + +Отображения, используемые как возвращаемое значение, должны содержать единственный публичный +конструктор. Это может быть либо конструктор по умолчанию, либо конструктор с параметрами. +Если Kora находит конструктор с параметрами, объект отображения создаётся на его основе. +В случае пустого конструктора поля заполняются [через сеттеры](https://docs.oracle.com/cd/E19316-01/819-3669/bnais/index.html). + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record Entity(String id, String name) {} + ``` === ":simple-kotlin: `Kotlin`" ```kotlin @@ -36,9 +136,9 @@ agent: ### Таблица { #table } -Можно указывать к какой таблице принадлежит сущность, это понадобится в случае использования [макросов]() при построении запросов. +Вы можете указать, к какой таблице относится отображение — это понадобится, если вы используете [макросы](#macros) при построении запросов. -В случае если таблица не указана, макросы будут использовать имя класса в [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) +Если таблица не указана, макросы используют имя класса в [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). ===! ":fontawesome-brands-java: `Java`" @@ -56,10 +156,10 @@ agent: ### Идентификатор { #identifier } -Так как все манипуляции с данными происходят посредствам преобразования сущности в запрос к драйверу, -то нет надобности выделять специально первичный ключ в рамках сущности для работы с сущностью. +Поскольку все манипуляции с данными выполняются путём преобразования отображения в запрос драйвера, +нет необходимости выделять внутри отображения специальный первичный ключ для работы с ней. -Обозначать что именно является первичным ключом, может пригодиться в рамках использования [макросов](#manual-query), +Определение того, что именно является первичным ключом, может быть полезно при использовании [макросов](#macros), для этого можно использовать аннотацию `@Id`. ===! ":fontawesome-brands-java: `Java`" @@ -76,23 +176,23 @@ agent: #### Последовательный { #sequential } -Рассмотрим создание идентификатора как последовательность чисел на примере Postgres, +Рассмотрим создание идентификатора в виде последовательности чисел на примере Postgres: Kora предлагает использовать механизм базы данных [identity column](https://www.tutorialsteacher.com/postgresql/identity-column). -Пример таблицы для такой сущности будет выглядеть так: +Пример таблицы для такого отображения выглядел бы так: ```sql CREATE TABLE IF NOT EXISTS entities ( - id BIGINT GENERATED ALWAYS AS IDENTITY, + id BIGINT GENERATED ALWAYS AS IDENTITY, name VARCHAR NOT NULL, PRIMARY KEY (id) ); ``` -Создаваться идентификатор будет на этапе вставки в базу данных, -а получать его в коде приложения подразумевается с помощью конструкции [возвращаемого значения идентификатора для JDBC или R2DBC](database-jdbc.md#generated-identifier) при вставке -либо использовать [специальные конструкции](https://postgrespro.ru/docs/postgresql/9.5/dml-returning) вашей базы данных: +Идентификатор будет создан на этапе вставки в базу данных, +а его получение в коде приложения предполагается выполнять с помощью [возврата значения идентификатора для JDBC или R2DBC](database-jdbc.md#generated-identifier) при вставке +либо использовать [специальные конструкции](https://www.postgresql.org/docs/current/dml-returning.html) вашей базы данных: ===! ":fontawesome-brands-java: `Java`" @@ -127,27 +227,75 @@ CREATE TABLE IF NOT EXISTS entities } ``` +Вместо специфичного для драйвера `RETURNING` первичный ключ, сгенерированный базой данных при вставке, можно вернуть, +пометив сам **метод** репозитория аннотацией `@Id` (аннотация применима как к полю отображения, так и к методу). +Точное поведение генерируемого идентификатора и поддерживаемые сигнатуры возвращаемого значения специфичны для драйвера и описаны +для [JDBC](database-jdbc.md#generated-identifier) и [R2DBC](database-r2dbc.md#generated-identifier): + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends JdbcRepository { + + @Table("entities") + public record Entity(@Id Long id, String name) {} + + @Id //(1)! + @Query("INSERT INTO %{entity#inserts -= id}") //(2)! + long insert(Entity entity); + } + ``` + + 1. Помечает метод так, чтобы возвращался идентификатор, сгенерированный базой данных. + 2. Разворачивается в запрос: + ```sql + INSERT INTO entities(name) VALUES(:entity.name) + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + @Table("entities") + data class Entity(@field:Id val id: Long, val name: String) + + @Id //(1)! + @Query("INSERT INTO %{entity#inserts -= id}") //(2)! + fun insert(entity: Entity): Long + } + ``` + + 1. Помечает метод так, чтобы возвращался идентификатор, сгенерированный базой данных. + 2. Разворачивается в запрос: + ```sql + INSERT INTO entities(name) VALUES(:entity.name) + ``` + #### Случайный { #random } Для создания случайного идентификатора предлагается использовать стандартный `UUID` из Java: -Пример таблицы для такой сущности будет выглядеть так: +Пример таблицы для такого отображения выглядел бы так: ```sql CREATE TABLE IF NOT EXISTS entities ( - id UUID NOT NULL, + id UUID NOT NULL, name VARCHAR NOT NULL, PRIMARY KEY (id) ); ``` -Создаваться идентификатор будет на этапе создания объекта в пользовательском коде приложения: +Идентификатор будет создан на этапе создания объекта в пользовательском коде приложения: ===! ":fontawesome-brands-java: `Java`" ```java - public record Entity(UUID id, String name) {} + public record Entity(UUID id, + String name) {} @Repository public interface EntityRepository extends JdbcRepository { @@ -164,7 +312,8 @@ CREATE TABLE IF NOT EXISTS entities === ":simple-kotlin: `Kotlin`" ```kotlin - data class Entity(val id: UUID, val name: String) + data class Entity(val id: UUID, + val name: String) @Repository interface EntityRepository : JdbcRepository { @@ -177,22 +326,20 @@ CREATE TABLE IF NOT EXISTS entities } ``` -#### Композитный { #composite } +#### Составной { #composite } -В случае если требуется использовать композитный ключ, -предполагается использовать аннотацию `@Embedded` для создания [вложенных полей](#embedded-fields). +Когда требуется составной ключ, для создания [встроенных полей](#embedded-fields) предполагается использовать аннотацию `@Embedded`. ### Именование { #naming } -По умолчанию имена полей сущностей переводятся в [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) при извлечении -результата. +По умолчанию имена полей отображения при получении результата преобразуются в [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). -Если требуется настроить сопоставление конкретных полей из базы данных с сущностью, то можно использовать аннотацию `@Column`: +Если вы хотите настроить отображение конкретных полей из базы данных на отображение, можно использовать аннотацию `@Column`: ===! ":fontawesome-brands-java: `Java`" ```java - public record Entity(@Column("ID") String id, + public record Entity(@Column("ID") String id, @Column("NAME") String name) {} ``` @@ -205,22 +352,22 @@ CREATE TABLE IF NOT EXISTS entities #### Стратегия именования { #naming-strategy } -Если требуется использовать стратегию именования для всей сущности, то предлагается создать реализацию `NameConverter` и затем использовать ее в аннотации `@NamingStrategy`. -Требуется чтобы реализация `NameConverter` имела конструктор без параметров. +Если вы хотите использовать стратегию именования для всего отображения, предлагается создать реализацию `NameConverter`, а затем использовать её в аннотации `@NamingStrategy`. +Требуется, чтобы реализация `NameConverter` имела конструктор без параметров. -Либо использовать доступные стратегии из Kora: +Либо используйте доступные стратегии из Kora: -- `NoopNameConverter` - стратегия использует имя поля по умолчанию. -- `SnakeCaseNameConverter` - стратегия использует [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). -- `SnakeCaseUpperNameConverter` - стратегия использует [SNAKE_UPPER_CASE](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). -- `PascalCaseNameConverter` - стратегия использует [PascalCase](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). -- `CamelCaseNameConverter` - стратегия использует [camelCase](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). +- `NoopNameConverter` — стратегия использует имя поля по умолчанию. +- `SnakeCaseNameConverter` — стратегия использует [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). +- `SnakeCaseUpperNameConverter` — стратегия использует [SNAKE_UPPER_CASE](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). +- `PascalCaseNameConverter` — стратегия использует [PascalCase](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). +- `CamelCaseNameConverter` — стратегия использует [camelCase](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/). ===! ":fontawesome-brands-java: `Java`" ```java @NamingStrategy(NoopNameConverter.class) - public record Entity(String id, + public record Entity(String id, String name) {} ``` @@ -236,7 +383,7 @@ CREATE TABLE IF NOT EXISTS entities ===! ":fontawesome-brands-java: `Java`" - По умолчанию все поля объявленные в сущности считаются **обязательными** (*NotNull*). + По умолчанию все поля, объявленные в отображении, считаются **обязательными** (*NotNull*). ```java public record Entity(String id, @@ -245,7 +392,7 @@ CREATE TABLE IF NOT EXISTS entities === ":simple-kotlin: `Kotlin`" - По умолчанию все поля объявленные в сущности которые не используют [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис считаются **обязательными** (*NotNull*). + По умолчанию все поля, объявленные в отображении и не использующие синтаксис [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html), считаются **обязательными** (*NotNull*). ```kotlin data class Entity(val id: String, @@ -256,23 +403,23 @@ CREATE TABLE IF NOT EXISTS entities ===! ":fontawesome-brands-java: `Java`" - В случае если поле в сущности является необязательным, то есть может отсутствовать то, - можно использовать аннотацию `@Nullable` для соответствия поля в Json и DTO. + Если поле отображения необязательное, то есть может отсутствовать, + используйте аннотацию `@Nullable`, чтобы явно его пометить. ```java - public record Entity(String id, + public record Entity(String id, @Nullable String name) {} //(1)! ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдёт любая аннотация `@Nullable`, например `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. - Также можно указывать необязательными параметры конструктора в случае если переопределен канонический конструктор у Record: + Также можно указать необязательные параметры конструктора в случае, если канонический конструктор Record переопределён: ```java public record Entity(String id, String name) { - public Entity(String id, + public Entity(String id, @Nullable String name) { //(1)! this.id = id; this.name = name; @@ -280,35 +427,34 @@ CREATE TABLE IF NOT EXISTS entities } ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдёт любая аннотация `@Nullable`, например `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. === ":simple-kotlin: `Kotlin`" - Предполагается использовать [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис и помечать такой параметр как Nullable: + Ожидается использование синтаксиса [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) и пометка такого параметра как Nullable: ```kotlin data class Entity(val id: String, val name: String?) ``` -### Вложенные поля { #embedded-fields } +### Встроенные поля { #embedded-fields } -В случае если требуется использовать вложенные поля, -то есть объединить колонки из таблицы в рамках отдельного класса внутри сущности, можно использовать аннотацию `@Embedded`. +Если вы хотите использовать вложенные поля, то есть преобразовать поля отображения в отдельные классы, можно использовать аннотацию `@Embedded`. -Предположим есть SQL таблица где имеется композитный ключ который мы хотим выразить отдельным классом: +Предположим, есть SQL-таблица, в которой присутствует составной ключ, который мы хотим выразить как отдельный класс: ```sql CREATE TABLE IF NOT EXISTS entities ( - name VARCHAR NOT NULL, - surname VARCHAR NOT NULL, - info VARCHAR NOT NULL, + name VARCHAR NOT NULL, + surname VARCHAR NOT NULL, + info VARCHAR NOT NULL, PRIMARY KEY (name, surname) ) ``` -Тогда сущность будет выглядеть так: +Тогда отображение будет выглядеть так: ===! ":fontawesome-brands-java: `Java`" @@ -325,7 +471,7 @@ CREATE TABLE IF NOT EXISTS entities ```kotlin data class Entity( @field:Id @field:Embedded val id: UserID, - @field:Column("name") val info: String + @field:Column("info") val info: String ) { data class UserID( @@ -335,7 +481,7 @@ CREATE TABLE IF NOT EXISTS entities } ``` -Тогда репозиторий для такой сущности будет выглядеть так: +Тогда репозиторий для такого отображения выглядел бы так: ===! ":fontawesome-brands-java: `Java`" @@ -370,7 +516,7 @@ CREATE TABLE IF NOT EXISTS entities WHERE name = :id.name AND surname = :id.surname; """ ) - fun findById(id: Entity.CompositeID): Entity? + fun findById(id: Entity.UserID): Entity? @Query( """ @@ -382,30 +528,30 @@ CREATE TABLE IF NOT EXISTS entities } ``` -В случае если бы поля имели общий префикс, его можно было бы указать в аннотации `@Embedded("user_")`: +Если поля имеют общий префикс, его можно указать в аннотации `@Embedded("user_")`: ```sql CREATE TABLE IF NOT EXISTS entities ( - user_name VARCHAR NOT NULL, - user_surname VARCHAR NOT NULL, - info VARCHAR NOT NULL, + user_name VARCHAR NOT NULL, + user_surname VARCHAR NOT NULL, + info VARCHAR NOT NULL, PRIMARY KEY (user_name, user_surname) ) ``` ## Репозиторий { #repository } -Главным инструментом для работы с базами данных в Kora является использование [шаблона проектирование репозиторий](https://gist.github.com/maestrow/594fd9aee859c809b043) при проектировании абстракции доступа к базе данных. -Интерфейс репозитория должен быть проаннотирован `@Repository`. -Запросы для методов репозиториев описываются с помощью `@Query` аннотации. -На этапе компиляции создается реализация, где каждый метод будет выполнять описанный запрос и эффективно производить сборку аргументов запроса и обработку результата. +Основной инструмент для работы с базами данных в Kora — использование [паттерна репозиторий](https://java-design-patterns.com/patterns/repository/#explanation) при проектировании абстракции доступа к базе данных. +Интерфейс репозитория должен быть помечен аннотацией `@Repository`. +Запросы для методов репозитория описываются с помощью аннотации `@Query`. +Реализация репозитория создаётся во время компиляции, все методы `@Query` будут выполнять описанный запрос, оптимально собирать аргументы запроса и обрабатывать результат. -Предполагается написание SQL запросов разработчиком, поскольку это повышает ответственность разработчика за план запроса, -дает больше понимание и контекста разработчику о том что он делает и как его запрос будет работать. -Для улучшения пользовательского опыта с перечислениями моделей можно использовать [макросы](#multiple-databases). +Предполагается, что `SQL`-запросы пишет разработчик, поскольку это повышает понимание разработчиком плана запроса, +даёт больше представления и контекста о том, что делает запрос и как он будет работать. +Вы можете использовать [макросы](#macros) для улучшения удобства работы, чтобы не писать все поля/столбцы модели. -Репозиторий должен являться наследником одной из реализаций, в примерах ниже рассматривается реализация [JDBC](database-jdbc.md) +Репозиторий должен наследовать одну из реализаций, в примерах ниже будет рассмотрена реализация [JDBC](database-jdbc.md): ===! ":fontawesome-brands-java: `Java`" @@ -415,15 +561,15 @@ CREATE TABLE IF NOT EXISTS entities public record Entity(String id, String name) { } - @Query("SELECT id, name FROM entities WHERE id = :id") //(2)! - @Nullable //(3)! + //(2)! + @Query("SELECT id, name FROM entities WHERE id = :id") + @Nullable Entity findById(String id); } ``` 1. Указывает, что интерфейс является репозиторием. - 2. Указывает, что нужно создать реализацию метода, выполняющую SQL запрос указанный в аннотации. - 3. Указывает, что возвращаемое значение может отстутствовать, можно также использовать `Optional` + 2. Указывает, что Kora должна создать реализацию метода, выполняющую `SQL`-запрос, указанный в аннотации. === ":simple-kotlin: `Kotlin`" @@ -433,23 +579,105 @@ CREATE TABLE IF NOT EXISTS entities data class Entity(val id: String, val name: String) - @Query("SELECT id, name FROM entities WHERE id = :id") //(2)! + //(2)! + @Query("SELECT id, name FROM entities WHERE id = :id") fun findById(id: String): Entity? } ``` 1. Указывает, что интерфейс является репозиторием. - 2. Указывает, что нужно создать реализацию метода, выполняющую SQL запрос указанный в аннотации. + 2. Указывает, что Kora должна создать реализацию метода, выполняющую `SQL`-запрос, указанный в аннотации. -### Пакетный запрос { #batch-query } +### Параметры запроса { #query-parameters } + +Параметры метода репозитория связываются с именованными параметрами в `@Query`. +Простой параметр указывается по имени параметра метода: `:id`, `:name`, `:status`. +Если параметр является отображением или `DTO`, к его полям можно обращаться через точечную нотацию: `:entity.id`, `:entity.name`, `:filter.status`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends JdbcRepository { + + @Query(""" + SELECT id, name FROM entities + WHERE id = :id AND name = :filter.name + """) + @Nullable + Entity findById(String id, Filter filter); + + record Filter(String name) {} + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { -Kora поддерживает пакетные запросы (batch-запросы) с помощью аннотации `@Batch`. + @Query( + """ + SELECT id, name FROM entities + WHERE id = :id AND name = :filter.name + """ + ) + fun findById(id: String, filter: Filter): Entity? -В отличие от последовательного выполнения SQL запросов, пакетная обработка даёт возможность отправить целый набор запросов за один вызов, -уменьшая количество требуемых сетевых подключений и позволяя выполнять какое-то количество запросов параллельно на стороне базы данных, -что может увеличить скорость выполнения. + data class Filter(val name: String) + } + ``` + +Если параметр встречается в запросе более одного раза, Kora связывает его с каждым вхождением. +Если параметр метода не используется в запросе и не является служебным параметром конкретного драйвера, компиляция завершается с ошибкой. + +### Отображатели { #mappers } -Пример использования: +Используйте аннотацию `@Mapping`, когда значению нужно нестандартное представление в базе данных. +Её можно разместить на поле отображения, параметре метода или методе репозитория: + +- на поле отображения — чтобы настроить чтение или запись конкретного столбца; +- на параметре метода — чтобы настроить запись конкретного параметра запроса; +- на методе репозитория — чтобы настроить обработку всего результата запроса или строки результата. + +Произвольный отображатель нельзя использовать в любом месте: его тип должен соответствовать месту применения. +Отображатель параметра применяется к параметру запроса, отображатель столбца — к полю отображения, а отображатель результата или строки — к методу репозитория. +Точный набор поддерживаемых интерфейсов зависит от драйвера: например, `JDBC` использует `JdbcRowMapper`, `JdbcResultSetMapper`, `JdbcResultColumnMapper` и `JdbcParameterColumnMapper`. +Похожие интерфейсы для `Cassandra`, `R2DBC` и `Vertx`, а также детали их использования описаны в документации для каждой реализации базы данных. +Все отображатели строк драйверов имеют общий маркерный интерфейс `RowMapper` (`ru.tinkoff.kora.database.common.RowMapper`), который является базовым типом для специфичных для драйвера отображателей, таких как `JdbcRowMapper` и `CassandraRowMapper`. +Сама аннотация `@Mapping` находится в основном модуле `common` (`ru.tinkoff.kora.common.Mapping`). +Если отображатель указан через `@Mapping`, Kora добавляет его как зависимость сгенерированного репозитория и использует вместо отображателя по умолчанию. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Table("entities") + public record Entity(@Id String id, + @Mapping(JsonParameterMapper.class) + @Column("payload") + String payload) {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Table("entities") + data class Entity( + @field:Id val id: String, + @field:Mapping(JsonParameterMapper::class) + @field:Column("payload") + val payload: String + ) + ``` + +### Пакетный запрос { #batch-query } + +Kora поддерживает пакетные запросы с помощью аннотации `@Batch`. + +В отличие от последовательного выполнения SQL-запросов, пакетная обработка позволяет отправить целый набор запросов за один вызов, +сокращая число требуемых сетевых обращений и позволяя выполнять часть запросов параллельно на стороне базы данных, +что может повысить скорость выполнения. ===! ":fontawesome-brands-java: `Java`" @@ -462,8 +690,8 @@ Kora поддерживает пакетные запросы (batch-запро } ``` - **Пакетный запрос** не может возвращать произвольные значения, такой метод может возвращать `void`, либо `UpdateCount`, - либо созданные базой данных индетификаторы для [JDBC](database-jdbc.md#generated-identifier) или [R2DBC](database-r2dbc.md#generated-identifier) драйверов. + **Пакетный запрос** не может возвращать произвольные значения — такой метод может возвращать `void`, либо `UpdateCount`, + либо генерируемые базой данных идентификаторы для драйверов [JDBC](database-jdbc.md#generated-identifier) или [R2DBC](database-r2dbc.md#generated-identifier). === ":simple-kotlin: `Kotlin`" @@ -476,13 +704,37 @@ Kora поддерживает пакетные запросы (batch-запро } ``` - **Пакетный запрос** не может возвращать произвольные значения, такой метод может возвращать `Unit`, либо `UpdateCount`, - либо созданные базой данных индетификаторы для [JDBC](database-jdbc.md#generated-identifier) или [R2DBC](database-r2dbc.md#generated-identifier) драйверов. + **Пакетный запрос** не может возвращать произвольные значения — такой метод может возвращать `Unit`, либо `UpdateCount`, + либо генерируемые базой данных идентификаторы для драйверов [JDBC](database-jdbc.md#generated-identifier) или [R2DBC](database-r2dbc.md#generated-identifier). + +`@Batch` ставится на параметр-коллекцию, и каждый элемент коллекции по очереди подставляется в один и тот же запрос. +Все остальные параметры метода, если они есть, являются общими для всех элементов пакета. +Например, в `INSERT INTO logs(tenant_id, id, value) VALUES (:tenantId, :entity.id, :entity.value)` +параметр `tenantId` одинаков для каждого элемента, тогда как поля `entity` берутся из каждого элемента коллекции. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Query("INSERT INTO logs(tenant_id, id, value) VALUES (:tenantId, :entity.id, :entity.value)") + UpdateCount insert(String tenantId, @Batch List entity); + ``` -### Счетчик обновлений { #affected-rows } +=== ":simple-kotlin: `Kotlin`" -Kora не обрабатывает содержимое запроса, результат метода всегда считается производным из строк, которые вернула база данных. -Если необходимо получить в качестве результата количество обновленных строк нужно использовать специальный тип `UpdateCount`. + ```kotlin + @Query("INSERT INTO logs(tenant_id, id, value) VALUES (:tenantId, :entity.id, :entity.value)") + fun insert(tenantId: String, @Batch entity: List): UpdateCount + ``` + +Метод должен иметь не более одного параметра, помеченного `@Batch`. +Поддержка генерируемых базой данных идентификаторов в пакетных запросах зависит от конкретного драйвера и описана в соответствующем разделе. + +### Затронутые строки { #affected-rows } + +Kora не обрабатывает содержимое запроса, результат метода всегда выводится из строк, возвращённых базой данных. +Если вы хотите получить в результате количество затронутых строк, используйте специальный тип `UpdateCount`. +Для обычного запроса `UpdateCount#value()` содержит количество строк, возвращённое драйвером для выполненного запроса. +Для пакетного запроса значение обычно является суммой результатов по всем элементам пакета; точное поведение зависит от драйвера базы данных. ===! ":fontawesome-brands-java: `Java`" @@ -506,13 +758,17 @@ Kora не обрабатывает содержимое запроса, резу } ``` -### Ручное управление { #manual-query } +### Ручной запрос { #manual-query } + +Если по какой-либо причине функциональности запросов в аннотации `@Query` недостаточно или требуется ручное управление соединением, +можно использовать встроенный метод фабрики соединений, чтобы создать метод с полностью ручным управлением. -В случае если не хватает функционала по каким то причинам с запросами в `@Query` аннотации или требуется ручное управление соединением, -можно использовать встроенный метод фабрики соединений для создания метода с полностью ручным управлением. +Внутри такого метода можно также использовать другие методы репозитория, и при необходимости они тоже будут выполняться в рамках одной транзакции. +Подробнее о транзакциях смотрите в документации для конкретной реализации репозитория. -Можно также использовать внутри метода другие методы репозитория и они также будут выполняться в рамках одной транзакции если это требуется. -Детальнее про транзакции стоит смотреть документацию по конкретной реализации репозитория. +Репозитории могут объявлять обычные методы с реализациями. +Это полезно, когда более сложную операцию стоит держать рядом с запросами: например, выполнение нескольких методов `@Query` в одной транзакции, +построение результата из нескольких запросов или хранение последовательности операций с базой данных внутри репозитория вместо переноса её в слой сервиса. ===! ":fontawesome-brands-java: `Java`" @@ -522,15 +778,17 @@ Kora не обрабатывает содержимое запроса, резу public record Entity(Long id, String name) {} - default int insert(Entity entity) { - return getJdbcConnectionFactory().inTx(connection -> { - String sql = "INSERT INTO entities(name) VALUES (?) RETURNING id"; - try(PreparedStatement preparedStatement = connection.prepareStatement(sql)) { - preparedStatement.setString(1, entity.name()); - try(ResultSet resultSet = preparedStatement.executeQuery()) { - return resultSet.getInt(1); - } - } + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + UpdateCount insert(Entity entity); + + @Query("UPDATE entities SET name = :name WHERE id = :id") + UpdateCount updateName(Long id, String name); + + default Entity saveAndRename(Entity entity, String name) { + return getJdbcConnectionFactory().inTx(() -> { + insert(entity); + updateName(entity.id(), name); + return new Entity(entity.id(), name); }); } } @@ -544,27 +802,37 @@ Kora не обрабатывает содержимое запроса, резу data class Entity(val id: Long, val name: String) - fun insert(entity: Entity): Int { - return jdbcConnectionFactory.inTx { connection -> - val sql = "INSERT INTO entities(name) VALUES (?) RETURNING id" - connection.prepareStatement(sql).use { preparedStatement -> - preparedStatement.setString(1, entity.name) - preparedStatement.executeQuery().use { resultSet -> resultSet.getInt(1) } - } + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + fun insert(entity: Entity): UpdateCount + + @Query("UPDATE entities SET name = :name WHERE id = :id") + fun updateName(id: Long, name: String): UpdateCount + + fun saveAndRename(entity: Entity, name: String): Entity { + return jdbcConnectionFactory.inTx { + insert(entity) + updateName(entity.id, name) + Entity(entity.id, name) } } } ``` +Когда вы строите `SQL` вручную через фабрику соединений драйвера, а не с помощью метода `@Query`, +запрос всё равно проходит через телеметрию Kora. Выполняемый запрос описывается общим +`QueryContext(queryId, sql, operation)`: `queryId` — это стабильный идентификатор запроса, передаваемый в телеметрию +(удобно использовать имя вида `Repository.method`), `sql` — итоговый текст запроса, а `operation` по умолчанию равно `db_query`. +Точный метод фабрики соединений и его сигнатура специфичны для драйвера — рабочий пример смотрите в [JDBC](database-jdbc.md#query). + ### Несколько баз данных { #multiple-databases } -Иногда требуется чтобы в рамках одного приложения был доступ к разным базам данных в разных репозиториях, +Иногда в рамках одного приложения нужно обращаться к разным базам данных в разных репозиториях, это можно решить следующим образом. -Требуется создать отдельный экземпляр базы данных и подключить его в репозиторий, -ниже будет показан пример для [JDBC](database-jdbc.md) базы данных, но принцип аналогичный и для других типов подключений. +Нужно создать отдельный экземпляр базы данных и подключить его к репозиторию, +ниже приведён пример для базы данных [JDBC](database-jdbc.md), но принцип аналогичен для других типов соединений. -Требуется скопировать фабрики создания `JdbcDatabase` и его конфигурации из модуля `JdbcDatabaseModule` -и указать им свой тег, который будет указывать что это соединения для другой базы данных. +Требуется скопировать фабрики создания `JdbcDatabase` и его конфигурацию из модуля `JdbcDatabaseModule` +и присвоить им собственный тег, который будет указывать, что это соединения для другой базы данных. ===! ":fontawesome-brands-java: `Java`" @@ -575,7 +843,7 @@ Kora не обрабатывает содержимое запроса, резу final class OtherDatabase { } @Tag(OtherDatabase.class) - default JdbcDatabaseConfig otherJdbcDataBaseConfig(Config config, + default JdbcDatabaseConfig otherJdbcDataBaseConfig(Config config, ConfigValueExtractor extractor) { var value = config.get("db.other"); return extractor.extract(value); @@ -618,7 +886,7 @@ Kora не обрабатывает содержимое запроса, резу } ``` -А в репозиториях, которые будут использовать эту базу данных теперь требуется указывать тег этого подключения: +А репозитории, которые будут использовать эту базу данных, теперь обязаны указывать тег этого соединения: ===! ":fontawesome-brands-java: `Java`" @@ -634,26 +902,26 @@ Kora не обрабатывает содержимое запроса, резу ```kotlin @Repository(executorTag = Tag(value = [OtherDatabase::class])) interface OtherJdbcRepository : JdbcRepository { - + } ``` -Репозитории с подключением к основной базе данных, не требуют тега. +Репозиториям с основным соединением к базе данных тег не требуется. ### Макросы { #macros } -Самой неприятной частью написания SQL запросов может быть перечисление и поддержание в соответствие колонок и полей сущности в актуальном состоянии. +Самой утомительной частью написания SQL-запросов может быть перечисление и поддержание в актуальном состоянии столбцов и полей отображения. -Чтобы решить эту проблему можно использовать специальные макрос конструкции внутри SQL запроса в рамках `@Query` аннотации. -Эти конструкции позволяют оперировать [сущностью](#entity) на которую указывают и раскрывать её в определенные SQL конструкции и легко дополнять SQL запросы. -Макрос является помощником при написании SQL запросов, раскрывается в конструкции которые пользователь смог бы написать собственными руками. +Чтобы решить эту проблему, используйте специальные макросы внутри `SQL`-запроса в аннотации `@Query`. +Эти конструкции оперируют целевым [отображением](#view), разворачивают её в конкретные `SQL`-конструкции и упрощают расширение `SQL`-запросов. +Макрос — это помощник для написания `SQL`-запросов, который разворачивается в конструкции, которые пользователь мог бы написать вручную. -Синтаксис макроса выглядит следующем образом: `%{return#selects}` +Синтаксис макросов выглядит следующим образом: `%{return#selects}`. 1. Макрос ограничен синтаксической конструкцией `%{` и `}` -2. Первым указывается цель макроса, это может быть как имя любого аргумента метода, так и возвращаемое значение с помощью ключевого слова `return` -3. Затем используется `#` символ для разделения цели и команды макроса -4. Затем указывается команда макроса, которая говорит в какую именно SQL конструкцию раскрывать сущность +2. Сначала указывается цель макроса — это может быть имя любого аргумента метода или возвращаемое значение с помощью ключевого слова `return` +3. Затем символ `#` используется для разделения цели макроса и команды макроса +4. После этого указывается команда макроса, которая сообщает, в какую SQL-конструкцию развернуть отображение ===! ":fontawesome-brands-java: `Java`" @@ -662,8 +930,8 @@ Kora не обрабатывает содержимое запроса, резу public interface EntityRepository extends JdbcRepository { @Table("entities") - public record Entity(@Id Long id, - @Column("entity_name") String name, + public record Entity(@Id Long id, + @Column("entity_name") String name, String code) {} @Query("SELECT %{return#selects} FROM %{return#table}") //(1)! @@ -671,7 +939,7 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql SELECT id, entity_name, code FROM entities ``` @@ -683,8 +951,8 @@ Kora не обрабатывает содержимое запроса, резу interface EntityRepository : JdbcRepository { @Table("entities") - data class Entity(@field:Id val id: Long, - @field:Column("entity_name") val name: String, + data class Entity(@field:Id val id: Long, + @field:Column("entity_name") val name: String, val code: String) @Query("SELECT %{return#selects} FROM %{return#table}") //(1)! @@ -692,7 +960,7 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql SELECT id, entity_name, code FROM entities ``` @@ -701,24 +969,24 @@ Kora не обрабатывает содержимое запроса, резу Доступные команды макросов: -- `table` - конструкция раскрывает значение сущности в [аннотации](#table) `@Table` либо если таковая отсутствует то, переводит имя сущности в [snake_lower_case](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) -- `selects` - создает конструкцию перечисления колонок сущности для `SELECT` запроса -- `inserts` - создает конструкцию таблицы, перечисления колонок и соответствующих полей сущности для `INSERT` запроса -- `updates` - создает конструкцию перечисления колонок и соответствующих полей сущности (за исключением `@id` поля) для `UPDATE` запроса -- `where` - создает конструкцию перечисления колонок со значением из сущности для `WHERE` части запроса +- `table` — разворачивает значение отображения из [аннотации](#table) `@Table`, либо, если она отсутствует, преобразует имя отображения в [`snake_lower_case`](https://www.freecodecamp.org/news/snake-case-vs-camel-case-vs-pascal-case-vs-kebab-case-whats-the-difference/) +- `selects` — создаёт конструкцию перечисления столбцов отображения для запроса `SELECT` +- `inserts` — создаёт конструкцию перечисления таблицы, столбцов и соответствующих полей отображения для запроса `INSERT` +- `updates` — создаёт конструкцию перечисления столбцов и соответствующих полей отображения для запроса `UPDATE` +- `where` — создаёт конструкцию перечисления столбцов со значением из отображения для части `WHERE` запроса #### Перечисление полей { #field-enumeration } -Макрос поддерживает дополнительный синтаксис по перечислению определенных полей в команде, -если вдруг требуется сделать частичное обновление или получение данных. -Для этого после команды используется специальная конструкция: `%{return#updates=name}` +Макрос поддерживает дополнительный синтаксис для перечисления определённых полей в команде, +если вдруг нужно выполнить частичное обновление или получение данных. +Для этого после команды используется специальная конструкция: `%{return#updates=name}`. -**Только** между полями перечисления и символом перечисления могут содержаться пробелы. +Пробелы можно ставить **только** между полями в перечислении или специальным символом перечисления. Доступны специальные символы перечисления: -1. `=` - только указанные после символа имя полей сущности будут участвовать в раскрытии команды -2. `-=` - все поля сущности за исключением указанных после символа будут участвовать в раскрытии команды +1. `=` — в разворачивании команды будут участвовать только поля отображения, имена которых указаны после символа +2. `-=` — в разворачивании команды будут участвовать все поля отображения, кроме указанных после символа ===! ":fontawesome-brands-java: `Java`" @@ -727,8 +995,8 @@ Kora не обрабатывает содержимое запроса, резу public interface EntityRepository extends JdbcRepository { @Table("entities") - public record Entity(@Id Long id, - @Column("entity_name") String name, + public record Entity(@Id Long id, + @Column("entity_name") String name, String code) {} @Query("INSERT INTO %{entity#inserts=name,code}") //(1)! @@ -736,9 +1004,9 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - INSERT INTO entities(entity_name, code) + INSERT INTO entities(entity_name, code) VALUES(:entity.name, :entity.code) ``` @@ -749,8 +1017,8 @@ Kora не обрабатывает содержимое запроса, резу interface EntityRepository : JdbcRepository { @Table("entities") - data class Entity(@field:Id val id: Long, - @field:Column("entity_name") val name: String, + data class Entity(@field:Id val id: Long, + @field:Column("entity_name") val name: String, val code: String) @Query("INSERT INTO %{entity#inserts=name,code}") //(1)! @@ -758,18 +1026,18 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - INSERT INTO entities(entity_name, code) + INSERT INTO entities(entity_name, code) VALUES(:entity.name, :entity.code) ``` ##### Идентификатор { #identifier-2 } -При перечислении полей в макросе возможно использовать специальное ключевое слово `@id` -для того чтобы ссылаться сразу идентификатор сущности проаннотированный [аннотацией](#identifier) `@Id`. +При перечислении полей в макросе можно использовать специальное ключевое слово `@id`, +чтобы сразу обратиться к идентификатору отображения, помеченному [аннотацией](#identifier) `@Id`. -Это может быть особенно полезно когда идентификатор является [составным ключом](#optional-fields), для перечисления сразу всех колонок. +Это может быть особенно полезно, когда идентификатор является [составным ключом](#embedded-fields), чтобы перечислить сразу все столбцы. ===! ":fontawesome-brands-java: `Java`" @@ -778,8 +1046,8 @@ Kora не обрабатывает содержимое запроса, резу public interface EntityRepository extends JdbcRepository { @Table("entities") - public record Entity(@Id Long id, - @Column("entity_name") String name, + public record Entity(@Id Long id, + @Column("entity_name") String name, String code) {} @Query("INSERT INTO %{entity#inserts-=@id}") //(1)! @@ -787,9 +1055,9 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - INSERT INTO entities(entity_name, code) + INSERT INTO entities(entity_name, code) VALUES(:entity.name, :entity.code) ``` @@ -800,8 +1068,8 @@ Kora не обрабатывает содержимое запроса, резу interface EntityRepository : JdbcRepository { @Table("entities") - data class Entity(@field:Id val id: Long, - @field:Column("entity_name") val name: String, + data class Entity(@field:Id val id: Long, + @field:Column("entity_name") val name: String, val code: String) @Query("INSERT INTO %{entity#inserts-=@id}") //(1)! @@ -809,15 +1077,15 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - INSERT INTO entities(entity_name, code) + INSERT INTO entities(entity_name, code) VALUES(:entity.name, :entity.code) ``` #### Пример репозитория { #repository-example } -Пример полного репозитория со всеми основными методами для оперирования сущностью для [Postgres SQL](https://postgrespro.ru/docs/postgresql): +Пример полного репозитория со всеми основными методами для работы с отображением для [Postgres SQL](https://postgrespro.com/docs/postgresql): ===! ":fontawesome-brands-java: `Java`" @@ -855,34 +1123,34 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - SELECT id, value1, value2, value3 - FROM entities + SELECT id, value1, value2, value3 + FROM entities WHERE id = :id ``` - 2. Раскрывается в запрос: + 2. Разворачивается в запрос: ```sql - SELECT id, value1, value2, value3 + SELECT id, value1, value2, value3 FROM entities ``` - 3. Раскрывается в запрос: + 3. Разворачивается в запрос: ```sql - INSERT INTO entities(id, value1, value2, value3) - VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.field1, :entity.value2, :entity.value3) ``` - 4. Раскрывается в запрос: + 4. Разворачивается в запрос: ```sql UPDATE entities - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 WHERE id = :entity.id ``` - 5. Раскрывается в запрос: + 5. Разворачивается в запрос: ```sql - INSERT INTO entities(id, value1, value2, value3) - VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) - ON CONFLICT (id) DO UPDATE - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.field1, :entity.value2, :entity.value3) + ON CONFLICT (id) DO UPDATE + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` === ":simple-kotlin: `Kotlin`" @@ -900,7 +1168,7 @@ Kora не обрабатывает содержимое запроса, резу ) @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! - fun findById(id: String): Entity? + fun findById(id: String?): Entity? @Query("SELECT %{return#selects} FROM %{return#table}") //(2)! fun findAll(): List @@ -921,41 +1189,40 @@ Kora не обрабатывает содержимое запроса, резу fun deleteAll(): UpdateCount } ``` - - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - SELECT id, value1, value2, value3 - FROM entities + SELECT id, value1, value2, value3 + FROM entities WHERE id = :id ``` - 2. Раскрывается в запрос: + 2. Разворачивается в запрос: ```sql - SELECT id, value1, value2, value3 + SELECT id, value1, value2, value3 FROM entities ``` - 3. Раскрывается в запрос: + 3. Разворачивается в запрос: ```sql - INSERT INTO entities(id, value1, value2, value3) - VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.field1, :entity.value2, :entity.value3) ``` - 4. Раскрывается в запрос: + 4. Разворачивается в запрос: ```sql UPDATE entities - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 WHERE id = :entity.id ``` - 5. Раскрывается в запрос: + 5. Разворачивается в запрос: ```sql - INSERT INTO entities(id, value1, value2, value3) - VALUES(:entity.id, :entity.value1, :entity.value2, :entity.value3) - ON CONFLICT (id) DO UPDATE - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + INSERT INTO entities(id, value1, value2, value3) + VALUES(:entity.id, :entity.field1, :entity.value2, :entity.value3) + ON CONFLICT (id) DO UPDATE + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` -#### Пример композитного { #composite-example } +#### Пример с составным идентификатором { #composite-example } -Пример репозитория с [композитным идентификатором](#composite) и основными методами для оперирования сущностью, -он почти что полностью идентичен предыдущему за исключением `WHERE` условий при поиске и удалении для [Postgres SQL](https://postgrespro.ru/docs/postgresql): +Пример репозитория с [составным идентификатором](#composite) и основными методами для работы с отображением, +он практически идентичен предыдущему за исключением условий `WHERE` для поиска и удаления для [Postgres SQL](https://postgrespro.com/docs/postgresql): ===! ":fontawesome-brands-java: `Java`" @@ -968,7 +1235,7 @@ Kora не обрабатывает содержимое запроса, резу @Column("value1") int field1, String value2, @Nullable String value3) { - + public record EntityId(String code, String type) { } } @@ -996,34 +1263,34 @@ Kora не обрабатывает содержимое запроса, резу } ``` - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - SELECT code, type, value1, value2, value3 - FROM entities + SELECT code, type, value1, value2, value3 + FROM entities WHERE code = :code AND type = :type ``` - 2. Раскрывается в запрос: + 2. Разворачивается в запрос: ```sql - SELECT code, type, value1, value2, value3 + SELECT code, type, value1, value2, value3 FROM entities ``` - 3. Раскрывается в запрос: + 3. Разворачивается в запрос: ```sql - INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) + INSERT INTO entities(code, type, value1, value2, value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) ``` - 4. Раскрывается в запрос: + 4. Разворачивается в запрос: ```sql UPDATE entities - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 WHERE code = :entity.id.code AND type = :entity.id.type ``` - 5. Раскрывается в запрос: + 5. Разворачивается в запрос: ```sql - INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) - ON CONFLICT (code, type) DO UPDATE - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + INSERT INTO entities(code, type, value1, value2, value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) + ON CONFLICT (code, type) DO UPDATE + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` === ":simple-kotlin: `Kotlin`" @@ -1065,40 +1332,39 @@ Kora не обрабатывает содержимое запроса, резу fun deleteAll(): UpdateCount } ``` - - 1. Раскрывается в запрос: + 1. Разворачивается в запрос: ```sql - SELECT code, type, value1, value2, value3 - FROM entities + SELECT code, type, value1, value2, value3 + FROM entities WHERE code = :code AND type = :type ``` - 2. Раскрывается в запрос: + 2. Разворачивается в запрос: ```sql - SELECT code, type, value1, value2, value3 + SELECT code, type, value1, value2, value3 FROM entities ``` - 3. Раскрывается в запрос: + 3. Разворачивается в запрос: ```sql - INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) + INSERT INTO entities(code, type, value1, value2, value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) ``` - 4. Раскрывается в запрос: + 4. Разворачивается в запрос: ```sql UPDATE entities - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 WHERE code = :entity.id.code AND type = :entity.id.type ``` - 5. Раскрывается в запрос: + 5. Разворачивается в запрос: ```sql - INSERT INTO entities(code, type, value1, value2, value3) - VALUES(:entity.code, :entity.type, :entity.value1, :entity.value2, :entity.value3) - ON CONFLICT (code, type) DO UPDATE - SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + INSERT INTO entities(code, type, value1, value2, value3) + VALUES(:entity.id.code, :entity.id.type, :entity.field1, :entity.value2, :entity.value3) + ON CONFLICT (code, type) DO UPDATE + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 ``` -#### Пример наследования { #inheritance-example } +#### Пример с наследованием { #inheritance-example } -Также можно создать абстрактный общий репозиторий и потом использовать его в наследовании для [Postgres SQL](https://postgrespro.ru/docs/postgresql): +Вы также можете создать абстрактный CRUD-репозиторий, а затем использовать его в наследовании для [Postgres SQL](https://postgrespro.com/docs/postgresql): ===! ":fontawesome-brands-java: `Java`" @@ -1202,3 +1468,61 @@ Kora не обрабатывает содержимое запроса, резу fun deleteAll(): UpdateCount } ``` + +## Телеметрия { #telemetry } + +Все драйверы баз данных используют общий контракт телеметрии для логирования, метрик и трассировки запросов. +Конкретные параметры конфигурации (секция `telemetry { logging / metrics / tracing }`) описаны в документации +для каждого драйвера, например [JDBC](database-jdbc.md#configuration); этот раздел документирует только общие точки расширения, +которые находятся в `ru.tinkoff.kora.database.common.telemetry`. + +Для каждого выполняемого запроса создаётся `DataBaseTelemetry.DataBaseTelemetryContext`, который закрывается по завершении запроса +(получая выброшенное исключение, если оно было). +Выполняемый запрос описывается `QueryContext(queryId, sql, operation)`, где `queryId` — это стабильный идентификатор запроса, +передаваемый в телеметрию, `sql` — итоговый текст запроса, а `operation` по умолчанию равно `db_query`. + +Фабрика по умолчанию `DefaultDataBaseTelemetryFactory` объединяет три необязательные вложенные фабрики: + +- `DataBaseLoggerFactory` строит `DataBaseLogger`, который логирует начало/конец запроса (`logQueryBegin` / `logQueryEnd`); +- `DataBaseMetricWriterFactory` строит `DataBaseMetricWriter`, который записывает метрики для каждого запроса (`recordQuery`); +- `DataBaseTracerFactory` строит `DataBaseTracer`, который создаёт спаны запроса и вызова для распределённой трассировки. + +Если ни одна из вложенных фабрик не создаёт реализацию (например, когда логирование, [метрики](metrics.md) и [трассировка](tracing.md) +все отключены в конфигурации), используется `DataBaseTelemetryFactory.EMPTY`, и телеметрия становится пустой операцией. + +Чтобы полностью настроить свою собственную телеметрию, предоставьте собственную `DataBaseTelemetryFactory` в [графе приложения](container.md), +которая [переопределяет](container.md#component-override) фабрику по умолчанию: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends JdbcDatabaseModule { + + default DataBaseTelemetryFactory dataBaseTelemetryFactory() { //(1)! + return (config, name, driverType, dbType, username) -> { + // build and return a custom DataBaseTelemetry + return DataBaseTelemetryFactory.EMPTY; + }; + } + } + ``` + + 1. Переопределяет фабрику `DataBaseTelemetryFactory` по умолчанию, предоставляемую `DataBaseModule`. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : JdbcDatabaseModule { + + fun dataBaseTelemetryFactory(): DataBaseTelemetryFactory { //(1)! + return DataBaseTelemetryFactory { config, name, driverType, dbType, username -> + // build and return a custom DataBaseTelemetry + DataBaseTelemetryFactory.EMPTY + } + } + } + ``` + + 1. Переопределяет фабрику `DataBaseTelemetryFactory` по умолчанию, предоставляемую `DataBaseModule`. diff --git a/mkdocs/docs/ru/documentation/database-jdbc.md b/mkdocs/docs/ru/documentation/database-jdbc.md index 1ac605e..700806d 100644 --- a/mkdocs/docs/ru/documentation/database-jdbc.md +++ b/mkdocs/docs/ru/documentation/database-jdbc.md @@ -4,10 +4,16 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora JDBC repositories, JDBC configuration, result and parameter mapping, generated identifiers, transactions, and repository method signatures; key triggers include @Repository, @Query, @EntityJdbc, @Table, @Id, @Column, @Batch, JdbcDatabaseModule, JdbcConnectionFactory, JdbcRepository." --- -Модуль предоставляет реализацию репозиториев на основе [JDBC](https://proselyte.net/tutorials/jdbc/introduction/) протокола работы с базами данных -и с использованием [Hikari](https://github.com/brettwooldridge/HikariCP) для управления набором соединений. +Модуль предоставляет реализацию репозитория на основе [JDBC](https://proselyte.net/tutorials/jdbc/introduction/) для +работы с реляционными базами данных и использует [Hikari](https://github.com/brettwooldridge/HikariCP) для управления пулом +соединений. +Вы описываете интерфейс репозитория и `SQL`-запросы с помощью `@Repository` и `@Query`, а `Kora` генерирует реализацию, +которая получает соединение из пула, связывает параметры, читает результат и участвует в транзакциях. -Если нужен пошаговый разбор перед справочным описанием, смотрите [База данных JDBC](../guides/database-jdbc.md) и [База данных JDBC продвинутая](../guides/database-jdbc-advanced.md). +Общие правила для отображений, `@Repository`, `@Query`, `@Batch`, `UpdateCount`, макросов, ручных запросов и других механизмов +репозитория описаны в разделе [Общие правила работы с базами данных](database-common.md). + +Если нужен пошаговый разбор перед справочным описанием, смотрите [База данных JDBC](../guides/database-jdbc.md) и [Продвинутая база данных JDBC](../guides/database-jdbc-advanced.md). ## Подключение { #dependency } @@ -37,77 +43,29 @@ agent: interface Application : JdbcDatabaseModule ``` -Также **требуется предоставить** реализацию драйвера базы данных как зависимость. +Также вы **обязаны предоставить** реализацию драйвера базы данных в качестве зависимости. ## Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `JdbcDatabaseConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации JDBC: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript db { jdbcUrl = "jdbc:postgresql://localhost:5432/postgres" //(1)! username = "postgres" //(2)! password = "postgres" //(3)! - schema = "public" //(4)! - poolName = "kora" //(5)! - maxPoolSize = 10 //(6)! - minIdle = 0 //(7)! - connectionTimeout = "10s" //(8)! - validationTimeout = "5s" //(9)! - idleTimeout = "10m" //(10)! - maxLifetime = "15m" //(11)! - leakDetectionThreshold = "0s" //(12)! - initializationFailTimeout = "0s" //(13)! - readinessProbe = false //(14)! - dsProperties { //(15)! - "hostRecheckSeconds": "2" - } - telemetry { - logging { - enabled = false //(16)! - } - metrics { - enabled = true //(17)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(18)! - tags = { // (19)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(20)! - attributes = { // (21)! - "key1" = "value1" - "key2" = "value2" - } - } - } + poolName = "kora" //(4)! + maxPoolSize = 10 //(5)! } ``` - 1. JDBC URL подключения к базе данных (**обязательный**) - 2. Имя пользователя для подключения (**обязательный**) - 3. Пароль пользователя для подключения (**обязательный**) - 4. Схема базы данных для подключения (по умолчанию отсутвует) - 5. Имя набора соединений к базе данных в Hikari (**обязательный**) - 6. Максимальный размер набора соединений к базе данных в Hikari - 7. Минимальный размер набора готовых соединений к базе данных в Hikari в режиме ожидания - 8. Максимальное время на установку соединения в Hikari - 9. Максимальное время на проверку соединения в Hikari - 10. Максимальное время на простой соединения в Hikari - 11. Максимальное время жизни соединения в Hikari - 12. Максимальное время соединение может отстуствовать в Hikari до того как будет считаться утечкой (по умолчанию отсутвует) - 13. Максимальное время ожидания инициализации соединения при старте сервиса (по умолчанию отсутвует) - 14. Включить ли [пробу готовности](probes.md#readiness) для соединения базы данных - 15. Дополнительные атрибуты JDBC соединения `dataSourceProperties` (ниже пример `hostRecheckSeconds` параметра) (по умолчанию отсутвует) - 16. Включает логгирование модуля (по умолчанию `false`) - 17. Включает метрики модуля (по умолчанию `true`) - 18. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 19. Настройка тегов для метрик (опционально) - 20. Включает трассировку модуля (по умолчанию `true`) - 21. Настройка атрибутов для трассировки (опционально) + 1. `JDBC URL` для подключения к базе данных (`обязательный`, по умолчанию: не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию: не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию: не указано) + 4. Имя пула соединений `Hikari` (`обязательный`, по умолчанию: не указано) + 5. Максимальный размер пула соединений `Hikari` (по умолчанию: `10`) === ":simple-yaml: `YAML`" @@ -116,81 +74,248 @@ agent: jdbcUrl: "jdbc:postgresql://localhost:5432/postgres" #(1)! username: "postgres" #(2)! password: "postgres" #(3)! - schema: "public" #(4)! - poolName: "kora" #(5)! - maxPoolSize: 10 #(6)! - minIdle: 0 #(7)! - connectionTimeout: "10s" #(8)! - validationTimeout: "5s" #(9)! - idleTimeout: "10m" #(10)! - maxLifetime: "15m" #(11)! - leakDetectionThreshold: "0s" #(12)! - initializationFailTimeout: "0s" //(13)! - readinessProbe: false //(14)! - dsProperties: #(15)! - hostRecheckSeconds: "1" - telemetry: - logging: - enabled: false #(16)! - metrics: - enabled: true #(17)! - slo: [ 2, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(18)! - tags: #(19)! - key1: value1 - key2: value2 - tracing: - enabled: true #(20)! - attributes: #(21)! - key1: value1 - key2: value2 - } - ``` - - 1. JDBC URL подключения к базе данных (**обязательный**) - 2. Имя пользователя для подключения (**обязательный**) - 3. Пароль пользователя для подключения (**обязательный**) - 4. Схема базы данных для подключения (по умолчанию отсутвует) - 5. Имя набора соединений к базе данных в Hikari (**обязательный**) - 6. Максимальный размер набора соединений к базе данных в Hikari - 7. Минимальный размер набора готовых соединений к базе данных в Hikari в режиме ожидания - 8. Максимальное время на установку соединения в Hikari - 9. Максимальное время на проверку соединения в Hikari - 10. Максимальное время на простой соединения в Hikari - 11. Максимальное время жизни соединения в Hikari - 12. Максимальное время соединение может отстуствовать в Hikari до того как будет считаться утечкой (по умолчанию отсутвует) - 13. Максимальное время ожидания инициализации соединения при старте сервиса (по умолчанию отсутвует) - 14. Включить ли [пробу готовности](probes.md#readiness) для соединения базы данных - 15. Дополнительные атрибуты JDBC соединения `dataSourceProperties` (ниже пример `hostRecheckSeconds` параметра) (по умолчанию отсутвует) - 16. Включает логгирование модуля (по умолчанию `false`) - 17. Включает метрики модуля (по умолчанию `true`) - 18. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 19. Настройка тегов для метрик (опционально) - 20. Включает трассировку модуля (по умолчанию `true`) - 21. Настройка атрибутов для трассировки (опционально) + poolName: "kora" #(4)! + maxPoolSize: 10 #(5)! + ``` + + 1. `JDBC URL` для подключения к базе данных (`обязательный`, по умолчанию: не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию: не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию: не указано) + 4. Имя пула соединений `Hikari` (`обязательный`, по умолчанию: не указано) + 5. Максимальный размер пула соединений `Hikari` (по умолчанию: `10`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `JdbcDatabaseConfig`: + + ===! ":material-code-json: `HOCON`" + + ```javascript + db { + jdbcUrl = "jdbc:postgresql://localhost:5432/postgres" //(1)! + username = "postgres" //(2)! + password = "postgres" //(3)! + schema = "public" //(4)! + poolName = "kora" //(5)! + maxPoolSize = 10 //(6)! + minIdle = 0 //(7)! + connectionTimeout = "10s" //(8)! + validationTimeout = "5s" //(9)! + idleTimeout = "10m" //(10)! + maxLifetime = "15m" //(11)! + leakDetectionThreshold = "0s" //(12)! + initializationFailTimeout = "0s" //(13)! + readinessProbe = false //(14)! + dsProperties { //(15)! + "hostRecheckSeconds": "2" + } + telemetry { + logging { + enabled = false //(16)! + } + metrics { + enabled = true //(17)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(18)! + tags = { // (19)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(20)! + attributes = { // (21)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. `JDBC URL` для подключения к базе данных (`обязательный`, по умолчанию: не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию: не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию: не указано) + 4. Схема базы данных для подключения (по умолчанию: не указано, необязательно) + 5. Имя пула соединений `Hikari` (`обязательный`, по умолчанию: не указано) + 6. Максимальный размер пула соединений `Hikari` (по умолчанию: `10`) + 7. Минимальное количество простаивающих готовых соединений в пуле `Hikari` (по умолчанию: `0`) + 8. Максимальное время ожидания соединения из пула `Hikari` (по умолчанию: `10s`) + 9. Максимальное время проверки соединения `Hikari` (по умолчанию: `5s`) + 10. Максимальное время простоя соединения `Hikari` (по умолчанию: `10m`) + 11. Максимальное время жизни соединения `Hikari` (по умолчанию: `15m`) + 12. Время, после которого занятое соединение считается возможной утечкой (по умолчанию: `0s`) + 13. Максимальное время ожидания инициализации соединения при запуске сервиса (по умолчанию: не указано, необязательно) + 14. Включать ли [пробу готовности](probes.md) для соединения с базой данных (по умолчанию: `false`) + 15. Дополнительные свойства соединения `JDBC`, передаваемые в `dataSourceProperties` `Hikari` (по умолчанию: `{}`) + 16. Включает логирование модуля (по умолчанию: `false`) + 17. Включает метрики модуля (по умолчанию: `true`) + 18. Настраивает [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Настраивает теги метрик (по умолчанию: `{}`) + 20. Включает трассировку модуля (по умолчанию: `true`) + 21. Настраивает атрибуты трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + db: + jdbcUrl: "jdbc:postgresql://localhost:5432/postgres" #(1)! + username: "postgres" #(2)! + password: "postgres" #(3)! + schema: "public" #(4)! + poolName: "kora" #(5)! + maxPoolSize: 10 #(6)! + minIdle: 0 #(7)! + connectionTimeout: "10s" #(8)! + validationTimeout: "5s" #(9)! + idleTimeout: "10m" #(10)! + maxLifetime: "15m" #(11)! + leakDetectionThreshold: "0s" #(12)! + initializationFailTimeout: "0s" #(13)! + readinessProbe: false #(14)! + dsProperties: #(15)! + hostRecheckSeconds: "1" + telemetry: + logging: + enabled: false #(16)! + metrics: + enabled: true #(17)! + slo: [ 2, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(18)! + tags: #(19)! + key1: value1 + key2: value2 + tracing: + enabled: true #(20)! + attributes: #(21)! + key1: value1 + key2: value2 + ``` + + 1. `JDBC URL` для подключения к базе данных (`обязательный`, по умолчанию: не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию: не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию: не указано) + 4. Схема базы данных для подключения (по умолчанию: не указано, необязательно) + 5. Имя пула соединений `Hikari` (`обязательный`, по умолчанию: не указано) + 6. Максимальный размер пула соединений `Hikari` (по умолчанию: `10`) + 7. Минимальное количество простаивающих готовых соединений в пуле `Hikari` (по умолчанию: `0`) + 8. Максимальное время ожидания соединения из пула `Hikari` (по умолчанию: `10s`) + 9. Максимальное время проверки соединения `Hikari` (по умолчанию: `5s`) + 10. Максимальное время простоя соединения `Hikari` (по умолчанию: `10m`) + 11. Максимальное время жизни соединения `Hikari` (по умолчанию: `15m`) + 12. Время, после которого занятое соединение считается возможной утечкой (по умолчанию: `0s`) + 13. Максимальное время ожидания инициализации соединения при запуске сервиса (по умолчанию: не указано, необязательно) + 14. Включать ли [пробу готовности](probes.md) для соединения с базой данных (по умолчанию: `false`) + 15. Дополнительные свойства соединения `JDBC`, передаваемые в `dataSourceProperties` `Hikari` (по умолчанию: `{}`) + 16. Включает логирование модуля (по умолчанию: `false`) + 17. Включает метрики модуля (по умолчанию: `true`) + 18. Настраивает [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Настраивает теги метрик (по умолчанию: `{}`) + 20. Включает трассировку модуля (по умолчанию: `true`) + 21. Настраивает атрибуты трассировки (по умолчанию: `{}`) + ## Использование { #usage } +Репозиторий `JDBC` объявляется как интерфейс, помеченный аннотацией `@Repository`, и должен наследовать `JdbcRepository`. +Каждый метод, помеченный `@Query`, содержит обычный `SQL`-запрос. Параметры метода связываются по имени с помощью +синтаксиса `:parameter`, а к полям объекта можно обращаться как `:entity.field`. + +Отображения описываются с помощью [общих аннотаций баз данных](database-common.md) и помечаются `@EntityJdbc`, +чтобы `Kora` сгенерировала отображатель на этапе компиляции (см. [Отображение](database-common.md#view)): + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends JdbcRepository { } + public interface EntityRepository extends JdbcRepository { + + @EntityJdbc + @Table("entities") + record Entity(@Id long id, + String name, + @Nullable String description) {} + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + @Nullable + Entity findById(long id); + + @Query("SELECT id, name, description FROM entities") //(2)! + List findAll(); + + @Query("INSERT INTO %{entity#inserts}") //(3)! + UpdateCount insert(Entity entity); + } ``` + 1. Использует макрос `%{return#selects}` и `%{return#table}`. Разворачивается в запрос: + ```sql + SELECT id, name, description + FROM entities + WHERE id = :id + ``` + Метод использует макросы для `SELECT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + 2. Поля перечислены вручную без использования макросов — это допустимо, но требует поддержки при изменении отображения. + 3. Использует макрос `%{entity#inserts}`. Разворачивается в запрос: + ```sql + INSERT INTO entities(id, name, description) + VALUES(:entity.id, :entity.name, :entity.description) + ``` + Метод использует макросы для `INSERT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : JdbcRepository - ``` + interface EntityRepository : JdbcRepository { -## Конвертация { #mapping } + @EntityJdbc + @Table("entities") + data class Entity( + @field:Id val id: Long, + val name: String, + val description: String? + ) + + @Query("SELECT %{return#selects} FROM %{return#table} WHERE id = :id") //(1)! + fun findById(id: Long): Entity? + + @Query("INSERT INTO %{entity#inserts}") //(3)! + fun insert(entity: Entity): UpdateCount + } + ``` -Возможно переопределять преобразование различных частей [сущности](database-common.md) и параметров запроса, для этого Kora предоставляет специальные интерфейсы. + 1. Использует макрос `%{return#selects}` и `%{return#table}`. Разворачивается в запрос: + ```sql + SELECT id, name, description + FROM entities + WHERE id = :id + ``` + Метод использует макросы для `SELECT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + 3. Использует макрос `%{entity#inserts}`. Разворачивается в запрос: + ```sql + INSERT INTO entities(id, name, description) + VALUES(:entity.id, :entity.name, :entity.description) + ``` + Метод использует макросы для `INSERT`. Подробнее: [Общие правила работы с базами данных — Макросы](database-common.md#macros) + +`SQL` остается под контролем разработчика: вы можете использовать специфичные для базы данных возможности, тогда как `Kora` +берет на себя только безопасное связывание параметров, выполнение запроса и отображение результата. +Общие правила для отображений, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch` и макросов описаны в разделе +[Общие правила работы с базами данных](database-common.md#macros). + +**Связывание параметров:** Kora выполняет типизированное внедрение аргументов в SQL-запрос на этапе компиляции. +Параметры запроса (например, `:id`, `:entity.name`) заменяются в сгенерированном коде на соответствующие вызовы `PreparedStatement`. +Например, для параметра `String name` будет сгенерировано что-то вроде `statement.setString(1, name)`, где индекс соответствует порядку параметра в запросе. +Это обеспечивает безопасность (защита от SQL-инъекций) и производительность (использование подготовленных запросов). + +## Отображение { #mapping } + +Вы можете переопределить отображение различных частей [отображения](database-common.md), результата запроса и параметров запроса. +Для этого `Kora` предоставляет несколько интерфейсов-отображателей. ### Результат { #result } -Если требуется преобразовать результат вручную, предлагается использовать `JdbcResultSetMapper`: +Используйте `JdbcResultSetMapper`, когда нужно вручную отобразить весь `ResultSet`. +Такой отображатель получает весь результат запроса и сам решает, сколько строк прочитать и что вернуть. ===! ":fontawesome-brands-java: `Java`" @@ -199,7 +324,7 @@ agent: @Override public UUID apply(ResultSet rs) throws SQLException { - // код преобразования + // mapping code } } @@ -214,14 +339,12 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin - class ResultMapper : JdbcResultSetMapper { + class ResultMapper : JdbcResultSetMapper { @Throws(SQLException::class) override fun apply(rs: ResultSet): UUID { - // код преобразования + // mapping code } } @@ -234,12 +357,16 @@ agent: } ``` -#### Сущность { #entity } +`JdbcResultSetMapper` также предоставляет статические вспомогательные методы `singleResultSetMapper`, `listResultSetMapper` +и `optionalResultSetMapper`, которые создают отображатель всего `ResultSet` из `JdbcRowMapper`. + +#### Отображение { #view } -Для оптимального преобразование сущности предполагается использовать аннотацию `@EntityJdbc` -для создания обработчиками аннотаций преобразователя результата. +Используйте аннотацию `@EntityJdbc` для оптимального отображения. +Аннотация позволяет обработчику аннотаций сгенерировать все необходимые отображатели за **один раунд** аннотационной обработки. +Без этой аннотации отображатели генерируются по требованию, что может потребовать **множества раундов** обработки и значительно увеличить время компиляции. -Для всех вложенных сущностей также предполагается использовать эту аннотацию +Ожидается, что все вложенные отображения также используют эту аннотацию. ===! ":fontawesome-brands-java: `Java`" @@ -257,8 +384,8 @@ agent: ### Строка { #row } -Если требуется преобразовать строку вручную, предлагается использовать `JdbcRowMapper`, -имейте в виду, что порядок колонок начинается с `1`: +Используйте `JdbcRowMapper`, когда нужно вручную отобразить одну строку. +Учтите, что в `JDBC` индексы столбцов в `ResultSet` начинаются с `1`: ===! ":fontawesome-brands-java: `Java`" @@ -282,8 +409,6 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin class RowMapper : JdbcRowMapper { @@ -302,9 +427,9 @@ agent: } ``` -### Колонка { #column } +### Столбец { #column } -Если требуется преобразовать значение колонки вручную, предлагается использовать `JdbcResultColumnMapper`: +Используйте `JdbcResultColumnMapper`, когда нужно вручную отобразить значение одного столбца: ===! ":fontawesome-brands-java: `Java`" @@ -331,8 +456,6 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin class ColumnMapper : JdbcResultColumnMapper { @@ -359,7 +482,7 @@ agent: ### Параметр { #parameter } -Если требуется преобразовать значение параметра запроса вручную, предлагается использовать `JdbcParameterColumnMapper`: +Используйте `JdbcParameterColumnMapper`, когда нужно вручную отобразить значение параметра запроса: ===! ":fontawesome-brands-java: `Java`" @@ -384,8 +507,6 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin class ParameterMapper : JdbcParameterColumnMapper { @@ -409,7 +530,8 @@ agent: ??? abstract "Список поддерживаемых типов для аргументов/возвращаемых значений из коробки" - Такие типы выбраны так как поддерживаются большинством популярных баз данных. + Эти типы выбраны потому, что поддерживаются большинством популярных баз данных. + `Kora` предоставляет для них встроенные отображатели строк, столбцов и параметров. * void * boolean / Boolean @@ -428,14 +550,19 @@ agent: * OffsetTime * OffsetDateTime + Поля отображения без явного `@Mapping` нативно поддерживают `boolean` / `Boolean`, `short` / `Short`, + `int` / `Integer`, `long` / `Long`, `double` / `Double`, `float` / `Float`, `byte[]`, `String`, + `BigDecimal`, `LocalDate` и `LocalDateTime`. + Для остальных типов используйте встроенные отображатели `JdbcResultColumnMapper` / `JdbcParameterColumnMapper` или объявите собственные отображатели. + ## Выборка по списку { #select-by-list } -Иногда требуется выборка по списку значений из базы, на уровне драйвера все эти параметры должны быть отдельно проставлены, так как длина списка не известна -это не самая очевидная задача так как Kora старается делать все преобразования во время компиляции и убирать любые преобразования строк особенно в SQL во время выполнения, -для такой функциональности потребуется добавить самостоятельный преобразователь параметров. +Иногда нужно выбрать строки по списку значений. +На уровне `JDBC` такие параметры должны подготавливаться драйвером отдельно, поскольку длина списка заранее неизвестна. +`Kora` старается выполнять отображения во время компиляции и не переписывает `SQL` во время выполнения, поэтому для таких параметров требуется собственный отображатель. -На данный момент точно известно, что можно легко добавить поддержку таких параметров без ручного управления в такие популярные базы данных как Postgres/Oracle. -Из коробки Kora не предоставляет конвертацию таких параметров, но его легко добавить самостоятельно, ниже показан пример для `Postgres`: +`Kora` не предоставляет отображение такого параметра из коробки, но его легко добавить самостоятельно. +В примере ниже показан `Postgres` через `JDBC Array`: ===! ":fontawesome-brands-java: `Java`" @@ -481,11 +608,137 @@ agent: } ``` -## Созданный идентификатор { #generated-identifier } +## JSON / JSONB { #json } + +Столбец `JSON` / `JSONB` можно отобразить на поле отображения, зарегистрировав обобщенные +`JdbcParameterColumnMapper` и `JdbcResultColumnMapper` как компоненты по умолчанию в `@Module`, помеченные `@Json`. +Эти отображатели связывают `JsonWriter` / `JsonReader` из модуля [JSON](json.md) со значением, специфичным для драйвера. +В примере для `Postgres` ниже значение сериализуется в `PGobject` типа `jsonb` при связывании параметра, +`null` обрабатывается через `setNull(index, Types.NULL)`, а столбец читается обратно как `String`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface JdbcJsonbMapperModule { + + @Json + default JdbcParameterColumnMapper jdbcJsonParameterColumnMapper(JsonWriter writer) { + return (stmt, index, value) -> { + if (value != null) { + PGobject jsonb = new PGobject(); + jsonb.setType("jsonb"); + jsonb.setValue(writer.toStringUnchecked(value)); + stmt.setObject(index, jsonb); + } else { + stmt.setNull(index, Types.NULL); + } + }; + } + + @Json + default JdbcResultColumnMapper jdbcJsonResultColumnMapper(JsonReader reader) { + return (row, index) -> { + var value = row.getString(index); + if (value == null) { + return null; + } else { + return reader.readUnchecked(value); + } + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface JdbcJsonbMapperModule { + + @Json + fun jdbcJsonParameterColumnMapper(writer: JsonWriter): JdbcParameterColumnMapper { + return JdbcParameterColumnMapper { stmt, index, value -> + if (value == null) { + stmt.setNull(index, Types.NULL) + } else { + val jsonb = PGobject() + jsonb.type = "jsonb" + jsonb.value = writer.toStringUnchecked(value) + stmt.setObject(index, jsonb) + } + } + } + + @Json + fun jdbcJsonResultColumnMapper(reader: JsonReader): JdbcResultColumnMapper { + return JdbcResultColumnMapper { row, index -> + val value = row.getString(index) + if (value == null) null else reader.readUnchecked(value) + } + } + } + ``` + +Пометьте поле отображения аннотацией `@Json` (и `@Column`, если имя столбца отличается), где тип поля сам является `@Json`-типом. +В `INSERT` используется приведение `::jsonb`, чтобы `Postgres` принял сериализованную строку как `JSONB`; +`findById` читает ее обратно через тот же отображатель столбца, помеченный `@Json`: -Если необходимо получить в качестве результата созданные базой данных первичные ключи сущности, -предлагается использовать аннотацию `@Id` над методом, где тип возвращаемого значения является идентификаторами. -Такой подход работает и для `@Batch` запросов. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface JdbcJsonbRepository extends JdbcRepository { + + @EntityJdbc + record Entity(UUID id, + @Column("value") @Json JsonbValue value) { + + @Json + record JsonbValue(String name, String surname) {} + } + + @Query("SELECT * FROM entities_jsonb WHERE id = :id") + @Nullable + Entity findById(UUID id); + + @Query("INSERT INTO entities_jsonb(id, value) VALUES (:entity.id, :entity.value::jsonb)") + void insert(Entity entity); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface JdbcJsonbRepository : JdbcRepository { + + @EntityJdbc + data class Entity( + val id: UUID, + @field:Column("value") @Json val value: JsonbValue + ) { + + @Json + data class JsonbValue(val name: String, val surname: String) + } + + @Query("SELECT * FROM entities_jsonb WHERE id = :id") + fun findById(id: UUID): Entity? + + @Query("INSERT INTO entities_jsonb(id, value) VALUES (:entity.id, :entity.value::jsonb)") + fun insert(entity: Entity) + } + ``` + +Зависимость модуля [JSON](json.md) обязательна, чтобы `Kora` мог сгенерировать `JsonWriter` / `JsonReader` для типа поля, +а `@Module` с отображателями должен быть добавлен в [граф приложения](container.md). + +## Сгенерированный идентификатор { #generated-identifier } + +Если нужно вернуть первичные ключи, сгенерированные базой данных, +используйте аннотацию `@Id` над методом. +Этот подход также работает для `@Batch`-запросов. ===! ":fontawesome-brands-java: `Java`" @@ -509,7 +762,7 @@ agent: interface EntityRepository : JdbcRepository { @EntityJdbc - public record Entity(Long id, String name) {} + data class Entity(val id: Long, val name: String) @Query("INSERT INTO entities(name) VALUES (:entity.name)") @Id @@ -517,73 +770,236 @@ agent: } ``` -## Транзакции { #transaction } +Сгенерированный ключ также можно вернуть как тип ключа отображения, а не как скалярное значение. +Когда идентификатор является составным ключом, описанным записью [`@Embedded`](database-common.md#embedded-fields), +метод `@Id` возвращает эту запись, а вставка `@Batch` возвращает `List` ключей — по одному на каждую вставленную строку: -Для выполнения блокирующих запросов в Kora есть интерфейс `JdbcConnectionFactory`, -который предоставляется в методе в рамках контракта `JdbcRepository`. -Все методы репозитория вызванные в рамках лямбды транзакции будут выполнены в этой самой транзакции. +===! ":fontawesome-brands-java: `Java`" -Для того чтобы выполнять запросы транзакционно, можно использовать контракт `inTx`: + ```java + @Repository + public interface EntityRepository extends JdbcRepository { + + @EntityJdbc + record Entity(@Id @Embedded EntityId id, @Column("name") String name) { + + @EntityJdbc + record EntityId(Long a, Long b) {} + } + + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + @Id + Entity.EntityId insertGenerated(Entity entity); + + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + @Id + List insertGenerated(@Batch List entities); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + @EntityJdbc + data class Entity( + @field:Id @field:Embedded val id: EntityId?, + @field:Column("name") val name: String + ) { + + @EntityJdbc + data class EntityId(val a: Long?, val b: Long?) + } + + @Id + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + fun insertGenerated(entity: Entity): Entity.EntityId + + @Id + @Query("INSERT INTO entities_composite(name) VALUES (:entity.name)") + fun insertGenerated(@Batch entities: List): List + } + ``` + +## Ручной запрос с телеметрией { #query } + +Если запрос сложно выразить одной статической `@Query`, вы можете создать обычный метод с реализацией и построить `SQL` вручную. +Используйте `JdbcConnectionFactory#query` для выполнения такого запроса. +Этот метод создает `PreparedStatement`, выполняет запрос через телеметрию Kora и использует то же соединение, что и другие методы репозитория. +Если `query` вызывается внутри активной транзакции `inTx`, запрос выполняется на текущем транзакционном соединении. + +`QueryContext` содержит идентификатор запроса и итоговый `SQL`. +Идентификатор запроса передается в телеметрию, поэтому удобно использовать стабильное имя, например `Repository.method`. +Значения должны передаваться через параметры `PreparedStatement`, а не конкатенироваться напрямую в строку запроса. ===! ":fontawesome-brands-java: `Java`" ```java - @Component - public final class SomeService { + @Repository + public interface EntityRepository extends JdbcRepository { - private final EntityRepository repository; + default List findByFilter(@Nullable String name, boolean onlyActive) { + var sql = new StringBuilder("SELECT id, name FROM entities WHERE 1 = 1"); + var params = new ArrayList(); - public SomeService(EntityRepository repository) { - this.repository = repository; + if (name != null) { + sql.append(" AND name = ?"); + params.add(name); + } + if (onlyActive) { + sql.append(" AND active = true"); + } + + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return getJdbcConnectionFactory().query(queryContext, statement -> { + for (int i = 0; i < params.size(); i++) { + statement.setString(i + 1, params.get(i)); + } + try (var resultSet = statement.executeQuery()) { + var result = new ArrayList(); + while (resultSet.next()) { + result.add(new Entity(resultSet.getLong("id"), resultSet.getString("name"))); + } + return result; + } + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : JdbcRepository { + + fun findByFilter(name: String?, onlyActive: Boolean): List { + val sql = StringBuilder("SELECT id, name FROM entities WHERE 1 = 1") + val params = mutableListOf() + + if (name != null) { + sql.append(" AND name = ?") + params += name + } + if (onlyActive) { + sql.append(" AND active = true") + } + + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return jdbcConnectionFactory.query(queryContext) { statement -> + params.forEachIndexed { index, value -> + statement.setString(index + 1, value) + } + statement.executeQuery().use { resultSet -> + val result = mutableListOf() + while (resultSet.next()) { + result += Entity(resultSet.getLong("id"), resultSet.getString("name")) + } + result + } + } } + } + ``` + +## Транзакции { #transaction } + +Для выполнения блокирующих запросов `Kora` предоставляет интерфейс `JdbcConnectionFactory` через контракт `JdbcRepository`. +Все методы репозитория, вызванные внутри лямбды транзакции, выполняются в этой же транзакции. + +Используйте `inTx` для транзакционного выполнения запросов. +Если в текущем потоке уже есть активная транзакция, вложенный вызов `inTx` использует то же соединение и не открывает +новую транзакцию. + +Транзакционную последовательность операций можно оставить внутри самого репозитория в виде обычного метода с реализацией. +Это удобно, когда несколько методов `@Query` или сложный ручной `SQL`-запрос должны находиться рядом с остальными запросами репозитория, +без переноса технической работы с базой данных в слой сервиса. +Внутри такого метода можно использовать как методы репозитория `@Query`, так и `JdbcConnectionFactory#query` для ручного запроса с телеметрией. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends JdbcRepository { + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + UpdateCount insert(Entity entity); + + @Query("UPDATE entities SET name = :name WHERE id = :id") + UpdateCount updateName(long id, String name); public List saveAll(Entity one, Entity two) { - return repository.getJdbcConnectionFactory().inTx(() -> { - repository.insert(one); //(1)! - // do some work - repository.insert(two); //(2)! + return getJdbcConnectionFactory().inTx(() -> { + insert(one); //(1)! + updateName(two.id(), two.name()); //(2)! return List.of(one, two); }); } } ``` - 1. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение - 2. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение + 1. Выполняется в рамках транзакции или откатывается, если вся лямбда выбрасывает исключение + 2. Выполняется в рамках транзакции или откатывается, если вся лямбда выбрасывает исключение === ":simple-kotlin: `Kotlin`" ```kotlin - @Component - class SomeService(private val repository: EntityRepository) { + @Repository + interface EntityRepository : JdbcRepository { - fun saveAll(one: List, two: List): List { - return repository.jdbcConnectionFactory.inTx(SqlFunction1 { - repository.insert(one) //(1)! - // do some work - repository.insert(two) //(2)! - one + two - }) + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insert(entity: Entity): UpdateCount + + @Query("UPDATE entities SET name = :name WHERE id = :id") + fun updateName(id: Long, name: String): UpdateCount + + fun saveAll(one: Entity, two: Entity): List { + return jdbcConnectionFactory.inTx> { + insert(one) //(1)! + updateName(two.id, two.name) //(2)! + listOf(one, two) + } } } ``` - 1. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение - 2. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение + 1. Выполняется в рамках транзакции или откатывается, если вся лямбда выбрасывает исключение + 2. Выполняется в рамках транзакции или откатывается, если вся лямбда выбрасывает исключение -Транзакция считается успешно зафиксированной после выполнения метода, если метод не выбросил исключение. -В случае если метод выбросил исключение, все изменения в БД в рамках транзакции не будут применены. +Транзакция считается успешно зафиксированной после завершения метода, если он не выбросил исключение. +Если метод выбрасывает исключение, все изменения в базе данных, сделанные в рамках транзакции, не применяются. -Уровень изоляции транзакции берется из конфигурации `dsProperties` пула Hikari, -либо можно самостоятельно поменять его через `java.sql.Connection` перед выполнением запросов. +Уровень изоляции транзакции берется из конфигурации `dsProperties` пула `Hikari`, +либо вы можете изменить его вручную через `java.sql.Connection` перед выполнением запросов. ```java connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); ``` -### Ручное управление { #connection } +### Ручное управление соединением { #connection } + +Если для запроса нужна более сложная логика или запросы вне репозитория, вы можете использовать `java.sql.Connection`. +Метод `withConnection` выполняет код с соединением, но сам по себе не открывает транзакцию. + +`withConnection` работает следующим образом: -Если для запроса нужна какая-то более сложная логика, либо запросы вне репозитория, можно использовать `java.sql.Connection`: +- если текущий `Context` уже содержит `ConnectionContext`, метод передает текущее соединение в лямбду; +- если текущий `Context` не содержит соединения, метод берет новое соединение из `DataSource`, сохраняет его в `ConnectionContext` на время выполнения лямбды и закрывает после завершения; +- вложенные вызовы `withConnection`, `JdbcConnectionFactory#query` и методов репозитория внутри этой лямбды используют то же текущее соединение; +- если исключение `JDBC` является `SQLException`, оно оборачивается в `RuntimeSqlException`. + +!!! note + + Ручные вызовы `query`, `withConnection` и `inTx` представляют сбой `JDBC` как непроверяемое исключение `RuntimeSqlException`, + которое оборачивает исходное `java.sql.SQLException`. Перехватывайте `RuntimeSqlException` (а не `SQLException`) в месте вызова + и используйте `getCause()`, чтобы добраться до исходного `SQLException`. + +Метод `inTx` открывает транзакцию и построен поверх `withConnection`. +Если текущее соединение уже находится в активной транзакции, то есть `autoCommit = false`, вложенный `inTx` использует ту же транзакцию. +Если активной транзакции нет, `inTx` отключает `autoCommit`, выполняет лямбду, а затем вызывает `commit` при успехе или `rollback` при исключении. +После завершения транзакции выполняются зарегистрированные обратные вызовы `addPostCommitAction` или `addPostRollbackAction`. ===! ":fontawesome-brands-java: `Java`" @@ -621,10 +1037,11 @@ connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); } ``` -### После коммит действия { #post-commit-actions } +### Действия после фиксации { #post-commit-actions } -В случае если требуется выполнить какие-либо действия после фиксации транзакции, -можно добавить соответствущие действия с помощью `addPostCommitAction`. +Если нужно выполнить действия после успешной фиксации транзакции, добавьте их с помощью `addPostCommitAction`. +Действие выполняется после `commit` и только если транзакция завершилась успешно. +Такие действия можно добавлять только внутри активной транзакции. ===! ":fontawesome-brands-java: `Java`" @@ -641,7 +1058,7 @@ connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); public List saveAll(Entity one, Entity two) { return repository.getJdbcConnectionFactory().inTx(connection -> { var ccc = repository.getJdbcConnectionFactory().currentConnectionContext(); - ccc.addPostCommitAction(conn) -> { + ccc.addPostCommitAction(conn -> { // do some work }); @@ -660,8 +1077,8 @@ connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); fun saveAll(one: Entity, two: Entity): List { return repository.jdbcConnectionFactory.inTx(SqlFunction1 { connection: Connection -> - val ccc = repository.jdbcConnectionFactory.currentConnectionContext() - ccc.addPostCommitAction { conn -> { + val ccc = repository.jdbcConnectionFactory.currentConnectionContext()!! + ccc.addPostCommitAction { conn -> // do some work } @@ -672,10 +1089,11 @@ connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); } ``` -### После откат действия { #post-rollback-actions } +### Действия после отката { #post-rollback-actions } -В случае если требуется выполнить какие-либо действия после отката транзакции, -можно добавить соответствущие действия с помощью `addPostRollbackAction`. +Если нужно выполнить действия после отката транзакции, добавьте их с помощью `addPostRollbackAction`. +Действие получает соединение и исключение, вызвавшее откат транзакции. +Такие действия можно добавлять только внутри активной транзакции. ===! ":fontawesome-brands-java: `Java`" @@ -711,7 +1129,7 @@ connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); fun saveAll(one: Entity, two: Entity): List { return repository.jdbcConnectionFactory.inTx(SqlFunction1 { connection: Connection -> - val ccc = repository.jdbcConnectionFactory.currentConnectionContext() + val ccc = repository.jdbcConnectionFactory.currentConnectionContext()!! ccc.addPostRollbackAction { conn, e -> // do some work } @@ -725,21 +1143,57 @@ connection.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED); ## Сигнатуры { #signatures } -Доступные сигнатуры для методов репозитория из коробки: +Доступные из коробки сигнатуры методов репозитория: ===! ":fontawesome-brands-java: `Java`" - Под `T` подразумевается тип возвращаемого значения, либо `List`, либо `Void`, либо `UpdateCount`. + `T` означает тип возвращаемого значения, либо `List`, либо `Void`, либо `UpdateCount`. + `CompletionStage`, `CompletableFuture` и `Mono` требуют компонент `Executor`. - `T myMethod()` - `@Nullable T myMethod()` - `Optional myMethod()` - - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) (надо предоставить `Executor`) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо предоставить `Executor` и подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) (требует `Executor`) + - `CompletableFuture myMethod()` [CompletableFuture](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html) (требует `Executor`) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (требует `Executor` и [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `List`, либо `Unit`, либо `UpdateCount`. + `T` означает тип возвращаемого значения, либо `T?`, либо `List`, либо `Unit`, либо `UpdateCount`. + Методы `suspend` требуют компонент `Executor`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо предоставить `Executor` и подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требует `Executor` и [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + +Для асинхронных методов вы можете указать отдельный тег `Executor` через параметр `executorTag` в `@Repository`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public final class BlockingJdbcExecutorTag {} + + @Repository(executorTag = @Tag(BlockingJdbcExecutorTag.class)) + public interface EntityRepository extends JdbcRepository { + + @Query("SELECT id, name FROM entities") + CompletionStage> findAll(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + class BlockingJdbcExecutorTag + + @Repository(executorTag = Tag(BlockingJdbcExecutorTag::class)) + interface EntityRepository : JdbcRepository { + + @Query("SELECT id, name FROM entities") + suspend fun findAll(): List + } + ``` + +## Телеметрия { #telemetry } + +Логирование, метрики и трассировка настраиваются через блок `telemetry` в [конфигурации](#configuration) и описаны в разделе [Справочник метрик](metrics.md#database). +Чтобы переопределить телеметрию полностью, можно предоставить собственные SPI-фабрики, подробнее в [Общей документации по Базам данных](database-common.md#telemetry). diff --git a/mkdocs/docs/ru/documentation/database-migration.md b/mkdocs/docs/ru/documentation/database-migration.md index accb72f..459900a 100644 --- a/mkdocs/docs/ru/documentation/database-migration.md +++ b/mkdocs/docs/ru/documentation/database-migration.md @@ -4,11 +4,21 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora database migration modules for Flyway and Liquibase, migration configuration, startup behavior, and database integration; key triggers include FlywayJdbcDatabaseInterceptor, LiquibaseJdbcDatabaseInterceptor, FlywayConfig, LiquibaseConfig, JdbcDatabaseModule." --- -Модуль для миграции базы данных вместе с запуском сервиса. +Миграции базы данных применяют изменения схемы и справочных данных в контролируемом порядке: они создают таблицы, индексы, ограничения и выполняют другие операции `SQL`, необходимые новой версии приложения. +В Kora модули миграций привязаны к инициализации `JdbcDatabase` через `GraphInterceptor`: при запуске приложения `JdbcDatabase` создается как компонент графа, а метод `init()` перехватчика выполняет миграции до того, как компонент будет опубликован для остальной части графа. +Если миграция завершается ошибкой, метод `init()` выбрасывает исключение, поэтому инициализация компонента `JdbcDatabase` и построение всего графа (запуск приложения) также завершаются неудачей. +Метод `release()` перехватчика ничего не делает: миграции никогда не откатываются и не выполняются повторно при остановке приложения. + +Такой подход удобен для локальной разработки, тестов и небольших установок, где приложение запускается в одном экземпляре. +Для окружений с несколькими репликами заранее выберите отдельный способ выполнения миграций, чтобы они не запускались одновременно из каждого экземпляра приложения. +Репозитории не создают схему базы данных сами: таблицы, индексы, ограничения и справочные данные должны создаваться миграциями или внешним процессом подготовки базы данных. ## Flyway { #flyway } Модуль для миграции базы данных с помощью инструмента [Flyway](https://documentation.red-gate.com/fd). +При инициализации `JdbcDatabase` модуль вызывает `Flyway.migrate()` с настройками из секции `flyway`. +Миграции запускает `FlywayJdbcDatabaseInterceptor`, который предоставляется модулем `FlywayJdbcDatabaseModule`. +`Flyway` подключен к `SLF4J` (`loggers("slf4j")`), поэтому вывод миграций и строка с замером времени `FlyWay migration applied in ...` (журналируемая на уровне `INFO`) попадают в обычные логи приложения. ### Подключение { #dependency } @@ -38,13 +48,14 @@ agent: interface Application : FlywayJdbcDatabaseModule ``` -Требует подключения [JDBC модуля](database-jdbc.md). +Требует подключения [`JDBC`-модуля](database-jdbc.md), так как миграции выполняются через `DataSource`. +В приложении обычно подключаются оба модуля: `JdbcDatabaseModule` создает `JdbcDatabase`, а `FlywayJdbcDatabaseModule` добавляет перехватчик миграций. ### Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `FlywayConfig` (указаны значения по умолчанию): +Пример полной конфигурации, описанной в классе `FlywayConfig`: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript flyway { @@ -57,16 +68,15 @@ agent: } ``` - 1. Включена ли миграция базы данных при старте приложения. Если `false`, миграции выполняться не будут. - 2. Пути директорий где искать скрипты миграции - 3. Выполнять ли миграции внутри транзакции - 4. Проверять ли контрольные суммы существующих миграций перед выполнением. Если не совпадают — будет ошибка - 5. Разрешать ли смешивание транзакционных и нетранзакционных SQL-операций в одной миграции. Если включено, - вся миграция будет выполняться **без транзакции**, чтобы избежать ошибок в БД, где некоторые операции не могут - выполняться внутри транзакции. - Эта настройка актуальна только для СУБД, которые не поддерживают выполнение отдельных операций внутри транзакции: - PostgreSQL, Aurora PostgreSQL, SQL Server и SQLite. - 6. Дополнительные свойства конфигурации в формате ключ-значение для `Flyway#configurationProperties` + 1. Включает выполнение миграций при инициализации `JdbcDatabase` (по умолчанию: `true`). Если указать `false`, модуль пропустит вызов `Flyway.migrate()`. + 2. Пути к директориям со скриптами миграций (по умолчанию: `["db/migration"]`). + 3. Выполняет миграции внутри транзакции, если это поддерживается базой данных и самими операциями `SQL` (по умолчанию: `true`). + 4. Проверяет контрольные суммы уже примененных миграций перед выполнением новых (по умолчанию: `true`). Если контрольные суммы не совпадают, запуск завершится ошибкой. + 5. Разрешает смешивать транзакционные и нетранзакционные операции `SQL` в одной миграции (по умолчанию: `false`). + Если настройка включена, вся миграция выполняется **без транзакции**, чтобы избежать ошибок в базах данных, где часть операций нельзя выполнять внутри транзакции. + Настройка актуальна для баз данных, которые не поддерживают выполнение отдельных операций внутри транзакции: PostgreSQL, Aurora PostgreSQL, SQL Server и SQLite. + 6. Дополнительные свойства `Flyway` в формате ключ-значение (по умолчанию: `{}`). + Через них можно передать настройки, у которых нет отдельной опции конфигурации Kora, например `schemas`, `baselineOnMigrate`, `placeholderReplacement` или `placeholders.*`. === ":simple-yaml: `YAML`" @@ -80,20 +90,44 @@ agent: configurationProperties: {} #(6)! ``` - 1. Включена ли миграция базы данных при старте приложения. Если `false`, миграции выполняться не будут. - 2. Пути директорий где искать скрипты миграции - 3. Выполнять ли миграции внутри транзакции - 4. Проверять ли контрольные суммы существующих миграций перед выполнением. Если не совпадают — будет ошибка - 5. Разрешать ли смешивание транзакционных и нетранзакционных SQL-операций в одной миграции. Если включено, - вся миграция будет выполняться **без транзакции**, чтобы избежать ошибок в БД, где некоторые операции не могут - выполняться внутри транзакции. - Эта настройка актуальна только для СУБД, которые не поддерживают выполнение отдельных операций внутри транзакции: - PostgreSQL, Aurora PostgreSQL, SQL Server и SQLite. - 6. Дополнительные свойства конфигурации в формате ключ-значение для `Flyway#configurationProperties` + 1. Включает выполнение миграций при инициализации `JdbcDatabase` (по умолчанию: `true`). Если указать `false`, модуль пропустит вызов `Flyway.migrate()`. + 2. Пути к директориям со скриптами миграций (по умолчанию: `["db/migration"]`). + 3. Выполняет миграции внутри транзакции, если это поддерживается базой данных и самими операциями `SQL` (по умолчанию: `true`). + 4. Проверяет контрольные суммы уже примененных миграций перед выполнением новых (по умолчанию: `true`). Если контрольные суммы не совпадают, запуск завершится ошибкой. + 5. Разрешает смешивать транзакционные и нетранзакционные операции `SQL` в одной миграции (по умолчанию: `false`). + Если настройка включена, вся миграция выполняется **без транзакции**, чтобы избежать ошибок в базах данных, где часть операций нельзя выполнять внутри транзакции. + Настройка актуальна для баз данных, которые не поддерживают выполнение отдельных операций внутри транзакции: PostgreSQL, Aurora PostgreSQL, SQL Server и SQLite. + 6. Дополнительные свойства `Flyway` в формате ключ-значение (по умолчанию: `{}`). + Через них можно передать настройки, у которых нет отдельной опции конфигурации Kora, например `schemas`, `baselineOnMigrate`, `placeholderReplacement` или `placeholders.*`. + +### Файлы миграций { #flyway-files } + +По умолчанию `Flyway` ищет миграции в `src/main/resources/db/migration`. +Обычный файл миграции имеет имя вида `V1__init_schema.sql`, где `V1` — версия, а часть после двойного подчеркивания — описание. + +```text +src/main/resources/db/migration/ + V1__init_users.sql + V2__add_user_status.sql +``` + +Пример простой миграции: + +```sql +CREATE TABLE users ( + id BIGSERIAL PRIMARY KEY, + name TEXT NOT NULL +); +``` + +При запуске `Flyway` создает служебную таблицу истории миграций и применяет только новые версии. +Если включена проверка `validateOnMigrate`, уже примененные файлы нельзя менять без отдельного процесса исправления истории миграций. ## Liquibase { #liquibase } Модуль для миграции базы данных с помощью инструмента [Liquibase](https://www.liquibase.com/supported-databases). +При инициализации `JdbcDatabase` модуль получает соединение из `DataSource`, создает экземпляр `Liquibase` и вызывает `update()`. +Миграции запускает `LiquibaseJdbcDatabaseInterceptor`, который предоставляется модулем `LiquibaseJdbcDatabaseModule`. ### Подключение { #dependency-2 } @@ -123,13 +157,14 @@ agent: interface Application : LiquibaseJdbcDatabaseModule ``` -Требует подключения [JDBC модуля](database-jdbc.md). +Требует подключения [`JDBC`-модуля](database-jdbc.md), так как миграции выполняются через `DataSource`. +В приложении обычно подключаются оба модуля: `JdbcDatabaseModule` создает `JdbcDatabase`, а `LiquibaseJdbcDatabaseModule` добавляет перехватчик миграций. ### Конфигурация { #configuration-2 } -Пример полной конфигурации, описанной в классе `LiquibaseConfig` (указаны значения по умолчанию): +Пример полной конфигурации, описанной в классе `LiquibaseConfig`: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript liquibase { @@ -137,7 +172,7 @@ agent: } ``` - 1. Путь до [мастер файла](https://docs.liquibase.com/concepts/changelogs/home.html) конфигурации миграций + 1. Путь к основному файлу [`changelog`](https://docs.liquibase.com/concepts/changelogs/home.html) с описанием миграций (по умолчанию: `db/changelog/db.changelog-master.xml`). === ":simple-yaml: `YAML`" @@ -146,16 +181,60 @@ agent: changelog: "db/changelog/db.changelog-master.xml" #(1)! ``` - 1. Путь до [мастер файла](https://docs.liquibase.com/concepts/changelogs/home.html) конфигурации миграций + 1. Путь к основному файлу [`changelog`](https://docs.liquibase.com/concepts/changelogs/home.html) с описанием миграций (по умолчанию: `db/changelog/db.changelog-master.xml`). + +В отличие от `Flyway`, у модуля `Liquibase` нет настройки `enabled`: если модуль подключен к графу приложения, миграции запускаются при инициализации `JdbcDatabase`. +Если миграция `Liquibase` завершается ошибкой, модуль оборачивает ее в `IllegalStateException`, и запуск приложения прерывается. + +### Файлы миграций { #liquibase-files } + +По умолчанию `Liquibase` ищет основной файл `changelog` в `src/main/resources/db/changelog/db.changelog-master.xml`. +`Liquibase` поддерживает разные форматы `changelog`, но в `SQL`-ориентированном проекте часто удобнее хранить миграции в форматированном `SQL`. +Основной файл может подключать такие миграции через `include`. + +```text +src/main/resources/db/changelog/ + db.changelog-master.xml + changes/ + 001-init-users.sql +``` + +Минимальный основной `changelog`: + +```xml + + + + +``` + +Пример подключенной миграции в форматированном `SQL`: + +```sql +--liquibase formatted sql + +--changeset app:001-init-users +CREATE TABLE users ( + id BIGSERIAL PRIMARY KEY, + name TEXT NOT NULL +); +``` + +## Рекомендации { #recommendations } + +???+ warning "Рекомендация" -## Совет { #recommendations } + **Модули миграций не рекомендуется** использовать для выполнения миграций на старте приложения в горизонтально масштабируемых окружениях, + где приложение запускается в нескольких репликах. Каждая реплика будет пытаться выполнить миграции при запуске. + Также учитывайте, что каждый перезапуск приложения снова приводит к запуску механизма миграций. -???+ warning "Совет" + В таких случаях для локальной разработки используйте [Flyway Gradle Plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway), + для тестов — запуск `Flyway` из кода после старта базы данных, + для промышленного окружения Kubernetes — [Kubernetes Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/), + либо выполняйте миграции отдельно из `CI`. - **Мы не советуем** использовать модули миграции для работы приложений в окружении где есть горизонтальное масштабирование - посредствам увелечения количества реплик рабочего приложения. Так как это будет вести за собой вызов миграции на каждую запущенную реплику. - Также имейте в виду что каждый перезапуск приложения также будет вызывать миграции. - В таких случаях советуем например использовать для локальной разработки [Flyway Gradle plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway), - для тестов использовать запуск Flyway из кода после запуска базы данных, для боевого окружения Kubernetes использовать [K8S Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) - либо миграцию из CI через [Flyway Gradle plugin](https://plugins.gradle.org/plugin/org.flywaydb.flyway). diff --git a/mkdocs/docs/ru/documentation/database-r2dbc.md b/mkdocs/docs/ru/documentation/database-r2dbc.md index 0e49b11..1acdc87 100644 --- a/mkdocs/docs/ru/documentation/database-r2dbc.md +++ b/mkdocs/docs/ru/documentation/database-r2dbc.md @@ -1,11 +1,17 @@ --- -description: "Explains Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and signatures. Use when working with @Repository, @Query, @EntityR2dbc, @Table, @Id, @Column, R2dbcDatabaseModule, R2dbcConnectionFactory." +description: "Explains Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and repository method signatures. Use when working with @Repository, @Query, @Table, @Id, @Column, @Batch, R2dbcDatabaseModule, R2dbcConnectionFactory." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and signatures; key triggers include @Repository, @Query, @EntityR2dbc, @Table, @Id, @Column, R2dbcDatabaseModule, R2dbcConnectionFactory, R2dbcRepository." + use_when: "Use this file for Kora docs or implementation questions about Kora R2DBC repositories, reactive database configuration, result and parameter mapping, transactions, generated identifiers, and repository method signatures; key triggers include @Repository, @Query, @Table, @Id, @Column, @Batch, R2dbcDatabaseModule, R2dbcConnectionFactory, R2dbcRepository." --- -Модуль предоставляет реализацию репозиториев на основе [R2DBC](https://r2dbc.io/) реактивного протокола работы с базами данных, -реализацией как пример является [Postgres R2DBC](https://github.com/pgjdbc/r2dbc-postgresql). +Модуль предоставляет реализацию репозиториев на основе реактивного протокола баз данных [R2DBC](https://r2dbc.io/); +реализацией драйвера может быть, например, [Postgres R2DBC](https://github.com/pgjdbc/r2dbc-postgresql). +Для управления соединениями используется пул соединений [io.r2dbc.pool](https://github.com/r2dbc/r2dbc-pool). +Вы описываете интерфейс репозитория и `SQL`-запросы через `@Repository` и `@Query`, а `Kora` генерирует реализацию, +которая получает реактивное соединение из пула, подставляет параметры, преобразует `Flux` и участвует в транзакциях. + +Общие правила для отображений, `@Repository`, `@Query`, `@Batch`, `UpdateCount`, макросов, ручных запросов и других +механизмов репозиториев описаны в разделе [Общие правила баз данных](database-common.md). ## Подключение { #dependency } @@ -39,9 +45,9 @@ agent: ## Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `R2dbcDatabaseConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации R2DBC: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript db { @@ -50,60 +56,14 @@ agent: password = "postgres" //(3)! poolName = "kora" //(4)! maxPoolSize = 10 //(5)! - minIdle = 0 //(6)! - acquireRetry = 3 //(7)! - connectionTimeout = "10s" //(8)! - connectionCreateTimeout = "30s" //(9)! - idleTimeout = "10m" //(10)! - maxLifetime = "0s" //(11)! - statementTimeout = "0s" //(12)! - readinessProbe = false //(13)! - options { //(14)! - "backgroundEvictionInterval": "PT120S" - } - telemetry { - logging { - enabled = false //(15)! - } - metrics { - enabled = true //(16)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(17)! - tags = { // (18)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(19)! - attributes = { // (20)! - "key1" = "value1" - "key2" = "value2" - } - } - } } ``` - 1. R2DBC URL подключения к базе данных (**обязательный**) - 2. Имя пользователя для подключения (**обязательный**) - 3. Пароль пользователя для подключения (**обязательный**) - 4. Имя набора соединений к базе данных (**обязательный**) - 5. Максимальный размер набора соединений к базе данных - 6. Минимальный размер набора готовых соединений к базе данных в режиме ожидания - 7. Максимальное количество попыток получения соединения - 8. Максимальное время на установку соединения - 9. Максимальное время на создание соединения - 10. Максимальное время на простой соединения - 11. Максимальное время жизни соединения (по умолчанию отсутвует) - 12. Максимальное время на выполнение запроса в базу данных (по умолчанию отсутвует) - 13. Включить ли [пробу готовности](probes.md#readiness) для соединения базы данных - 14. Дополнительные атрибуты R2DBC соединения (по умолчанию отсутвует) - 15. Включает логгирование модуля (по умолчанию `false`) - 16. Включает метрики модуля (по умолчанию `true`) - 17. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 18. Настройка тегов для метрик (опционально) - 19. Включает трассировку модуля (по умолчанию `true`) - 20. Настройка атрибутов для трассировки (опционально) + 1. `R2DBC URL` подключения к базе данных (`обязательный`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию не указано) + 4. Имя пула соединений (`обязательный`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) === ":simple-yaml: `YAML`" @@ -114,76 +74,203 @@ agent: password: "postgres" #(3)! poolName: "kora" #(4)! maxPoolSize: 10 #(5)! - minIdle: 0 #(6)! - acquireRetry: 3 #(7)! - connectionTimeout: "10s" #(8)! - connectionCreateTimeout: "30s" #(9)! - idleTimeout: "10m" #(10)! - maxLifetime: "0s" #(11)! - statementTimeout: "0ms" #(12)! - readinessProbe: false #(13)! - options: #(14)! - backgroundEvictionInterval: "PT120S" - telemetry: - logging: - enabled: false #(15)! - metrics: - enabled: true #(16)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(17)! - tags: #(18)! - key1: value1 - key2: value2 - tracing: - enabled: true #(19)! - attributes: #(20)! - key1: value1 - key2: value2 ``` - 1. R2DBC URL подключения к базе данных (**обязательный**) - 2. Имя пользователя для подключения (**обязательный**) - 3. Пароль пользователя для подключения (**обязательный**) - 4. Имя набора соединений к базе данных (**обязательный**) - 5. Максимальный размер набора соединений к базе данных - 6. Минимальный размер набора готовых соединений к базе данных в режиме ожидания - 7. Максимальное количество попыток получения соединения - 8. Максимальное время на установку соединения - 9. Максимальное время на создание соединения - 10. Максимальное время на простой соединения - 11. Максимальное время жизни соединения (по умолчанию отсутвует) - 12. Максимальное время на выполнение запроса в базу данных (по умолчанию отсутвует) - 13. Включить ли [пробу готовности](probes.md#readiness) для соединения базы данных - 14. Дополнительные атрибуты R2DBC соединения (по умолчанию отсутвует) - 15. Включает логгирование модуля (по умолчанию `false`) - 16. Включает метрики модуля (по умолчанию `true`) - 17. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 18. Настройка тегов для метрик (опционально) - 19. Включает трассировку модуля (по умолчанию `true`) - 20. Настройка атрибутов для трассировки (опционально) + 1. `R2DBC URL` подключения к базе данных (`обязательный`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию не указано) + 4. Имя пула соединений (`обязательный`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `R2dbcDatabaseConfig`: + + ===! ":material-code-json: `HOCON`" + + ```javascript + db { + r2dbcUrl = "r2dbc:postgresql://localhost:5432/postgres" //(1)! + username = "postgres" //(2)! + password = "postgres" //(3)! + poolName = "kora" //(4)! + maxPoolSize = 10 //(5)! + minIdle = 0 //(6)! + acquireRetry = 3 //(7)! + connectionTimeout = "10s" //(8)! + connectionCreateTimeout = "30s" //(9)! + idleTimeout = "10m" //(10)! + maxLifetime = "0s" //(11)! + statementTimeout = "0s" //(12)! + readinessProbe = false //(13)! + options { //(14)! + "backgroundEvictionInterval": "PT120S" + } + telemetry { + logging { + enabled = false //(15)! + } + metrics { + enabled = true //(16)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(17)! + tags = { // (18)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(19)! + attributes = { // (20)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. `R2DBC URL` подключения к базе данных (`обязательная`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательная`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательная`, по умолчанию не указано) + 4. Имя пула соединений (`обязательная`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) + 6. Минимальное количество готовых соединений в пуле в режиме ожидания (по умолчанию: `0`) + 7. Максимальное количество попыток получить соединение (по умолчанию: `3`) + 8. Максимальное время получения соединения из пула (по умолчанию: `10s`) + 9. Максимальное время создания нового физического соединения (по умолчанию: `30s`) + 10. Максимальное время простоя соединения (по умолчанию: `10m`) + 11. Максимальное время жизни соединения, `0s` означает отсутствие ограничения (по умолчанию: `0s`) + 12. Максимальное время выполнения запроса к базе данных (по умолчанию не указано, необязательно) + 13. Включить ли [пробу готовности](probes.md#readiness) для соединения с базой данных (по умолчанию: `false`) + 14. Дополнительные параметры соединения `R2DBC`, передаваемые в драйвер (по умолчанию: `{}`) + 15. Включает логирование модуля (по умолчанию: `false`) + 16. Включает метрики модуля (по умолчанию: `true`) + 17. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Настройка тегов для метрик (по умолчанию: `{}`) + 19. Включает трассировку модуля (по умолчанию: `true`) + 20. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + db: + r2dbcUrl: "r2dbc:postgresql://localhost:5432/postgres" #(1)! + username: "postgres" #(2)! + password: "postgres" #(3)! + poolName: "kora" #(4)! + maxPoolSize: 10 #(5)! + minIdle: 0 #(6)! + acquireRetry: 3 #(7)! + connectionTimeout: "10s" #(8)! + connectionCreateTimeout: "30s" #(9)! + idleTimeout: "10m" #(10)! + maxLifetime: "0s" #(11)! + statementTimeout: "0s" #(12)! + readinessProbe: false #(13)! + options: #(14)! + backgroundEvictionInterval: "PT120S" + telemetry: + logging: + enabled: false #(15)! + metrics: + enabled: true #(16)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(17)! + tags: #(18)! + key1: value1 + key2: value2 + tracing: + enabled: true #(19)! + attributes: #(20)! + key1: value1 + key2: value2 + ``` + + 1. `R2DBC URL` подключения к базе данных (`обязательная`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательная`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательная`, по умолчанию не указано) + 4. Имя пула соединений (`обязательная`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) + 6. Минимальное количество готовых соединений в пуле в режиме ожидания (по умолчанию: `0`) + 7. Максимальное количество попыток получить соединение (по умолчанию: `3`) + 8. Максимальное время получения соединения из пула (по умолчанию: `10s`) + 9. Максимальное время создания нового физического соединения (по умолчанию: `30s`) + 10. Максимальное время простоя соединения (по умолчанию: `10m`) + 11. Максимальное время жизни соединения, `0s` означает отсутствие ограничения (по умолчанию: `0s`) + 12. Максимальное время выполнения запроса к базе данных (по умолчанию не указано, необязательно) + 13. Включить ли [пробу готовности](probes.md#readiness) для соединения с базой данных (по умолчанию: `false`) + 14. Дополнительные параметры соединения `R2DBC`, передаваемые в драйвер (по умолчанию: `{}`) + 15. Включает логирование модуля (по умолчанию: `false`) + 16. Включает метрики модуля (по умолчанию: `true`) + 17. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Настройка тегов для метрик (по умолчанию: `{}`) + 19. Включает трассировку модуля (по умолчанию: `true`) + 20. Настройка атрибутов для трассировки (по умолчанию: `{}`) ## Использование { #usage } +`R2DBC`-репозиторий объявляется интерфейсом с аннотацией `@Repository` и должен наследовать `R2dbcRepository`. +Каждый метод с `@Query` содержит обычный `SQL`-запрос. Параметры метода подставляются в запрос по имени через +синтаксис `:parameter`, а поля объекта можно указывать через `:entity.field`. + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends R2dbcRepository { } + public interface EntityRepository extends R2dbcRepository { + + @Query("SELECT id, name FROM entities WHERE id = :id") + Mono findById(String id); + + @Query("SELECT id, name FROM entities") + Flux findAll(); + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + Mono insert(Entity entity); + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + Mono insertBatch(@Batch List entities); + } ``` === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : R2dbcRepository + interface EntityRepository : R2dbcRepository { + + @Query("SELECT id, name FROM entities WHERE id = :id") + fun findById(id: String): Mono + + @Query("SELECT id, name FROM entities") + fun findAll(): Flux + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insert(entity: Entity): Mono + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insertBatch(@Batch entities: List): Mono + } ``` -## Конвертация { #mapping } +`SQL` остается под контролем разработчика: можно использовать специфичные возможности конкретной базы данных, а `Kora` +занимается только безопасной подстановкой параметров, выполнением запроса и преобразованием результата. +Общие правила отображений, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch` и макросов описаны в разделе +[Общие правила баз данных](database-common.md). + +Реактивные возвращаемые значения `Mono` и `Flux` являются нативными сигнатурами для этого модуля. +Блокирующие возвращаемые значения, такие как `Entity`, `List`, `void` и `UpdateCount`, также поддерживаются, но они +блокируют вызывающий поток до завершения реактивного результата, поэтому в реактивном контексте предпочтительнее использовать +реактивные сигнатуры. + +## Преобразование { #mapping } -Возможно переопределять преобразование различных частей [сущности](database-common.md) и параметров запроса, для этого Kora предоставляет специальные интерфейсы. +Можно переопределять преобразование разных частей [отображения](database-common.md), результата и параметров запроса. +Для этого `Kora` предоставляет несколько интерфейсов преобразователей. ### Результат { #result } -Если требуется преобразовать результат вручную, предлагается использовать `R2dbcResultFluxMapper`: +Используйте `R2dbcResultFluxMapper`, когда требуется управлять всем `Flux`. +Такой преобразователь получает весь реактивный поток результата и сам решает, как его прочитать и что вернуть. ===! ":fontawesome-brands-java: `Java`" @@ -192,7 +279,8 @@ agent: @Override public Flux apply(Flux resultFlux) { - // код преобразования + return resultFlux.flatMap(result -> result.map((row, meta) -> + UUID.fromString(row.get(0, String.class)))); } } @@ -207,12 +295,12 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin class ResultMapper : R2dbcResultFluxMapper> { override fun apply(resultFlux: Flux): Flux { - // код преобразования + return resultFlux.flatMap { result -> + result.map { row, _ -> UUID.fromString(row.get(0, String::class.java)) } + } } } @@ -225,18 +313,24 @@ agent: } ``` +В большинстве случаев управлять всем `Flux` не требуется. +Достаточно предоставить [R2dbcRowMapper](#row), и `Kora` автоматически адаптирует его к типу возвращаемого значения метода: +модуль предоставляет готовые преобразователи потока результата `mono` (`Mono`), `monoList` (`Mono>`) и `flux` (`Flux`), +построенные на основе одного преобразователя строки. Также есть вспомогательный метод `R2dbcResultFluxMapper.monoOptional`, который адаптирует преобразователь строки к `Mono>`. + ### Строка { #row } -Если требуется преобразовать строку вручную, предлагается использовать `R2dbcRowMapper`: +Используйте `R2dbcRowMapper`, когда требуется вручную преобразовать одну строку результата. +Колонки читаются из `io.r2dbc.spi.Row` по индексу (начиная с `0`) или по имени: ===! ":fontawesome-brands-java: `Java`" ```java - final class RowMapper implements R2dbcRowMapper { + final class RowMapper implements R2dbcRowMapper { @Override - public UUID apply(Row row) { - return UUID.fromString(rs.get(0, String.class)); + public EntityPart apply(Row row) { + return new EntityPart(row.get(0, String.class), row.get(1, Integer.class)); } } @@ -244,20 +338,18 @@ agent: public interface EntityRepository extends R2dbcRepository { @Mapping(RowMapper.class) - @Query("SELECT id FROM entities") - Flux findAll(); + @Query("SELECT id, value1 FROM entities") + Flux findAllParts(); } ``` === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin - class RowMapper : R2dbcRowMapper { + class RowMapper : R2dbcRowMapper { - override fun apply(row: Row): UUID { - return UUID.fromString(rs.get(0, String.class)) + override fun apply(row: Row): EntityPart { + return EntityPart(row.get(0, String::class.java), row.get(1, Integer::class.java)) } } @@ -265,14 +357,14 @@ agent: interface EntityRepository : R2dbcRepository { @Mapping(RowMapper::class) - @Query("SELECT id FROM entities") - fun findAll(): Flux + @Query("SELECT id, value1 FROM entities") + fun findAllParts(): Flux } ``` ### Колонка { #column } -Если требуется преобразовать значение колонки вручную, предлагается использовать `R2dbcResultColumnMapper`: +Используйте `R2dbcResultColumnMapper`, когда требуется вручную преобразовать значение отдельной колонки по её имени: ===! ":fontawesome-brands-java: `Java`" @@ -298,13 +390,11 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin class ColumnMapper : R2dbcResultColumnMapper { override fun apply(row: Row, label: String): UUID { - return UUID.fromString(row.get(label, String.class)) + return UUID.fromString(row.get(label, String::class.java)) } } @@ -324,7 +414,8 @@ agent: ### Параметр { #parameter } -Если требуется преобразовать значение параметра запроса вручную, предлагается использовать `R2dbcParameterColumnMapper`: +Используйте `R2dbcParameterColumnMapper`, когда требуется вручную привязать значение параметра запроса к `io.r2dbc.spi.Statement`. +Значение привязывается через `stmt.bind(index, value)`, а для `null` используется `stmt.bindNull(index, type)`: ===! ":fontawesome-brands-java: `Java`" @@ -332,7 +423,7 @@ agent: public final class ParameterMapper implements R2dbcParameterColumnMapper { @Override - public void set(Statement stmt, int index, @Nullable UUID value) { + public void apply(Statement stmt, int index, @Nullable UUID value) { if (value != null) { stmt.bind(index, value.toString()); } @@ -349,12 +440,10 @@ agent: === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin class ParameterMapper : R2dbcParameterColumnMapper { - override fun set(stmt: Statement, index: Int, value: UUID?) { + override fun apply(stmt: Statement, index: Int, value: UUID?) { if (value != null) { stmt.bind(index, value.toString()) } @@ -369,11 +458,35 @@ agent: } ``` +Преобразователь колонки результата и преобразователь параметра можно сочетать на одном поле отображения. +Это удобно для преобразования, например, перечисления как при чтении строки, так и при привязке параметра: + +===! ":fontawesome-brands-java: `Java`" + + ```java + record Entity(String id, + @Mapping(FieldTypeResultMapper.class) + @Mapping(FieldTypeParameterMapper.class) + @Column("value1") FieldType field1) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class Entity( + val id: String, + @Mapping(FieldTypeResultMapper::class) + @Mapping(FieldTypeParameterMapper::class) + @Column("value1") val field1: FieldType + ) + ``` + ### Поддерживаемые типы { #supported-types } ??? abstract "Список поддерживаемых типов для аргументов/возвращаемых значений из коробки" Такие типы выбраны так как поддерживаются большинством популярных баз данных. + Для них `Kora` предоставляет встроенные преобразователи строк, колонок и параметров. * void * boolean / Boolean @@ -393,11 +506,14 @@ agent: * OffsetTime * OffsetDateTime + Для остальных типов используйте собственные преобразователи `R2dbcResultColumnMapper` / `R2dbcParameterColumnMapper` + либо `R2dbcRowMapper` / `R2dbcResultFluxMapper`. + ## Созданный идентификатор { #generated-identifier } -Если необходимо получить в качестве результата созданные базой данных первичные ключи сущности, -предлагается использовать аннотацию `@Id` над методом, где тип возвращаемого значения является идентификаторами. -Такой подход работает и для `@Batch` запросов. +Если необходимо получить в качестве результата первичные ключи, созданные базой данных, +используйте аннотацию `@Id` над методом, тип возвращаемого значения которого является идентификатором. +Такой подход работает и для `@Batch` запросов, в этом случае метод возвращает список созданных идентификаторов. ===! ":fontawesome-brands-java: `Java`" @@ -405,11 +521,15 @@ agent: @Repository public interface EntityRepository extends R2dbcRepository { - public record Entity(Long id, String name) {} + record Entity(Long id, String name) {} @Query("INSERT INTO entities(name) VALUES (:entity.name)") @Id Mono insert(Entity entity); + + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + @Id + Mono> insertBatch(@Batch List entities); } ``` @@ -419,71 +539,203 @@ agent: @Repository interface EntityRepository : R2dbcRepository { - public record Entity(Long id, String name) {} + data class Entity(val id: Long?, val name: String) @Query("INSERT INTO entities(name) VALUES (:entity.name)") @Id fun insert(entity: Entity): Mono + + @Query("INSERT INTO entities(name) VALUES (:entity.name)") + @Id + fun insertBatch(@Batch entities: List): Mono> } ``` -## Транзакции { #transactions } +В качестве альтернативы можно явно вернуть созданные колонки через `RETURNING` и преобразовать их как обычный результат, +без аннотации `@Id`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Query("INSERT INTO entities(name) VALUES (:entity.name) RETURNING id") + Mono insert(Entity entity); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Query("INSERT INTO entities(name) VALUES (:entity.name) RETURNING id") + fun insert(entity: Entity): Mono + ``` -Для выполнения ручных запросов в Kora есть интерфейс `ru.tinkoff.kora.database.r2dbc.R2dbcConnectionFactory`, -который предоставляется в методе в рамках контракта `R2dbcRepository`. -Все методы репозитория вызванные в рамках лямбды транзакции будут выполнены в этой самой транзакции. +## Ручной запрос с телеметрией { #query } -Для того чтобы выполнять запросы транзакционно, можно использовать контракт `inTx`: +Если запрос сложно выразить одним статическим `@Query`, можно сделать обычный метод с реализацией и самостоятельно собрать `SQL`. +Для выполнения такого запроса используйте `R2dbcConnectionFactory#query`. +Этот метод создает `io.r2dbc.spi.Statement`, проводит запрос через телеметрию `Kora` и использует то же соединение, что и остальные методы репозитория. +Если `query` вызывается внутри активной транзакции `inTx`, запрос будет выполнен на текущем транзакционном соединении. + +`query` принимает три аргумента: + +- `QueryContext` с идентификатором запроса и итоговым `SQL`. Идентификатор запроса попадает в телеметрию, поэтому для него удобно использовать стабильное имя вида `Repository.method`; +- `Consumer`, который привязывает значения параметров. Значения нужно привязывать через `Statement` (`stmt.bind(...)`), а не подставлять в строку запроса напрямую; +- `Function, Mono>`, который читает реактивный результат и формирует возвращаемое значение. ===! ":fontawesome-brands-java: `Java`" ```java - @Component - public final class SomeService { + @Repository + public interface EntityRepository extends R2dbcRepository { - private final EntityRepository repository; + default Mono> findByFilter(@Nullable String name, boolean onlyActive) { + var sql = new StringBuilder("SELECT id, name FROM entities WHERE 1 = 1"); + var params = new ArrayList(); - public SomeService(EntityRepository repository) { - this.repository = repository; + if (name != null) { + params.add(name); + sql.append(" AND name = $").append(params.size()); + } + if (onlyActive) { + sql.append(" AND active = true"); + } + + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return getR2dbcConnectionFactory().query( + queryContext, + statement -> { + for (int i = 0; i < params.size(); i++) { + statement.bind(i, params.get(i)); + } + }, + resultFlux -> resultFlux + .flatMap(result -> result.map((row, meta) -> + new Entity(row.get("id", String.class), row.get("name", String.class)))) + .collectList() + ); } + } + ``` - public Mono> saveAll(Entity one, Entity two) { - return repository.getR2dbcConnectionFactory().inTx(connection -> { - // do some work - return repository.insert(one) //(1)! - .zipWith(repository.insert(two), //(2)! - (r1, r2) -> List.of(one, two)); - }); +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : R2dbcRepository { + + fun findByFilter(name: String?, onlyActive: Boolean): Mono> { + val sql = StringBuilder("SELECT id, name FROM entities WHERE 1 = 1") + val params = mutableListOf() + + if (name != null) { + params += name + sql.append(" AND name = $").append(params.size) + } + if (onlyActive) { + sql.append(" AND active = true") + } + + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return r2dbcConnectionFactory.query( + queryContext, + { statement -> + params.forEachIndexed { index, value -> + statement.bind(index, value) + } + }, + { resultFlux -> + resultFlux + .flatMap { result -> result.map { row, _ -> + Entity(row.get("id", String::class.java), row.get("name", String::class.java)) + } } + .collectList() + } + ) + } + } + ``` + +## Транзакции { #transactions } + +Для выполнения ручных запросов и объединения запросов в транзакцию `Kora` предоставляет интерфейс `R2dbcConnectionFactory` +в рамках контракта `R2dbcRepository`, получаемый через `getR2dbcConnectionFactory()`. +Все методы репозитория, вызванные внутри лямбды транзакции, выполняются в этой самой транзакции. + +Для того чтобы выполнять запросы транзакционно, используйте `inTx`. +Если на текущем реактивном `Context` уже есть активная транзакция, вложенный вызов `inTx` использует то же соединение и не открывает +новую транзакцию. + +Транзакционную последовательность операций можно оставлять внутри самого репозитория с помощью обычного метода с реализацией. +Такой подход удобен, когда нужно оставить несколько `@Query`-методов или сложный самостоятельный `SQL`-запрос рядом с остальными запросами репозитория, +не вынося техническую работу с базой данных в сервисный слой. +Внутри такого метода можно использовать и `@Query`-методы репозитория, и `R2dbcConnectionFactory#query` для ручного запроса с телеметрией. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends R2dbcRepository { + + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + Mono insert(Entity entity); + + @Query("UPDATE entities SET name = :name WHERE id = :id") + Mono updateName(String id, String name); + + default Mono> saveAll(Entity one, Entity two) { + return getR2dbcConnectionFactory().inTx(connection -> + insert(one) //(1)! + .then(updateName(two.id(), two.name())) //(2)! + .thenReturn(List.of(one, two))); } } ``` - 1. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение - 2. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение + 1. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке + 2. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке === ":simple-kotlin: `Kotlin`" ```kotlin - @Component - class SomeService(private val repository: EntityRepository) { + @Repository + interface EntityRepository : R2dbcRepository { - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.r2dbcConnectionFactory.inTx { - repository.insert(one).zipWith(repository.insert(two)) //(1)! - { r1: UpdateCount, r2: UpdateCount -> listOf(one, two) } + @Query("INSERT INTO entities(id, name) VALUES (:entity.id, :entity.name)") + fun insert(entity: Entity): Mono + + @Query("UPDATE entities SET name = :name WHERE id = :id") + fun updateName(id: String, name: String): Mono + + fun saveAll(one: Entity, two: Entity): Mono> { + return r2dbcConnectionFactory.inTx { _ -> + insert(one) //(1)! + .then(updateName(two.id, two.name)) //(2)! + .thenReturn(listOf(one, two)) } } } ``` - 1. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение + 1. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке + 2. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке + +Транзакция фиксируется при успешном завершении возвращаемого `Mono`. +Если `Mono` сигнализирует об ошибке, транзакция откатывается, а ошибка пробрасывается дальше, поэтому все изменения в базе данных, +сделанные в рамках транзакции, не применяются. + +### Ручное управление соединением { #connection } + +Если для запроса нужна более сложная логика или запросы вне репозитория, можно работать напрямую с `io.r2dbc.spi.Connection`. +Метод `withConnection` выполняет код с соединением, но сам по себе не открывает транзакцию. -### Ручное управление { #connection } +`withConnection` работает так: -Если для запроса нужна какая-то более сложная логика, либо запросы вне репозитория, можно использовать `io.r2dbc.spi.Connection`: +- если в текущем реактивном `Context` уже есть соединение, метод передает в лямбду это текущее соединение; +- если соединения в текущем `Context` нет, метод берет новое соединение из пула, кладет его в `Context` на время выполнения лямбды и закрывает после завершения; +- повторные вызовы `withConnection`, `R2dbcConnectionFactory#query` и методы репозитория внутри этой лямбды используют то же текущее соединение. + +`withConnection` возвращает `Mono`. Для результатов, которые естественным образом являются потоком строк, используйте `withConnectionFlux` — +вариант с возвратом `Flux` и той же семантикой работы с соединением. +Метод `inTx` открывает транзакцию и построен поверх `withConnection`. ===! ":fontawesome-brands-java: `Java`" @@ -497,9 +749,15 @@ agent: this.repository = repository; } - public Mono> saveAll(Entity one, Entity two) { - return repository.getR2dbcConnectionFactory().inTx(connection -> { - // do some work + public Mono> loadAll() { + return repository.getR2dbcConnectionFactory().withConnection(connection -> { + // do some work, returns Mono + }); + } + + public Flux streamAll() { + return repository.getR2dbcConnectionFactory().withConnectionFlux(connection -> { + // do some work, returns Flux }); } } @@ -511,22 +769,72 @@ agent: @Component class SomeService(private val repository: EntityRepository) { - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.r2dbcConnectionFactory.inTx { connection -> - // do some work + fun loadAll(): Mono> { + return repository.r2dbcConnectionFactory.withConnection { connection -> + // do some work, returns Mono + } + } + + fun streamAll(): Flux { + return repository.r2dbcConnectionFactory.withConnectionFlux { connection -> + // do some work, returns Flux } } } ``` -## Сигнатуры { #signatures } +## Выборка по списку { #select-by-list } + +Иногда требуется выборка строк по списку значений. +`Kora` старается делать преобразования во время компиляции и не переписывать `SQL` во время работы, поэтому для параметра-списка +нужен собственный преобразователь, который привязывает всю коллекцию как одно значение. + +Пример ниже показывает `Postgres` через массив, привязываемый с помощью `ANY(:ids)`. +Обратите внимание, что многие драйверы `R2DBC` (например, `Postgres`) могут привязывать Java-массив напрямую, поэтому такой преобразователь часто короткий: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + class ListOfStringR2dbcParameterMapper implements R2dbcParameterColumnMapper> { + + @Override + public void apply(Statement stmt, int index, @Nullable List value) { + stmt.bind(index, value.toArray(String[]::new)); + } + } + + @Repository + public interface EntityRepository extends R2dbcRepository { -Под `T` подразумевается тип возвращаемого значения, либо `Void`, либо `UpdateCount`. + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + Flux findAllByIds(@Mapping(ListOfStringR2dbcParameterMapper.class) List ids); + } + ``` -Доступные сигнатуры для методов репозитория из коробки: +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ListOfStringR2dbcParameterMapper : R2dbcParameterColumnMapper> { + + override fun apply(stmt: Statement, index: Int, value: List?) { + stmt.bind(index, value!!.toTypedArray()) + } + } + + @Repository + interface EntityRepository : R2dbcRepository { + + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + fun findAllByIds(@Mapping(ListOfStringR2dbcParameterMapper::class) ids: List): Flux + } + ``` + +## Сигнатуры { #signatures } + +Доступные сигнатуры для методов репозитория из коробки. +Поскольку `R2DBC` реактивен нативно, для асинхронных сигнатур не требуется компонент `Executor`. ===! ":fontawesome-brands-java: `Java`" @@ -545,3 +853,8 @@ agent: - `myMethod(): T` - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + +## Телеметрия { #telemetry } + +Логирование, метрики и трассировка настраиваются через блок `telemetry` в [конфигурации](#configuration) и описаны в разделе [Справочник метрик](metrics.md#database). +Чтобы переопределить телеметрию полностью, можно предоставить собственные SPI-фабрики, подробнее в [Общей документации по Базам данных](database-common.md#telemetry). diff --git a/mkdocs/docs/ru/documentation/database-vertx.md b/mkdocs/docs/ru/documentation/database-vertx.md index 26b1983..fd5476c 100644 --- a/mkdocs/docs/ru/documentation/database-vertx.md +++ b/mkdocs/docs/ru/documentation/database-vertx.md @@ -1,10 +1,17 @@ --- -description: "Explains Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures. Use when working with @Repository, @Query, @EntityVertx, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory." +description: "Explains Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures. Use when working with @Repository, @Query, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures; key triggers include @Repository, @Query, @EntityVertx, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory, VertxRepository." + use_when: "Use this file for Kora docs or implementation questions about Kora Vert.x database repositories, Vert.x SQL client configuration, mapping, transactions, and repository signatures; key triggers include @Repository, @Query, @Table, @Id, @Column, VertxDatabaseModule, VertxConnectionFactory, VertxRepository." --- -Модуль предоставляет реализацию репозиториев на основе [Vertx](https://vertx.io/docs/#databases) реактивного протокола работы с базой данных. +Модуль предоставляет реализацию репозиториев на основе реактивного `SQL`-клиента [Vert.x](https://vertx.io/docs/#databases). +[Пул](https://vertx.io/docs/vertx-pg-client/java/#_using_connection_pool) соединений Vert.x работает поверх транспорта +[Netty](netty.md). Вы описываете интерфейс репозитория и `SQL`-запросы через `@Repository` и `@Query`, а `Kora` генерирует +реализацию, которая привязывает Vert.x `Tuple`, выполняет подготовленный запрос через телеметрию, преобразует +`RowSet` и участвует в транзакциях. + +Общие правила для отображений, `@Repository`, `@Query`, `@Batch`, `UpdateCount`, макросов и других механизмов +репозиториев описаны в разделе [Общие правила баз данных](database-common.md). ## Подключение { #dependency } @@ -34,15 +41,33 @@ agent: interface Application : VertxDatabaseModule ``` -Также **требуется предоставить** реализацию драйвера как зависимость версии не выше [4.3.8](https://mvnrepository.com/artifact/io.vertx/vertx-pg-client/4.3.8) +Также **требуется предоставить** реализацию драйвера Vert.x как зависимость, версии не выше +[4.3.8](https://mvnrepository.com/artifact/io.vertx/vertx-pg-client/4.3.8), например +[vertx-pg-client](https://mvnrepository.com/artifact/io.vertx/vertx-pg-client/4.3.8) для `PostgreSQL`: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + implementation "io.vertx:vertx-pg-client:4.3.8" + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + implementation("io.vertx:vertx-pg-client:4.3.8") + ``` -В отдельных случаях как например с [PostgreSQL](https://postgrespro.ru/docs/postgresql), требуется также добавить [зависимость](https://mvnrepository.com/artifact/com.ongres.scram/client/2.1) +Для базы данных [PostgreSQL](https://postgrespro.ru/docs/postgresql), использующей аутентификацию `SCRAM`, также +необходимо добавить зависимость [com.ongres.scram:client](https://mvnrepository.com/artifact/com.ongres.scram/client/2.1). + +Зависимость [io.projectreactor:reactor-core](https://mvnrepository.com/artifact/io.projectreactor/reactor-core) +требуется только если вы используете сигнатуры методов `Mono`/`Flux`. ## Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `VertxDatabaseConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации Vert.x: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript db { @@ -51,136 +76,259 @@ agent: password = "postgres" //(3)! poolName = "kora" //(4)! maxPoolSize = 10 //(5)! - connectionTimeout = "10s" //(6)! - acquireTimeout = "0s" //(7)! - idleTimeout = "10m" //(8)! - cachePreparedStatements = true //(9)! - initializationFailTimeout = "0s" //(10)! - readinessProbe = false //(11)! - telemetry { - logging { - enabled = false //(12)! - } - metrics { - enabled = true //(13)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(14)! - tags = { // (15)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(16)! - attributes = { // (17)! - "key1" = "value1" - "key2" = "value2" - } - } - } } ``` - 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) подключения к базе данных (**обязательный**) - 2. Имя пользователя для подключения (**обязательный**) - 3. Пароль пользователя для подключения (**обязательный**) - 4. Имя набора соединений к базе данных (**обязательный**) - 5. Максимальный размер набора соединений к базе данных - 6. Максимальное время на установку соединения - 7. Максимальное время на получение соединения из набора соединений (по умолчанию отсутвует) - 7. Максимальное время на простой соединения - 9. Кэшировать ли подготовленные запросы - 10. Максимальное время ожидания инициализации соединения при старте сервиса (по умолчанию отсутвует) - 11. Включить ли [пробу готовности](probes.md#readiness) для соединения базы данных - 12. Включает логгирование модуля (по умолчанию `false`) - 13. Включает метрики модуля (по умолчанию `true`) - 14. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 15. Настройка тегов для метрик (опционально) - 16. Включает трассировку модуля (по умолчанию `true`) - 17. Настройка атрибутов для трассировки (опционально) + 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) подключения к базе данных (`обязательный`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию не указано) + 4. Имя пула соединений (`обязательный`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) === ":simple-yaml: `YAML`" ```yaml db: - connectionUri = "postgresql://localhost:5432/postgres" #(1)! + connectionUri: "postgresql://localhost:5432/postgres" #(1)! username: "postgres" #(2)! password: "postgres" #(3)! poolName: "kora" #(4)! maxPoolSize: 10 #(5)! - connectionTimeout: "10s" #(6)! - acquireTimeout: "10s" #(7)! - idleTimeout: "10m" #(8)! - cachePreparedStatements: true #(9)! - initializationFailTimeout: "0s" #(10)! - readinessProbe: false #(11)! - telemetry: - logging: - enabled: false #(12)! - metrics: - enabled: true #(13)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(14)! - tags: #(15)! - key1: value1 - key2: value2 - tracing: - enabled: true #(16)! - attributes: #(17)! - key1: value1 - key2: value2 ``` - 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) подключения к базе данных (**обязательный**) - 2. Имя пользователя для подключения (**обязательный**) - 3. Пароль пользователя для подключения (**обязательный**) - 4. Имя набора соединений к базе данных (**обязательный**) - 5. Максимальный размер набора соединений к базе данных - 6. Максимальное время на установку соединения - 7. Максимальное время на получение соединения из набора соединений (по умолчанию отсутвует) - 7. Максимальное время на простой соединения - 9. Кэшировать ли подготовленные запросы - 10. Максимальное время ожидания инициализации соединения при старте сервиса (по умолчанию отсутвует) - 11. Включить ли [пробу готовности](probes.md#readiness) для соединения базы данных - 12. Включает логгирование модуля (по умолчанию `false`) - 13. Включает метрики модуля (по умолчанию `true`) - 14. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 15. Настройка тегов для метрик (опционально) - 16. Включает трассировку модуля (по умолчанию `true`) - 17. Настройка атрибутов для трассировки (опционально) - -Можно также настроить [Netty транспорт](netty.md). + 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) подключения к базе данных (`обязательный`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательный`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательный`, по умолчанию не указано) + 4. Имя пула соединений (`обязательный`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `VertxDatabaseConfig` (указаны примеры значений или значения по умолчанию): + + ===! ":material-code-json: `HOCON`" + + ```javascript + db { + connectionUri = "postgresql://localhost:5432/postgres" //(1)! + username = "postgres" //(2)! + password = "postgres" //(3)! + poolName = "kora" //(4)! + maxPoolSize = 10 //(5)! + connectionTimeout = "10s" //(6)! + acquireTimeout = "10s" //(7)! + idleTimeout = "10m" //(8)! + cachePreparedStatements = true //(9)! + initializationFailTimeout = "10s" //(10)! + readinessProbe = false //(11)! + telemetry { + logging { + enabled = false //(12)! + } + metrics { + enabled = true //(13)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(14)! + tags = { // (15)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(16)! + attributes = { // (17)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) подключения к базе данных (`обязательная`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательная`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательная`, по умолчанию не указано) + 4. Имя пула соединений (`обязательная`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) + 6. Максимальное время установления физического соединения (по умолчанию: `10s`) + 7. Максимальное время получения соединения из пула; если не задано, вместо него используется `connectionTimeout` (по умолчанию не указано, необязательно) + 8. Максимальное время простоя соединения (по умолчанию: `10m`) + 9. Кэшировать ли подготовленные выражения (по умолчанию: `true`) + 10. Максимальное время ожидания проверки соединения `SELECT 1` при запуске сервиса; если не задано, проверка при запуске не выполняется (по умолчанию не указано, необязательно) + 11. Включить ли [пробу готовности](probes.md#readiness) для соединения с базой данных (по умолчанию: `false`) + 12. Включает логирование модуля (по умолчанию: `false`) + 13. Включает метрики модуля (по умолчанию: `true`) + 14. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Настройка тегов для метрик (по умолчанию: `{}`) + 16. Включает трассировку модуля (по умолчанию: `true`) + 17. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + db: + connectionUri: "postgresql://localhost:5432/postgres" #(1)! + username: "postgres" #(2)! + password: "postgres" #(3)! + poolName: "kora" #(4)! + maxPoolSize: 10 #(5)! + connectionTimeout: "10s" #(6)! + acquireTimeout: "10s" #(7)! + idleTimeout: "10m" #(8)! + cachePreparedStatements: true #(9)! + initializationFailTimeout: "10s" #(10)! + readinessProbe: false #(11)! + telemetry: + logging: + enabled: false #(12)! + metrics: + enabled: true #(13)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(14)! + tags: #(15)! + key1: value1 + key2: value2 + tracing: + enabled: true #(16)! + attributes: #(17)! + key1: value1 + key2: value2 + ``` + + 1. [URI](https://vertx.io/docs/vertx-pg-client/java/#_connection_uri) подключения к базе данных (`обязательная`, по умолчанию не указано) + 2. Имя пользователя для подключения (`обязательная`, по умолчанию не указано) + 3. Пароль пользователя для подключения (`обязательная`, по умолчанию не указано) + 4. Имя пула соединений (`обязательная`, по умолчанию не указано) + 5. Максимальный размер пула соединений (по умолчанию: `10`) + 6. Максимальное время установления физического соединения (по умолчанию: `10s`) + 7. Максимальное время получения соединения из пула; если не задано, вместо него используется `connectionTimeout` (по умолчанию не указано, необязательно) + 8. Максимальное время простоя соединения (по умолчанию: `10m`) + 9. Кэшировать ли подготовленные выражения (по умолчанию: `true`) + 10. Максимальное время ожидания проверки соединения `SELECT 1` при запуске сервиса; если не задано, проверка при запуске не выполняется (по умолчанию не указано, необязательно) + 11. Включить ли [пробу готовности](probes.md#readiness) для соединения с базой данных (по умолчанию: `false`) + 12. Включает логирование модуля (по умолчанию: `false`) + 13. Включает метрики модуля (по умолчанию: `true`) + 14. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Настройка тегов для метрик (по умолчанию: `{}`) + 16. Включает трассировку модуля (по умолчанию: `true`) + 17. Настройка атрибутов для трассировки (по умолчанию: `{}`) + +Поскольку пул работает поверх транспорта [Netty](netty.md), вы также можете отдельно настроить [транспорт Netty](netty.md). ## Использование { #usage } +Репозиторий Vert.x объявляется интерфейсом с аннотацией `@Repository` и должен наследовать `VertxRepository`. +Каждый метод с аннотацией `@Query` содержит обычный `SQL`-запрос. Параметры метода подставляются в запрос по имени через +синтаксис `:parameter`, а поля объекта можно указывать через `:entity.field`. + ===! ":fontawesome-brands-java: `Java`" ```java @Repository - public interface EntityRepository extends VertxRepository { } + public interface EntityRepository extends VertxRepository { + + record Entity(String id, @Column("value1") int field1, String value2, @Nullable String value3) {} + + @Query("SELECT * FROM entities WHERE id = :id") + Mono findById(String id); + + @Query("SELECT * FROM entities") + Flux findAll(); + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + Mono insert(Entity entity); + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + Mono insertBatch(@Batch List entities); + + @Query(""" + UPDATE entities + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + WHERE id = :entity.id + """) + Mono update(Entity entity); + + @Query("DELETE FROM entities WHERE id = :id") + Mono deleteById(String id); + } ``` === ":simple-kotlin: `Kotlin`" ```kotlin @Repository - interface EntityRepository : VertxRepository + interface EntityRepository : VertxRepository { + + data class Entity(val id: String, @Column("value1") val field1: Int, val value2: String, val value3: String?) + + @Query("SELECT * FROM entities WHERE id = :id") + fun findById(id: String): Mono + + @Query("SELECT * FROM entities") + fun findAll(): Flux + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + fun insert(entity: Entity): Mono + + @Query(""" + INSERT INTO entities(id, value1, value2, value3) + VALUES (:entity.id, :entity.field1, :entity.value2, :entity.value3) + """) + fun insertBatch(@Batch entities: List): Mono + + @Query(""" + UPDATE entities + SET value1 = :entity.field1, value2 = :entity.value2, value3 = :entity.value3 + WHERE id = :entity.id + """) + fun update(entity: Entity): Mono + + @Query("DELETE FROM entities WHERE id = :id") + fun deleteById(id: String): Mono + } ``` -## Конвертация { #mapping } +`SQL` остается под контролем разработчика: можно использовать специфичные возможности конкретной базы данных, а `Kora` +занимается только безопасной подстановкой параметров, выполнением запроса и преобразованием результата. +Общие правила отображений, `@Table`, `@Column`, `@Id`, `@Embedded`, `@Batch` и макросов описаны в разделе +[Общие правила баз данных](database-common.md). + +Реактивные возвращаемые значения `Mono` и `Flux` являются нативными сигнатурами для этого модуля, поскольку клиент Vert.x +асинхронный. Блокирующие возвращаемые значения, такие как `Entity`, `List`, `void` и `UpdateCount`, также +поддерживаются, но они блокируют вызывающий поток до завершения асинхронного результата, поэтому в реактивном контексте +предпочтительнее использовать реактивные сигнатуры. + +## Преобразование { #mapping } -Возможно переопределять преобразование различных частей [сущности](database-common.md) и параметров запроса, для этого Kora предоставляет специальные интерфейсы. +Можно переопределять преобразование разных частей [отображения](database-common.md), результата запроса и параметров запроса. +Для этого `Kora` предоставляет несколько интерфейсов преобразователей. ### Результат { #result } -Если требуется преобразовать результат вручную, предлагается использовать `VertxRowSetMapper`: +Используйте `VertxRowSetMapper`, когда требуется управлять всем `io.vertx.sqlclient.RowSet`. +Такой преобразователь получает весь набор результата и сам решает, как его прочитать и что вернуть: ===! ":fontawesome-brands-java: `Java`" ```java - final class ResultMapper implements VertxRowSetMapper> { + final class ResultMapper implements VertxRowSetMapper>> { @Override - public List apply(RowSet rows) { - // код преобразования + public Map> apply(RowSet rows) { + var result = new LinkedHashMap>(rows.size()); + for (Row row : rows) { + var entityPart = new EntityPart(row.getString(0), row.getInteger(1)); + var entityParts = result.computeIfAbsent(entityPart.field1(), k -> new ArrayList<>()); + entityParts.add(entityPart); + } + return result; } } @@ -188,19 +336,22 @@ agent: public interface EntityRepository extends VertxRepository { @Mapping(ResultMapper.class) - @Query("SELECT id FROM entities") - Mono> getIds(); + @Query("SELECT id, value1 FROM entities") + Mono>> findAllParts(); } ``` === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin - class ResultMapper : VertxRowSetMapper> { - override fun apply(rows: RowSet): List { - // код преобразования + class ResultMapper : VertxRowSetMapper>> { + override fun apply(rows: RowSet): Map> { + val result = LinkedHashMap>(rows.size()) + for (row in rows) { + val entityPart = EntityPart(row.getString(0), row.getInteger(1)) + result.computeIfAbsent(entityPart.field1) { ArrayList() }.add(entityPart) + } + return result } } @@ -208,23 +359,29 @@ agent: interface EntityRepository : VertxRepository { @Mapping(ResultMapper::class) - @Query("SELECT id FROM entities") - fun getIds(): Mono> + @Query("SELECT id, value1 FROM entities") + fun findAllParts(): Mono>> } ``` +В большинстве случаев управлять всем `RowSet` не требуется. +Достаточно предоставить [VertxRowMapper](#row), и `Kora` автоматически адаптирует его к типу возвращаемого значения метода +с помощью встроенных вспомогательных методов `VertxRowSetMapper`: `singleRowSetMapper` (одиночный `T`), `listRowSetMapper` +(`List`) и `optionalRowSetMapper` (`Optional`). Также есть `VertxRowSetMapper.extractUpdateCount`, который адаптирует результат к `UpdateCount`. + ### Строка { #row } -Если требуется преобразовать строку вручную, предлагается использовать `VertxRowMapper`: +Используйте `VertxRowMapper`, когда требуется вручную преобразовать одну строку результата. +Колонки читаются из `io.vertx.sqlclient.Row` по типу и индексу (начиная с `0`): ===! ":fontawesome-brands-java: `Java`" ```java - final class RowMapper implements VertxRowMapper { + final class RowMapper implements VertxRowMapper { @Override - public UUID apply(Row row) { - return UUID.fromString(rs.get(0, String.class)); + public EntityPart apply(Row row) { + return new EntityPart(row.get(String.class, 0), row.get(Integer.class, 1)); } } @@ -232,20 +389,18 @@ agent: public interface EntityRepository extends VertxRepository { @Mapping(RowMapper.class) - @Query("SELECT id FROM entities") - Flux findAll(); + @Query("SELECT id, value1 FROM entities") + Flux findAllParts(); } ``` === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin - class RowMapper : VertxRowMapper { + class RowMapper : VertxRowMapper { - override fun apply(row: Row): UUID { - return UUID.fromString(rs.get(0, String.class)) + override fun apply(row: Row): EntityPart { + return EntityPart(row.get(String::class.java, 0), row.get(Integer::class.java, 1)) } } @@ -253,110 +408,161 @@ agent: interface EntityRepository : VertxRepository { @Mapping(RowMapper::class) - @Query("SELECT id FROM entities") - fun findAll(): Flux + @Query("SELECT id, value1 FROM entities") + fun findAllParts(): Flux } ``` ### Колонка { #column } -Если требуется преобразовать значение колонки вручную, предлагается использовать `VertxResultColumnMapper`: +Используйте `VertxResultColumnMapper`, когда требуется вручную преобразовать значение отдельной колонки по её индексу: ===! ":fontawesome-brands-java: `Java`" ```java - public final class ColumnMapper implements VertxResultColumnMapper { + public final class ColumnMapper implements VertxResultColumnMapper { + private static final Entity.FieldType[] ALL = Entity.FieldType.values(); + + @Nullable @Override - public UUID apply(Row row, int index) { - return UUID.fromString(row.get(String.class, index)); + public Entity.FieldType apply(Row row, int index) { + var fieldAsInt = row.get(Integer.class, index); + if (fieldAsInt == null) { + return null; + } + for (var type : ALL) { + if (type.code() == fieldAsInt) { + return type; + } + } + return Entity.FieldType.UNKNOWN; } } @Table("entities") - public record Entity(@Mapping(ColumnMapper.class) @Id UUID id, String name) { } + public record Entity(String id, @Mapping(ColumnMapper.class) @Column("value1") FieldType field1) { + + enum FieldType { + UNKNOWN(-10), ONE(1), TWO(2); + + private final int code; + + FieldType(int code) { this.code = code; } + + public int code() { return code; } + } + } @Repository public interface EntityRepository extends VertxRepository { - @Query("SELECT id, name FROM entities") + @Query("SELECT id, value1 FROM entities") Flux findAll(); } ``` === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin - class ColumnMapper : VertxResultColumnMapper { + class ColumnMapper : VertxResultColumnMapper { - override fun apply(row: Row, index: Int): UUID { - return UUID.fromString(row.get(String.class, index)) + override fun apply(row: Row, index: Int): Entity.FieldType? { + val fieldAsInt = row.get(Integer::class.java, index) ?: return null + return Entity.FieldType.entries.firstOrNull { it.code == fieldAsInt } ?: Entity.FieldType.UNKNOWN } } @Table("entities") data class Entity( - @Id @Mapping(ColumnMapper::class) val id: UUID, - val name: String - ) + val id: String, + @Mapping(ColumnMapper::class) @Column("value1") val field1: FieldType + ) { + enum class FieldType(val code: Int) { UNKNOWN(-10), ONE(1), TWO(2) } + } @Repository interface EntityRepository : VertxRepository { - @Query("SELECT id, name FROM entities") + @Query("SELECT id, value1 FROM entities") fun findAll(): Flux } ``` +В отличие от некоторых других модулей баз данных, здесь преобразователь колонки читает по числовому `index`, а не по имени колонки. + ### Параметр { #parameter } -Если требуется преобразовать значение параметра запроса вручную, предлагается использовать `VertxParameterColumnMapper`: +Используйте `VertxParameterColumnMapper`, когда требуется вручную преобразовать значение параметра запроса. +Преобразователь возвращает исходное значение, которое `Kora` привязывает в Vert.x `Tuple`; для значения `null` возвращайте `null`: ===! ":fontawesome-brands-java: `Java`" ```java - public final class ParameterMapper implements VertxParameterColumnMapper { + public final class ParameterMapper implements VertxParameterColumnMapper { + @Nullable @Override - public Object apply(@Nullable UUID value) { - return value.toString(); + public Object apply(@Nullable Entity.FieldType fieldType) { + return (fieldType == null) ? null : fieldType.code(); } } @Repository public interface EntityRepository extends VertxRepository { - @Query("SELECT id, name FROM entities WHERE id = :id") - Flux findById(@Mapping(ParameterMapper.class) UUID id); + @Query("UPDATE entities SET value1 = :fieldType WHERE id = :id") + UpdateCount updateFieldType(String id, @Mapping(ParameterMapper.class) Entity.FieldType fieldType); } ``` === ":simple-kotlin: `Kotlin`" - Для Kotlin писать преобразователи надо только для `T?` типов, так в интерфейсах тип указан как `@Nullable`. - ```kotlin - class ParameterMapper : VertxParameterColumnMapper { - override fun apply(value: UUID?): Any { - return value.toString() + class ParameterMapper : VertxParameterColumnMapper { + + override fun apply(fieldType: Entity.FieldType?): Any? { + return fieldType?.code } } @Repository interface EntityRepository : VertxRepository { - @Query("SELECT id, name FROM entities WHERE id = :id") - fun findById(@Mapping(ParameterMapper::class) id: UUID): Flux + @Query("UPDATE entities SET value1 = :fieldType WHERE id = :id") + fun updateFieldType(id: String, @Mapping(ParameterMapper::class) fieldType: Entity.FieldType): UpdateCount } ``` +Преобразователь колонки результата и преобразователь параметра можно сочетать на одном поле отображения. +Это удобно для преобразования, например, перечисления как при чтении строки, так и при привязке параметра: + +===! ":fontawesome-brands-java: `Java`" + + ```java + record Entity(String id, + @Mapping(FieldTypeColumnMapper.class) + @Mapping(FieldTypeParameterMapper.class) + @Column("value1") FieldType field1) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class Entity( + val id: String, + @Mapping(FieldTypeColumnMapper::class) + @Mapping(FieldTypeParameterMapper::class) + @Column("value1") val field1: FieldType + ) + ``` + ### Поддерживаемые типы { #supported-types } ??? abstract "Список поддерживаемых типов для аргументов/возвращаемых значений из коробки" Такие типы выбраны так как поддерживаются большинством популярных баз данных. + Для них `Kora` предоставляет встроенные преобразователи строк, колонок и параметров. * void * boolean / Boolean @@ -365,71 +571,97 @@ agent: * long / Long * double / Double * float / Float - * Buffer + * Buffer (`io.vertx.core.buffer.Buffer`) * String - * BigDecimal * BigInteger + * BigDecimal * UUID - * LocalTime + * LocalDate * LocalDateTime + Для остальных типов используйте собственные преобразователи `VertxResultColumnMapper` / `VertxParameterColumnMapper` + либо `VertxRowMapper` / `VertxRowSetMapper`. + ## Транзакции { #transactions } -Для выполнения ручных запросов в Kora есть интерфейс `ru.tinkoff.kora.database.vertx.VertxConnectionFactory`, -который предоставляется в методе в рамках контракта `VertxRepository`. -Все методы репозитория вызванные в рамках лямбды транзакции будут выполнены в этой самой транзакции. +Для объединения запросов в транзакцию `Kora` предоставляет интерфейс `VertxConnectionFactory` +в рамках контракта `VertxRepository`, получаемый через `getVertxConnectionFactory()`. +Все методы репозитория, вызванные внутри лямбды транзакции, выполняются в этой самой транзакции. + +Для того чтобы выполнять запросы транзакционно, используйте `inTx`. Лямбда получает `io.vertx.sqlclient.SqlConnection` +и должна вернуть `java.util.concurrent.CompletionStage`, поэтому реактивные результаты `Mono` из методов репозитория +нужно преобразовывать через `.toFuture()`. Если на текущем `Context` уже есть активная транзакция, вложенный вызов `inTx` +использует то же соединение и не открывает новую транзакцию. -Для того чтобы выполнять запросы транзакционно, можно использовать контракт `inTx`: +Транзакционную последовательность операций можно оставлять внутри самого репозитория с помощью обычного метода с реализацией. +Такой подход удобен, когда нужно оставить несколько `@Query`-методов рядом с остальными запросами репозитория, +не вынося техническую работу с базой данных в сервисный слой. ===! ":fontawesome-brands-java: `Java`" ```java - @Component - public final class SomeService { + @Repository + public interface EntityRepository extends VertxRepository { - private final EntityRepository repository; + @Query("INSERT INTO entities(id, value2) VALUES (:entity.id, :entity.value2)") + Mono insert(Entity entity); - public SomeService(EntityRepository repository) { - this.repository = repository; - } + @Query("UPDATE entities SET value2 = :value2 WHERE id = :id") + Mono updateValue(String id, String value2); - public Mono> saveAll(Entity one, Entity two) { - return repository.getVertxConnectionFactory().inTx(connection -> { - // do some work - return repository.insert(one) //(1)! - .zipWith(repository.insert(two), //(2)! - (r1, r2) -> List.of(one, two)); - }); + default CompletionStage> saveAll(Entity one, Entity two) { + return getVertxConnectionFactory().inTx(connection -> + insert(one) //(1)! + .then(updateValue(two.id(), two.value2())) //(2)! + .thenReturn(List.of(one, two)) + .toFuture()); } } ``` - 1. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение - 2. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение + 1. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке + 2. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке === ":simple-kotlin: `Kotlin`" ```kotlin - @Component - class SomeService(private val repository: EntityRepository) { + @Repository + interface EntityRepository : VertxRepository { + + @Query("INSERT INTO entities(id, value2) VALUES (:entity.id, :entity.value2)") + fun insert(entity: Entity): Mono - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.getVertxConnectionFactory.inTx { - repository.insert(one).zipWith(repository.insert(two)) //(1)! - { r1: UpdateCount, r2: UpdateCount -> listOf(one, two) } + @Query("UPDATE entities SET value2 = :value2 WHERE id = :id") + fun updateValue(id: String, value2: String): Mono + + fun saveAll(one: Entity, two: Entity): CompletionStage> { + return vertxConnectionFactory.inTx { _ -> + insert(one) //(1)! + .then(updateValue(two.id, two.value2)) //(2)! + .thenReturn(listOf(one, two)) + .toFuture() } } } ``` - 1. Будет выполнено в рамках транзакции либо откатится если вся лямбра выкинет исключение + 1. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке + 2. Будет выполнено в рамках транзакции или откатится, если вся цепочка сигнализирует об ошибке + +Транзакция фиксируется при успешном завершении возвращаемого `CompletionStage`. +Если `CompletionStage` завершается с ошибкой, транзакция откатывается, а ошибка пробрасывается дальше, поэтому все изменения +в базе данных, сделанные в рамках транзакции, не применяются. -### Ручное управление { #connection } +### Ручное управление соединением { #connection } -Если для запроса нужна какая-то более сложная логика, либо запросы вне репозитория, можно использовать `io.r2dbc.spi.Connection`: +Если для запроса нужна более сложная логика или запросы вне репозитория, можно работать напрямую с `io.vertx.sqlclient.SqlConnection`. +Метод `withConnection` выполняет лямбду с соединением, но сам по себе не открывает транзакцию: + +- если в текущем `Context` уже есть соединение, метод передает в лямбду это текущее соединение; +- если соединения в текущем `Context` нет, метод берет новое соединение из пула, кладет его в `Context` на время выполнения лямбды и закрывает после завершения; +- повторные вызовы `withConnection` и методы репозитория внутри этой лямбды используют то же текущее соединение. + +И `withConnection`, и `inTx` возвращают `CompletionStage`, а метод `inTx` построен поверх `withConnection`. ===! ":fontawesome-brands-java: `Java`" @@ -443,9 +675,9 @@ agent: this.repository = repository; } - public Mono> saveAll(Entity one, Entity two) { - return repository.getVertxConnectionFactory().inTx(connection -> { - // do some work + public CompletionStage> loadAll() { + return repository.getVertxConnectionFactory().withConnection(connection -> { + // do some work, returns CompletionStage }); } } @@ -457,20 +689,152 @@ agent: @Component class SomeService(private val repository: EntityRepository) { - fun saveAll( - one: Entity, - two: Entity - ): Mono> { - return repository.getVertxConnectionFactory.inTx { connection -> - // do some work + fun loadAll(): CompletionStage> { + return repository.vertxConnectionFactory.withConnection { connection -> + // do some work, returns CompletionStage + } + } + } + ``` + +`VertxConnectionFactory` также предоставляет более низкоуровневые методы доступа: `currentConnection()` возвращает соединение, +привязанное к текущему `Context` (или `null`, если его нет), `newConnection()` получает новый `CompletionStage` +из пула, `pool()` возвращает исходный `io.vertx.sqlclient.Pool`, а `telemetry()` возвращает телеметрию базы данных. + +## Ручной запрос с телеметрией { #query } + +Если запрос сложно выразить одним статическим `@Query`, можно сделать обычный метод с реализацией и самостоятельно собрать +`SQL`. У фабрики нет метода `query`; вместо этого используйте вспомогательный класс `VertxRepositoryHelper`, чтобы выполнить +запрос через телеметрию Kora на текущем или новом соединении. + +`VertxRepositoryHelper.completionStage` принимает четыре аргумента: + +- `VertxConnectionFactory` (из `getVertxConnectionFactory()`); +- `QueryContext` с идентификатором запроса и итоговым `SQL`. Идентификатор попадает в телеметрию, поэтому для него удобно использовать стабильное имя вида `Repository.method`; +- Vert.x `Tuple` с привязанными значениями параметров; +- `VertxRowSetMapper`, который читает `RowSet` и формирует возвращаемое значение. + +Если соединение уже привязано к текущему `Context` (например, внутри `inTx` или `withConnection`), запрос выполняется на этом +соединении; иначе новое соединение берется из пула и закрывается после завершения. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Repository + public interface EntityRepository extends VertxRepository { + + default CompletionStage> findByFilter(@Nullable String name, boolean onlyActive) { + var sql = new StringBuilder("SELECT id, name FROM entities WHERE 1 = 1"); + var params = new ArrayList(); + + if (name != null) { + params.add(name); + sql.append(" AND name = $").append(params.size()); + } + if (onlyActive) { + sql.append(" AND active = true"); + } + + var queryContext = new QueryContext("EntityRepository.findByFilter", sql.toString()); + return VertxRepositoryHelper.completionStage( + getVertxConnectionFactory(), + queryContext, + Tuple.from(params), + rows -> { + var result = new ArrayList(rows.size()); + for (var row : rows) { + result.add(new Entity(row.getString(0), row.getString(1))); + } + return result; + } + ); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Repository + interface EntityRepository : VertxRepository { + + fun findByFilter(name: String?, onlyActive: Boolean): CompletionStage> { + val sql = StringBuilder("SELECT id, name FROM entities WHERE 1 = 1") + val params = mutableListOf() + + if (name != null) { + params += name + sql.append(" AND name = $").append(params.size) + } + if (onlyActive) { + sql.append(" AND active = true") + } + + val queryContext = QueryContext("EntityRepository.findByFilter", sql.toString()) + return VertxRepositoryHelper.completionStage( + vertxConnectionFactory, + queryContext, + Tuple.from(params) + ) { rows -> + rows.map { row -> Entity(row.getString(0), row.getString(1)) } } } } ``` +Если вы предпочитаете реактивные типы возвращаемых значений, используйте вместо этого `VertxRepositoryHelper.Reactor.mono` +(возвращает `Mono` с `VertxRowSetMapper`) или `VertxRepositoryHelper.Reactor.flux` (возвращает `Flux` с +`VertxRowMapper`). Это продвинутый путь; для статических запросов предпочтительнее обычные `@Query`-методы. + +## Выборка по списку { #select-by-list } + +Иногда требуется выборка строк по списку значений. +`Kora` выполняет преобразования во время компиляции и не переписывает `SQL` во время работы, поэтому для параметра-списка +нужен собственный преобразователь, который привязывает всю коллекцию как одно значение. + +Пример ниже показывает `Postgres` через массив, привязываемый с помощью `ANY(:ids)`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public final class ListOfStringParameterMapper implements VertxParameterColumnMapper> { + + @Override + public Object apply(@Nullable List value) { + return value == null ? null : value.toArray(String[]::new); + } + } + + @Repository + public interface EntityRepository extends VertxRepository { + + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + Flux findAllByIds(@Mapping(ListOfStringParameterMapper.class) List ids); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + class ListOfStringParameterMapper : VertxParameterColumnMapper?> { + + override fun apply(value: List?): Any? { + return value?.toTypedArray() + } + } + + @Repository + interface EntityRepository : VertxRepository { + + @Query("SELECT id, name FROM entities WHERE id = ANY(:ids)") + fun findAllByIds(@Mapping(ListOfStringParameterMapper::class) ids: List): Flux + } + ``` + ## Сигнатуры { #signatures } -Доступные сигнатуры для методов репозитория из коробки: +Доступные сигнатуры для методов репозитория из коробки. +Поскольку клиент Vert.x реактивен нативно, для асинхронных сигнатур не требуется компонент `Executor`. ===! ":fontawesome-brands-java: `Java`" @@ -479,6 +843,7 @@ agent: - `T myMethod()` - `@Nullable T myMethod()` - `Optional myMethod()` + - `CompletionStage myMethod()` - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) @@ -489,3 +854,8 @@ agent: - `myMethod(): T` - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + +## Телеметрия { #telemetry } + +Логирование, метрики и трассировка настраиваются через блок `telemetry` в [конфигурации](#configuration) и описаны в разделе [Справочник метрик](metrics.md#database). +Чтобы переопределить телеметрию полностью, можно предоставить собственные SPI-фабрики, подробнее в [Общей документации по Базам данных](database-common.md#telemetry). diff --git a/mkdocs/docs/ru/documentation/general.md b/mkdocs/docs/ru/documentation/general.md index 3358a38..b373958 100644 --- a/mkdocs/docs/ru/documentation/general.md +++ b/mkdocs/docs/ru/documentation/general.md @@ -1,100 +1,113 @@ --- description: "Explains Kora framework fundamentals, annotation processors, compatibility, Gradle build setup, dependencies, application runtime, and terminology. Use when working with @KoraApp, annotation processors, Gradle, BOM, kora-parent, application plugin." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora framework fundamentals, annotation processors, compatibility, Gradle build setup, dependencies, application runtime, and terminology; key triggers include @KoraApp, annotation processors, Gradle, BOM, kora-parent, application plugin." + use_when: "Use this file for Kora docs or implementation questions about Kora framework fundamentals, annotation processors, Gradle, BOM, kora-parent, application plugin." --- -Kora является облачно ориентированным серверным фреймворком написанным на Java и предлагает -множество различных модулей для быстрого создания приложений такие как HTTP сервер и клиент, Kafka потребители, -абстракция над базой данных в виде репозиториев, S3-клиент, gRPC сервер и клиент, интеграции с Camunda, -телеметрию всех модулей, модули отказоустойчивости и многое другое. +Kora - облачно ориентированный серверный фреймворк, написанный на `Java`, для приложений на `Java` и `Kotlin`. +Эта страница описывает базовые принципы Kora, требования к окружению, подключение обработчиков аннотаций, минимальную настройку `Gradle`, управление зависимостями и запуск приложения. -Прочитать про основные характеристики Kora можно [на главной](../index.md). +Kora предоставляет набор модулей для быстрого создания серверных приложений: `HTTP`-сервер и `HTTP`-клиент, потребители `Kafka`, репозитории для работы с базами данных, `S3`-клиент, `gRPC`-сервер и `gRPC`-клиент, интеграции с `Camunda`, телеметрию модулей, отказоустойчивость и другие возможности. +Основные характеристики фреймворка описаны [на главной странице](../index.md). -Kora предоставляет все необходимые для современной Java или Kotlin серверной разработки инструменты: +Kora предоставляет инструменты, которые обычно нужны современной серверной разработке: -- Внедрение и инверсию зависимостей посредствам аннотаций -- Достаточно высокоуровневые простые абстракции и инструменты разработки -- Аспектно-ориентированное программирование посредствам аннотаций -- Большой набор пред-сконфигурированных интеграций -- Телеметрия, трассировка, метрики по `OpenTelemetry` стандарту и логирование всех модулей -- Легкое и быстрое тестирование с помощью [JUnit5](junit5.md) -- Простая и подробная документация подкрепленная [примерами и руководствами рабочих сервисов](../guides/home.md) +- внедрение зависимостей через аннотации; +- инверсию управления без отдельного контейнера во время выполнения; +- аспектно-ориентированное программирование через аннотации; +- достаточно высокоуровневые простые абстракции и инструменты разработки; +- большой набор заранее настроенных интеграций; +- телеметрию, трассировку, метрики по стандарту `OpenTelemetry` и логирование модулей; +- быстрое тестирование с помощью [JUnit5](junit5.md); +- рабочие [примеры и руководства](../guides/home.md). -Для достижения высокопроизводительного и эффективного кода, Kora стоит на таких принципах: +Для высокопроизводительного, эффективного и предсказуемого кода Kora следует нескольким принципам: -- Отказ от использования Reflection API во время работы -- Отказ от динамических прокси во время работы -- Отказ от генерации байт-кода во время компиляции и работы -- Создание исходного кода посредствам обработчиков аннотаций во время компиляции -- Тонкие абстракции над модулями -- Бесплатные аспекты -- Использование только наиболее эффективных реализаций для интеграций -- Поощрение и использование наиболее эффективных принципов разработки и естественных конструкций языка +- не использует `Reflection` во время работы приложения; +- не использует `динамический прокси` во время работы приложения; +- не генерирует байт-код во время компиляции или работы приложения; +- создает исходный код на этапе компиляции через обработчики аннотаций; +- оставляет тонкие абстракции над интеграциями; +- предоставляет бесплатные аспекты: без дополнительной стоимости во время работы приложения; +- использует только наиболее эффективные реализации для интеграций; +- поощряет и использует наиболее эффективные принципы разработки и естественные конструкции языка. Если нужен пошаговый разбор перед справочным описанием, смотрите [Создание первого приложения на Kora](../guides/getting-started.md) и [Введение во внедрение зависимостей](../guides/dependency-injection-introduction.md). -## Обработчики аннотаций { #annotation-handlers } +## Обработчики аннотаций { #annotation-processor } -Главный столп на котором строится фреймворк Kora это обработчики аннотаций. +Kora строит приложение на этапе компиляции: обработчики читают аннотации, проверяют код и генерируют исходные файлы, которые затем компилируются вместе с кодом приложения. +За счет этого граф зависимостей, аспекты, `HTTP`-обработчики, репозитории и другие компоненты становятся обычным скомпилированным кодом без `Reflection` во время работы. ===! ":fontawesome-brands-java: `Java`" - Аннотация - это конструкция, связанная с элементами исходного кода Java, такими как классы, методы и переменные. - Аннотации предоставляют программе информацию во время компиляции на основе которой программа может предпринять дальнейшие действия. - [Процессор аннотаций](https://docs.oracle.com/en/java/javase/17/docs/api/java.compiler/javax/annotation/processing/Processor.html) обрабатывает эти аннотации во время компиляции для обеспечения таких функций, как генерация кода, проверка ошибок и т.д. + Аннотация - это конструкция, связанная с элементами исходного кода `Java`: классами, методами, параметрами и полями. + [Обработчик аннотаций](https://docs.oracle.com/en/java/javase/17/docs/api/java.compiler/javax/annotation/processing/Processor.html) запускается компилятором, читает эти аннотации и может сгенерировать дополнительный исходный код или остановить компиляцию с понятной ошибкой. - Kora предоставляет в рамках одной зависимости все [обработчики аннотаций](https://docs.oracle.com/en/java/javase/17/docs/api/java.compiler/javax/annotation/processing/Processor.html), которые потребуются для работы со всеми модулями, - процессоры не тянут за собой никакие лишние зависимости которые протекали бы во время компиляции или выполнения приложения. + Kora предоставляет все обработчики аннотаций в одной зависимости: + + ```groovy + annotationProcessor "ru.tinkoff.kora:annotation-processors" + ``` + + Эта зависимость нужна только на этапе компиляции и не добавляет лишние библиотеки в путь классов времени выполнения приложения. === ":simple-kotlin: `Kotlin`" - [Kotlin Symbol Processing (KSP)](https://kotlinlang.org/docs/ksp-overview.html) - это API, который можно использовать для разработки легких плагинов для компиляторов. - KSP представляет собой упрощенный API для плагинов к компиляторам, который позволяет использовать возможности Kotlin - при минимальной кривой обучения. По сравнению с kapt, процессоры аннотаций, использующие KSP, могут работать в два раза быстрее (но в разы медленее чем Java). + Для `Kotlin` используется [`KSP`](https://kotlinlang.org/docs/ksp-overview.html) (`Kotlin Symbol Processing`). + `KSP` читает символы исходного кода `Kotlin`, передает их процессорам Kora и позволяет генерировать код до основной компиляции. + + Kora предоставляет `KSP`-обработчики в одной зависимости: + + ```kotlin + ksp("ru.tinkoff.kora:symbol-processors") + ``` - Другой способ представить [KSP](https://kotlinlang.org/docs/ksp-overview.html) как препроцессорный фреймворк программ на языке Kotlin. Если рассматривать плагины на базе KSP как символьные процессоры, или просто процессоры, то поток данных при компиляции можно описать следующими шагами: + При этом обработка `Kotlin` обычно медленнее обработки аннотаций в `Java`. - - Процессоры читают и анализируют исходные программы и ресурсы. - - Процессоры генерируют код или другие формы вывода. - - Компилятор Kotlin компилирует исходные программы вместе со сгенерированным кодом. +### `KSP` { #ksp } -Такой подход позволяет использовать привычную всем парадигму программирования посредствам создания с помощью аннотаций HTTP-обработчиков, -Kafka продюсеров, репозиториев баз данных и так далее, но дает большой прирост в производительности и прозрачности относительно известных JVM фреймворков. +`KSP` нужен только для `Kotlin`-проектов. +Если приложение написано на `Java`, используйте обычный `annotationProcessor`; если приложение написано на `Kotlin`, подключайте `com.google.devtools.ksp` и зависимость `ru.tinkoff.kora:symbol-processors`. ## Совместимость { #compatibility } ===! ":fontawesome-brands-java: `Java`" - Требуется версия не ниже [JDK 17](https://openjdk.org/projects/jdk/17/). + Требуется версия не ниже [JDK 17](https://openjdk.org/projects/jdk/17/), рекомендуется [`JDK` `21`](https://openjdk.org/projects/jdk/21/) - версия, которую используют официальные примеры и шаблоны приложений. - Конфигурация в `build.gradle`: + Минимальная конфигурация в `build.gradle`: ```groovy plugins { id "java" - } + } - sourceCompatibility = JavaVersion.VERSION_17 - targetCompatibility = JavaVersion.VERSION_17 + java { + toolchain { + languageVersion = JavaLanguageVersion.of(21) + vendor = JvmVendorSpec.ADOPTIUM + } + } ``` -=== ":simple-kotlin: `Kotlin`" + Указание `vendor` необязательно и просто соответствует набору инструментов `Adoptium`, который используется в примерах проектов; его можно опустить или выбрать другого поставщика. - Требуется версия не ниже [JDK 17](https://openjdk.org/projects/jdk/17/). +=== ":simple-kotlin: `Kotlin`" - Рекомендуемая версия [Kotlin `1.9+`](https://github.com/JetBrains/kotlin/releases), совместимость с версией `1.8+` и `2+` не гарантируется. + Требуется версия не ниже [JDK 17](https://openjdk.org/projects/jdk/17/), рекомендуется [`JDK` `21`](https://openjdk.org/projects/jdk/21/) из-за совместимости с `Kotlin`. - Рекомендуемая версия [KSP `1.9+`](https://github.com/google/ksp/releases) соответсвующая версии Kotlin. + Рекомендуемая версия [`Kotlin` `1.9+`](https://github.com/JetBrains/kotlin/releases), совместимость с версиями `1.8+` и `2+` не гарантируется. + Рекомендуемая версия [`KSP` `1.9+`](https://github.com/google/ksp/releases) должна соответствовать версии `Kotlin`. - Конфигурация в `build.gradle.kts`: - ```groovy + Минимальная конфигурация в `build.gradle.kts`: + ```kotlin plugins { - kotlin("jvm") version ("1.9.25") - id("com.google.devtools.ksp") version ("1.9.25-1.0.20") + kotlin("jvm") version "1.9.25" + id("com.google.devtools.ksp") version "1.9.25-1.0.20" } kotlin { - jvmToolchain { languageVersion.set(JavaLanguageVersion.of("17")) } + jvmToolchain { languageVersion.set(JavaLanguageVersion.of("21")) } sourceSets.main { kotlin.srcDir("build/generated/ksp/main/kotlin") } sourceSets.test { kotlin.srcDir("build/generated/ksp/test/kotlin") } } @@ -102,104 +115,97 @@ Kafka продюсеров, репозиториев баз данных и та ## Система сборки { #build-system } -Поскольку основным столпом являются обработчики аннотаций, то подразумевается что вы будете использовать именно систему сборки [Gradle](https://gradle.org/guides/), -так как она лучше других поддерживает обработчики аннотаций, инкрементальную сборку и является наиболее совершенной системой сборки в JVM экосистеме. -Требуется версия Gradle `7+`. +Kora рассчитана на сборку через [Gradle](https://gradle.org/guides/), потому что `Gradle` хорошо поддерживает обработчики аннотаций, `KSP`, инкрементальную сборку и управление зависимостями. +Требуется версия `Gradle` `7+`, рекомендуется `Gradle` `9.5+`. -Для того чтобы не прописывать версии для каждой зависимости, предполагается использовать [BOM](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import) -зависимость `ru.tinkoff.kora:kora-parent` в которой требуется один раз указать версию для всех Kora зависимостей разом. +Чтобы не указывать версии для каждой зависимости Kora отдельно, используется [`BOM`](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import) `ru.tinkoff.kora:kora-parent`. +Версия `BOM` задается один раз, а остальные зависимости Kora подключаются без явного указания версии. ===! ":fontawesome-brands-java: `Java`" - Kora поддерживает инкрементальную сборку из нескольких раундов на этапе обработки аннотаций, - более подробное описание по работе с Gradle и Java можно почитать в их [официальной документации](https://docs.gradle.org/current/userguide/java_plugin.html). - - Ниже будет представлена минимально необходимая конфигурация приложения `build.gradle`: + Минимальная конфигурация приложения в `build.gradle`: ```groovy plugins { id "java" id "application" - } - - sourceCompatibility = JavaVersion.VERSION_17 - targetCompatibility = JavaVersion.VERSION_17 + } - configurations { - koraBom - annotationProcessor.extendsFrom(koraBom) - compileOnly.extendsFrom(koraBom) - implementation.extendsFrom(koraBom) - api.extendsFrom(koraBom) + java { + toolchain { + languageVersion = JavaLanguageVersion.of(21) + vendor = JvmVendorSpec.ADOPTIUM + } } dependencies { - koraBom platform("ru.tinkoff.kora:kora-parent:1.2.18") - annotationProcessor "ru.tinkoff.kora:annotation-processors" + annotationProcessor "ru.tinkoff.kora:annotation-processors:1.2.18" + implementation(platform("ru.tinkoff.kora:kora-parent:1.2.18")) } ``` - Также можно ознакомиться с [руководством по созданию первого приложения](../guides/getting-started.md) с более подробным пошаговым описанием. + Более подробный пример есть в [руководстве по созданию первого приложения](../guides/getting-started.md). === ":simple-kotlin: `Kotlin`" - Kora поддерживает инкрементальную сборку из нескольких раундов на этапе обработки аннотаций, - более подробное описание по работе с Gradle и Kotlin можно почитать в их [официальной документации](https://kotlinlang.org/docs/get-started-with-jvm-gradle-project.html), - а также будет полезно ознакомиться с работой [KSP в Gradle](https://kotlinlang.org/docs/ksp-quickstart.html) - - Для Kotlin предполагается что будет использоваться [Gradle Kotlin DSL](https://docs.gradle.org/current/userguide/kotlin_dsl.html), - так что все примеры для Kotlin будут даваться именно в этом синтаксисе, если вы используете Groovy синтаксис, то используйте Java примеры. + Для `Kotlin` предполагается [Gradle Kotlin DSL](https://docs.gradle.org/current/userguide/kotlin_dsl.html). + Если проект использует `Groovy DSL`, ориентируйтесь на примеры для `Java`. - Ниже будет представлена минимально необходимая конфигурация приложения `build.gradle.kts`: - ```groovy + Минимальная конфигурация приложения в `build.gradle.kts`: + ```kotlin plugins { id("application") - kotlin("jvm") version ("1.9.25") - id("com.google.devtools.ksp") version ("1.9.25-1.0.20") + kotlin("jvm") version "1.9.25" + id("com.google.devtools.ksp") version "1.9.25-1.0.20" } kotlin { - jvmToolchain { languageVersion.set(JavaLanguageVersion.of("17")) } + jvmToolchain { languageVersion.set(JavaLanguageVersion.of("21")) } sourceSets.main { kotlin.srcDir("build/generated/ksp/main/kotlin") } sourceSets.test { kotlin.srcDir("build/generated/ksp/test/kotlin") } } - val koraBom: Configuration by configurations.creating - configurations { - ksp.get().extendsFrom(koraBom) - compileOnly.get().extendsFrom(koraBom) - api.get().extendsFrom(koraBom) - implementation.get().extendsFrom(koraBom) - } - dependencies { - koraBom(platform("ru.tinkoff.kora:kora-parent:1.2.18")) - ksp("ru.tinkoff.kora:symbol-processors") + ksp("ru.tinkoff.kora:symbol-processors:1.2.18") + implementation(platform("ru.tinkoff.kora:kora-parent:1.2.18")) } ``` - Также можно ознакомиться с [руководством по созданию первого приложения](../guides/getting-started.md) с более подробным пошаговым описанием. + Более подробный пример есть в [руководстве по созданию первого приложения](../guides/getting-started.md). + +Записи `testAnnotationProcessor` (`Java`) и `kspTest` (`Kotlin`) подключают те же обработчики Kora к набору тестовых исходников. +Они необходимы, чтобы генерировались исходники, создаваемые во время тестов, - например те, что производятся для [`@KoraAppTest`](junit5.md); без них при компиляции тестов не будет сгенерирован нужный код Kora. + +В реальных проектах версию `BOM` обычно выносят в свойство `gradle.properties` (например `koraVersion`) и ссылаются на нее как `platform("ru.tinkoff.kora:kora-parent:$koraVersion")`, чтобы версия объявлялась в одном месте, а не была прописана в каждом модуле. + +!!! note "Доступ к внутренностям компилятора" + + Некоторые обработчики аннотаций `Java` читают внутренние компоненты `jdk.compiler`. На новых версиях `JDK` для этого может потребоваться экспортировать соответствующие пакеты компилятору. + Если компиляция завершается ошибками `IllegalAccessError` или `module jdk.compiler does not export ...`, добавьте в `gradle.properties` следующие аргументы `JVM`: + + ```properties + org.gradle.jvmargs=--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED \ + --add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED + ``` + + ## Зависимости { #dependencies } -Так как основным столпом на котором строится Kora являются обработчики аннотаций, то они являются обязательной зависимостью, -также не стоит забывать и о [BOM зависимости](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import): +В документации модулей Kora обычно показывается только зависимость конкретного модуля. +Но в приложении также должны быть подключены [`BOM`](https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import) и обработчики, показанные ниже. ===! ":fontawesome-brands-java: `Java`" `build.gradle`: ```groovy - configurations { - koraBom - annotationProcessor.extendsFrom(koraBom) - compileOnly.extendsFrom(koraBom) - implementation.extendsFrom(koraBom) - api.extendsFrom(koraBom) - } - dependencies { - koraBom platform("ru.tinkoff.kora:kora-parent:1.2.18") - annotationProcessor "ru.tinkoff.kora:annotation-processors" + annotationProcessor "ru.tinkoff.kora:annotation-processors:1.2.18" + implementation(platform("ru.tinkoff.kora:kora-parent:1.2.18")) } ``` @@ -207,36 +213,45 @@ Kafka продюсеров, репозиториев баз данных и та `build.gradle.kts`: - ```groovy - val koraBom: Configuration by configurations.creating - configurations { - ksp.get().extendsFrom(koraBom) - compileOnly.get().extendsFrom(koraBom) - api.get().extendsFrom(koraBom) - implementation.get().extendsFrom(koraBom) - } - + ```kotlin dependencies { - koraBom(platform("ru.tinkoff.kora:kora-parent:1.2.18")) - ksp("ru.tinkoff.kora:symbol-processors") + ksp("ru.tinkoff.kora:symbol-processors:1.2.18") + implementation(platform("ru.tinkoff.kora:kora-parent:1.2.18")) } ``` + + +После этого зависимости модулей можно указывать без версии, например: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + implementation "ru.tinkoff.kora:http-server-undertow" + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + implementation("ru.tinkoff.kora:http-server-undertow") + ``` + + + ## Запуск { #run } -Запускать и работать с приложением через систему сборки, предполагается с помощью [application плагина](https://docs.gradle.org/current/userguide/application_plugin.html) -который предоставляет Gradle. +Для локального запуска и сборки исполняемого архива обычно используется [плагин `application`](https://docs.gradle.org/current/userguide/application_plugin.html). ===! ":fontawesome-brands-java: `Java`" - Требуется указать плагин в `build.gradle`: + Подключите плагин в `build.gradle`: ```groovy plugins { id "application" } ``` - Можно указывать как системные переменные, так и переменные окружения при локальном запуске приложения в `build.gradle`: + Системные свойства и переменные окружения для локального запуска можно задать в задаче `run`: ```groovy run { jvmArgs += [ @@ -249,12 +264,12 @@ Kafka продюсеров, репозиториев баз данных и та } ``` - Запускать надо с помощью команды: + Запуск: ```shell ./gradlew run ``` - Настроить сборку артефакта можно таким способом в `build.gradle`: + Настройка сборки архива: ```groovy application { applicationName = "application" @@ -267,26 +282,26 @@ Kafka продюсеров, репозиториев баз данных и та } ``` - Собирать артефакт можно командой: + Сборка архива: ```shell ./gradlew distTar ``` - Пример настроенного приложения можно посмотреть [тут](https://github.com/kora-projects/kora-java-template/blob/master/build.gradle) + Пример настроенного приложения можно посмотреть в [шаблоне Java-приложения](https://github.com/kora-projects/kora-java-template/blob/master/build.gradle). === ":simple-kotlin: `Kotlin`" - Требуется указать плагин в `build.gradle`: - ```groovy + Подключите плагин в `build.gradle.kts`: + ```kotlin plugins { id("application") - kotlin("jvm") version ("1.9.25") - id("com.google.devtools.ksp") version ("1.9.25-1.0.20") + kotlin("jvm") version "1.9.25" + id("com.google.devtools.ksp") version "1.9.25-1.0.20" } ``` - Можно указывать как системные переменные, так и переменные окружения при локальном запуске приложения в `build.gradle.kts`: - ```groovy + Системные свойства и переменные окружения для локального запуска можно задать в задачах `JavaExec`: + ```kotlin tasks.withType { jvmArgs( "-Xmx256m", @@ -298,13 +313,13 @@ Kafka продюсеров, репозиториев баз данных и та } ``` - Запускать надо с помощью команды: + Запуск: ```shell ./gradlew run ``` - Настроить сборку артефакта можно таким способом: - ```groovy + Настройка сборки архива: + ```kotlin application { applicationName = "application" mainClass.set("ru.tinkoff.kora.kotlin.ApplicationKt") @@ -316,22 +331,24 @@ Kafka продюсеров, репозиториев баз данных и та } ``` - Собирать артефакт можно командой: + Сборка архива: ```shell ./gradlew distTar ``` - Пример настроенного приложения можно посмотреть [тут](https://github.com/kora-projects/kora-kotlin-template/blob/master/build.gradle.kts) + Пример настроенного приложения можно посмотреть в [шаблоне Kotlin-приложения](https://github.com/kora-projects/kora-kotlin-template/blob/master/build.gradle.kts). ## Терминология { #terminology } -В данной секции описываются основные термины которые встречаются во всей документации и в рамках фреймворка Kora: +В этой секции описаны базовые термины, которые встречаются в документации Kora: -- Фабрика - под фабрикой подразумевается метод который, создает экземпляры компонент/классов/зависимостей. -- Модуль - под [модулем](container.md#external-module-factory) понимается подключаемая зависимость, зачастую внешняя которая предоставляет какие-то фабричные методы и новый функционал в приложение. -- Компонент - под [компонентом](container.md#components) понимается класс в одном экземпляре (`Singleton`) который реализует какую то логику и является зависимостью в контейнере зависимостей. -- Аспект - под аспектом подразумевается логика которая будет расширять стандартное поведение метода посредствам какой-либо аннотации до и/или после его выполнения. +- Фабрика - метод, который создает и возвращает экземпляр компонента или зависимости. +- [Модуль](container.md#external-module-factory) - подключаемая зависимость или интерфейс с фабричными методами, которые добавляют в приложение новые компоненты. +- [Компонент](container.md#components) - объект в графе зависимостей Kora. Обычно это единственный экземпляр класса, который реализует часть логики приложения. +- Аспект - логика, которая расширяет поведение метода до, после или вокруг его выполнения на основании аннотации. +- Граф зависимостей - набор компонентов приложения и связей между ними, построенный Kora на этапе компиляции. ## Первое руководство -После общего обзора переходите к руководству [Создание первого приложения на Kora](../guides/getting-started.md). В нем базовая структура приложения показана на небольшом HTTP-сервисе, который можно собрать и запустить. +После общего обзора переходите к руководству [Создание первого приложения на Kora](../guides/getting-started.md). +В нем базовая структура приложения показана на небольшом `HTTP`-сервисе, который можно собрать и запустить. diff --git a/mkdocs/docs/ru/documentation/graalvm-native.md b/mkdocs/docs/ru/documentation/graalvm-native.md index c0c4ab1..7ba2d05 100644 --- a/mkdocs/docs/ru/documentation/graalvm-native.md +++ b/mkdocs/docs/ru/documentation/graalvm-native.md @@ -4,20 +4,35 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora GraalVM Native Image notes and native build considerations for Kora applications; key triggers include GraalVM, native-image, reflection config, AOT, native build." --- -Kora создает свои вспомогательные классы во время компиляции, -не использует Reflection API во время работы, -не использует динамических прокси, -не использует генерацию байт-кода во время компиляции и во время работы, -так что проблем для сборки нативного образа со стороны самой Kora нет. +GraalVM Native Image — это инструмент для `AOT-компиляции`, который собирает Java-приложение заранее в отдельный `нативный образ` для целевой платформы. +Такой образ запускается без обычного прогрева JVM, но требует, чтобы часть сведений о коде, ресурсах и отражении была известна уже во время сборки. -Пример сборки нативного образа с помощью [плагина](https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html) для `gradle`: +Kora создает вспомогательные классы во время компиляции, +не использует `Reflection API` во время выполнения, +не использует `динамические прокси`, +не использует генерацию байт-кода во время компиляции и во время выполнения. +Это упрощает сборку приложений Kora в `нативный образ`, который быстрее запускается и обычно потребляет меньше памяти, чем приложение на обычной JVM. +Основные ограничения при такой сборке чаще связаны не с Kora, а со сторонними библиотеками, которым могут понадобиться дополнительные настройки отражения, ресурсов или инициализации классов. + +Поэтому со стороны самой Kora обычно не требуется дополнительная настройка для сборки `нативного образа`. + +## Требования { #requirements } + +Для нативной сборки требуется JDK [GraalVM](https://www.graalvm.org/): **GraalVM Community Edition** или **Oracle GraalVM** версии 21. +[Gradle-плагин](https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html) выбирает такой набор инструментов через блок `javaLauncher`, показанный в разделе [Сборка](#build) (`JvmVendorSpec.matching("GraalVM Community")`), +поэтому саму сборку может запускать обычный JDK, а `native-image` при этом выполняется на GraalVM. +При сборке вне плагина (например, командой `native-image` внутри сборочного образа [Docker](#docker)) инструмент `native-image` должен быть доступен в `PATH` — официальные контейнерные образы GraalVM уже содержат его. + +## Сборка { #build } + +Пример сборки `нативного образа` с помощью [Gradle-плагина](https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html): ===! ":fontawesome-brands-java: `Java`" Плагин `build.gradle`: ```groovy plugins { - id "org.graalvm.buildtools.native" version "0.11.0" + id "org.graalvm.buildtools.native" version "0.11.5" } ``` @@ -32,7 +47,7 @@ Kora создает свои вспомогательные классы во в verbose = true buildArgs.add("--report-unsupported-elements-at-runtime") javaLauncher = javaToolchains.launcherFor { - languageVersion = JavaLanguageVersion.of(17) + languageVersion = JavaLanguageVersion.of(21) vendor = JvmVendorSpec.matching("GraalVM Community") } } @@ -48,7 +63,7 @@ Kora создает свои вспомогательные классы во в Плагин `build.gradle.kts`: ```groovy plugins { - id("org.graalvm.buildtools.native") version("0.11.0") + id("org.graalvm.buildtools.native") version("0.11.5") } ``` @@ -63,7 +78,7 @@ Kora создает свои вспомогательные классы во в verbose.set(true) buildArgs.add("--report-unsupported-elements-at-runtime") javaLauncher = javaToolchains.launcherFor { - languageVersion = JavaLanguageVersion.of(17) + languageVersion = JavaLanguageVersion.of(21) vendor = JvmVendorSpec.matching("GraalVM Community") } } @@ -74,31 +89,269 @@ Kora создает свои вспомогательные классы во в } ``` -Некоторые библиотеки требуют дополнительной конфигурации, часть конфигураций сделана в Kora. +Значения, добавленные в `buildArgs`, передаются напрямую в `native-image`. Наиболее распространённые из них: + +- `--report-unsupported-elements-at-runtime` — откладывает ошибки о неподдерживаемых возможностях на время выполнения вместо того, чтобы прерывать сборку (используется в примере выше). +- `--no-fallback` — никогда не создавать `резервный` образ (в который незаметно встраивается JVM); вместо этого прервать сборку, если что-то не удаётся скомпилировать заранее. Этот флаг используется при прямом вызове `native-image` (см. [Docker](#docker)). +- `debug` / `verbose` — дополнительная диагностика сборки; для релизных сборок их можно убрать. + +Флаги, необходимые самой Kora, добавляются её модулями автоматически, и их **не** нужно указывать вручную: + +- `ru.tinkoff.kora:application-graph` поставляет `--install-exit-handlers` и `--initialize-at-build-time` для держателя исполнителя виртуальных потоков. +- `ru.tinkoff.kora:common` поставляет `--initialize-at-run-time` для `Context` и хука контекста Reactor. -Проверенные модули, которые должны работать без дополнительной конфигурации: +Они берутся из ресурсов `META-INF/native-image` внутри JAR-файлов модулей и подмешиваются в сборку, как только зависимость оказывается в classpath (см. [Метаданные](#metadata)). + +### Fat JAR { #build-jar } + +`native-image` компилирует в бинарный файл единый classpath, поэтому приложение Kora обычно сначала собирают в один `fat JAR`. +Kora опирается на объединённые файлы `META-INF/services` (сгенерированные во время компиляции модули и расширения), поэтому JAR нужно собирать с объединением сервисных файлов — например, с помощью плагина [Shadow](https://gradleup.com/shadow/): + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + plugins { + id "application" + id "com.gradleup.shadow" version "9.4.1" + } + + jar.enabled = false + shadowJar { + mergeServiceFiles() + manifest { + attributes "Main-Class": application.mainClass + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + plugins { + id("application") + id("com.gradleup.shadow") version("9.4.1") + } + + tasks.jar { + enabled = false + } + tasks.shadowJar { + mergeServiceFiles() + manifest { + attributes["Main-Class"] = "ru.tinkoff.kora.example.Application" + } + } + ``` + +Плагин Shadow создаёт `*-all.jar` в каталоге `build/libs`, который может использовать как Gradle-плагин, так и прямой вызов `native-image`. + +## Docker { #docker } + +В CI и на проде `нативный образ` обычно создают двухэтапной сборкой Docker: этап-сборщик (`builder`) на GraalVM компилирует [fat JAR](#build-jar) в бинарный файл, а компактный этап выполнения поставляет только этот бинарный файл. +Именно так собирают свои образы [примеры](https://github.com/kora-projects/kora-examples/tree/master/examples/graalvm), и это не зависит от того, написано приложение на Java или Kotlin: + +```dockerfile +FROM ghcr.io/graalvm/native-image-community:21 AS builder + +ARG TARGET_DIR=/opt/app +ARG SOURCE_DIR=build/libs +WORKDIR $TARGET_DIR + +COPY $SOURCE_DIR/*-all.jar $TARGET_DIR/application.jar +RUN native-image --no-fallback -classpath $TARGET_DIR/application.jar + +FROM ubuntu:noble AS runner + +ARG TARGET_DIR=/opt/app +WORKDIR $TARGET_DIR + +COPY --from=builder $TARGET_DIR/application $TARGET_DIR/application + +ARG DOCKER_USER=app +RUN groupadd -r $DOCKER_USER && useradd -rg $DOCKER_USER $DOCKER_USER +RUN chmod +x application +USER $DOCKER_USER + +EXPOSE 8080/tcp +EXPOSE 8085/tcp +CMD "/opt/app/application" +``` + +Этап-сборщик компилирует `application.jar` в нативный бинарный файл с именем `application`, а этап выполнения запускает его от имени пользователя без прав root. +Сначала соберите fat JAR (`./gradlew shadowJar`), затем выполните `docker build .`. + +## Метаданные { #metadata } + +Некоторым библиотекам требуется дополнительная конфигурация для `нативного образа`, и `native-image` видит только то, что объявлено как `метаданные достижимости`. +Kora поставляет метаданные для собственных модулей в виде ресурсов `META-INF/native-image///` внутри JAR-файла каждого модуля, поэтому они применяются автоматически, как только зависимость оказывается в classpath. + +Распространённые случаи покрываются тремя видами файлов: + +- **`native-image.properties`** — аргументы времени сборки, в первую очередь флаги инициализации классов `--initialize-at-build-time` и `--initialize-at-run-time`. Например, модуль `common` в Kora инициализирует `ru.tinkoff.kora.common.Context` *во время выполнения* (его состояние потока/контекста не должно запекаться в образ), а `application-graph` инициализирует держатель исполнителя виртуальных потоков *во время сборки*. +- **`reflect-config.json`** — классы, методы и поля, к которым обращаются через отражение. Например, Kora регистрирует `Thread.ofVirtual` / `Executors.newVirtualThreadPerTaskExecutor`, чтобы виртуальные потоки Loom работали в `нативном образе`. +- **`resource-config.json`** — ресурсы, которые нужно встроить в бинарный файл. Например, Kora включает `reference.conf` / `application.conf`, чтобы конфигурация HOCON была доступна для чтения во время выполнения. + +### Репозиторий { #metadata-repository } + +Если приложение использует сторонние библиотеки, которым нужны `метаданные достижимости`, не поставляемые ими самими, включите их загрузку из [репозитория метаданных достижимости GraalVM](https://github.com/oracle/graalvm-reachability-metadata): + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + graalvmNative { + metadataRepository { + enabled = true + } + } + + processResources.dependsOn tasks.collectReachabilityMetadata + sourceSets.main { resources.srcDirs += "$buildDir/native-reachability-metadata" } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + graalvmNative { + metadataRepository { + enabled.set(true) + } + } + + tasks.processResources { + dependsOn(tasks.collectReachabilityMetadata) + } + kotlin.sourceSets.main { + resources.srcDir(layout.buildDirectory.dir("native-reachability-metadata")) + } + ``` + +### Пользовательские метаданные { #metadata-custom } + +Когда класс не покрыт ни Kora, ни репозиторием, задайте метаданные вручную: положите `native-image.properties`, `reflect-config.json` и/или `resource-config.json` в каталог `src/main/resources/META-INF/native-image///` вашего собственного приложения — `native-image` объединяет все такие файлы, найденные в classpath. + +Например, чтобы встроить конфигурацию Logback и файл конфигурации HOCON в бинарный файл, приложение может поставить `resource-config.json`: + +```json title="src/main/resources/META-INF/native-image/ru.tinkoff.kora.examples/logback/resource-config.json" +{ + "resources": { + "includes": [ + { "pattern": "\\Qlogback.xml\\E" }, + { "pattern": "\\Qapplication.conf\\E" } + ] + } +} +``` + +Сегменты пути `/` произвольны, но должны быть уникальными (обычно это group и модуль вашего приложения), чтобы файлы из разных зависимостей не конфликтовали. + +### Агент { #metadata-agent } + +Для сторонних библиотек, которые не покрыты репозиторием, стандартный способ обнаружить необходимые метаданные — `агент трассировки` GraalVM. +Запустите приложение на обычной JVM с подключённым агентом, пройдите по путям кода, которые используют отражение, ресурсы или прокси, и агент запишет соответствующие файлы конфигурации: + +```bash +java -agentlib:native-image-agent=config-output-dir=src/main/resources/META-INF/native-image// \ + -jar build/libs/application-all.jar +``` + +Зафиксируйте сгенерированные файлы как [пользовательские метаданные](#metadata-custom). +Это обычный резервный вариант, когда нативная сборка падает во время выполнения с ошибкой об отсутствующем отражении или отсутствующем ресурсе. + +### Подсказки через аннотации { #metadata-hints } + +Официальные примеры генерируют часть метаданных из аннотаций с помощью сторонней библиотеки [GraalVM Hint Processor](https://github.com/GoodforGod/graalvm-hint). +Это **не** API Kora — это внешнее, необязательное удобство, взаимозаменяемое с написанными вручную [пользовательскими метаданными](#metadata-custom) выше. + +Добавьте процессор и аннотации: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + dependencies { + annotationProcessor "io.goodforgod:graalvm-hint-processor:1.2.0" + compileOnly "io.goodforgod:graalvm-hint-annotations:1.2.0" + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + plugins { + kotlin("kapt") + } + + dependencies { + kapt("io.goodforgod:graalvm-hint-processor:1.2.0") + compileOnly("io.goodforgod:graalvm-hint-annotations:1.2.0") + } + ``` + +Затем разметьте интерфейс `@KoraApp`, чтобы объявить точку входа и ресурсы для встраивания — процессор сгенерирует соответствующую конфигурацию `native-image` во время компиляции: + +===! ":fontawesome-brands-java: `Java`" + + ```java + import io.goodforgod.graalvm.hint.annotation.NativeImageHint; + import io.goodforgod.graalvm.hint.annotation.ResourceHint; + + @ResourceHint(include = {"openapi/http-server.yaml"}) + @NativeImageHint(name = "application", entrypoint = Application.class) + @KoraApp + public interface Application { + + static void main(String[] args) { + KoraApplication.run(ApplicationGraph::graph); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import io.goodforgod.graalvm.hint.annotation.NativeImageHint + import io.goodforgod.graalvm.hint.annotation.ResourceHint + + @ResourceHint(include = ["openapi/http-server.yaml"]) + @NativeImageHint(name = "application", entrypoint = Application::class) + @KoraApp + interface Application { + + companion object { + @JvmStatic + fun main(args: Array) { + KoraApplication.run(ApplicationGraph::graph) + } + } + } + ``` + +## Модули { #modules } + +Модули, для которых в Kora уже предусмотрена необходимая часть настроек для `нативного образа`: - [Конфигурация](config.md) -- [Json](json.md) -- [Logback](logging-slf4j.md) +- [JSON](json.md) +- [Логирование Logback](logging-slf4j.md) - [Пробы](probes.md) - [Метрики](metrics.md) -- [Трасcировка](tracing.md) -- [HTTP сервер](http-server.md) -- [HTTP клиент](http-client.md) -- [OpenAPI генерация](openapi-codegen.md) -- [OpenAPI отображение](openapi-management.md) +- [Трассировка](tracing.md) +- [HTTP-сервер](http-server.md) +- [HTTP-клиент](http-client.md) +- [Генерация OpenAPI-кода](openapi-codegen.md) +- [Отображение OpenAPI](openapi-management.md) - [База данных JDBC (Postgres)](database-jdbc.md) - [База данных R2DBC (Postgres)](database-r2dbc.md) -- [База данных Vertx (Postgres)](database-vertx.md) +- [База данных Vert.x (Postgres)](database-vertx.md) - [База данных Cassandra](database-cassandra.md) - [Kafka](kafka.md) -- [gRPC сервер](grpc-server.md) -- [gRPC клиент](grpc-client.md) +- [gRPC-сервер](grpc-server.md) +- [gRPC-клиент](grpc-client.md) - [Отказоустойчивость](resilient.md) -- [Кеш](cache.md) +- [Кэш](cache.md) - [Валидация](validation.md) - [Планировщик](scheduling.md) - [Логирование](logging-aspect.md) -Посмотреть примеры рабочих нативных сервисов можно в [репозитории с примерами](https://github.com/kora-projects/kora-examples). +Каждый из этих модулей поставляет свою конфигурацию `META-INF/native-image` внутри собственного JAR, поэтому настройки применяются автоматически, как только зависимость оказывается в classpath; основные флаги инициализации классов и виртуальных потоков берутся из `ru.tinkoff.kora:application-graph` и `ru.tinkoff.kora:common`. + +Готовые примеры сборки через Gradle и Docker можно посмотреть в [репозитории с примерами](https://github.com/kora-projects/kora-examples/tree/master/examples/graalvm). diff --git a/mkdocs/docs/ru/documentation/grpc-client.md b/mkdocs/docs/ru/documentation/grpc-client.md index 679e441..ad66b7d 100644 --- a/mkdocs/docs/ru/documentation/grpc-client.md +++ b/mkdocs/docs/ru/documentation/grpc-client.md @@ -4,9 +4,16 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora gRPC client generation, protobuf Gradle plugin setup, client configuration, generated services, interceptors, and mapping; key triggers include GrpcClientModule, @GrpcClient, @InterceptWith, GrpcClientConfig, GrpcClientInterceptor, protobuf plugin." --- -Модуль для подключения gRPC клиентов на основе функционала [grpc.io](https://grpc.io/docs/languages/java/basics/) +`gRPC-клиент` вызывает удалённые службы, используя контракт `protobuf` и транспорт `HTTP/2`. +В Kora клиент строится поверх сгенерированных классов `stub` библиотеки `grpc-java`: модуль создаёт `ManagedChannel`, подключает перехватчики и регистрирует готовые к использованию экземпляры `stub` в графе приложения. -Если нужен пошаговый разбор перед справочным описанием, смотрите [gRPC клиент](../guides/grpc-client.md) и [gRPC клиент продвинутый](../guides/grpc-client-advanced.md). +Для каждой службы Kora делает доступными для внедрения сгенерированные stub-классы (`BlockingStub`, `FutureStub`, асинхронный `Stub` и корутинный stub для Kotlin), +исходный `io.grpc.Channel` и итоговый `GrpcClientConfig`, различая каждый клиент по `@Tag` сгенерированного класса службы +(например, `@Tag(SimpleServiceGrpc.class)`). + +Транспорт gRPC-клиента использует Netty, поэтому общие настройки `event loop` и транспорта можно задать в разделе [Netty](netty.md). + +Если нужен пошаговый разбор перед справочным описанием, смотрите [gRPC-клиент](../guides/grpc-client.md) и [продвинутый gRPC-клиент](../guides/grpc-client-advanced.md). ## Подключение { #dependency } @@ -15,7 +22,7 @@ agent: [Зависимость](general.md#dependencies) `build.gradle`: ```groovy implementation "ru.tinkoff.kora:grpc-client" - implementation "io.grpc:grpc-protobuf:1.62.2" + implementation "io.grpc:grpc-protobuf:1.74.0" implementation "javax.annotation:javax.annotation-api:1.3.2" ``` @@ -30,7 +37,7 @@ agent: [Зависимость](general.md#dependencies) `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:grpc-client") - implementation("io.grpc:grpc-protobuf:1.62.2") + implementation("io.grpc:grpc-protobuf:1.74.0") implementation("javax.annotation:javax.annotation-api:1.3.2") ``` @@ -42,7 +49,8 @@ agent: ### Плагин { #plugin } -Код для gRPC-клиента создается с помощью [protobuf gradle plugin](https://github.com/google/protobuf-gradle-plugin). +Код `gRPC-клиента` создаётся с помощью [Gradle-плагина protobuf](https://github.com/google/protobuf-gradle-plugin). +Плагин генерирует классы Java-сообщений из контракта `protobuf` и gRPC-классы `stub`, которые затем используются Kora. ===! ":fontawesome-brands-java: `Java`" @@ -55,7 +63,7 @@ agent: protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { all()*.plugins { grpc {} } @@ -83,7 +91,7 @@ agent: protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { ofSourceSet("main").forEach { it.plugins { id("grpc") { } } } @@ -100,93 +108,290 @@ agent: ## Конфигурация { #configuration } -Сервис gRPC с именем `SimpleService` будет иметь конфигурацию с путем `grpcClient.SimpleService`. +`gRPC-клиент` для службы `SimpleService` будет иметь путь конфигурации `grpcClient.SimpleService`. -Пример полной конфигурации, описанной в классе `GrpcClientConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации: ===! ":material-code-json: `Hocon`" ```javascript grpcClient { SimpleService { - url = "grpc://localhost:8090" //(1)! + url = "http://localhost:8090" //(1)! timeout = "10s" //(2)! - keepAliveTime = "0s" //(3)! - keepAliveTimeout = "0s" //(4)! - loadBalancingPolicy = "pick_first" //(5)! - telemetry { - logging { - enabled = false //(6)! + } + } + ``` + + 1. `URL` сервера, куда будут отправляться запросы (`обязательный`, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию не указано, необязательно). Значение применяется как `deadline`, если у вызова ещё нет собственного `deadline`. + +=== ":simple-yaml: `YAML`" + + ```yaml + grpcClient: + SimpleService: + url: "http://localhost:8090" #(1)! + timeout: "10s" #(2)! + ``` + + 1. `URL` сервера, куда будут отправляться запросы (`обязательный`, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию не указано, необязательно). Значение применяется как `deadline`, если у вызова ещё нет собственного `deadline`. + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `GrpcClientConfig`: + + ===! ":material-code-json: `Hocon`" + + ```javascript + grpcClient { + SimpleService { + url = "http://localhost:8090" //(1)! + timeout = "10s" //(2)! + keepAliveTime = "0s" //(3)! + keepAliveTimeout = "0s" //(4)! + loadBalancingPolicy = "pick_first" //(5)! + defaultServiceConfig { //(6)! + loadBalancingConfig = [ + { + round_robin = {} + } + ] } - metrics { - enabled = true //(7)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(8)! - tags = { // (9)! - "key1" = "value1" - "key2" = "value2" + telemetry { + logging { + enabled = false //(7)! } - } - tracing { - enabled = true //(10)! - attributes = { // (11)! - "key1" = "value1" - "key2" = "value2" + metrics { + enabled = true //(8)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(9)! + tags = { // (10)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(11)! + attributes = { // (12)! + "key1" = "value1" + "key2" = "value2" + } } } } } + ``` + + 1. `URL` сервера, куда будут отправляться запросы (`обязательная`, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию не указано, необязательно). Значение применяется как `deadline`, если у вызова ещё нет собственного `deadline`. + 3. Интервал между gRPC-фреймами `PING` (по умолчанию не указано, необязательно). + 4. Время ожидания подтверждения фрейма `PING` (по умолчанию не указано, необязательно). Если подтверждение не получено за это время, соединение закрывается. + 5. Политика балансировки нагрузки для `ManagedChannelBuilder` (по умолчанию не указано, необязательно). + 6. Стандартная конфигурация службы gRPC, передаваемая в `ManagedChannelBuilder.defaultServiceConfig` (по умолчанию не указано, необязательно). + 7. Включает логирование модуля (по умолчанию: `false`). + 8. Включает метрики модуля (по умолчанию: `true`). + 9. Настраивает [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 10. Дополнительные теги для метрик (по умолчанию: `{}`). + 11. Включает трассировку модуля (по умолчанию: `true`). + 12. Дополнительные атрибуты для трассировки (по умолчанию: `{}`). + + === ":simple-yaml: `YAML`" + + ```yaml + grpcClient: + SimpleService: + url: "http://localhost:8090" #(1)! + timeout: "10s" #(2)! + keepAliveTime: "0s" #(3)! + keepAliveTimeout: "0s" #(4)! + loadBalancingPolicy: "pick_first" #(5)! + defaultServiceConfig: #(6)! + loadBalancingConfig: + - round_robin: {} + telemetry: + logging: + enabled: false #(7)! + metrics: + enabled: true #(8)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! + tags: #(10)! + key1: value1 + key2: value2 + tracing: + enabled: true #(11)! + attributes: #(12)! + key1: value1 + key2: value2 + ``` + + 1. `URL` сервера, куда будут отправляться запросы (`обязательная`, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию не указано, необязательно). Значение применяется как `deadline`, если у вызова ещё нет собственного `deadline`. + 3. Интервал между gRPC-фреймами `PING` (по умолчанию не указано, необязательно). + 4. Время ожидания подтверждения фрейма `PING` (по умолчанию не указано, необязательно). Если подтверждение не получено за это время, соединение закрывается. + 5. Политика балансировки нагрузки для `ManagedChannelBuilder` (по умолчанию не указано, необязательно). + 6. Стандартная конфигурация службы gRPC, передаваемая в `ManagedChannelBuilder.defaultServiceConfig` (по умолчанию не указано, необязательно). + 7. Включает логирование модуля (по умолчанию: `false`). + 8. Включает метрики модуля (по умолчанию: `true`). + 9. Настраивает [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 10. Дополнительные теги для метрик (по умолчанию: `{}`). + 11. Включает трассировку модуля (по умолчанию: `true`). + 12. Дополнительные атрибуты для трассировки (по умолчанию: `{}`). + +### Транспорт и TLS { #transport-tls } + +Схема в `url` выбирает транспорт при создании `ManagedChannel` (`ManagedChannelLifecycle`): + +- `http` — незашифрованный транспорт `plaintext` (`usePlaintext()` у построителя), порт по умолчанию `80`, если порт не указан. +- `https` — транспорт `TLS`, порт по умолчанию `443`, если порт не указан. +- любая другая схема — порт должен быть указан явно, иначе при определении порта по умолчанию во время запуска будет выброшено исключение `IllegalArgumentException` с сообщением `Unknown scheme ''`. Режим `plaintext` включается только для схемы `http`, поэтому остальные схемы используют транспорт gRPC по умолчанию (`TLS`), если недоступен порт `plaintext`. + +===! ":material-code-json: `Hocon`" + + ```javascript + grpcClient { + SimpleService { + url = "http://localhost:8090" // plaintext, порт по умолчанию 80 + } } ``` - 1. URL сервера куда делать запросы (**обязательный**) - 2. Максимальное время запроса (по умолчанию отсутвует) - 3. Устанавливает интервал времени между PING фреймами - 4. Таймаут времени для подтверждения PING фрейма. Если отправитель не получил подтверждение за данное время, соединение будет закрыто - 5. Устанавливает политику балансировки - 6. Включает логгирование модуля (по умолчанию `false`) - 7. Включает метрики модуля (по умолчанию `true`) - 8. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 9. Настройка тегов для метрик (опционально) - 10. Включает трассировку модуля (по умолчанию `true`) - 11. Настройка атрибутов для трассировки (опционально) - === ":simple-yaml: `YAML`" ```yaml grpcClient: SimpleService: - url: "grpc://localhost:8090" //(1)! - timeout: "10s" //(2)! - keepAliveTime: "0s" //(3)! - keepAliveTimeout: "0s" //(4)! - loadBalancingPolicy: "pick_first" //(5)! - telemetry: - logging: - enabled: false #(6)! - metrics: - enabled: true #(7)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(8)! - telemetry: - enabled: true #(9)! - ``` - - 1. URL сервера куда делать запросы (**обязательный**) - 2. Максимальное время запроса (по умолчанию отсутвует) - 3. Устанавливает интервал времени между PING фреймами - 4. Таймаут времени для подтверждения PING фрейма. Если отправитель не получил подтверждение за данное время, соединение будет закрыто - 5. Устанавливает политику балансировки - 6. Включает логгирование модуля (по умолчанию `false`) - 7. Включает метрики модуля (по умолчанию `true`) - 8. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 9. Включает трассировку модуля (по умолчанию `true`) - -Можно также настроить [Netty транспорт](netty.md). + url: "http://localhost:8090" # plaintext, порт по умолчанию 80 + ``` + +Для нестандартного `TLS` (`mTLS`, собственное хранилище доверенных сертификатов или транспорт, отличный от Netty) зарегистрируйте свой компонент `GrpcClientChannelFactory`. +Он создаёт `ManagedChannelBuilder` и может передавать `io.grpc.ChannelCredentials`; реализация по умолчанию — `GrpcNettyClientChannelFactory`, +которая привязывает канал к общей для Kora группе Netty `EventLoopGroup`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class TlsGrpcClientChannelFactory implements GrpcClientChannelFactory { + + @Override + public ManagedChannelBuilder forAddress(SocketAddress serverAddress) { + return NettyChannelBuilder.forAddress(serverAddress); + } + + @Override + public ManagedChannelBuilder forAddress(SocketAddress serverAddress, ChannelCredentials creds) { + return NettyChannelBuilder.forAddress(serverAddress, creds); + } + + @Override + public ManagedChannelBuilder forTarget(String target) { + return NettyChannelBuilder.forTarget(target); + } + + @Override + public ManagedChannelBuilder forTarget(String target, ChannelCredentials creds) { + return NettyChannelBuilder.forTarget(target, creds); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class TlsGrpcClientChannelFactory : GrpcClientChannelFactory { + + override fun forAddress(serverAddress: SocketAddress): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forAddress(serverAddress) + } + + override fun forAddress(serverAddress: SocketAddress, creds: ChannelCredentials): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forAddress(serverAddress, creds) + } + + override fun forTarget(target: String): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forTarget(target) + } + + override fun forTarget(target: String, creds: ChannelCredentials): ManagedChannelBuilder<*> { + return NettyChannelBuilder.forTarget(target, creds) + } + } + ``` + +Реализация для промышленного окружения также должна привязывать построитель к общей для Kora группе Netty `EventLoopGroup` и `NettyChannelFactory` +так же, как это делает `GrpcNettyClientChannelFactory`, вместо того чтобы позволять Netty создавать собственный event loop. + +### Ограничение по времени { #timeouts } + +Значение `timeout` применяется всегда включённым перехватчиком `GrpcClientConfigInterceptor` как `deadline` вызова, но **только если у вызова нет собственного deadline**. +Заданный для конкретного вызова deadline через stub всегда имеет приоритет над настроенным `timeout`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + // uses the configured grpcClient.SimpleService.timeout as the deadline + var response = stub.createUser(request); + + // overrides the configured timeout for this single call + var responseWithDeadline = stub.withDeadlineAfter(2, TimeUnit.SECONDS).createUser(request); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // uses the configured grpcClient.SimpleService.timeout as the deadline + val response = stub.createUser(request) + + // overrides the configured timeout for this single call + val responseWithDeadline = stub.withDeadlineAfter(2, TimeUnit.SECONDS).createUser(request) + ``` + +Когда истекает deadline, вызов завершается с ошибкой `StatusRuntimeException`, несущей `Status.DEADLINE_EXCEEDED`. + +### Конфигурация канала { #channel-config } + +- `keepAliveTime` / `keepAliveTimeout` соответствуют настройкам `PING` у `ManagedChannelBuilder`. `keepAliveTime` — интервал между фреймами `PING` протокола `HTTP/2` на простаивающем соединении; `keepAliveTimeout` — как долго ждать подтверждения `PING` перед закрытием соединения. Оба отключены, если не заданы. +- `loadBalancingPolicy` соответствует `ManagedChannelBuilder.defaultLoadBalancingPolicy`. Значение по умолчанию в gRPC — `pick_first` (единственное соединение с первым разрешённым адресом); `round_robin` распределяет вызовы по всем разрешённым адресам и обычно используется с DNS-адресами, возвращающими несколько записей `A`/`AAAA`. +- `defaultServiceConfig` передаётся как есть в `ManagedChannelBuilder.defaultServiceConfig` и содержит нативную карту [конфигурации службы](https://github.com/grpc/grpc/blob/master/doc/service_config.md) gRPC (`loadBalancingConfig`, `methodConfig` для отдельных методов с политикой повторов/хеджирования и т. д.). Она описана обёрткой `DefaultServiceConfig` над `Map`. + +### Настройщик построителя канала { #builder-configurer } + +Если конфигурации через файл недостаточно, можно зарегистрировать компонент `GrpcClientBuilderConfigurer`. +Он получает уже подготовленный `ManagedChannelBuilder` и позволяет настроить канал в коде до его создания. +Сначала применяются настройки из `GrpcClientConfig`, затем вызывается `GrpcClientBuilderConfigurer`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class CustomGrpcClientBuilderConfigurer implements GrpcClientBuilderConfigurer { + @Override + public ManagedChannelBuilder configure(ManagedChannelBuilder builder) { + return builder.maxInboundMessageSize(8 * 1024 * 1024); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class CustomGrpcClientBuilderConfigurer : GrpcClientBuilderConfigurer { + override fun configure(builder: ManagedChannelBuilder<*>): ManagedChannelBuilder<*> { + return builder.maxInboundMessageSize(8 * 1024 * 1024) + } + } + ``` + +Также можно настроить [транспорт Netty](netty.md). Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#grpc-client). -## Сервис { #service } +## Служба { #service } -Созданные gRPC сервисы можно внедрять как зависимости: +Созданные экземпляры gRPC `stub` можно внедрять как зависимости: ===! ":fontawesome-brands-java: `Java`" @@ -194,7 +399,7 @@ agent: @KoraApp public interface Application extends HoconConfigModule, GrpcClientModule { - default SomeService(SimpleServiceGrpc.SimpleServiceBlockingStub grpcService) { + default SomeService someService(SimpleServiceGrpc.SimpleServiceBlockingStub grpcService) { return new SomeService(grpcService); } } @@ -205,25 +410,220 @@ agent: ```kotlin @KoraApp interface Application : HoconConfigModule, GrpcClientModule { - fun SomeService(grpcService: SimpleServiceGrpc.SimpleServiceBlockingStub?) { + fun someService(grpcService: SimpleServiceGrpc.SimpleServiceBlockingStub): SomeService { return SomeService(grpcService) } } ``` +stub также можно внедрить напрямую в конструктор `@Component`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final SimpleServiceGrpc.SimpleServiceBlockingStub grpcService; + + public SomeService(SimpleServiceGrpc.SimpleServiceBlockingStub grpcService) { + this.grpcService = grpcService; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val grpcService: SimpleServiceGrpc.SimpleServiceBlockingStub) + ``` + +### Типы реализаций { #stub-types } + +Плагин `protobuf` генерирует для одной службы (`SimpleService`) несколько stub-классов. Каждый доступен для внедрения простым объявлением соответствующего типа; +на самом stub указывать `@Tag` не нужно (Kora самостоятельно разрешает помеченный тегом `Channel`): + +| Тип stub | Модель вызова | Когда использовать | +|----------------------------------------|------------------------------------------------------------------------|------------------------------------------------------| +| `SimpleServiceBlockingStub` | Синхронная; возвращает ответ напрямую (или `Iterator` для серверной потоковой передачи) | Блокирующий код, простейший стиль вызова | +| `SimpleServiceFutureStub` | Асинхронная; возвращает `ListenableFuture` (только унарные вызовы) | Неблокирующий код с использованием `ListenableFuture`| +| `SimpleServiceStub` (асинхронный) | Асинхронная; передаёт результаты через обратные вызовы `StreamObserver` | Любая потоковая передача, асинхронные вызовы на обратных вызовах | +| Корутинный stub для Kotlin | `suspend`-функции и `Flow` | Идиоматичные корутины Kotlin | + +`BlockingStub`, `FutureStub` и асинхронный `Stub` связываются расширением обработчика аннотаций (`GrpcClientExtension`), которое обнаруживает stub-типы `@GrpcGenerated` +и вызывает сгенерированную фабрику `newBlockingStub` / `newFutureStub` / `newStub` для помеченного тегом `Channel`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends HoconConfigModule, GrpcClientModule { + + default BlockingCaller blockingCaller(SimpleServiceGrpc.SimpleServiceBlockingStub stub) { + return new BlockingCaller(stub); + } + + default FutureCaller futureCaller(SimpleServiceGrpc.SimpleServiceFutureStub stub) { + return new FutureCaller(stub); + } + + default AsyncCaller asyncCaller(SimpleServiceGrpc.SimpleServiceStub stub) { + return new AsyncCaller(stub); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : HoconConfigModule, GrpcClientModule { + + fun blockingCaller(stub: SimpleServiceGrpc.SimpleServiceBlockingStub) = BlockingCaller(stub) + + fun futureCaller(stub: SimpleServiceGrpc.SimpleServiceFutureStub) = FutureCaller(stub) + + fun asyncCaller(stub: SimpleServiceGrpc.SimpleServiceStub) = AsyncCaller(stub) + } + ``` + +Для вызовов на корутинах Kotlin сгенерируйте корутинный stub с помощью [генератора gRPC Kotlin](https://github.com/grpc/grpc-kotlin) +(`io.grpc:protoc-gen-grpc-kotlin`). Сгенерированный stub наследуется от `io.grpc.kotlin.AbstractCoroutineStub` и помечен аннотацией `@StubFor`; +далее обработчик символов KSP генерирует модуль Kora, который предоставляет его как `@DefaultComponent`, привязанный к помеченному тегом `Channel`, поэтому он внедряется таким же образом: + +===! ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : HoconConfigModule, GrpcClientModule { + + fun coroutineCaller(stub: SimpleServiceGrpcKt.SimpleServiceCoroutineStub) = CoroutineCaller(stub) + } + ``` + +### Стили вызова { #call-styles } + +Форма `rpc` в контракте `.proto` (одиночный или `stream` запрос/ответ) определяет сигнатуру сгенерированного метода. +В примерах ниже базовый контракт дополнен всеми четырьмя стилями вызова: + +```protobuf +service SimpleService { + rpc unary(RequestEvent) returns (ResponseEvent) {} // унарный + rpc serverStream(RequestEvent) returns (stream ResponseEvent) {} // серверная потоковая передача + rpc clientStream(stream RequestEvent) returns (ResponseEvent) {} // клиентская потоковая передача + rpc biDiStream(stream RequestEvent) returns (stream ResponseEvent) {} // двунаправленная потоковая передача +} +``` + +Запросы создаются с помощью сгенерированных построителей сообщений (`RequestEvent.newBuilder()`). +В Java унарные вызовы и серверная потоковая передача доступны у `BlockingStub`, тогда как клиентская потоковая передача и двунаправленные вызовы требуют асинхронного `Stub` +(у них нет блокирующего варианта). Корутинный stub Kotlin выражает каждый стиль через `suspend`-функции и `Flow`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var request = RequestEvent.newBuilder().setName("bob").setCode("b1").build(); + + // унарный — BlockingStub + ResponseEvent unary = blockingStub.unary(request); + + // унарный — FutureStub + ListenableFuture future = futureStub.unary(request); + + // серверная потоковая передача — BlockingStub возвращает итератор + Iterator responses = blockingStub.serverStream(request); + + // серверная потоковая передача — асинхронный Stub передаёт результаты в StreamObserver + asyncStub.serverStream(request, new StreamObserver<>() { + @Override public void onNext(ResponseEvent value) { /* ... */ } + @Override public void onError(Throwable t) { /* ... */ } + @Override public void onCompleted() { /* ... */ } + }); + + // клиентская потоковая передача — асинхронный Stub, пишем запросы, читаем один ответ + StreamObserver responseObserver = new StreamObserver<>() { + @Override public void onNext(ResponseEvent value) { /* единственный ответ */ } + @Override public void onError(Throwable t) { /* ... */ } + @Override public void onCompleted() { /* ... */ } + }; + StreamObserver requestObserver = asyncStub.clientStream(responseObserver); + requestObserver.onNext(request); + requestObserver.onCompleted(); + + // двунаправленная потоковая передача — асинхронный Stub, потоки с обеих сторон + StreamObserver bidi = asyncStub.biDiStream(responseObserver); + bidi.onNext(request); + bidi.onCompleted(); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = RequestEvent.newBuilder().setName("bob").setCode("b1").build() + + // унарный — suspend-функция + val response: ResponseEvent = coroutineStub.unary(request) + + // серверная потоковая передача — возвращает Flow + val serverFlow: Flow = coroutineStub.serverStream(request) + serverFlow.collect { event -> /* ... */ } + + // клиентская потоковая передача — принимает Flow, возвращает один ответ + val clientResponse: ResponseEvent = coroutineStub.clientStream(flowOf(request)) + + // двунаправленная потоковая передача — Flow на вход, Flow на выход + val biDiFlow: Flow = coroutineStub.biDiStream(flowOf(request)) + biDiFlow.collect { event -> /* ... */ } + ``` + +### Внедрение Channel и конфигурации { #inject-channel-config } + +Для продвинутого или ручного создания stub можно внедрить исходный `io.grpc.Channel` и итоговый `GrpcClientConfig`, +пометив их сгенерированным классом службы. Оба предоставляются `GrpcClientExtension`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends HoconConfigModule, GrpcClientModule { + + default SomeService someService(@Tag(SimpleServiceGrpc.class) Channel channel, + @Tag(SimpleServiceGrpc.class) GrpcClientConfig config) { + return new SomeService(SimpleServiceGrpc.newBlockingStub(channel), config.url()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : HoconConfigModule, GrpcClientModule { + + fun someService( + @Tag(SimpleServiceGrpc::class) channel: Channel, + @Tag(SimpleServiceGrpc::class) config: GrpcClientConfig + ): SomeService = SomeService(SimpleServiceGrpc.newBlockingStub(channel), config.url()) + } + ``` + ## Перехватчики { #interceptors } -[Перехватчики](https://grpc.github.io/grpc-java/javadoc/io/grpc/ClientInterceptor.html) позволяют перехватывать запросы перед тем, как они будут переданы сервисам. +[Перехватчики](https://grpc.github.io/grpc-java/javadoc/io/grpc/ClientInterceptor.html) позволяют перехватывать запросы до того, как они будут переданы службам. -### Стандартные { #default } +### По умолчанию { #default } -При запуске клиента по-умолчанию используются следующие перехватчики: +По умолчанию при запуске клиента используются следующие перехватчики: -- `GrpcClientConfigInterceptor` +- `GrpcClientConfigInterceptor` — применяет `timeout` как `deadline` вызова, если у вызова его нет. +- `GrpcClientTelemetryInterceptor`, если для клиента доступна телеметрия. ### Собственные { #custom } -Для добавления собственного перехватчика требуется зарегистрировать перехватчика как компонент с тегом сервиса. +В отличие от [HTTP-клиента](http-client.md#interceptors), у перехватчиков gRPC нет уровней метода/класса/глобального уровня. +Каждый перехватчик действует **в пределах одного клиента** — за счёт пометки компонента сгенерированным классом службы (`@Tag(SimpleServiceGrpc.class)`). +Зарегистрируйте перехватчик как компонент с этим тегом: ===! ":fontawesome-brands-java: `Java`" @@ -245,7 +645,7 @@ agent: @Tag(SimpleServiceGrpc::class) @Component class MyClientInterceptor : ClientInterceptor { - fun interceptCall( + override fun interceptCall( method: MethodDescriptor, callOptions: CallOptions, next: Channel @@ -255,4 +655,310 @@ agent: } ``` -Либо можно модифицировать сервис посредствам [GraphInterceptor](container.md#indirect-dependency). +Чтобы применить один компонент-перехватчик к нескольким клиентам («общий» перехватчик), укажите ему несколько значений `@Tag` — по одному сгенерированному классу службы на каждый клиент: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag({SimpleServiceGrpc.class, OtherServiceGrpc.class}) + @Component + public final class SharedInterceptor implements ClientInterceptor { + @Override + public ClientCall interceptCall(MethodDescriptor method, CallOptions callOptions, Channel next) { + return next.newCall(method, callOptions); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(SimpleServiceGrpc::class, OtherServiceGrpc::class) + @Component + class SharedInterceptor : ClientInterceptor { + override fun interceptCall( + method: MethodDescriptor, + callOptions: CallOptions, + next: Channel + ): ClientCall { + return next.newCall(method, callOptions) + } + } + ``` + +**Порядок выполнения:** + +`ManagedChannelLifecycle` собирает все перехватчики, помеченные для службы, как `All` и применяет их в фиксированном порядке: +сначала ваши собственные перехватчики, затем перехватчик телеметрии (если телеметрия включена), и последним — перехватчик конфигурации/deadline. + +``` +Запрос → Собственные перехватчики → Перехватчик телеметрии → Перехватчик конфигурации (deadline) → gRPC-сервер +``` + +Поскольку перехватчик deadline выполняется последним, deadline, установленный собственным перехватчиком в `CallOptions`, сохраняется, а настроенный `timeout` +применяется только тогда, когда его не задал ни один более ранний перехватчик (или `withDeadlineAfter` для конкретного вызова). + +В качестве альтернативы можно изменить `stub` с помощью [GraphInterceptor](container.md#component-inspection). + +## Авторизация { #authorization } + +В gRPC нет отдельного модуля авторизации: авторизация выполняется с помощью `ClientInterceptor`, помеченного классом службы, который добавляет заголовок +`Authorization` (или API-ключа) в `Metadata` исходящего вызова. Перехватчик оборачивает вызов в `ForwardingClientCall.SimpleForwardingClientCall` +и помещает заголовок в `start(...)`, до отправки запроса. + +### Bearer { #bearer } + +Перехватчик [Bearer](https://swagger.io/docs/specification/authentication/bearer-authentication/) читает токен из вашего собственного поставщика и +помещает его в заголовок `Authorization` каждого вызова. `TokenProvider` ниже — ваш собственный компонент: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(SimpleServiceGrpc.class) + @Component + public final class BearerAuthInterceptor implements ClientInterceptor { + + private static final Metadata.Key AUTHORIZATION = + Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER); + + private final TokenProvider tokenProvider; + + public BearerAuthInterceptor(TokenProvider tokenProvider) { + this.tokenProvider = tokenProvider; + } + + @Override + public ClientCall interceptCall(MethodDescriptor method, CallOptions callOptions, Channel next) { + return new ForwardingClientCall.SimpleForwardingClientCall<>(next.newCall(method, callOptions)) { + @Override + public void start(Listener responseListener, Metadata headers) { + headers.put(AUTHORIZATION, "Bearer " + tokenProvider.getToken()); + super.start(responseListener, headers); + } + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(SimpleServiceGrpc::class) + @Component + class BearerAuthInterceptor(private val tokenProvider: TokenProvider) : ClientInterceptor { + + override fun interceptCall( + method: MethodDescriptor, + callOptions: CallOptions, + next: Channel + ): ClientCall { + return object : ForwardingClientCall.SimpleForwardingClientCall(next.newCall(method, callOptions)) { + override fun start(responseListener: Listener, headers: Metadata) { + headers.put(AUTHORIZATION, "Bearer " + tokenProvider.getToken()) + super.start(responseListener, headers) + } + } + } + + companion object { + private val AUTHORIZATION: Metadata.Key = + Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER) + } + } + ``` + +### ApiKey { #apikey } + +Перехватчик [API-ключа](https://swagger.io/docs/specification/authentication/api-keys/) помещает статический ключ в собственный заголовок метаданных (например, `X-API-KEY`). +Ключ читается из интерфейса [`@ConfigSource`](config.md), внедрённого в перехватчик: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(SimpleServiceGrpc.class) + @Component + public final class ApiKeyInterceptor implements ClientInterceptor { + + private static final Metadata.Key API_KEY = + Metadata.Key.of("X-API-KEY", Metadata.ASCII_STRING_MARSHALLER); + + private final String apiKey; + + public ApiKeyInterceptor(ApiKeyConfig config) { //(1)! + this.apiKey = config.apiKey(); + } + + @Override + public ClientCall interceptCall(MethodDescriptor method, CallOptions callOptions, Channel next) { + return new ForwardingClientCall.SimpleForwardingClientCall<>(next.newCall(method, callOptions)) { + @Override + public void start(Listener responseListener, Metadata headers) { + headers.put(API_KEY, apiKey); + super.start(responseListener, headers); + } + }; + } + } + ``` + + 1. Любой интерфейс `@ConfigSource`, предоставляющий API-ключ, например `String apiKey();` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(SimpleServiceGrpc::class) + @Component + class ApiKeyInterceptor(config: ApiKeyConfig) : ClientInterceptor { //(1)! + + private val apiKey: String = config.apiKey() + + override fun interceptCall( + method: MethodDescriptor, + callOptions: CallOptions, + next: Channel + ): ClientCall { + return object : ForwardingClientCall.SimpleForwardingClientCall(next.newCall(method, callOptions)) { + override fun start(responseListener: Listener, headers: Metadata) { + headers.put(API_KEY, apiKey) + super.start(responseListener, headers) + } + } + } + + companion object { + private val API_KEY: Metadata.Key = + Metadata.Key.of("X-API-KEY", Metadata.ASCII_STRING_MARSHALLER) + } + } + ``` + + 1. Любой интерфейс `@ConfigSource`, предоставляющий API-ключ, например `fun apiKey(): String` + +## Обработка ошибок { #error-handling } + +Неуспешный вызов gRPC выбрасывает `io.grpc.StatusRuntimeException`. Его `getStatus()` несёт `Status.Code` +([коды статусов](https://grpc.io/docs/guides/status-codes/)), например `UNAVAILABLE` (сервер недоступен), `DEADLINE_EXCEEDED` (истёк `timeout`/deadline), +`UNAUTHENTICATED` (отклонённые учётные данные) или `INVALID_ARGUMENT`. Метаданные ответа доступны через `getTrailers()`. + +**Причины:** + +- `UNAVAILABLE` — неверный `url`, несоответствие plaintext/TLS или сервер не работает. +- `DEADLINE_EXCEEDED` — превышен настроенный `timeout` или `withDeadlineAfter` для конкретного вызова. +- `UNAUTHENTICATED` / `PERMISSION_DENIED` — отсутствуют или недействительны метаданные авторизации. + +**Рекомендации:** + +- Ветвитесь по `e.getStatus().getCode()`, а не по типу исключения. +- Используйте аспекты [отказоустойчивости](resilient.md) (`@Retry`, `@CircuitBreaker`, `@Timeout`) на оборачивающем методе службы для временных сбоев. + +===! ":fontawesome-brands-java: `Java`" + + ```java + try { + var response = stub.createUser(request); + } catch (StatusRuntimeException e) { + var code = e.getStatus().getCode(); + if (code == Status.Code.DEADLINE_EXCEEDED) { + // превышен timeout / deadline + } else if (code == Status.Code.UNAVAILABLE) { + // сервер недоступен + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + try { + val response = stub.createUser(request) + } catch (e: StatusRuntimeException) { + when (e.status.code) { + Status.Code.DEADLINE_EXCEEDED -> { /* превышен timeout / deadline */ } + Status.Code.UNAVAILABLE -> { /* сервер недоступен */ } + else -> throw e + } + } + ``` + +## Тестирование { #testing } + +gRPC-клиент тестируется как любой другой компонент Kora с помощью [`@KoraAppTest`](junit5.md). +Реализуйте `KoraAppTestConfigModifier`, чтобы задать `url` (например, через подстановку переменной окружения `GRPC_URL`, используемую в примере), +внедрите службу на основе stub через `@TestComponent`, соберите запрос сгенерированным построителем и проверьте `StatusRuntimeException`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class GrpcClientTests implements KoraAppTestConfigModifier { + + @TestComponent + private RootService service; + + @Override + public KoraConfigModification config() { + return KoraConfigModification.ofSystemProperty("GRPC_URL", "grpc://localhost:8090"); + } + + @Test + void createUser() { + var event = Message.RequestEvent.newBuilder() + .setName("bob") + .setCode("b1") + .build(); + + var stub = service.service(); + assertThrows(StatusRuntimeException.class, () -> stub.createUser(event)); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class GrpcClientTests : KoraAppTestConfigModifier { + + @TestComponent + lateinit var service: RootService + + override fun config(): KoraConfigModification = + KoraConfigModification.ofSystemProperty("GRPC_URL", "grpc://localhost:8090") + + @Test + fun createUser() { + val event = Message.RequestEvent.newBuilder() + .setName("bob") + .setCode("b1") + .build() + + val stub = service.service() + assertThrows(StatusRuntimeException::class.java) { stub.createUser(event) } + } + } + ``` + +## Телеметрия { #telemetry } + +Логирование, метрики и трассировка по умолчанию настраиваются через блок `telemetry` в [конфигурации](#configuration) и +описаны в разделе [Справочник метрик](metrics.md#grpc-client). + +Чтобы настроить собираемые сигналы, переопределите SPI-фабрики телеметрии как компоненты: `GrpcClientTelemetryFactory` +(вся телеметрия), `GrpcClientLoggerFactory`, `GrpcClientMetricsFactory` или `GrpcClientTracerFactory`. +Реализации по умолчанию связываются `GrpcClientModule`; предоставление собственного компонента заменяет соответствующую реализацию по умолчанию. + +## Телеметрия { #telemetry } + +gRPC Client использует контракт телеметрии для логирования, метрик и трассировки вызовов. +Конфигурация телеметрии (секция `telemetry { logging / metrics / tracing }`) описана в разделе [Конфигурация](#configuration). +Точки расширения находятся в `ru.tinkoff.kora.grpc.client.common.telemetry`. + +Для каждого gRPC-вызова создаётся `GrpcClientTelemetry.GrpcClientTelemetryContext`, который закрывается по завершении вызова. +Вызов описывается через параметры обработчика телеметрии, включая сервис, метод, статус ответа и длительность. + +Фабрика по умолчанию `DefaultGrpcClientTelemetryFactory` объединяет три фабрики: +- `GrpcClientLoggerFactory` строит `GrpcClientLogger` для логирования начала/конца вызова; +- `GrpcClientMetricsFactory` строит `GrpcClientMetrics` для записи метрик вызовов; +- `GrpcClientTracerFactory` строит `GrpcClientTracer` для распределённой трассировки. + +Метрики и трассировка описаны в разделе [Справочник метрик](metrics.md#grpc-client). diff --git a/mkdocs/docs/ru/documentation/grpc-server.md b/mkdocs/docs/ru/documentation/grpc-server.md index d442fdf..84295f2 100644 --- a/mkdocs/docs/ru/documentation/grpc-server.md +++ b/mkdocs/docs/ru/documentation/grpc-server.md @@ -1,12 +1,16 @@ --- -description: "Explains Kora gRPC server generation, protobuf Gradle plugin setup, server configuration, handlers, interceptors, reflection, and debugging. Use when working with GrpcServerModule, @GrpcService, @InterceptWith, GrpcServerConfig, GrpcServerInterceptor, Server Reflection." +description: "Explains Kora gRPC server: protobuf Gradle plugin setup, server configuration, unary and streaming handlers, io.grpc.Status error handling, ServerInterceptor interceptors and their execution order, lifecycle and readiness, telemetry, and reflection. Use when working with GrpcServerModule, GrpcServerConfig, GrpcServerBuilderConfigurer, ServerInterceptor, StreamObserver, reflectionEnabled, Server Reflection." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora gRPC server generation, protobuf Gradle plugin setup, server configuration, handlers, interceptors, reflection, and debugging; key triggers include GrpcServerModule, @GrpcService, @InterceptWith, GrpcServerConfig, GrpcServerInterceptor, Server Reflection." + use_when: "Use this file for Kora docs or implementation questions about the Kora gRPC server: protobuf Gradle plugin setup, server configuration, unary and streaming handlers, io.grpc.Status error handling, ServerInterceptor interceptors and their execution order, scoping and metadata authorization, lifecycle and readiness, telemetry, and reflection; key triggers include GrpcServerModule, GrpcServerConfig, GrpcServerBuilderConfigurer, ServerInterceptor, StreamObserver, reflectionEnabled, Server Reflection. Note: gRPC server interceptors are global io.grpc.ServerInterceptor beans only; there is no @GrpcService or @InterceptWith annotation in this module." --- -Модуль для подключения gRPC серверных обработчиков на основе функционала [grpc.io](https://grpc.io/docs/languages/java/basics/) +Модуль запускает `gRPC-сервер` на основе [`grpc-java`](https://grpc.io/docs/languages/java/basics/) и подключает к нему обработчики из графа приложения. +Обработчик — это `BindableService`, обычно класс, который наследует сгенерированный `...ImplBase` и реализует унарные или потоковые методы `RPC`. -Если нужен пошаговый разбор перед справочным описанием, смотрите [gRPC сервер](../guides/grpc-server.md) и [gRPC сервер продвинутый](../guides/grpc-server-advanced.md). +Kora создает `NettyServerBuilder`, добавляет сервисы сервера, пользовательские и стандартные `ServerInterceptor`, управляет жизненным циклом сервера и участвует в проверках готовности приложения. +Если параметров конфигурации недостаточно, итоговый `NettyServerBuilder` можно дополнительно настроить в коде через `GrpcServerBuilderConfigurer`. + +Если нужен пошаговый разбор перед справочным описанием, смотрите [gRPC-сервер](../guides/grpc-server.md) и [продвинутый gRPC-сервер](../guides/grpc-server-advanced.md). ## Подключение { #dependency } @@ -15,7 +19,7 @@ agent: [Зависимость](general.md#dependencies) `build.gradle`: ```groovy implementation "ru.tinkoff.kora:grpc-server" - implementation "io.grpc:grpc-protobuf:1.62.2" + implementation "io.grpc:grpc-protobuf:1.74.0" implementation "javax.annotation:javax.annotation-api:1.3.2" ``` @@ -30,7 +34,7 @@ agent: [Зависимость](general.md#dependencies) `build.gradle.kts`: ```groovy implementation("ru.tinkoff.kora:grpc-server") - implementation("io.grpc:grpc-protobuf:1.62.2") + implementation("io.grpc:grpc-protobuf:1.74.0") implementation("javax.annotation:javax.annotation-api:1.3.2") ``` @@ -42,11 +46,11 @@ agent: ### Плагин { #plugin } -Код для gRPC-сервера создается с помощью [protobuf gradle plugin](https://github.com/google/protobuf-gradle-plugin). +Код для `gRPC-сервера` генерируется с помощью [gradle-плагина protobuf](https://github.com/google/protobuf-gradle-plugin). ===! ":fontawesome-brands-java: `Java`" - Плагин `build.gradle`: + Плагин в `build.gradle`: ```groovy plugins { id "com.google.protobuf" version "0.9.4" @@ -55,7 +59,7 @@ agent: protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + grpc { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { all()*.plugins { grpc {} } @@ -72,7 +76,7 @@ agent: === ":simple-kotlin: `Kotlin`" - Плагин `build.gradle.kts`: + Плагин в `build.gradle.kts`: ```groovy import com.google.protobuf.gradle.id @@ -83,7 +87,7 @@ agent: protobuf { protoc { artifact = "com.google.protobuf:protoc:3.25.3" } plugins { - id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.62.2" } + id("grpc") { artifact = "io.grpc:protoc-gen-grpc-java:1.74.0" } } generateProtoTasks { ofSourceSet("main").forEach { it.plugins { id("grpc") { } } } @@ -100,7 +104,29 @@ agent: ## Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `GrpcServerConfig` (указаны примеры значений или значения по умолчанию): +Обычно нужно задать только `port`; все остальные параметры имеют значения по умолчанию. +Минимальная конфигурация, которая привязывает порт и включает логирование: + +===! ":material-code-json: `Hocon`" + + ```javascript + grpcServer { + port = 8090 + telemetry.logging.enabled = true + } + ``` + +=== ":simple-yaml: `YAML`" + + ```yaml + grpcServer: + port: 8090 + telemetry: + logging: + enabled: true + ``` + +Основные параметры конфигурации: ===! ":material-code-json: `Hocon`" @@ -109,48 +135,13 @@ agent: port = 8090 //(1)! maxMessageSize = "4MiB" //(2)! reflectionEnabled = false //(3)! - shutdownWait = "30s" //(4)! - maxConnectionAge = "0s" //(5)! - maxConnectionAgeGrace = "0s" //(6)! - keepAliveTime = "0s" //(7)! - keepAliveTimeout = "0s" //(8)! - telemetry { - logging { - enabled = false //(9)! - } - metrics { - enabled = true //(10)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(11)! - tags = { // (12)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(13)! - attributes = { // (14)! - "key1" = "value1" - "key2" = "value2" - } - } - } } ``` - 1. Порт gRPC сервера - 2. Максимальный размер входящего сообщения (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 3. Включает сервис [gRPC Server Reflection](#reflection) - 4. Время ожидания обработки перед выключением сервера в случае [штатного завершения](container.md#component-lifecycle) - 5. Устанавливает пользовательский максимальный возраст соединения, при превышении которого соединение будет изящно прервано. К нему будет добавлен случайный коэфициент +/-10%. - 6. Устанавливает пользовательское штатное время для штатного завершения соединения. После достижения максимального возраста соединения у RPC будет штатное время для завершения. RPC, не завершившиеся вовремя, будут отменены, что позволит завершить соединение. - 7. Устанавливает интервал времени между PING фреймами - 8. Таймаут времени для подтверждения PING фрейма. Если отправитель не получил подтверждение за данное время, соединение будет закрыто - 9. Включает логгирование модуля (по умолчанию `false`) - 10. Включает метрики модуля (по умолчанию `true`) - 11. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 12. Настройка тегов для метрик (опционально) - 13. Включает трассировку модуля (по умолчанию `true`) - 14. Настройка атрибутов для трассировки (опционально) + 1. Порт `gRPC-сервера` (по умолчанию: `8090`). + 2. Максимальный размер входящего сообщения (по умолчанию: `4MiB`). + 3. Включает сервис [`gRPC Server Reflection`](#reflection) (по умолчанию: `false`). + 4. Включает виртуальные потоки для обработки вызовов, требует `Java 21+` (по умолчанию: `false`). === ":simple-yaml: `YAML`" @@ -159,74 +150,480 @@ agent: port: 8090 #(1)! maxMessageSize: "4MiB" #(2)! reflectionEnabled: false #(3)! - shutdownWait: "30s" #(4)! - maxConnectionAge: "0s" #(5)! - maxConnectionAgeGrace: "0s" #(6)! - keepAliveTime: "0s" #(7)! - keepAliveTimeout: "0s" #(8)! - telemetry: - logging: - enabled: false #(9)! - metrics: - enabled: true #(10)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(11)! - telemetry: - enabled: true #(12)! ``` - 1. Порт gRPC сервера - 2. Максимальный размер входящего сообщения (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 3. Включает сервис [gRPC Server Reflection](#reflection) - 4. Время ожидания обработки перед выключением сервера в случае [штатного завершения](container.md#component-lifecycle) - 5. Устанавливает пользовательский максимальный возраст соединения, при превышении которого соединение будет изящно прервано. К нему будет добавлен случайный коэфициент +/-10%. - 6. Устанавливает пользовательское штатное время для штатного завершения соединения. После достижения максимального возраста соединения у RPC будет штатное время для завершения. RPC, не завершившиеся вовремя, будут отменены, что позволит завершить соединение. - 7. Устанавливает интервал времени между PING фреймами - 8. Таймаут времени для подтверждения PING фрейма. Если отправитель не получил подтверждение за данное время, соединение будет закрыто - 9. Включает логгирование модуля (по умолчанию `false`) - 10. Включает метрики модуля (по умолчанию `true`) - 11. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 12. Включает трассировку модуля (по умолчанию `true`) - -Можно также настроить [Netty транспорт](netty.md). + 1. Порт `gRPC-сервера` (по умолчанию: `8090`). + 2. Максимальный размер входящего сообщения (по умолчанию: `4MiB`). + 3. Включает сервис [`gRPC Server Reflection`](#reflection) (по умолчанию: `false`). + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `GrpcServerConfig`: + + ===! ":material-code-json: `Hocon`" + + ```javascript + grpcServer { + port = 8090 //(1)! + maxMessageSize = "4MiB" //(2)! + reflectionEnabled = false //(3)! + shutdownWait = "30s" //(4)! + maxConnectionAge = "0s" //(5)! + maxConnectionAgeGrace = "0s" //(6)! + keepAliveTime = "0s" //(7)! + keepAliveTimeout = "0s" //(8)! + telemetry { + logging { + enabled = false //(9)! + } + metrics { + enabled = true //(10)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(11)! + tags = { // (12)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(13)! + attributes = { // (14)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Порт `gRPC-сервера` (по умолчанию: `8090`). + 2. Максимальный размер входящего сообщения (по умолчанию: `4MiB`). Может быть указан в виде числа байт или как `4MiB`, `4MB`, `1000Kb` и подобных значений. + 3. Включает сервис [`gRPC Server Reflection`](#reflection) (по умолчанию: `false`). + 4. Время ожидания обработки перед выключением сервера при [штатном завершении](container.md#graceful-shutdown) (по умолчанию: `30s`). + 5. Задает пользовательское максимальное время жизни соединения, после которого соединение штатно завершается (по умолчанию: не задано, опционально). К значению добавляется случайное отклонение +/-10%. + 6. Задает дополнительное время для штатного завершения соединения после достижения максимального времени жизни соединения (по умолчанию: не задано, опционально). Вызовы `RPC`, которые не успевают завершиться, отменяются, чтобы соединение могло завершиться. + 7. Задает интервал между кадрами `PING` (по умолчанию: не задано, опционально). + 8. Тайм-аут подтверждения кадра `PING` (по умолчанию: не задано, опционально). Если подтверждение не получено за это время, соединение закрывается. + 9. Включает логирование модуля (по умолчанию: `false`). + 10. Включает метрики модуля (по умолчанию: `true`). + 11. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 12. Теги метрик (по умолчанию: `{}`). + 13. Включает трассировку модуля (по умолчанию: `true`). + 14. Атрибуты трассировки (по умолчанию: `{}`). + + === ":simple-yaml: `YAML`" + + ```yaml + grpcServer: + port: 8090 #(1)! + maxMessageSize: "4MiB" #(2)! + reflectionEnabled: false #(3)! + shutdownWait: "30s" #(4)! + maxConnectionAge: "0s" #(5)! + maxConnectionAgeGrace: "0s" #(6)! + keepAliveTime: "0s" #(7)! + keepAliveTimeout: "0s" #(8)! + telemetry: + logging: + enabled: false #(9)! + metrics: + enabled: true #(10)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(11)! + tags: #(12)! + key1: value1 + key2: value2 + tracing: + enabled: true #(13)! + attributes: #(14)! + key1: value1 + key2: value2 + ``` + + 1. Порт `gRPC-сервера` (по умолчанию: `8090`). + 2. Максимальный размер входящего сообщения (по умолчанию: `4MiB`). Может быть указан в виде числа байт или как `4MiB`, `4MB`, `1000Kb` и подобных значений. + 3. Включает сервис [`gRPC Server Reflection`](#reflection) (по умолчанию: `false`). + 4. Время ожидания обработки перед выключением сервера при [штатном завершении](container.md#graceful-shutdown) (по умолчанию: `30s`). + 5. Задает пользовательское максимальное время жизни соединения, после которого соединение штатно завершается (по умолчанию: не задано, опционально). К значению добавляется случайное отклонение +/-10%. + 6. Задает дополнительное время для штатного завершения соединения после достижения максимального времени жизни соединения (по умолчанию: не задано, опционально). Вызовы `RPC`, которые не успевают завершиться, отменяются, чтобы соединение могло завершиться. + 7. Задает интервал между кадрами `PING` (по умолчанию: не задано, опционально). + 8. Тайм-аут подтверждения кадра `PING` (по умолчанию: не задано, опционально). Если подтверждение не получено за это время, соединение закрывается. + 9. Включает логирование модуля (по умолчанию: `false`). + 10. Включает метрики модуля (по умолчанию: `true`). + 11. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`). + 12. Теги метрик (по умолчанию: `{}`). + 13. Включает трассировку модуля (по умолчанию: `true`). + 14. Атрибуты трассировки (по умолчанию: `{}`). -Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#grpc-server). ## Обработчики { #handlers } -Созданные gRPC сервисы требуется пометить аннотацией `@Component`: +Обработчик — это класс, который наследует сгенерированный `...ImplBase` и регистрируется в графе приложения с помощью аннотации `@Component`. +Класс `...ImplBase` создается из контракта `proto` с помощью [gradle-плагина protobuf](#plugin); вы переопределяете его методы `RPC`, чтобы реализовать поведение сервера. +Обычные компоненты Kora, такие как сервисы и репозитории, можно внедрить в обработчик через его конструктор. + +Рассмотрим контракт `proto` с единственным унарным методом: + +```protobuf title="src/main/proto/message.proto" +syntax = "proto3"; + +package ru.tinkoff.kora.generated.grpc; + +service UserService { + rpc createUser(RequestEvent) returns (ResponseEvent) {} //(1)! +} + +message RequestEvent { + string name = 1; + string code = 2; +} + +message ResponseEvent { + bytes id = 1; +} +``` + +1. Унарный `RPC`: одно сообщение запроса порождает одно сообщение ответа. + +Плагин генерирует `UserServiceGrpc.UserServiceImplBase`, а обработчик переопределяет сгенерированный метод. +Сгенерированный метод получает сообщение запроса и [`StreamObserver`](https://grpc.github.io/grpc-java/javadoc/io/grpc/stub/StreamObserver.html), +который используется для отправки ответов обратно клиенту: ===! ":fontawesome-brands-java: `Java`" ```java @Component - public class ExampleService extends ExampleGrpc.ExampleImplBase {} + public final class UserService extends UserServiceGrpc.UserServiceImplBase { + + @Override + public void createUser(Message.RequestEvent request, StreamObserver responseObserver) { //(1)! + var response = Message.ResponseEvent.newBuilder() + .setId(ByteString.copyFromUtf8(UUID.randomUUID().toString())) + .build(); + + responseObserver.onNext(response); //(2)! + responseObserver.onCompleted(); //(3)! + } + } ``` + 1. Сгенерированный метод получает сообщение запроса и `StreamObserver` для отправки ответа + 2. Отправляет клиенту одно сообщение ответа + 3. Сигнализирует о завершении вызова; для унарного метода вызывается ровно один раз, после единственного `onNext` + === ":simple-kotlin: `Kotlin`" ```kotlin @Component - class ExampleService : ExampleGrpc.ExampleImplBase {} + class UserService : UserServiceGrpc.UserServiceImplBase() { + + override fun createUser(request: Message.RequestEvent, responseObserver: StreamObserver) { //(1)! + val response = Message.ResponseEvent.newBuilder() + .setId(ByteString.copyFromUtf8(UUID.randomUUID().toString())) + .build() + + responseObserver.onNext(response) //(2)! + responseObserver.onCompleted() //(3)! + } + } + ``` + + 1. Сгенерированный метод получает сообщение запроса и `StreamObserver` для отправки ответа + 2. Отправляет клиенту одно сообщение ответа + 3. Сигнализирует о завершении вызова; для унарного метода вызывается ровно один раз, после единственного `onNext` + +### Серверная потоковая передача { #server-streaming } + +Для серверного потокового `RPC` (`returns (stream ...)` в `proto`) клиент отправляет один запрос, а сервер возвращает много сообщений. +Вызывайте `onNext` для каждого сообщения, а затем один раз `onCompleted` в конце: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public void getAllUsers(Message.RequestEvent request, StreamObserver responseObserver) { + for (var user : userService.findAll()) { + responseObserver.onNext(toResponse(user)); //(1)! + } + responseObserver.onCompleted(); //(2)! + } + ``` + + 1. Отправляет одно из нескольких сообщений ответа + 2. Завершает поток ответа после последнего сообщения + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun getAllUsers(request: Message.RequestEvent, responseObserver: StreamObserver) { + userService.findAll().forEach { responseObserver.onNext(toResponse(it)) } //(1)! + responseObserver.onCompleted() //(2)! + } + ``` + + 1. Отправляет одно из нескольких сообщений ответа + 2. Завершает поток ответа после последнего сообщения + +### Клиентская потоковая передача { #client-streaming } + +Для клиентского потокового `RPC` (`rpc method(stream ...)`) клиент отправляет много сообщений, а сервер отвечает один раз в конце. +Сгенерированный метод **возвращает** `StreamObserver`, который получает входящие сообщения запроса; итоговый ответ формируется в `onCompleted`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public StreamObserver createUsers(StreamObserver responseObserver) { + return new StreamObserver<>() { + private final List received = new ArrayList<>(); + + @Override + public void onNext(Message.RequestEvent value) { + received.add(value); //(1)! + } + + @Override + public void onError(Throwable t) { + responseObserver.onError(t); //(2)! + } + + @Override + public void onCompleted() { + responseObserver.onNext(aggregate(received)); //(3)! + responseObserver.onCompleted(); + } + }; + } + ``` + + 1. Собирает каждое входящее сообщение запроса + 2. Пробрасывает ошибку потока со стороны клиента + 3. Формирует единственный агрегированный ответ после того, как клиент завершил отправку + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun createUsers(responseObserver: StreamObserver): StreamObserver { + return object : StreamObserver { + private val received = mutableListOf() + + override fun onNext(value: Message.RequestEvent) { + received += value //(1)! + } + + override fun onError(t: Throwable) { + responseObserver.onError(t) //(2)! + } + + override fun onCompleted() { + responseObserver.onNext(aggregate(received)) //(3)! + responseObserver.onCompleted() + } + } + } + ``` + + 1. Собирает каждое входящее сообщение запроса + 2. Пробрасывает ошибку потока со стороны клиента + 3. Формирует единственный агрегированный ответ после того, как клиент завершил отправку + +### Двунаправленная потоковая передача { #bidirectional-streaming } + +Для двунаправленного потокового `RPC` (`rpc method(stream ...) returns (stream ...)`) обе стороны обмениваются множеством сообщений в рамках одного вызова. +Метод возвращает `StreamObserver` для входящих запросов и может отправлять ответы в любой момент через `responseObserver`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public StreamObserver updateUsers(StreamObserver responseObserver) { + return new StreamObserver<>() { + @Override + public void onNext(Message.RequestEvent value) { + responseObserver.onNext(process(value)); //(1)! + } + + @Override + public void onError(Throwable t) { + responseObserver.onError(t); + } + + @Override + public void onCompleted() { + responseObserver.onCompleted(); //(2)! + } + }; + } + ``` + + 1. Отвечает на каждое входящее сообщение по мере его поступления + 2. Завершает поток ответа, когда клиент прекращает отправку + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun updateUsers(responseObserver: StreamObserver): StreamObserver { + return object : StreamObserver { + override fun onNext(value: Message.RequestEvent) { + responseObserver.onNext(process(value)) //(1)! + } + + override fun onError(t: Throwable) { + responseObserver.onError(t) + } + + override fun onCompleted() { + responseObserver.onCompleted() //(2)! + } + } + } + ``` + + 1. Отвечает на каждое входящее сообщение по мере его поступления + 2. Завершает поток ответа, когда клиент прекращает отправку + +### Обработка ошибок { #error-handling } + +**Описание**: gRPC представляет ошибки вызова с помощью кода [`io.grpc.Status`](https://grpc.github.io/grpc-java/javadoc/io/grpc/Status.html) +и необязательного описания, а не с помощью кодов ответа HTTP. +Чтобы завершить вызов с ошибкой, завершите observer ответа вызовом `responseObserver.onError(status.asRuntimeException())` +или выбросьте `StatusRuntimeException` из обработчика. +Автоматически зарегистрированный [`TelemetryInterceptor`](#default) наблюдает финальный `Status` при закрытии вызова +(в `close`, `onHalfClose`, `onCancel` и `onComplete`) и соответствующим образом записывает логирование, метрики и трассировку. + +**Причины**: выбирайте код `Status`, соответствующий сбою, — например, `Status.NOT_FOUND` для отсутствующей сущности, +`Status.INVALID_ARGUMENT` для некорректных входных данных, `Status.UNAUTHENTICATED` или `Status.PERMISSION_DENIED` для сбоев авторизации +и `Status.INTERNAL` для непредвиденных ошибок сервера. + +**Рекомендации**: + +- Прикрепляйте понятное человеку сообщение через `withDescription(...)` и сохраняйте исходное исключение через `withCause(...)`, чтобы телеметрия могла его записать. +- Завершайте вызов ровно один раз: никогда не вызывайте `onError` после `onCompleted` и не вызывайте ни один из них дважды. +- Не раскрывайте клиентам внутренние детали исключений; сначала сопоставьте их с подходящим `Status`. + +**Пример обработки**: унарный обработчик, который возвращает `NOT_FOUND`, когда сущность отсутствует, и сопоставляет непредвиденные сбои с `INTERNAL`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Override + public void getUser(Message.RequestEvent request, StreamObserver responseObserver) { + try { + var user = userService.getUser(request.getName()) + .orElseThrow(() -> Status.NOT_FOUND + .withDescription("User not found: " + request.getName()) + .asRuntimeException()); //(1)! + responseObserver.onNext(toResponse(user)); + responseObserver.onCompleted(); + } catch (StatusRuntimeException e) { + responseObserver.onError(e); //(2)! + } catch (Exception e) { + responseObserver.onError(Status.INTERNAL + .withDescription("Failed to get user") + .withCause(e) //(3)! + .asRuntimeException()); + } + } ``` + 1. Строит ошибку `NOT_FOUND` с описанием + 2. Передает клиенту уже сопоставленную ошибку `Status` + 3. Сохраняет исходное исключение в качестве причины, чтобы телеметрия могла его записать + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + override fun getUser(request: Message.RequestEvent, responseObserver: StreamObserver) { + try { + val user = userService.getUser(request.name) + ?: throw Status.NOT_FOUND + .withDescription("User not found: ${request.name}") + .asRuntimeException() //(1)! + responseObserver.onNext(toResponse(user)) + responseObserver.onCompleted() + } catch (e: StatusRuntimeException) { + responseObserver.onError(e) //(2)! + } catch (e: Exception) { + responseObserver.onError( + Status.INTERNAL + .withDescription("Failed to get user") + .withCause(e) //(3)! + .asRuntimeException() + ) + } + } + ``` + + 1. Строит ошибку `NOT_FOUND` с описанием + 2. Передает клиенту уже сопоставленную ошибку `Status` + 3. Сохраняет исходное исключение в качестве причины, чтобы телеметрия могла его записать + +### Сигнатуры { #signatures } + +Форма метода обработчика определяется контрактом `proto` и сгенерированным `...ImplBase`: + +===! ":fontawesome-brands-java: `Java`" + + Под `Req` и `Resp` подразумеваются сгенерированные типы сообщений запроса и ответа. + + - Унарный: `void myMethod(Req request, StreamObserver responseObserver)` + - Серверная потоковая передача: `void myMethod(Req request, StreamObserver responseObserver)` (несколько `onNext`, один `onCompleted`) + - Клиентская потоковая передача: `StreamObserver myMethod(StreamObserver responseObserver)` + - Двунаправленная потоковая передача: `StreamObserver myMethod(StreamObserver responseObserver)` + + Сгенерированный метод возвращает `void` (или `StreamObserver` для запроса), поэтому результаты доставляются асинхронно через колбэки `StreamObserver`; + ответы могут завершаться из другого потока. + +=== ":simple-kotlin: `Kotlin`" + + Под `Req` и `Resp` подразумеваются сгенерированные типы сообщений запроса и ответа. + + - Унарный: `myMethod(request: Req, responseObserver: StreamObserver)` + - Серверная потоковая передача: `myMethod(request: Req, responseObserver: StreamObserver)` (несколько `onNext`, один `onCompleted`) + - Клиентская потоковая передача: `myMethod(responseObserver: StreamObserver): StreamObserver` + - Двунаправленная потоковая передача: `myMethod(responseObserver: StreamObserver): StreamObserver` + + Когда вы генерируете корутинные заглушки с помощью плагина [`grpc-kotlin`](https://github.com/grpc/grpc-kotlin) (`io.grpc:protoc-gen-grpc-kotlin`) + и наследуете сгенерированный `...CoroutineImplBase`, методы обработчика могут быть `suspend`-функциями (а потоковые методы могут использовать `Flow`). + Kora автоматически регистрирует [`CoroutineContextInjectInterceptor`](#default), который внедряет `Context` Kora в `CoroutineContext` обработчика; + он активируется только при наличии `kotlinx-coroutines` в classpath. + ## Перехватчики { #interceptors } -[Перехватчики](https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerInterceptor.html) позволяют перехватывать запросы перед тем, как они будут переданы обработчикам. +[`io.grpc.ServerInterceptor`](https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerInterceptor.html) обрабатывает вызов до того, как он будет передан в `gRPC-сервис`. +Перехватчики подходят для сквозной логики: логирования, авторизации, трассировки, работы с `Metadata` и сопоставления ошибок. + +В отличие от [HTTP-сервера](http-server.md#interceptors), модуль gRPC-сервера **не** имеет аннотаций `@GrpcService` или `@InterceptWith`: +каждый `ServerInterceptor`, зарегистрированный как `@Component`, применяется **глобально** ко всем сервисам на сервере. +Чтобы ограничить перехватчик одним сервисом или методом, анализируйте вызов во время выполнения — смотрите [Ограничение области и авторизация](#authorization). ### Стандартные { #default } -При запуске сервера по-умолчанию используются следующие перехватчики: +При запуске сервера Kora добавляет стандартные перехватчики: -- `ContextServerInterceptor` -- `CoroutineContextInjectInterceptor` -- `MetricCollectorServerInterceptor` -- `LoggingServerInterceptor` +- `TelemetryInterceptor` — включает телеметрию сервера (логирование, метрики, трассировку) в зависимости от подключенных модулей и настроек `grpcServer.telemetry`, а также сопоставляет финальный `Status`/исключение при закрытии вызова +- `ContextServerInterceptor` — пробрасывает `Context` Kora в обработку вызова, чтобы он был доступен внутри обработчика +- `CoroutineContextInjectInterceptor` — добавляет поддержку `CoroutineContext` для корутинных обработчиков на `Kotlin` (активен только при наличии `kotlinx-coroutines` в classpath) -Для переопределения списка перехватчиков по умолчанию можно переопределить метод `serverBuilder` из класса `GrpcModule` +Пользовательские бины `ServerInterceptor` из графа приложения добавляются в `NettyServerBuilder` перед стандартными перехватчиками. +Для полной настройки `NettyServerBuilder` используйте [GrpcServerBuilderConfigurer](#builder-configurer). -### Собственные { #custom } +### Порядок выполнения { #execution-order } -Для добавления собственного перехватчика требуется создать наследника `ServerInterceptor` с аннотацией `@Component`: +gRPC вызывает перехватчики в **обратном** порядке их регистрации, поэтому последний добавленный перехватчик выполняется первым (самый внешний). +Поскольку Kora регистрирует пользовательские перехватчики первыми, а стандартные — последними, входящий вызов обрабатывается в таком порядке: + +``` +CoroutineContextInjectInterceptor -> ContextServerInterceptor -> TelemetryInterceptor -> user interceptors -> handler +``` + +Следствия такого порядка: + +- `Context` Kora и `CoroutineContext` Kotlin устанавливаются вокруг ваших перехватчиков и обработчика, поэтому они доступны внутри колбэков слушателя обработчика. +- `TelemetryInterceptor` оборачивает ваши перехватчики и обработчик, поэтому он наблюдает финальный `Status` (включая ошибки, выброшенные или переданные через observer ответа). +- Когда пользовательских перехватчиков несколько, они выполняются в порядке, обратном порядку их регистрации в графе; не полагайтесь на конкретный порядок между ними для корректности. + +### Пользовательские { #custom } + +Чтобы добавить пользовательский перехватчик, создайте реализацию `ServerInterceptor` с аннотацией `@Component`: ===! ":fontawesome-brands-java: `Java`" @@ -240,7 +637,7 @@ agent: ServerCallHandler serverCallHandler) { // do something - return serverCallHandler.startCall(serverCall, metadata): + return serverCallHandler.startCall(serverCall, metadata); } } ``` @@ -263,37 +660,176 @@ agent: } ``` -## Отладка { #reflection } +### Ограничение области и авторизация { #authorization } + +Поскольку перехватчик глобальный, ограничьте его конкретным сервисом или методом, анализируя `call.getMethodDescriptor()`: +`getServiceName()` возвращает имя сервиса (сгенерированную константу `...Grpc.SERVICE_NAME`), а `getFullMethodName()` возвращает `service/method`. + +Заголовки запроса поступают в виде [`Metadata`](https://grpc.github.io/grpc-java/javadoc/io/grpc/Metadata.html). +Читайте заголовок с помощью `Metadata.Key`, а отклоняйте вызов, закрывая его с помощью `Status` и возвращая пустой слушатель, чтобы обработчик никогда не вызывался. +Пример ниже применяет авторизацию по API-ключу только к одному сервису: -Поддерживается [gRPC Server Reflection](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md) -который предоставляет информацию об общедоступных gRPC-сервисах на сервере -и помогает клиентам во время выполнения строить запросы и ответы RPC без предварительно скомпилированной информации о сервисе. -Он используется инструментом командной строки gRPC (gRPC CLI), с помощью которого можно исследовать proto-файлы сервера и отправлять/получать тестовые RPC. -Reflection поддерживается только для сервисов, основанных на proto. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ApiKeyServerInterceptor implements ServerInterceptor { -Подробнее о работе с gRPC Server Reflection можно ознакомится [тут](https://github.com/grpc/grpc-java/blob/master/documentation/server-reflection-tutorial.md#enable-server-reflection). + private static final Metadata.Key AUTHORIZATION = + Metadata.Key.of("authorization", Metadata.ASCII_STRING_MARSHALLER); //(1)! -### Подключение { #dependency-2 } + @Override + public ServerCall.Listener interceptCall(ServerCall call, + Metadata headers, + ServerCallHandler next) { + if (!UserServiceGrpc.SERVICE_NAME.equals(call.getMethodDescriptor().getServiceName())) { //(2)! + return next.startCall(call, headers); + } -Требуется дополнительно подключить зависимость [gRPC Server Reflection](https://mvnrepository.com/artifact/io.grpc/grpc-services). + var apiKey = headers.get(AUTHORIZATION); //(3)! + if (apiKey == null || !apiKey.equals("secret")) { + call.close(Status.UNAUTHENTICATED.withDescription("Invalid API key"), new Metadata()); //(4)! + return new ServerCall.Listener<>() {}; //(5)! + } + + return next.startCall(call, headers); + } + } + ``` + + 1. `Metadata.Key` для чтения заголовка `authorization` как ASCII-строки + 2. Применяет перехватчик только к `UserService`; другие сервисы проходят без изменений + 3. Читает значение заголовка из `Metadata` запроса + 4. Отклоняет вызов со статусом `UNAUTHENTICATED` + 5. Возвращает пустой слушатель, чтобы обработчик никогда не вызывался + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ApiKeyServerInterceptor : ServerInterceptor { + + override fun interceptCall( + call: ServerCall, + headers: Metadata, + next: ServerCallHandler + ): ServerCall.Listener { + if (UserServiceGrpc.SERVICE_NAME != call.methodDescriptor.serviceName) { //(2)! + return next.startCall(call, headers) + } + + val apiKey = headers.get(AUTHORIZATION) //(3)! + if (apiKey != "secret") { + call.close(Status.UNAUTHENTICATED.withDescription("Invalid API key"), Metadata()) //(4)! + return object : ServerCall.Listener() {} //(5)! + } + + return next.startCall(call, headers) + } + + companion object { + private val AUTHORIZATION: Metadata.Key = + Metadata.Key.of("authorization", Metadata.ASCII_STRING_MARSHALLER) //(1)! + } + } + ``` + + 1. `Metadata.Key` для чтения заголовка `authorization` как ASCII-строки + 2. Применяет перехватчик только к `UserService`; другие сервисы проходят без изменений + 3. Читает значение заголовка из `Metadata` запроса + 4. Отклоняет вызов со статусом `UNAUTHENTICATED` + 5. Возвращает пустой слушатель, чтобы обработчик никогда не вызывался + +## Жизненный цикл и готовность { #lifecycle } + +Сервером управляет компонент `GrpcNettyServer`, который создается как компонент [`@Root`](container.md#root-component) +и следует [жизненному циклу приложения](container.md#component-lifecycle): + +- При запуске он создает и стартует сервер `Netty` на настроенном `port`. Если порт уже занят, запуск завершается с понятной ошибкой. +- При выключении он выполняет [штатное завершение](container.md#graceful-shutdown): перестает принимать новые вызовы и ждет до `shutdownWait` завершения выполняющихся вызовов, затем принудительно завершает оставшиеся вызовы. + +`GrpcNettyServer` также реализует [пробу готовности](probes.md): сервер сообщает о **неготовности** во время запуска или выключения +и о **готовности** только во время работы. В развертывании `Kubernetes` это позволяет пробе готовности отражать реальное состояние сервера и сливать трафик во время штатного завершения. + +## Телеметрия { #telemetry } + +Наблюдаемость сервера обеспечивается `TelemetryInterceptor` через фасад `GrpcServerTelemetry` и настраивается в [`grpcServer.telemetry`](#configuration). +Метрики описаны в разделе [Справочник метрик](metrics.md#grpc-server). + +Каждая часть телеметрии — это заменяемый компонент: значения по умолчанию регистрируются как компоненты по умолчанию, поэтому предоставление собственного `@Component` переопределяет их: + +- `GrpcServerTelemetry` — агрегирующий фасад телеметрии (`createContext`, возвращающий контекст с `sendMessage`/`receiveMessage`/`close`) +- `GrpcServerLogger` — логирование вызовов (`logBegin`/`logEnd`/`logSendMessage`/`logReceiveMessage`); по умолчанию используется `Slf4jGrpcServerLogger` +- `GrpcServerTracer` — спаны трассировки вокруг вызовов +- `GrpcServerMetricsFactory` — сбор метрик по каждому вызову + +Например, чтобы полностью настроить логирование, зарегистрируйте `@Component`, реализующий `GrpcServerLogger`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public interface GrpcServerLogger { + + boolean isEnabled(); + + void logBegin(ServerCall call, Metadata headers, String serviceName, String methodName); + + void logEnd(String serviceName, String methodName, @Nullable Status status, @Nullable Throwable exception, long processingTime); + + void logSendMessage(String serviceName, String methodName, Object message); + + void logReceiveMessage(String serviceName, String methodName, Object message); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + interface GrpcServerLogger { + + fun isEnabled(): Boolean + + fun logBegin(call: ServerCall<*, *>, headers: Metadata, serviceName: String, methodName: String) + + fun logEnd(serviceName: String, methodName: String, status: Status?, exception: Throwable?, processingTime: Long) + + fun logSendMessage(serviceName: String, methodName: String, message: Any) + + fun logReceiveMessage(serviceName: String, methodName: String, message: Any) + } + ``` + +## Рефлексия { #reflection } + +Поддерживается [`gRPC Server Reflection`](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md), которая предоставляет информацию о доступных `gRPC-сервисах` на сервере. +Рефлексия помогает клиентам и инструментам формировать запросы `RPC` во время выполнения без предварительно скомпилированной информации о сервисах. +Например, ее использует `gRPC CLI`, который может исследовать описания `proto` сервера и отправлять тестовые вызовы `RPC`. +`gRPC Server Reflection` поддерживается только для сервисов на основе `proto`. + +Подробнее о `gRPC Server Reflection` можно узнать в [руководстве grpc-java](https://github.com/grpc/grpc-java/blob/master/documentation/server-reflection-tutorial.md#enable-server-reflection). + +### Зависимость { #dependency-2 } + +Необходимо дополнительно добавить зависимость [`gRPC Server Reflection`](https://mvnrepository.com/artifact/io.grpc/grpc-services). ===! ":fontawesome-brands-java: `Java`" [Зависимость](general.md#dependencies) `build.gradle`: ```groovy - implementation "io.grpc:grpc-services:1.62.2" + implementation "io.grpc:grpc-services:1.74.0" ``` === ":simple-kotlin: `Kotlin`" [Зависимость](general.md#dependencies) `build.gradle.kts`: ```groovy - implementation("io.grpc:grpc-services:1.62.2") + implementation("io.grpc:grpc-services:1.74.0") ``` ### Конфигурация { #configuration-2 } -Требуется также включить сервис gRPC Server Reflection в конфигурации: +Также необходимо включить сервис `gRPC Server Reflection` в конфигурации. +Kora добавляет его на сервер только при наличии в приложении класса `io.grpc.protobuf.services.ProtoReflectionService`, поэтому одной конфигурации без зависимости недостаточно. ===! ":material-code-json: `Hocon`" @@ -303,7 +839,7 @@ Reflection поддерживается только для сервисов, о } ``` - 1. Включает сервис gRPC Server Reflection + 1. Включает сервис `gRPC Server Reflection` (по умолчанию: `false`). === ":simple-yaml: `YAML`" @@ -312,4 +848,36 @@ Reflection поддерживается только для сервисов, о reflectionEnabled: false #(1)! ``` - 1. Включает сервис gRPC Server Reflection + 1. Включает сервис `gRPC Server Reflection` (по умолчанию: `false`). + +### Использование { #reflection-usage } + +При включенной рефлексии инструменты вроде [`grpcurl`](https://github.com/fullstorydev/grpcurl) могут обнаруживать сервисы и отправлять вызовы `RPC` без предварительно скомпилированного клиента. +Для сервера, слушающего порт `8090`: + +```bash +grpcurl -plaintext localhost:8090 list #(1)! +grpcurl -plaintext localhost:8090 describe ru.tinkoff.kora.generated.grpc.UserService #(2)! +grpcurl -plaintext -d '{"name": "Bob", "code": "123"}' \ + localhost:8090 ru.tinkoff.kora.generated.grpc.UserService/createUser #(3)! +``` + +1. Выводит список сервисов, предоставляемых сервером +2. Описывает сервис и его методы +3. Отправляет унарный `RPC`; `-plaintext` используется, потому что у сервера из примера нет `TLS` + +## Телеметрия { #telemetry } + +gRPC Server использует контракт телеметрии для логирования, метрик и трассировки вызовов. +Конфигурация телеметрии (секция `telemetry { logging / metrics / tracing }`) описана в разделе [Конфигурация](#configuration). +Точки расширения находятся в `ru.tinkoff.kora.grpc.server.common.telemetry`. + +Для каждого gRPC-вызова создаётся `GrpcServerTelemetry.GrpcServerTelemetryContext`, который закрывается по завершении вызова. +Вызов описывается через параметры обработчика телеметрии, включая сервис, метод, статус ответа и длительность. + +Фабрика по умолчанию `DefaultGrpcServerTelemetryFactory` объединяет три фабрики: +- `GrpcServerLoggerFactory` строит `GrpcServerLogger` для логирования начала/конца вызова; +- `GrpcServerMetricsFactory` строит `GrpcServerMetrics` для записи метрик вызовов; +- `GrpcServerTracerFactory` строит `GrpcServerTracer` для распределённой трассировки. + +Метрики и трассировка описаны в разделе [Справочник метрик](metrics.md#grpc-server). diff --git a/mkdocs/docs/ru/documentation/http-client.md b/mkdocs/docs/ru/documentation/http-client.md index 88c2160..902196f 100644 --- a/mkdocs/docs/ru/documentation/http-client.md +++ b/mkdocs/docs/ru/documentation/http-client.md @@ -4,22 +4,27 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora HTTP clients, OkHttp, AsyncHttpClient, Java native client, declarative client annotations, request and response mapping, interceptors, and authorization; key triggers include @HttpClient, @HttpRoute, @Path, @Query, @Header, @Cookie, @Json, @InterceptWith, HttpClientModule, OkHttp." --- -Модуль предоставляет тонкий слой абстракции для создания HTTP-клиентов -с помощью аннотаций в декларативном стиле, либо использование клиентов в императивном стиле. +Модуль `HTTP-клиента` описывает исходящие HTTP-вызовы приложения: от выбора транспортной реализации до преобразования запроса, +преобразования ответа, телеметрии и перехватчиков. В Kora можно описывать типизированные клиенты декларативно через `@HttpClient` +и `@HttpRoute` с тонким слоем абстракции, либо использовать общий интерфейс `HttpClient` напрямую, когда запрос нужно собрать в коде. + +Декларативный подход подходит для большинства интеграций с внешними службами: контракт метода становится контрактом удаленного вызова, +а Kora во время компиляции создает реализацию без использования `Reflection` во время работы. Императивный подход полезен для низкоуровневых +или динамических сценариев, где путь, заголовки, параметры или тело запроса удобнее собирать вручную. ???+ tip "Совет" - **Мы советуем** использовать подход когда первичен контракт в формате OpenAPI - и из него создаются клиенты по средствам генератора. - Такой подход позволяет достигнуть консистентности контракта между потребителем и собственником контракта - и быстро обновляться в случае нового контракта по средствам замены файла контракта. - Подробнее про генератор в [секции про генерации из OpenAPI](openapi-codegen.md). + **Мы советуем** использовать подход, при котором первичен контракт в формате `OpenAPI`, + а клиенты создаются с помощью генератора. + Такой подход помогает сохранить согласованность контракта между потребителем и владельцем контракта + и быстрее обновлять клиент при изменении контракта за счет замены файла описания. + Подробнее про генератор смотрите в [разделе про генерацию из OpenAPI](openapi-codegen.md). -Если нужен пошаговый разбор перед справочным описанием, смотрите [HTTP клиент](../guides/http-client.md) и [HTTP клиент продвинутый](../guides/http-client-advanced.md). +Если нужен пошаговый разбор перед справочным описанием, смотрите [HTTP-клиент](../guides/http-client.md) и [продвинутый HTTP-клиент](../guides/http-client-advanced.md). ## OkHttp { #okhttp } -Реализация HTTP клиента основанная на библиотеке [OkHttp](https://github.com/square/okhttp). +Реализация `HTTP`-клиента основана на библиотеке [OkHttp](https://github.com/square/okhttp). Учитывайте что реализация написана на Kotlin и использует соответствующие зависимости. Лучше всего подходит для Kotlin сервисов, либо Java сервисов где нужна высокая производительность, либо требуется поддержка HTTP 3, либо поддержка GZip сжатия, либо другие специфичные HTTP опции. @@ -54,134 +59,161 @@ agent: ### Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `OkHttpClientConfig` и `HttpClientConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации OkHttp клиента: ===! ":material-code-json: `Hocon`" ```javascript httpClient { - ok { - followRedirects = true //(1)! - httpVersion = "HTTP_1_1" //(2)! - retryOnConnectionFailure = true //(3)! - } - connectTimeout = "5s" //(4)! - readTimeout = "2m" //(5)! - useEnvProxy = false //(6)! - proxy { - host = "localhost" //(7)! - port = 8090 //(8)! - user = "user" //(9)! - password = "password" //(10)! - nonProxyHosts = [ "host1", "host2" ] //(11)! - } - telemetry { - logging { - enabled = false //(12)! - mask = "***" //(13)! - maskQueries = [ ] //(14)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(15)! - pathTemplate = true //(16)! - } - metrics { - enabled = true //(17)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(18)! - tags = { // (19)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(20)! - attributes = { // (21)! - "key1" = "value1" - "key2" = "value2" - } - } - } + connectTimeout = "5s" //(1)! + readTimeout = "2m" //(2)! } ``` - 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) - 2. Максимальная используемая версия HTTP протокола (доступные значения: `HTTP_1_1` / `HTTP_2` / `HTTP_3`) - 3. Пробовать повторно выполнить запрос при ошибке соединения или нет, может влиять на предельное время на установление соединения - 4. Максимальное время на установление соединения - 5. Максимальное время на чтение ответа - 6. Использовать ли переменные окружения для настройки прокси - 7. Адрес прокси (по умолчанию отсутвует) - 8. Порт прокси (по умолчанию отсутвует) - 9. Пользователь для прокси (по умолчанию отсутвует) - 10. Пароль для прокси (по умолчанию отсутвует) - 11. Хосты которые следует исключить из проксирования (по умолчанию отсутвует) - 12. Включает логгирование модуля (по умолчанию `false`) - 13. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 14. Список параметров запроса которые следует скрывать - 15. Список заголовков запроса/ответа которые следует скрывать - 16. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 17. Включает метрики модуля (по умолчанию `true`) - 18. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 19. Настройка тегов для метрик (опционально) - 20. Включает трассировку модуля (по умолчанию `true`) - 21. Настройка атрибутов для трассировки (опционально) + 1. Максимальное время на установление соединения (по умолчанию: `5s`) + 2. Максимальное время на чтение ответа (по умолчанию: `2m`) === ":simple-yaml: `YAML`" ```yaml httpClient: - ok: - followRedirects: true #(1)! - httpVersion: "HTTP_1_1" #(2)! - retryOnConnectionFailure: true #(3)! - connectTimeout: "5s" #(4)! - readTimeout: "2m" #(5)! - useEnvProxy: false #(6)! - proxy: - host: "localhost" #(7)! - port: 8090 #(8)! - user: "user" #(9)! - password: "password" #(10)! - nonProxyHosts: [ "host1", "host2" ] #(11)! - telemetry: - logging: - enabled: false #(12)! - mask: "***" #(13)! - maskQueries: [ ] #(14)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(15)! - pathTemplate: true #(16)! - metrics: - enabled: true #(17)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(18)! - tags: #(19)! - key1: value1 - key2: value2 - tracing: - enabled: true #(20)! - attributes: #(21)! - key1: value1 - key2: value2 - ``` - - 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) - 2. Максимальная используемая версия HTTP протокола (доступные значения: `HTTP_1_1` / `HTTP_2` / `HTTP_3`) - 3. Пробовать повторно выполнить запрос при ошибке соединения или нет, может влиять на предельное время на установление соединения - 4. Максимальное время на установление соединения - 5. Максимальное время на чтение ответа - 6. Использовать ли переменные окружения для настройки прокси - 7. Адрес прокси (по умолчанию отсутвует) - 8. Порт прокси (по умолчанию отсутвует) - 9. Пользователь для прокси (по умолчанию отсутвует) - 10. Пароль для прокси (по умолчанию отсутвует) - 11. Хосты которые следует исключить из проксирования (по умолчанию отсутвует) - 12. Включает логгирование модуля (по умолчанию `false`) - 13. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 14. Список параметров запроса которые следует скрывать - 15. Список заголовков запроса/ответа которые следует скрывать - 16. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 17. Включает метрики модуля (по умолчанию `true`) - 18. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 19. Настройка тегов для метрик (опционально) - 20. Включает трассировку модуля (по умолчанию `true`) - 21. Настройка атрибутов для трассировки (опционально) + connectTimeout: "5s" #(1)! + readTimeout: "2m" #(2)! + ``` + + 1. Максимальное время на установление соединения (по умолчанию: `5s`) + 2. Максимальное время на чтение ответа (по умолчанию: `2m`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `OkHttpClientConfig` и `HttpClientConfig` (указаны примеры значений или значения по умолчанию): + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + ok { + followRedirects = true //(1)! + httpVersion = "HTTP_1_1" //(2)! + retryOnConnectionFailure = true //(3)! + } + connectTimeout = "5s" //(4)! + readTimeout = "2m" //(5)! + useEnvProxy = false //(6)! + proxy { + host = "localhost" //(7)! + port = 8090 //(8)! + user = "user" //(9)! + password = "password" //(10)! + nonProxyHosts = [ "host1", "host2" ] //(11)! + } + telemetry { + logging { + enabled = false //(12)! + mask = "***" //(13)! + maskQueries = [ ] //(14)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(15)! + pathTemplate = true //(16)! + } + metrics { + enabled = true //(17)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(18)! + tags = { // (19)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(20)! + attributes = { // (21)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) (по умолчанию: `true`) + 2. Максимальная используемая версия `HTTP`-протокола, доступные значения: `HTTP_1_1` / `HTTP_2` / `HTTP_3` (по умолчанию: `HTTP_1_1`) + 3. Пробовать ли повторно выполнить запрос при ошибке соединения; может влиять на предельное время установления соединения (по умолчанию: `true`) + 4. Максимальное время на установление соединения (по умолчанию: `5s`) + 5. Максимальное время на чтение ответа (по умолчанию: `2m`) + 6. Использовать ли переменные окружения `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` и `no_proxy` / `NO_PROXY` для настройки прокси (по умолчанию: `false`) + 7. Адрес прокси (`обязательная`, по умолчанию не указано) + 8. Порт прокси (`обязательная`, по умолчанию не указано) + 9. Пользователь для прокси (по умолчанию не указано, необязательно) + 10. Пароль для прокси (по умолчанию не указано, необязательно) + 11. Узлы, которые следует исключить из проксирования (по умолчанию не указано, необязательно) + 12. Включает логирование модуля (по умолчанию: `false`) + 13. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 14. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 15. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 16. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 17. Включает метрики модуля (по умолчанию: `true`) + 18. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Настройка тегов для метрик (по умолчанию: `{}`) + 20. Включает трассировку модуля (по умолчанию: `true`) + 21. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + ok: + followRedirects: true #(1)! + httpVersion: "HTTP_1_1" #(2)! + retryOnConnectionFailure: true #(3)! + connectTimeout: "5s" #(4)! + readTimeout: "2m" #(5)! + useEnvProxy: false #(6)! + proxy: + host: "localhost" #(7)! + port: 8090 #(8)! + user: "user" #(9)! + password: "password" #(10)! + nonProxyHosts: [ "host1", "host2" ] #(11)! + telemetry: + logging: + enabled: false #(12)! + mask: "***" #(13)! + maskQueries: [ ] #(14)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(15)! + pathTemplate: true #(16)! + metrics: + enabled: true #(17)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(18)! + tags: #(19)! + key1: value1 + key2: value2 + tracing: + enabled: true #(20)! + attributes: #(21)! + key1: value1 + key2: value2 + ``` + + 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) (по умолчанию: `true`) + 2. Максимальная используемая версия `HTTP`-протокола, доступные значения: `HTTP_1_1` / `HTTP_2` / `HTTP_3` (по умолчанию: `HTTP_1_1`) + 3. Пробовать ли повторно выполнить запрос при ошибке соединения; может влиять на предельное время установления соединения (по умолчанию: `true`) + 4. Максимальное время на установление соединения (по умолчанию: `5s`) + 5. Максимальное время на чтение ответа (по умолчанию: `2m`) + 6. Использовать ли переменные окружения `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` и `no_proxy` / `NO_PROXY` для настройки прокси (по умолчанию: `false`) + 7. Адрес прокси (`обязательная`, по умолчанию не указано) + 8. Порт прокси (`обязательная`, по умолчанию не указано) + 9. Пользователь для прокси (по умолчанию не указано, необязательно) + 10. Пароль для прокси (по умолчанию не указано, необязательно) + 11. Узлы, которые следует исключить из проксирования (по умолчанию не указано, необязательно) + 12. Включает логирование модуля (по умолчанию: `false`) + 13. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 14. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 15. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 16. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 17. Включает метрики модуля (по умолчанию: `true`) + 18. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 19. Настройка тегов для метрик (по умолчанию: `{}`) + 20. Включает трассировку модуля (по умолчанию: `true`) + 21. Настройка атрибутов для трассировки (по умолчанию: `{}`) Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#http-client). @@ -215,7 +247,7 @@ agent: ## AsyncHttpClient { #asynchttpclient } -Реализация HTTP клиента основанная на библиотеке [Async HTTP Client](https://github.com/AsyncHttpClient/async-http-client). +Реализация `HTTP`-клиента основана на библиотеке [Async HTTP Client](https://github.com/AsyncHttpClient/async-http-client). Подходит для Java сервисов, где преобладают асинхронные вызовы. ### Подключение { #dependency-2 } @@ -248,125 +280,160 @@ agent: ### Конфигурация { #configuration-2 } -Пример полной конфигурации, описанной в классе `AsyncHttpClientConfig` и `HttpClientConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации AsyncHttpClient: ===! ":material-code-json: `Hocon`" ```javascript httpClient { - async { - followRedirects = true //(1)! - } - connectTimeout = "5s" //(2)! - readTimeout = "2m" //(3)! - useEnvProxy = false //(4)! - proxy { - host = "localhost" //(5)! - port = 8090 //(6)! - user = "user" //(7)! - password = "password" //(8)! - nonProxyHosts = [ "host1", "host2" ] //(9)! - } - telemetry { - logging { - enabled = false //(10)! - mask = "***" //(11)! - maskQueries = [ ] //(12)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(13)! - pathTemplate = true //(14)! - } - metrics { - enabled = true //(15)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(16)! - tags = { // (17)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(18)! - attributes = { // (19)! - "key1" = "value1" - "key2" = "value2" - } - } - } + connectTimeout = "5s" //(1)! + readTimeout = "2m" //(2)! } ``` - 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) - 2. Максимальное время на установление соединения - 3. Максимальное время на чтение ответа - 4. Использовать ли переменные окружения для настройки прокси - 5. Адрес прокси (по умолчанию отсутвует) - 6. Порт прокси (по умолчанию отсутвует) - 7. Пользователь для прокси (по умолчанию отсутвует) - 8. Пароль для прокси (по умолчанию отсутвует) - 9. Хосты которые следует исключить из проксирования (по умолчанию отсутвует) - 10. Включает логгирование модуля (по умолчанию `false`) - 11. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 12. Список параметров запроса которые следует скрывать - 13. Список заголовков запроса/ответа которые следует скрывать - 14. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 15. Включает метрики модуля (по умолчанию `true`) - 16. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 17. Настройка тегов для метрик (опционально) - 18. Включает трассировку модуля (по умолчанию `true`) - 19. Настройка атрибутов для трассировки (опционально) + 1. Максимальное время на установление соединения (по умолчанию: `5s`) + 2. Максимальное время на чтение ответа (по умолчанию: `2m`) === ":simple-yaml: `YAML`" ```yaml httpClient: - async: - followRedirects: true #(1)! - connectTimeout: "5s" #(2)! - readTimeout: "2m" #(3)! - useEnvProxy: false #(4)! - proxy: - host: "localhost" #(5)! - port: 8090 #(6)! - user: "user" #(7)! - password: "password" #(8)! - nonProxyHosts: [ "host1", "host2" ] #(9)! - telemetry: - logging: - enabled: false #(10)! - mask: "***" #(11)! - maskQueries: [ ] #(12)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(13)! - pathTemplate: true #(14)! - metrics: - enabled: true #(15)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(16)! - telemetry: - enabled: true #(17)! - ``` - - 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) - 2. Максимальное время на установление соединения - 3. Максимальное время на чтение ответа - 4. Использовать ли переменные окружения для настройки прокси - 5. Адрес прокси (по умолчанию отсутвует) - 6. Порт прокси (по умолчанию отсутвует) - 7. Пользователь для прокси (по умолчанию отсутвует) - 8. Пароль для прокси (по умолчанию отсутвует) - 9. Хосты которые следует исключить из проксирования (по умолчанию отсутвует) - 10. Включает логгирование модуля (по умолчанию `false`) - 11. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 12. Список параметров запроса которые следует скрывать - 13. Список заголовков запроса/ответа которые следует скрывать - 14. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 15. Включает метрики модуля (по умолчанию `true`) - 16. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 17. Включает трассировку модуля (по умолчанию `true`) + connectTimeout: "5s" #(1)! + readTimeout: "2m" #(2)! + ``` + + 1. Максимальное время на установление соединения (по умолчанию: `5s`) + 2. Максимальное время на чтение ответа (по умолчанию: `2m`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `AsyncHttpClientConfig` и `HttpClientConfig` (указаны примеры значений или значения по умолчанию): + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + async { + followRedirects = true //(1)! + } + connectTimeout = "5s" //(2)! + readTimeout = "2m" //(3)! + useEnvProxy = false //(4)! + proxy { + host = "localhost" //(5)! + port = 8090 //(6)! + user = "user" //(7)! + password = "password" //(8)! + nonProxyHosts = [ "host1", "host2" ] //(9)! + } + telemetry { + logging { + enabled = false //(10)! + mask = "***" //(11)! + maskQueries = [ ] //(12)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(13)! + pathTemplate = true //(14)! + } + metrics { + enabled = true //(15)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(16)! + tags = { // (17)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(18)! + attributes = { // (19)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) (по умолчанию: `true`) + 2. Максимальное время на установление соединения (по умолчанию: `5s`) + 3. Максимальное время на чтение ответа (по умолчанию: `2m`) + 4. Использовать ли переменные окружения `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` и `no_proxy` / `NO_PROXY` для настройки прокси (по умолчанию: `false`) + 5. Адрес прокси (`обязательная`, по умолчанию не указано) + 6. Порт прокси (`обязательная`, по умолчанию не указано) + 7. Пользователь для прокси (по умолчанию не указано, необязательно) + 8. Пароль для прокси (по умолчанию не указано, необязательно) + 9. Узлы, которые следует исключить из проксирования (по умолчанию не указано, необязательно) + 10. Включает логирование модуля (по умолчанию: `false`) + 11. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 12. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 13. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 14. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 15. Включает метрики модуля (по умолчанию: `true`) + 16. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 17. Настройка тегов для метрик (по умолчанию: `{}`) + 18. Включает трассировку модуля (по умолчанию: `true`) + 19. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + async: + followRedirects: true #(1)! + connectTimeout: "5s" #(2)! + readTimeout: "2m" #(3)! + useEnvProxy: false #(4)! + proxy: + host: "localhost" #(5)! + port: 8090 #(6)! + user: "user" #(7)! + password: "password" #(8)! + nonProxyHosts: [ "host1", "host2" ] #(9)! + telemetry: + logging: + enabled: false #(10)! + mask: "***" #(11)! + maskQueries: [ ] #(12)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(13)! + pathTemplate: true #(14)! + metrics: + enabled: true #(15)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(16)! + tags: #(17)! + key1: value1 + key2: value2 + tracing: + enabled: true #(18)! + attributes: #(19)! + key1: value1 + key2: value2 + ``` + + 1. Следовать ли по [перенаправлениям в HTTP](https://developer.mozilla.org/ru/docs/Web/HTTP/Redirections) (по умолчанию: `true`) + 2. Максимальное время на установление соединения (по умолчанию: `5s`) + 3. Максимальное время на чтение ответа (по умолчанию: `2m`) + 4. Использовать ли переменные окружения `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` и `no_proxy` / `NO_PROXY` для настройки прокси (по умолчанию: `false`) + 5. Адрес прокси (`обязательная`, по умолчанию не указано) + 6. Порт прокси (`обязательная`, по умолчанию не указано) + 7. Пользователь для прокси (по умолчанию не указано, необязательно) + 8. Пароль для прокси (по умолчанию не указано, необязательно) + 9. Узлы, которые следует исключить из проксирования (по умолчанию не указано, необязательно) + 10. Включает логирование модуля (по умолчанию: `false`) + 11. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 12. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 13. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 14. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 15. Включает метрики модуля (по умолчанию: `true`) + 16. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 17. Настройка тегов для метрик (по умолчанию: `{}`) + 18. Включает трассировку модуля (по умолчанию: `true`) + 19. Настройка атрибутов для трассировки (по умолчанию: `{}`) Можно также настроить [Netty транспорт](netty.md). ## Java клиент { #native-client } -Реализация HTTP клиента на основании встроенного Java клиента поставляемого в [JDK](https://openjdk.org/groups/net/httpclient/intro.html). -Лучше всего подходит для Java сервисов где не требуется производительность так как работает прилично медленнее, +Реализация `HTTP`-клиента основана на встроенном Java-клиенте, поставляемом в [JDK](https://openjdk.org/groups/net/httpclient/intro.html). +Лучше всего подходит для Java-сервисов, где не требуется максимальная производительность, и хочется минимизировать количество внешних библиотек. ### Подключение { #dependency-3 } @@ -399,125 +466,164 @@ agent: ### Конфигурация { #configuration-3 } -Пример полной конфигурации, описанной в классе `JdkHttpClientConfig` и `HttpClientConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации JDK HttpClient: ===! ":material-code-json: `Hocon`" ```javascript httpClient { - jdk { - threads = 2 //(1)! - httpVersion = "HTTP_1_1" //(2)! - } - connectTimeout = "5s" //(3)! - useEnvProxy = false //(4)! - proxy { - host = "localhost" //(5)! - port = 8090 //(6)! - user = "user" //(7)! - password = "password" //(8)! - nonProxyHosts = [ "host1", "host2" ] //(9)! - } - telemetry { - logging { - enabled = false //(10)! - mask = "***" //(11)! - maskQueries = [ ] //(12)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(13)! - pathTemplate = true //(14)! - } - metrics { - enabled = true //(15)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(16)! - tags = { // (17)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(18)! - attributes = { // (19)! - "key1" = "value1" - "key2" = "value2" - } - } - } + connectTimeout = "5s" //(1)! + readTimeout = "2m" //(2)! } ``` - 1. Количество потоков для HTTP клиента, по умолчанию равен кол-во ядер процессора умноженных на 2 - 2. Какую версию HTTP протокола использовать (доступные значения: `HTTP_1_1` / `HTTP_2`) - 3. Максимальное время на установление соединения - 4. Использовать ли переменные окружения для настройки прокси - 5. Адрес прокси (по умолчанию отсутвует) - 6. Порт прокси (по умолчанию отсутвует) - 7. Пользователь для прокси (по умолчанию отсутвует) - 8. Пароль для прокси (по умолчанию отсутвует) - 9. Хосты которые следует исключить из проксирования (по умолчанию отсутвует) - 10. Включает логгирование модуля (по умолчанию `false`) - 11. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 12. Список параметров запроса которые следует скрывать - 13. Список заголовков запроса/ответа которые следует скрывать - 14. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 15. Включает метрики модуля (по умолчанию `true`) - 16. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 17. Настройка тегов для метрик (опционально) - 18. Включает трассировку модуля (по умолчанию `true`) - 19. Настройка атрибутов для трассировки (опционально) + 1. Максимальное время на установление соединения (по умолчанию: `5s`) + 2. Максимальное время на чтение ответа (по умолчанию: `2m`) === ":simple-yaml: `YAML`" ```yaml httpClient: - jdk: - threads: 2 #(1)! - httpVersion: "HTTP_1_1" #(2)! - connectTimeout: "2s" #(3)! - useEnvProxy: false #(4)! - proxy: - host: "localhost" #(5)! - port: 8090 #(6)! - user: "user" #(7)! - password: "password" #(8)! - nonProxyHosts: [ "host1", "host2" ] #(9)! - telemetry: - logging: - enabled: false #(10)! - mask: "***" #(11)! - maskQueries: [ ] #(12)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(13)! - pathTemplate: true #(14)! - metrics: - enabled: true #(15)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(16)! - telemetry: - enabled: true #(17)! - ``` - - 1. Количество потоков для HTTP клиента, по умолчанию равен кол-во ядер процессора умноженных на 2 - 2. Какую версию HTTP протокола использовать (доступные значения: `HTTP_1_1` / `HTTP_2`) - 3. Максимальное время на установление соединения - 4. Использовать ли переменные окружения для настройки прокси - 5. Адрес прокси (по умолчанию отсутвует) - 6. Порт прокси (по умолчанию отсутвует) - 7. Пользователь для прокси (по умолчанию отсутвует) - 8. Пароль для прокси (по умолчанию отсутвует) - 9. Хосты которые следует исключить из проксирования (по умолчанию отсутвует) - 10. Включает логгирование модуля (по умолчанию `false`) - 11. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 12. Список параметров запроса которые следует скрывать - 13. Список заголовков запроса/ответа которые следует скрывать - 14. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 15. Включает метрики модуля (по умолчанию `true`) - 16. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 17. Включает трассировку модуля (по умолчанию `true`) - -## Клиент декларативный { #client-declarative } + connectTimeout: "5s" #(1)! + readTimeout: "2m" #(2)! + ``` + + 1. Максимальное время на установление соединения (по умолчанию: `5s`) + 2. Максимальное время на чтение ответа (по умолчанию: `2m`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `JdkHttpClientConfig` и `HttpClientConfig` (указаны примеры значений или значения по умолчанию): + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + jdk { + threads = 2 //(1)! + httpVersion = "HTTP_1_1" //(2)! + } + connectTimeout = "5s" //(3)! + readTimeout = "2m" //(4)! + useEnvProxy = false //(5)! + proxy { + host = "localhost" //(6)! + port = 8090 //(7)! + user = "user" //(8)! + password = "password" //(9)! + nonProxyHosts = [ "host1", "host2" ] //(10)! + } + telemetry { + logging { + enabled = false //(11)! + mask = "***" //(12)! + maskQueries = [ ] //(13)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(14)! + pathTemplate = true //(15)! + } + metrics { + enabled = true //(16)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(17)! + tags = { // (18)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(19)! + attributes = { // (20)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Количество потоков для `HTTP`-клиента (по умолчанию: количество доступных процессоров, умноженное на `2`) + 2. Какую версию `HTTP`-протокола использовать, доступные значения: `HTTP_1_1` / `HTTP_2` (по умолчанию: `HTTP_1_1`) + 3. Максимальное время на установление соединения (по умолчанию: `5s`) + 4. Максимальное время на чтение ответа (по умолчанию: `2m`) + 5. Использовать ли переменные окружения `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` и `no_proxy` / `NO_PROXY` для настройки прокси (по умолчанию: `false`) + 6. Адрес прокси (`обязательная`, по умолчанию не указано) + 7. Порт прокси (`обязательная`, по умолчанию не указано) + 8. Пользователь для прокси (по умолчанию не указано, необязательно) + 9. Пароль для прокси (по умолчанию не указано, необязательно) + 10. Узлы, которые следует исключить из проксирования (по умолчанию не указано, необязательно) + 11. Включает логирование модуля (по умолчанию: `false`) + 12. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 13. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 14. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 15. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 16. Включает метрики модуля (по умолчанию: `true`) + 17. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Настройка тегов для метрик (по умолчанию: `{}`) + 19. Включает трассировку модуля (по умолчанию: `true`) + 20. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + jdk: + threads: 2 #(1)! + httpVersion: "HTTP_1_1" #(2)! + connectTimeout: "5s" #(3)! + readTimeout: "2m" #(4)! + useEnvProxy: false #(5)! + proxy: + host: "localhost" #(6)! + port: 8090 #(7)! + user: "user" #(8)! + password: "password" #(9)! + nonProxyHosts: [ "host1", "host2" ] #(10)! + telemetry: + logging: + enabled: false #(11)! + mask: "***" #(12)! + maskQueries: [ ] #(13)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(14)! + pathTemplate: true #(15)! + metrics: + enabled: true #(16)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(17)! + tags: #(18)! + key1: value1 + key2: value2 + tracing: + enabled: true #(19)! + attributes: #(20)! + key1: value1 + key2: value2 + ``` + + 1. Количество потоков для `HTTP`-клиента (по умолчанию: количество доступных процессоров, умноженное на `2`) + 2. Какую версию `HTTP`-протокола использовать, доступные значения: `HTTP_1_1` / `HTTP_2` (по умолчанию: `HTTP_1_1`) + 3. Максимальное время на установление соединения (по умолчанию: `5s`) + 4. Максимальное время на чтение ответа (по умолчанию: `2m`) + 5. Использовать ли переменные окружения `https_proxy` / `HTTPS_PROXY` / `http_proxy` / `HTTP_PROXY` и `no_proxy` / `NO_PROXY` для настройки прокси (по умолчанию: `false`) + 6. Адрес прокси (`обязательная`, по умолчанию не указано) + 7. Порт прокси (`обязательная`, по умолчанию не указано) + 8. Пользователь для прокси (по умолчанию не указано, необязательно) + 9. Пароль для прокси (по умолчанию не указано, необязательно) + 10. Узлы, которые следует исключить из проксирования (по умолчанию не указано, необязательно) + 11. Включает логирование модуля (по умолчанию: `false`) + 12. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 13. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 14. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 15. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 16. Включает метрики модуля (по умолчанию: `true`) + 17. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 18. Настройка тегов для метрик (по умолчанию: `{}`) + 19. Включает трассировку модуля (по умолчанию: `true`) + 20. Настройка атрибутов для трассировки (по умолчанию: `{}`) + +## Декларативный клиент { #client-declarative } Предлагается использовать специальные аннотации для создания декларативного клиента: -* `@HttpClient` — указывает что интерфейс является декларативным HTTP клиентом -* `@HttpRoute` — указывает [тип HTTP запроса](https://developer.mozilla.org/ru/docs/Web/HTTP/Methods) и путь запроса +* `@HttpClient` — указывает, что интерфейс является декларативным `HTTP`-клиентом +* `@HttpRoute` — указывает [тип HTTP-запроса](https://developer.mozilla.org/ru/docs/Web/HTTP/Methods) и путь запроса ===! ":fontawesome-brands-java: `Java`" @@ -543,8 +649,8 @@ agent: ### Конфигурация клиента { #client-configuration } -Конфигурация конкретной реализации `@HttpClient` по умолчанию для поиска конфигурации использует следующий путь `httpClient.{имя класса в нижнем регистре}`, -либо указывается в параметре `configPath` в аннотации: +Конфигурация конкретной реализации `@HttpClient` по умолчанию ищется по пути `httpClient.{имя класса в нижнем регистре}`. +Если нужно задать путь явно, используйте параметр `configPath` в аннотации: ===! ":fontawesome-brands-java: `Java`" @@ -572,7 +678,42 @@ agent: 1. Путь до конфигурации конкретно этого клиента -Пример конфигурации в случае пути `httpClient.someClient` описанной в классе `DeclarativeHttpClientConfig`: +В `@HttpClient` также можно указать теги для внедряемых компонентов: + +* `httpClientTag` — тег для выбора конкретного транспортного `HttpClient`, если в графе есть несколько реализаций с разными `@Tag` +* `telemetryTag` — тег для выбора конкретной фабрики телеметрии клиента + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient( + configPath = "httpClient.someClient", + httpClientTag = CustomTransport.class, + telemetryTag = CustomTelemetry.class + ) + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + void hello(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient( + configPath = "httpClient.someClient", + httpClientTag = [CustomTransport::class], + telemetryTag = [CustomTelemetry::class] + ) + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + fun hello() + } + ``` + +Основные параметры конфигурации декларативного клиента: ===! ":material-code-json: `Hocon`" @@ -581,46 +722,12 @@ agent: someClient { url = "https://localhost:8090" //(1)! requestTimeout = "10s" //(2)! - telemetry { - logging { - enabled = false //(3)! - mask = "***" //(4)! - maskQueries = [ ] //(5)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(6)! - pathTemplate = true //(7)! - } - metrics { - enabled = true //(8)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(9)! - tags = { // (10)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(11)! - attributes = { // (12)! - "key1" = "value1" - "key2" = "value2" - } - } - } } } ``` - 1. URL сервиса куда будут отправляться запросы - 2. Максимальное время запроса, может включать все стадии, разрешение DNS, подключение, запись тела запроса, обработку сервера и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода. - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 5. Список параметров запроса которые следует скрывать - 6. Список заголовков запроса/ответа которые следует скрывать - 7. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 8. Включает метрики модуля (по умолчанию `true`) - 9. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 10. Настройка тегов для метрик (опционально) - 11. Включает трассировку модуля (по умолчанию `true`) - 12. Настройка атрибутов для трассировки (опционально) + 1. Базовый `URL` сервиса, куда будут отправляться запросы (`обязательная`, по умолчанию не указано) + 2. Максимальное время запроса (по умолчанию не указано, необязательно) === ":simple-yaml: `YAML`" @@ -629,63 +736,41 @@ agent: someClient: url: "https://localhost:8090" #(1)! requestTimeout: "10s" #(2)! - telemetry: - logging: - enabled: false #(3)! - mask: "***" #(4)! - maskQueries: [ ] #(5)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(6)! - pathTemplate: true #(7)! - metrics: - enabled: true #(8)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! - telemetry: - enabled: true #(10)! ``` - 1. URL сервиса куда будут отправляться запросы - 2. Максимальное время запроса, может включать все стадии, разрешение DNS, подключение, запись тела запроса, обработку сервера и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода. - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 5. Список параметров запроса которые следует скрывать - 6. Список заголовков запроса/ответа которые следует скрывать - 7. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 8. Включает метрики модуля (по умолчанию `true`) - 9. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 10. Включает трассировку модуля (по умолчанию `true`) + 1. Базовый `URL` сервиса, куда будут отправляться запросы (`обязательная`, по умолчанию не указано) + 2. Максимальное время запроса (по умолчанию не указано, необязательно) -### Конфигурация метода { #method-configuration } +??? note "Полная конфигурация" -На примере выше рассмотренного HTTP клиента, можно настроить отдельно часть параметров для определенного метода, путь к конфигурации -определяется путем к клиенту и именем метода, в примере выше конфигурация `httpClient.someClient` -и метода `hello` финальный путь будет `httpClient.someClient.hello` + Пример конфигурации в случае пути `httpClient.someClient` описанной в классе `DeclarativeHttpClientConfig`: -===! ":material-code-json: `Hocon`" + ===! ":material-code-json: `Hocon`" - ```javascript - httpClient { - someClient { - hello { - requestTimeout = "10s" //(1)! + ```javascript + httpClient { + someClient { + url = "https://localhost:8090" //(1)! + requestTimeout = "10s" //(2)! telemetry { logging { - enabled = false //(2)! - mask = "***" //(3)! - maskQueries = [ ] //(4)! - maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(5)! - pathTemplate = true //(6)! + enabled = false //(3)! + mask = "***" //(4)! + maskQueries = [ ] //(5)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(6)! + pathTemplate = true //(7)! } metrics { - enabled = true //(7)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(8)! - tags = { // (9)! + enabled = true //(8)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(9)! + tags = { // (10)! "key1" = "value1" "key2" = "value2" } } tracing { - enabled = true //(10)! - attributes = { // (11)! + enabled = true //(11)! + attributes = { // (12)! "key1" = "value1" "key2" = "value2" } @@ -693,20 +778,84 @@ agent: } } } + ``` + + 1. Базовый `URL` сервиса, куда будут отправляться запросы (`обязательная`, по умолчанию не указано) + 2. Максимальное время запроса: может включать разрешение `DNS`, подключение, запись тела запроса, обработку сервером и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода (по умолчанию не указано, необязательно) + 3. Включает логирование модуля (по умолчанию: `false`) + 4. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 5. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 6. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 7. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 8. Включает метрики модуля (по умолчанию: `true`) + 9. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 10. Настройка тегов для метрик (по умолчанию: `{}`) + 11. Включает трассировку модуля (по умолчанию: `true`) + 12. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + someClient: + url: "https://localhost:8090" #(1)! + requestTimeout: "10s" #(2)! + telemetry: + logging: + enabled: false #(3)! + mask: "***" #(4)! + maskQueries: [ ] #(5)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(6)! + pathTemplate: true #(7)! + metrics: + enabled: true #(8)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! + tags: #(10)! + key1: value1 + key2: value2 + tracing: + enabled: true #(11)! + attributes: #(12)! + key1: value1 + key2: value2 + ``` + + 1. Базовый `URL` сервиса, куда будут отправляться запросы (`обязательная`, по умолчанию не указано) + 2. Максимальное время запроса: может включать разрешение `DNS`, подключение, запись тела запроса, обработку сервером и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода (по умолчанию не указано, необязательно) + 3. Включает логирование модуля (по умолчанию: `false`) + 4. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 5. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 6. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 7. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 8. Включает метрики модуля (по умолчанию: `true`) + 9. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 10. Настройка тегов для метрик (по умолчанию: `{}`) + 11. Включает трассировку модуля (по умолчанию: `true`) + 12. Настройка атрибутов для трассировки (по умолчанию: `{}`) + +### Конфигурация метода { #method-configuration } + +Для конкретного метода можно отдельно настроить часть параметров. Путь к конфигурации метода определяется путем к клиенту и именем метода: +если путь клиента `httpClient.someClient`, то для метода `hello` итоговый путь будет `httpClient.someClient.hello`. + +Конфигурация метода накладывается поверх конфигурации клиента: `requestTimeout` метода заменяет клиентское значение, а настройки телеметрии метода +переопределяют только явно указанные поля. + +Основные параметры конфигурации метода: + +===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + someClient { + hello { + requestTimeout = "10s" //(1)! + } + } } ``` - 1. Максимальное время запроса, может включать все стадии, разрешение DNS, подключение, запись тела запроса, обработку сервера и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода. - 2. Включает логгирование модуля (по умолчанию `false`) - 3. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 4. Список параметров запроса которые следует скрывать - 5. Список заголовков запроса/ответа которые следует скрывать - 6. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 7. Включает метрики модуля (по умолчанию `true`) - 8. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 9. Настройка тегов для метрик (опционально) - 10. Включает трассировку модуля (по умолчанию `true`) - 11. Настройка атрибутов для трассировки (опционально) + 1. Максимальное время запроса (по умолчанию не указано, необязательно) === ":simple-yaml: `YAML`" @@ -715,35 +864,176 @@ agent: someClient: hello: requestTimeout: "10s" #(1)! - telemetry: - logging: - enabled: false #(2)! - mask: "***" #(3)! - maskQueries: [ ] #(4)! - maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(5)! - pathTemplate: true #(6)! - metrics: - enabled: true #(7)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(8)! - telemetry: - enabled: true #(9)! ``` - 1. Максимальное время запроса, может включать все стадии, разрешение DNS, подключение, запись тела запроса, обработку сервера и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода. - 2. Включает логгирование модуля (по умолчанию `false`) - 3. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 4. Список параметров запроса которые следует скрывать - 5. Список заголовков запроса/ответа которые следует скрывать - 6. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 7. Включает метрики модуля (по умолчанию `true`) - 8. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 9. Включает трассировку модуля (по умолчанию `true`) + 1. Максимальное время запроса (по умолчанию не указано, необязательно) + +??? note "Полная конфигурация" + + Пример полной конфигурации метода: + + ===! ":material-code-json: `Hocon`" + + ```javascript + httpClient { + someClient { + hello { + requestTimeout = "10s" //(1)! + telemetry { + logging { + enabled = false //(2)! + mask = "***" //(3)! + maskQueries = [ ] //(4)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(5)! + pathTemplate = true //(6)! + } + metrics { + enabled = true //(7)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(8)! + tags = { // (9)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(10)! + attributes = { // (11)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + } + ``` + + 1. Максимальное время запроса: может включать разрешение `DNS`, подключение, запись тела запроса, обработку сервером и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода (по умолчанию не указано, необязательно) + 2. Включает логирование модуля (по умолчанию: `false`) + 3. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 4. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 5. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 6. Использовать ли шаблон пути запроса при логировании; если не указано, наследуется значение клиента (по умолчанию не указано, необязательно) + 7. Включает метрики модуля (по умолчанию: `true`) + 8. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 9. Настройка тегов для метрик (по умолчанию: `{}`) + 10. Включает трассировку модуля (по умолчанию: `true`) + 11. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + httpClient: + someClient: + hello: + requestTimeout: "10s" #(1)! + telemetry: + logging: + enabled: false #(2)! + mask: "***" #(3)! + maskQueries: [ ] #(4)! + maskHeaders: [ "authorization", "cookie", "set-cookie" ] #(5)! + pathTemplate: true #(6)! + metrics: + enabled: true #(7)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(8)! + tags: #(9)! + key1: value1 + key2: value2 + tracing: + enabled: true #(10)! + attributes: #(11)! + key1: value1 + key2: value2 + ``` + + 1. Максимальное время запроса: может включать разрешение `DNS`, подключение, запись тела запроса, обработку сервером и чтение тела ответа. Если вызов требует перенаправления или повторных попыток, все они должны завершиться в течение одного периода (по умолчанию не указано, необязательно) + 2. Включает логирование модуля (по умолчанию: `false`) + 3. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 4. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 5. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 6. Использовать ли шаблон пути запроса при логировании; если не указано, наследуется значение клиента (по умолчанию не указано, необязательно) + 7. Включает метрики модуля (по умолчанию: `true`) + 8. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 9. Настройка тегов для метрик (по умолчанию: `{}`) + 10. Включает трассировку модуля (по умолчанию: `true`) + 11. Настройка атрибутов для трассировки (по умолчанию: `{}`) ### Запрос { #request } -Секция описывает преобразования HTTP запроса у декларативного HTTP клиента. +Раздел описывает преобразования `HTTP`-запроса у декларативного `HTTP`-клиента. Предлагается использовать специальные аннотации для указания параметров запроса. +#### Преобразование параметров в строку { #string-parameter-converter } + +`StringParameterConverter` преобразует значение параметра в строку перед тем, как Kora подставит его в путь, параметр запроса, +заголовок или куки. Интерфейс состоит из одного метода: + +```java +public interface StringParameterConverter { + String convert(T value); +} +``` + +Преобразователь ищется как обычный компонент графа по точному типу параметра. Если параметр имеет тип `Map`, +то преобразователь ищется для типа значения `T`; если используется `Map>`, он применяется к каждому элементу списка. + +Из коробки доступны преобразователи для `Boolean`, `Short`, `Integer`, `Long`, `Double`, `Float`, `UUID`, `BigDecimal`, `BigInteger`, +`Duration`, `OffsetTime`, `OffsetDateTime`, `LocalTime`, `LocalDate`, `LocalDateTime`, `ZonedDateTime` и `Instant`. +Типы даты и времени записываются в `ISO`-формате. Для собственных типов нужно предоставить компонент `StringParameterConverter`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserId(long value) {} + + @Module + public interface UserIdModule { + + default StringParameterConverter userIdStringParameterConverter() { + return value -> Long.toString(value.value()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserId(val value: Long) + + @Module + interface UserIdModule { + + fun userIdStringParameterConverter(): StringParameterConverter { + return StringParameterConverter { value -> value.value.toString() } + } + } + ``` + +После этого тип можно использовать в параметрах клиента: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + User get(@Path("id") UserId id); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + fun get(@Path("id") id: UserId): User + } + ``` + #### Параметр пути { #path-parameter } `@Path` — обозначает значение части пути запроса, сам параметр указывается в `{кавычках}` в пути @@ -774,6 +1064,8 @@ agent: #### Параметр запроса { #query-parameter } `@Query` — значение параметра запроса, имя параметра указывается в `value` либо по умолчанию равно имени аргумента метода. +Поддерживаются одиночные значения, `List`, `Set`, `Collection`, а также `Map` и `Map>`. +Для значений, которые не являются строками, используется доступный `StringParameterConverter`. ===! ":fontawesome-brands-java: `Java`" @@ -800,7 +1092,9 @@ agent: ``` Можно отправлять параметры запроса в формате ключ и значение, для этого предполагается использовать тип `Map`, -где ключом является имя параметра и обязано иметь тип `String`, а значение параметра может быть любым типом и будет обработано через `String.valueOf()`: +где ключом является имя параметра и обязательно имеет тип `String`. +Если значение `Map` является списком, каждый элемент списка будет отправлен как отдельное значение того же параметра. +Если элемент списка равен `null`, параметр будет отправлен без значения. ===! ":fontawesome-brands-java: `Java`" @@ -827,6 +1121,7 @@ agent: #### Заголовок { #header } `@Header` — значение [заголовка запроса](https://developer.mozilla.org/ru/docs/Web/HTTP/Headers), имя параметра указывается в `value` либо по умолчанию равно имени аргумента метода. +Поддерживаются одиночные значения, `List`, `Set`, `Collection`, `Map` и готовый объект `HttpHeaders`. ===! ":fontawesome-brands-java: `Java`" @@ -852,8 +1147,9 @@ agent: } ``` -Можно отправлять параметры запроса в формате ключ и значение, для этого предполагается использовать `HttpHeaders` тип либо тип `Map`, -где ключом является имя параметра и обязано иметь тип `String`, а значение параметра может быть любым типом и будет обработано через `String.valueOf()`: +Можно отправлять заголовки в формате ключ и значение, для этого предполагается использовать тип `HttpHeaders` либо `Map`, +где ключом является имя заголовка и обязательно имеет тип `String`. +Для значений, которые не являются строками, используется доступный `StringParameterConverter`: ===! ":fontawesome-brands-java: `Java`" @@ -882,10 +1178,10 @@ agent: Для указания тела запроса требуется использовать аргумент метода без специальных аннотации, по умолчанию поддерживаются такие типы как `byte[]`, `ByteBuffer` или `String`. -##### Json { #json } +##### JSON { #json } -Для указания, что тело является Json и ему требуется автоматически создать такого писателя и внедрить его, -требуется использовать специальную тег аннотацию `@Json`: +Чтобы указать, что тело является `JSON` и для него требуется автоматически создать и внедрить `JsonWriter`, +используется тег-аннотация `@Json`: ===! ":fontawesome-brands-java: `Java`" @@ -900,7 +1196,7 @@ agent: } ``` - 1. Указывает что тело должно быть записано как Json + 1. Указывает, что тело должно быть записано как `JSON` === ":simple-kotlin: `Kotlin`" @@ -915,9 +1211,9 @@ agent: } ``` - 1. Указывает что тело должно быть записано как Json + 1. Указывает, что тело должно быть записано как `JSON` -Требуется подключить модуль [Json](json.md). +Требуется подключить модуль [JSON](json.md). ##### Текстовая форма { #text-form } @@ -1067,9 +1363,52 @@ agent: } ``` +**Пример: Protobuf сериализация** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface ProtobufClient { + + final class ProtobufRequestMapper implements HttpClientRequestMapper { + + @Override + public HttpBodyOutput apply(Context ctx, MyMessage value) { + byte[] protobufBytes = value.toByteArray(); + return HttpBody.of(protobufBytes, "application/x-protobuf"); + } + } + + @HttpRoute(method = HttpMethod.POST, path = "/message") + void sendMessage(@Mapping(ProtobufRequestMapper.class) MyMessage message); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface ProtobufClient { + + class ProtobufRequestMapper : HttpClientRequestMapper { + + override fun apply(ctx: Context, value: MyMessage): HttpBodyOutput { + val protobufBytes = value.toByteArray() + return HttpBody.of(protobufBytes, "application/x-protobuf") + } + } + + @HttpRoute(method = HttpMethod.POST, path = "/message") + fun sendMessage(@Mapping(ProtobufRequestMapper::class) message: MyMessage) + } + ``` + #### Куки { #cookie } `@Cookie` — значение [Cookie](https://developer.mozilla.org/ru/docs/Glossary/Cookie), имя параметра указывается в `value` либо по умолчанию равно имени аргумента метода. +Поддерживаются одиночные значения, `List`, `Set`, `Collection`, `Map` и готовый объект `Cookie`. +Куки добавляются в заголовок `Cookie`; для коллекций каждое значение превращается в отдельное значение куки с тем же именем. ===! ":fontawesome-brands-java: `Java`" @@ -1136,15 +1475,15 @@ agent: ### Ответ { #response } -Секция описывает преобразование HTTP ответа от декларативного HTTP клиента. +Раздел описывает преобразование `HTTP`-ответа от декларативного `HTTP`-клиента. #### Тело ответа { #response-body } По умолчанию можно использовать стандартные типы возвращаемых значений тела ответа, такие как `void`, `byte[]`, `ByteBuffer` либо `String`. -##### Json { #json-2 } +##### JSON { #json-2 } -Если предполагается читать тело как Json, то требуется использовать аннотацию `@Json` над методом. +Если предполагается читать тело как `JSON`, то требуется использовать аннотацию `@Json` над методом. ===! ":fontawesome-brands-java: `Java`" @@ -1160,7 +1499,7 @@ agent: } ``` - 1. Указывает что ответ должен быть прочитан как Json + 1. Указывает, что ответ должен быть прочитан как `JSON` === ":simple-kotlin: `Kotlin`" @@ -1176,16 +1515,16 @@ agent: } ``` - 1. Указывает что ответ должен быть прочитан как Json + 1. Указывает, что ответ должен быть прочитан как `JSON` -Требуется подключить модуль [Json](json.md). +Требуется подключить модуль [JSON](json.md). ##### Сущность ответа { #response-entity } Если предполагается читать тело и получить также заголовки и статус код ответа, то предполагается использовать `HttpResponseEntity`, это обертка над телом ответа. -Ниже показан пример аналогичный примеру Json вместе с оберткой `HttpResponseEntity`: +Ниже показан пример, аналогичный примеру `JSON`, вместе с оберткой `HttpResponseEntity`: ===! ":fontawesome-brands-java: `Java`" @@ -1271,17 +1610,127 @@ agent: } ``` -#### Ошибка ответа { #response-error } +**Пример: Обработка ошибок в маппере** -По умолчанию когда не указан ни какой тег преобразователя ни сам преобразователь, то преобразование будет применяться только для `2хх` HTTP статусов кодов, -для всех остальных будет выбрасываться исключение `HttpClientResponseException`, которое содержит [HTTP статус код](https://developer.mozilla.org/ru/docs/Web/HTTP/Status), тело ответа и заголовки ответа. +===! ":fontawesome-brands-java: `Java`" -#### Преобразование по коду { #conversion-by-code } + ```java + @HttpClient + public interface ApiClient { + + record ApiResponse(String status, Object data) {} + + final class SafeResponseMapper implements HttpClientResponseMapper { + + private final JsonReader jsonReader; + + public SafeResponseMapper(JsonReader jsonReader) { + this.jsonReader = jsonReader; + } + + @Override + public ApiResponse apply(HttpClientResponse response) throws IOException { + int statusCode = response.statusCode(); + byte[] body = response.body(); + + if (statusCode >= 400) { + // Обработка ошибки: логирование или выброс исключения + throw new HttpClientResponseException(statusCode, body, response.headers()); + } + + if (body == null || body.length == 0) { + return null; + } + + return jsonReader.read(body); + } + } + + @HttpRoute(method = HttpMethod.GET, path = "/api/data") + @Mapping(SafeResponseMapper.class) + ApiResponse getData(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface ApiClient { + + data class ApiResponse(val status: String, val data: Any?) + + class SafeResponseMapper( + private val jsonReader: JsonReader + ) : HttpClientResponseMapper { + + @Throws(IOException::class) + override fun apply(response: HttpClientResponse): ApiResponse { + val statusCode = response.statusCode() + val body = response.body() + + if (statusCode >= 400) { + // Обработка ошибки: логирование или выброс исключения + throw HttpClientResponseException(statusCode, body, response.headers()) + } + + if (body == null || body.isEmpty()) { + return null + } + + return jsonReader.read(body) + } + } + + @HttpRoute(method = HttpMethod.GET, path = "/api/data") + @Mapping(SafeResponseMapper::class) + fun getData(): ApiResponse + } + ``` + +#### Ошибка ответа { #response-error } + +По умолчанию, когда не указан ни тег преобразователя, ни сам преобразователь, преобразование применяется только для `2xx` HTTP-кодов ответа. +Для всех остальных кодов будет выброшено исключение `HttpClientResponseException`, которое содержит [HTTP-код ответа](https://developer.mozilla.org/ru/docs/Web/HTTP/Status), тело ответа и заголовки ответа. + +#### Исключения клиента { #client-exceptions } + +Все штатные исключения `HTTP`-клиента наследуются от `HttpClientException`, который является `RuntimeException`. +Это позволяет перехватывать как конкретный вид ошибки, так и все ошибки клиента одним общим типом: -Если требуются специфичные преобразование в зависимости от [HTTP статус кода](https://developer.mozilla.org/ru/docs/Web/HTTP/Status) ответа, можно использовать аннотацию `@ResponseCodeMapper` для указания -соответствия HTTP статус кода и преобразователя `HttpClientResponseMapper`. +```java +try { + client.getUser("123"); +} catch (HttpClientResponseException e) { + var code = e.getCode(); + var headers = e.getHeaders(); + var body = e.getBytes(); +} catch (HttpClientException e) { + throw e; +} +``` + +Основные типы исключений: + +* `HttpClientResponseException` — ответ получен, но его код не был обработан как успешный. Содержит `getCode()`, `getHeaders()` и `getBytes()`. +* `HttpClientTimeoutException` — истекло время ожидания запроса, соединения или чтения. +* `HttpClientConnectionException` — ошибка установления или поддержания соединения с удаленным узлом. +* `HttpClientEncoderException` — ошибка преобразования пользовательского значения в тело запроса. +* `HttpClientDecoderException` — ошибка преобразования тела ответа в пользовательский тип. +* `HttpClientUnknownException` — прочая ошибка транспортного клиента, которая не попала в более точную категорию. + +`HttpClientResponseException` создается после чтения тела ответа в массив байт. Если тело не удалось прочитать полностью, +ошибка чтения добавляется как `suppressed`-исключение, а в `getBytes()` попадает то тело, которое удалось собрать. + +#### Преобразование по коду { #conversion-by-code } + +Если требуется особое преобразование в зависимости от [HTTP-кода ответа](https://developer.mozilla.org/ru/docs/Web/HTTP/Status), можно использовать аннотацию `@ResponseCodeMapper` для указания +соответствия HTTP-кода и преобразователя `HttpClientResponseMapper`. -Также можно использовать `ResponseCodeMapper.DEFAULT` как указание поведения по умолчанию для всех не перечисленных статус кодов. +Также можно использовать `ResponseCodeMapper.DEFAULT` как указание поведения по умолчанию для всех неперечисленных HTTP-кодов. +Если для кода указан параметр `mapper`, будет использован конкретный `HttpClientResponseMapper`. +Если указан параметр `type`, Kora подберет преобразователь ответа для этого типа и затем приведет результат к возвращаемому типу метода. +Это удобно для закрытых иерархий ответов, где разные HTTP-статусы соответствуют разным подтипам результата. ===! ":fontawesome-brands-java: `Java`" @@ -1326,9 +1775,57 @@ agent: В примере выше для статуса кода `200` будет использовать `ResponseSuccessMapper`, а для всех остальных статус кодов будет использован `ResponseErrorMapper`. +Пример с параметром `type`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface SomeClient { + + @Json + sealed interface UserResponse permits Success, Error {} + + @Json + record Success(String id) implements UserResponse {} + + @Json + record Error(String message) implements UserResponse {} + + @Json + @ResponseCodeMapper(code = 200, type = Success.class) + @ResponseCodeMapper(code = 404, type = Error.class) + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + UserResponse get(@Path String id); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface SomeClient { + + @Json + sealed interface UserResponse + + @Json + data class Success(val id: String) : UserResponse + + @Json + data class Error(val message: String) : UserResponse + + @Json + @ResponseCodeMapper(code = 200, type = Success::class) + @ResponseCodeMapper(code = 404, type = Error::class) + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + fun get(@Path id: String): UserResponse + } + ``` + ### Сигнатуры { #signatures } -Доступные сигнатуры для методов декларативного HTTP клиента из коробки: +Доступные сигнатуры для методов декларативного `HTTP`-клиента из коробки: ===! ":fontawesome-brands-java: `Java`" @@ -1347,8 +1844,202 @@ agent: ## Перехватчики { #interceptors } -Можно создавать перехватчики для изменения поведения либо создания дополнительного поведения используя класс `HttpClientInterceptor`. -Перехватчики можно подключить на определенные методы либо весь `@HttpClient` класс целиком: +Можно создавать перехватчики для изменения поведения либо создания дополнительного поведения используя интерфейс `HttpClientInterceptor`. +Перехватчики можно подключить на определенные методы либо весь `@HttpClient` класс целиком с помощью аннотации `@InterceptWith`. + +**Перехватчик на метод:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + public interface SomeClient { + + final class MethodInterceptor implements HttpClientInterceptor { + + private final Component1 component1; + + private MethodInterceptor(Component1 component1) { + this.component1 = component1; + } + + @Override + public CompletionStage processRequest(Context ctx, InterceptChain chain, HttpClientRequest request) throws Exception { + component1.doSomething(); + return chain.process(ctx, request); + } + } + + @InterceptWith(MethodInterceptor.class) + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + void hello(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + interface SomeClient { + + class MethodInterceptor(val component1: Component1) : HttpClientInterceptor { + + @Throws(Exception::class) + override fun processRequest( + ctx: Context, + chain: HttpClientInterceptor.InterceptChain, + request: HttpClientRequest + ): CompletionStage { + component1.doSomething() + return chain.process(ctx, request) + } + } + + @InterceptWith(MethodInterceptor::class) + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + fun hello() + } + ``` + +**Перехватчик на весь класс:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @InterceptWith(LoggingInterceptor.class) // Применяется ко всем методам клиента + @HttpClient + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello") + void hello(); + + @HttpRoute(method = HttpMethod.POST, path = "/world") + void world(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @InterceptWith(LoggingInterceptor::class) // Применяется ко всем методам клиента + @HttpClient + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello") + fun hello() + + @HttpRoute(method = HttpMethod.POST, path = "/world") + fun world() + } + ``` + +**Порядок выполнения перехватчиков:** + +Перехватчики выполняются в порядке объявления (слева направо). Каждый перехватчик может: +- Модифицировать запрос перед отправкой +- Вызвать следующий перехватчик в цепочке (`chain.process()`) +- Модифицировать ответ после получения +- Выбросить исключение и прервать цепочку + +``` +Запрос → Interceptor1 → Interceptor2 → Interceptor3 → HTTP сервер +Ответ ← Interceptor1 ← Interceptor2 ← Interceptor3 ← HTTP сервер +``` + +### Перехватчик на клиент { #interceptor-global } + +Для применения перехватчика ко всем клиентам можно зарегистрировать его как компонент без `@InterceptWith`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class GlobalInterceptor implements HttpClientInterceptor { + + @Override + public CompletionStage processRequest(Context ctx, InterceptChain chain, HttpClientRequest request) throws Exception { + // Применяется ко всем HTTP клиентам + return chain.process(ctx, request); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class GlobalInterceptor : HttpClientInterceptor { + + @Throws(Exception::class) + override fun processRequest( + ctx: Context, + chain: HttpClientInterceptor.InterceptChain, + request: HttpClientRequest + ): CompletionStage { + // Применяется ко всем HTTP клиентам + return chain.process(ctx, request) + } + } + ``` + +### Базовый URL { #root-uri-interceptor } + +`RootUriInterceptor` — готовый перехватчик, который добавляет базовый `URL` к относительным запросам. +Если запрос уже содержит схему (`http://` или `https://`), перехватчик оставляет его без изменений. +Если запрос относительный, `RootUriInterceptor` добавляет к нему корневой адрес и гарантирует один разделитель `/` между корнем и путем. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface ClientModule { + + default RootUriInterceptor rootUriInterceptor() { + return new RootUriInterceptor("https://api.example.com"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface ClientModule { + + fun rootUriInterceptor(): RootUriInterceptor { + return RootUriInterceptor("https://api.example.com") + } + } + ``` + +После регистрации перехватчика его можно подключить к клиенту: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + @InterceptWith(RootUriInterceptor.class) + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + User get(@Path String id); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + @InterceptWith(RootUriInterceptor::class) + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/users/{id}") + fun get(@Path id: String): User + } + ``` + +Для декларативных клиентов обычно удобнее задавать базовый `URL` через конфигурацию `DeclarativeHttpClientConfig.url`. +`RootUriInterceptor` полезен для императивного `HttpClient` или для случаев, когда общий корневой адрес нужно добавить как отдельное сквозное поведение. ===! ":fontawesome-brands-java: `Java`" @@ -1402,9 +2093,37 @@ agent: } ``` +Если перехватчик нужен для всех методов клиента, `@InterceptWith` можно поставить на интерфейс: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @HttpClient + @InterceptWith(ClientInterceptor.class) + public interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + void hello(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @HttpClient + @InterceptWith(ClientInterceptor::class) + interface SomeClient { + + @HttpRoute(method = HttpMethod.GET, path = "/hello/world") + fun hello() + } + ``` + +Если перехватчики указаны и на клиенте, и на методе, для конкретного вызова будут применены оба набора перехватчиков. + ### Авторизация { #authorization } -Kora предоставляет готовые перехватчики которые можно использовать для авторизации посредствам [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/) +Kora предоставляет готовые перехватчики, которые можно использовать для авторизации с помощью [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/) #### Basic { #basic } @@ -1452,7 +2171,7 @@ Kora предоставляет готовые перехватчики кото Также в конструктор можно предоставить собственную реализацию `HttpClientTokenProvider` если правила получения секретов другие. -Затем подключить перехватчик для всего HTTP клиента либо определенных методов. +Затем подключить перехватчик для всего `HTTP`-клиента либо определенных методов. ===! ":fontawesome-brands-java: `Java`" @@ -1518,7 +2237,7 @@ Kora предоставляет готовые перехватчики кото } ``` -Затем подключить перехватчик для всего HTTP клиента либо определенных методов. +Затем подключить перехватчик для всего `HTTP`-клиента либо определенных методов. ===! ":fontawesome-brands-java: `Java`" @@ -1582,7 +2301,7 @@ public interface HttpClientTokenProvider { } ``` -Затем подключить перехватчик для всего HTTP клиента либо определенных методов. +Затем подключить перехватчик для всего `HTTP`-клиента либо определенных методов. ===! ":fontawesome-brands-java: `Java`" @@ -1610,9 +2329,263 @@ public interface HttpClientTokenProvider { #### OAuth { #oauth } -Авторизация посредствам [OAuth](https://swagger.io/docs/specification/authentication/oauth2/) аналогично [Bearer](#bearer), +Авторизация с помощью [OAuth](https://swagger.io/docs/specification/authentication/oauth2/) аналогична [Bearer](#bearer), требуется самостоятельно реализовать `HttpClientTokenProvider` и подложить его в контейнер зависимостей. +#### Предоставление токена { #token-provider } + +`HttpClientTokenProvider` — интерфейс для предоставления токенов авторизации динамически. +Используется когда токен нужно обновлять или получать из внешнего источника (например, OAuth2 token endpoint). + +**Пример реализации:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class MyTokenProvider implements HttpClientTokenProvider { + + private final OAuthClient oauthClient; + private volatile String cachedToken; + private volatile long tokenExpiry; + + public MyTokenProvider(OAuthClient oauthClient) { + this.oauthClient = oauthClient; + } + + @Override + public CompletionStage getToken(HttpClientRequest request) { + if (cachedToken != null && System.currentTimeMillis() < tokenExpiry) { + return CompletableFuture.completedFuture(cachedToken); + } + + // Получить новый токен + return oauthClient.refreshToken() + .thenApply(response -> { + this.cachedToken = response.accessToken(); + this.tokenExpiry = System.currentTimeMillis() + response.expiresIn() * 1000; + return this.cachedToken; + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyTokenProvider( + private val oauthClient: OAuthClient + ) : HttpClientTokenProvider { + + private var cachedToken: String? = null + private var tokenExpiry: Long = 0 + + override fun getToken(request: HttpClientRequest): CompletionStage { + if (cachedToken != null && System.currentTimeMillis() < tokenExpiry) { + return CompletableFuture.completedFuture(cachedToken) + } + + // Получить новый токен + return oauthClient.refreshToken() + .thenApply { response -> + cachedToken = response.accessToken() + tokenExpiry = System.currentTimeMillis() + response.expiresIn() * 1000 + cachedToken!! + } + } + } + ``` + +**Использование с BearerAuthHttpClientInterceptor:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface AuthModule { + + default BearerAuthHttpClientInterceptor bearerAuthInterceptor(HttpClientTokenProvider tokenProvider) { + return new BearerAuthHttpClientInterceptor(tokenProvider); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface AuthModule { + + fun bearerAuthInterceptor(tokenProvider: HttpClientTokenProvider): BearerAuthHttpClientInterceptor { + return BearerAuthHttpClientInterceptor(tokenProvider) + } + } + ``` + +## Обработка исключений { #exception-handling } + +При выполнении HTTP запросов могут возникать различные исключения. Все исключения наследуются от базового `HttpClientException`. + +**Иерархия исключений:** + +``` +HttpClientException +├── HttpClientTimeoutException +├── HttpClientConnectionException +├── HttpClientResponseException +├── HttpClientEncoderException +├── HttpClientDecoderException +└── HttpClientUnknownException +``` + +**Пример обработки:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + class SomeService { + private final SomeClient client; + + public SomeService(SomeClient client) { + this.client = client; + } + + public void call() { + try { + client.hello(); + } catch (HttpClientTimeoutException e) { + // Таймаут: логирование, повторная попытка + } catch (HttpClientConnectionException e) { + // Ошибка соединения: проверка доступности сервиса + } catch (HttpClientResponseException e) { + // Ошибка ответа: statusCode, body, headers + int statusCode = e.getStatusCode(); + byte[] body = e.getBody(); + } catch (HttpClientEncoderException e) { + // Ошибка сериализации: проверка данных + } catch (HttpClientDecoderException e) { + // Ошибка десериализации: логирование + } catch (HttpClientUnknownException e) { + // Неизвестная ошибка: e.getCause() + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val client: SomeClient + ) { + fun call() { + try { + client.hello() + } catch (e: HttpClientTimeoutException) { + // Таймаут: логирование, повторная попытка + } catch (e: HttpClientConnectionException) { + // Ошибка соединения: проверка доступности сервиса + } catch (e: HttpClientResponseException) { + // Ошибка ответа: statusCode, body, headers + val statusCode = e.statusCode + val body = e.body + } catch (e: HttpClientEncoderException) { + // Ошибка сериализации: проверка данных + } catch (e: HttpClientDecoderException) { + // Ошибка десериализации: логирование + } catch (e: HttpClientUnknownException) { + // Неизвестная ошибка: e.cause + } + } + } + ``` + +#### Время ожидания { #timeout-exception } + +Выбрасывается когда запрос превышает установленное время ожидания (`requestTimeout` или `connectTimeout`). + +**Причины:** +- Сервер не отвечает в течение `requestTimeout` +- Превышено время установления соединения (`connectTimeout`) +- Сетевые задержки + +**Рекомендации:** +- Настройте адекватные таймауты в конфигурации +- Реализуйте retry-логику для временных сбоев +- Используйте circuit breaker для защиты от cascading failures + +#### Ошибка соединения { #connection-exception } + +Выбрасывается когда не удалось установить соединение с сервером. + +**Причины:** +- DNS не разрешается +- Сервер недоступен (port closed, firewall) +- Соединение отклонено +- SSL/TLS handshake failed + +**Рекомендации:** +- Проверьте доступность сервиса (health check) +- Используйте fallback на резервный сервис +- Настройте retry с exponential backoff + +#### Ошибка клиента и сервера { #response-exception } + +Выбрасывается когда сервер вернул HTTP статус код ошибки (4xx или 5xx) и не указан кастомный маппер через `@ResponseCodeMapper`. + +**Доступные данные:** +- `statusCode` — HTTP статус код (400, 404, 500, etc.) +- `body` — тело ответа (может содержать детали ошибки) +- `headers` — заголовки ответа + +**Рекомендации:** +- Используйте `@ResponseCodeMapper` для кастомной обработки статусов +- Логируйте statusCode и body для отладки +- Различайте клиентские (4xx) и серверные (5xx) ошибки + +#### Ошибка запроса { #encoder-exception } + +Выбрасывается когда произошла ошибка при сериализации тела запроса. + +**Причины:** +- Ошибка JSON/XML сериализации +- Невалидные данные в объекте запроса +- Отсутствие сериализатора для типа + +**Рекомендации:** +- Валидируйте данные перед отправкой +- Проверьте наличие Json-аннотаций на классах +- Логируйте оригинальное исключение в `cause` + +#### Ошибка ответа { #decoder-exception } + +Выбрасывается когда произошла ошибка при десериализации тела ответа. + +**Причины:** +- Невалидный JSON/XML в ответе сервера +- Несоответствие схемы (сервер вернул неожиданные поля) +- Отсутствие десериализатора для типа + +**Рекомендации:** +- Проверьте совместимость версий API +- Логируйте тело ответа для отладки +- Используйте `@ResponseCodeMapper` для обработки ошибок формата + +#### Ошибка неизвестная { #unknown-exception } + +Выбрасывается когда произошла неизвестная ошибка, не подпадающая под другие категории. + +**Доступные данные:** +- `cause` — оригинальное исключение + +**Рекомендации:** +- Всегда логируйте `cause` для диагностики +- Проверьте логи HTTP клиента на уровне DEBUG/TRACE +- Сообщите о баге если исключение воспроизводится + ## Клиент императивный { #client-imperative } Базовый клиент представляет собой интерфейс `HttpClient` и доступен для внедрения: @@ -1652,3 +2625,207 @@ public interface HttpClient { .body(HttpBody.plaintext("refresh")) .build() ``` + +### Построитель запроса { #request-builder } + +`HttpClientRequestBuilder` позволяет строить HTTP запросы вручную. + +===! ":fontawesome-brands-java: `Java`" + + ```java + HttpClientRequest request = HttpClientRequest.of("POST", "http://localhost:8090/pets/{petId}") + .templateParam("petId", "1") + .queryParam("page", 1) + .header("token", "12345") + .body(HttpBody.plaintext("refresh")) + .build(); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = HttpClientRequest.of("POST", "http://localhost:8090/pets/{petId}") + .templateParam("petId", "1") + .queryParam("page", 1) + .header("token", "12345") + .body(HttpBody.plaintext("refresh")) + .build() + ``` + +### Построитель URI { #uri-query-builder } + +`UriQueryBuilder` помогает строить URI с параметрами запроса. + +===! ":fontawesome-brands-java: `Java`" + + ```java + UriQueryBuilder builder = new UriQueryBuilder() + .path("/api/users") + .queryParam("page", 1) + .queryParam("size", 10) + .queryParam("sort", "name"); + + String uri = builder.build(); + // /api/users?page=1&size=10&sort=name + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val builder = UriQueryBuilder() + .path("/api/users") + .queryParam("page", 1) + .queryParam("size", 10) + .queryParam("sort", "name") + + val uri = builder.build() + // /api/users?page=1&size=10&sort=name + ``` + +### Тело запроса { #http-body-input } + +`HttpBodyInput` — интерфейс который описывает тело HTTP запроса как поток данных (Flow.Publisher). +Используется для стриминга больших данных без загрузки в память. + +**Методы:** + +| Метод | Возвращает | Описание | +|-------|------------|----------| +| `asInputStream()` | `InputStream` | Представляет тело как InputStream для чтения | +| `asBufferStage()` | `CompletionStage` | Асинхронно читает всё тело в ByteBuffer | +| `asArrayStage()` | `CompletionStage` | Асинхронно читает всё тело в byte[] | + +### Ответ клиента { #http-client-response } + +`HttpClientResponse` — интерфейс который представляет HTTP ответ от сервера. + +**Методы:** + +| Метод | Возвращает | Описание | +|-------|------------|----------| +| `statusCode()` | `int` | HTTP статус код (200, 404, 500, etc.) | +| `body()` | `byte[]` | Тело ответа как массив байтов | +| `headers()` | `HttpHeaders` | Заголовки ответа | +| `cookies()` | `Cookies` | Cookies из ответа | + +### Заголовки { #http-headers-imperative } + +`HttpHeaders` предоставляет доступ к заголовкам запроса и ответа в императивном клиенте. + +**Чтение заголовков:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + HttpClientRequest request = HttpClientRequest.of("GET", "http://localhost:8090/api/data") + .build(); + + httpClient.execute(request).thenAccept(response -> { + HttpHeaders headers = response.headers(); + String contentType = headers.getFirst("Content-Type"); + List allValues = headers.get("X-Custom-Header"); + boolean hasHeader = headers.contains("Authorization"); + }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = HttpClientRequest.of("GET", "http://localhost:8090/api/data").build() + + httpClient.execute(request).thenAccept { response -> + val headers = response.headers + val contentType = headers.getFirst("Content-Type") + val allValues = headers.get("X-Custom-Header") + val hasHeader = headers.contains("Authorization") + } + ``` + +**Добавление заголовков:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + MutableHttpHeaders headers = new MutableHttpHeaders(); + headers.add("Authorization", "Bearer token123"); + headers.add("X-Custom-Header", "value"); + headers.set("Content-Type", "application/json"); + + HttpClientRequest request = HttpClientRequest.of("POST", "http://localhost:8090/api/data") + .headers(headers) + .body(HttpBody.plaintext("body")) + .build(); + + httpClient.execute(request); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val headers = MutableHttpHeaders() + headers.add("Authorization", "Bearer token123") + headers.add("X-Custom-Header", "value") + headers.set("Content-Type", "application/json") + + val request = HttpClientRequest.of("POST", "http://localhost:8090/api/data") + .headers(headers) + .body(HttpBody.plaintext("body")) + .build() + + httpClient.execute(request) + ``` + +### Cookies { #cookies-imperative } + +`Cookies` предоставляет доступ к cookies запроса и ответа в императивном клиенте. + +**Чтение cookies:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + HttpClientRequest request = HttpClientRequest.of("GET", "http://localhost:8090/api/profile") + .build(); + + httpClient.execute(request).thenAccept(response -> { + Cookies cookies = response.cookies(); + Cookie sessionCookie = cookies.get("SESSIONID"); + if (sessionCookie != null) { + String value = sessionCookie.value(); + String domain = sessionCookie.domain(); + String path = sessionCookie.path(); + } + }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val request = HttpClientRequest.of("GET", "http://localhost:8090/api/profile").build() + + httpClient.execute(request).thenAccept { response -> + val cookies = response.cookies + val sessionCookie = cookies.get("SESSIONID") + if (sessionCookie != null) { + val value = sessionCookie.value() + val domain = sessionCookie.domain() + val path = sessionCookie.path() + } + } + ``` + +## Телеметрия { #telemetry } + +HTTP Client использует контракт телеметрии для логирования, метрик и трассировки запросов. +Конфигурация телеметрии (секция `telemetry { logging / metrics / tracing }`) описана в разделе [Конфигурация](#configuration). +Точки расширения находятся в `ru.tinkoff.kora.http.client.common.telemetry`. + +Для каждого HTTP-запроса создаётся `HttpClientTelemetry.HttpClientTelemetryContext`, который закрывается по завершении запроса. +Запрос описывается через параметры обработчика телеметрии, включая метод, URL, статус ответа и длительность. + +Фабрика по умолчанию `DefaultHttpClientTelemetryFactory` объединяет три фабрики: +- `HttpClientLoggerFactory` строит `HttpClientLogger` для логирования начала/конца запроса; +- `HttpClientMetricsFactory` строит `HttpClientMetrics` для записи метрик запросов; +- `HttpClientTracerFactory` строит `HttpClientTracer` для распределённой трассировки. + +Метрики и трассировка описаны в разделе [Справочник метрик](metrics.md#http-client). diff --git a/mkdocs/docs/ru/documentation/http-server.md b/mkdocs/docs/ru/documentation/http-server.md index cf7c8ac..c512afa 100644 --- a/mkdocs/docs/ru/documentation/http-server.md +++ b/mkdocs/docs/ru/documentation/http-server.md @@ -4,24 +4,29 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora HTTP server, declarative and imperative controllers, routing, request and response mapping, interceptors, error handling, and Undertow configuration; key triggers include @HttpController, @HttpRoute, @Path, @Query, @Header, @Cookie, @Json, @InterceptWith, HttpServerModule, UndertowHttpServerModule." --- -Модуль предоставляет тонкий слой абстракции над библиотеками HTTP-сервера для создания обработчиков HTTP-запросов -с помощью аннотаций в декларативном стиле, так и в императивном стиле. +Модуль `HTTP-сервера` описывает входящую HTTP-границу приложения: прием запроса, разбор параметров, чтение тела, +выбор обработчика, формирование ответа, телеметрию и перехватчики. В Kora можно описывать контроллеры декларативно +через `@HttpController` и `@HttpRoute` как тонкий слой абстракции, либо регистрировать обработчики императивно через `HttpServerRequestHandler`. + +Декларативный подход подходит для большинства API: сигнатура метода описывает HTTP-контракт, а Kora во время компиляции +создает обработчик. Императивный подход полезен для низкоуровневых +или динамических маршрутов, где запрос удобнее обрабатывать вручную. ???+ tip "Совет" - **Мы советуем** использовать подход когда первичен контракт в формате OpenAPI - и из него создаются контроллеры по средствам генератора. - Такой подход позволяет достигнуть консистентности контракта между потребителем и собственником, - и позволять делиться этим контрактом для создания клиентов для него по средствам такого же подхода. - Подробнее про генератор в [секции про генерации из OpenAPI](openapi-codegen.md). + **Мы советуем** использовать подход, при котором первичен контракт в формате `OpenAPI`, + а контроллеры создаются с помощью генератора. + Такой подход помогает сохранить согласованность контракта между потребителем и владельцем контракта + и позволяет использовать тот же контракт для генерации клиентов. + Подробнее про генератор смотрите в [разделе про генерацию из OpenAPI](openapi-codegen.md). -Если нужен пошаговый разбор перед справочным описанием, смотрите [HTTP сервер](../guides/http-server.md) и [HTTP сервер продвинутый](../guides/http-server-advanced.md). +Если нужен пошаговый разбор перед справочным описанием, смотрите [HTTP-сервер](../guides/http-server.md) и [продвинутый HTTP-сервер](../guides/http-server-advanced.md). ## Подключение { #dependency } -Реализация основанная на [Undertow](https://undertow.io/). -Undertow — это легковесный веб-сервер с открытым исходным кодом для Java-приложений. -Он построен на асинхронных и неблокирующих I/O-операциях с использованием NIO, +Реализация основана на [Undertow](https://undertow.io/). +`Undertow` — это легковесный веб-сервер с открытым исходным кодом для `Java`-приложений. +Он построен на асинхронных и неблокирующих операциях ввода-вывода с использованием `NIO`, что обеспечивает высокую производительность и низкое потребление ресурсов. ===! ":fontawesome-brands-java: `Java`" @@ -52,7 +57,38 @@ Undertow — это легковесный веб-сервер с открыты ## Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `HttpServerConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации HTTP-сервера: + +===! ":material-code-json: `Hocon`" + + ```javascript + httpServer { + publicApiHttpPort = 8080 //(1)! + privateApiHttpPort = 8085 //(2)! + maxRequestBodySize = "256MiB" //(3)! + } + ``` + + 1. Порт публичного `HTTP`-сервера (по умолчанию: `8080`) + 2. Порт служебного `HTTP`-сервера (по умолчанию: `8085`) + 3. Максимально допустимый размер тела входящего запроса (по умолчанию: `256MiB`) + +=== ":simple-yaml: `YAML`" + + ```yaml + httpServer: + publicApiHttpPort: 8080 #(1)! + privateApiHttpPort: 8085 #(2)! + maxRequestBodySize: "256MiB" #(3)! + ``` + + 1. Порт публичного `HTTP`-сервера (по умолчанию: `8080`) + 2. Порт служебного `HTTP`-сервера (по умолчанию: `8085`) + 3. Максимально допустимый размер тела входящего запроса (по умолчанию: `256MiB`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `HttpServerConfig` (указаны примеры значений или значения по умолчанию): ===! ":material-code-json: `Hocon`" @@ -78,8 +114,8 @@ Undertow — это легковесный веб-сервер с открыты enabled = false //(16)! stacktrace = true //(17)! mask = "***" //(18)! - maskqueries = [ ] //(19)! - maskheaders = [ "authorization", "cookie", "set-cookie" ] //(20)! + maskQueries = [ ] //(19)! + maskHeaders = [ "authorization", "cookie", "set-cookie" ] //(20)! pathTemplate = true //(21)! } metrics { @@ -101,32 +137,32 @@ Undertow — это легковесный веб-сервер с открыты } ``` - 1. Порт публичного HTTP-сервера - 2. Порт служебного HTTP-сервера - 3. Путь для получения [метрик](metrics.md) на служебном сервере - 4. Путь для получения статуса [проб готовности](probes.md) на служебном сервере - 5. Путь для получения статуса [проб жизнеспособности](probes.md) на служебном сервере - 6. Игнорировать ли слэш в окончании пути, если включен то `/my/path` и `/my/path/` будут интерпритироваться одинакого, по умолчанию выключен - 7. Количество потоков сетевых обработчиков, по умолчанию равен кол-во ядер процессора либо минимум `2` - 8. Количество потоков обработчиков запросов, по умолчанию равен кол-во ядер процессора умноженных на 2 либо минимум `2` потока - 9. Время ожидания обработки перед выключением сервера в случае [штатного завершения](container.md#component-lifecycle) - 10. Максимальное время жизни потока обработчика запроса - 11. Максимальное время ожидания чтения данные из сокета/соединения - 12. Максимальное время ожидания записи данных в сокет/соединение - 13. Отсылать ли сообщения `keep-alive` во время жизни сокета/соединения TCP - 14. Включает поддержку виртуальных потоков для обработки запросов (вместо `blockingThreads`), требует Java 21+ - 15. Максимально допустимый размер тела входящего запроса - 16. Включает логгирование модуля (по умолчанию `false`) - 17. Включает логгирование стэка вызовов в случае исключения - 18. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 19. Список параметров запроса которые следует скрывать - 20. Список заголовков запроса/ответа которые следует скрывать - 21. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 22. Включает метрики модуля (по умолчанию `true`) - 23. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 24. Настройка тегов для метрик (опционально) - 25. Включает трассировку модуля (по умолчанию `true`) - 26. Настройка атрибутов для трассировки (опционально) + 1. Порт публичного `HTTP`-сервера (по умолчанию: `8080`) + 2. Порт служебного `HTTP`-сервера (по умолчанию: `8085`) + 3. Путь для получения [метрик](metrics.md) на служебном сервере (по умолчанию: `/metrics`) + 4. Путь для получения статуса [проб готовности](probes.md) на служебном сервере (по умолчанию: `/system/readiness`) + 5. Путь для получения статуса [проб жизнеспособности](probes.md) на служебном сервере (по умолчанию: `/system/liveness`) + 6. Игнорировать ли завершающий `/` в пути: если включено, `/my/path` и `/my/path/` будут считаться одним маршрутом (по умолчанию: `false`) + 7. Количество потоков сетевого ввода-вывода (по умолчанию: количество доступных процессоров, но не меньше `2`) + 8. Количество потоков для блокирующей обработки запросов (по умолчанию: `min(max(доступные процессоры, 2) * 8, 200)`) + 9. Время ожидания обработки перед выключением сервера при [штатном завершении](container.md#component-lifecycle) (по умолчанию: `30s`) + 10. Максимальное время жизни потока обработчика запроса без работы (по умолчанию: `60s`) + 11. Максимальное время ожидания чтения данных из сокета или соединения; `0s` отключает тайм-аут (по умолчанию: `0s`) + 12. Максимальное время ожидания записи данных в сокет или соединение; `0s` отключает тайм-аут (по умолчанию: `0s`) + 13. Включать ли `TCP keep-alive` для сокета или соединения (по умолчанию: `false`) + 14. Включает виртуальные потоки для блокирующей обработки запросов вместо пула `blockingThreads`, требует `Java 21+` (по умолчанию: `false`) + 15. Максимально допустимый размер тела входящего запроса (по умолчанию: `256MiB`) + 16. Включает логирование модуля (по умолчанию: `false`) + 17. Включает логирование стека вызовов при исключении (по умолчанию: `true`) + 18. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 19. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 20. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 21. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 22. Включает метрики модуля (по умолчанию: `true`) + 23. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 24. Настройка тегов для метрик (по умолчанию: `{}`) + 25. Включает трассировку модуля (по умолчанию: `true`) + 26. Настройка атрибутов для трассировки (по умолчанию: `{}`) === ":simple-yaml: `YAML`" @@ -158,40 +194,47 @@ Undertow — это легковесный веб-сервер с открыты metrics: enabled: true #(22)! slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(23)! - telemetry: - enabled: true #(24)! - ``` - - 1. Порт публичного HTTP-сервера - 2. Порт служебного HTTP-сервера - 3. Путь для получения [метрик](metrics.md) на служебном сервере - 4. Путь для получения статуса [проб готовности](probes.md) на служебном сервере - 5. Путь для получения статуса [проб жизнеспособности](probes.md) на служебном сервере - 6. Игнорировать ли слэш в окончании пути, если включен то `/my/path` и `/my/path/` будут интерпритироваться одинакого, по умолчанию выключен - 7. Количество потоков сетевых обработчиков, по умолчанию равен кол-во ядер процессора либо минимум `2` - 8. Количество потоков обработчиков запросов, по умолчанию равен кол-во ядер процессора умноженных на 2 либо минимум `2` потока - 9. Время ожидания обработки перед выключением сервера в случае [штатного завершения](container.md#component-lifecycle) - 10. Максимальное время жизни потока обработчика запроса - 11. Максимальное время ожидания чтения данные из сокета/соединения - 12. Максимальное время ожидания записи данных в сокет/соединение - 13. Отсылать ли сообщения `keep-alive` во время жизни сокета/соединения TCP - 14. Включает поддержку виртуальных потоков для обработки запросов (вместо `blockingThreads`), требует Java 21+ - 15. Включает логгирование модуля (по умолчанию `false`) - 15. Максимально допустимый размер тела входящего запроса - 16. Включает логгирование модуля (по умолчанию `false`) - 17. Включает логгирование стэка вызовов в случае исключения - 18. Маска которая используется для скрытия указанных заголовков и параметров запроса/ответа - 19. Список параметров запроса которые следует скрывать - 20. Список заголовков запроса/ответа которые следует скрывать - 21. Использовать ли всегда шаблон пути запроса при логгировании. По умолчанию используется всегда шаблон пути, за исключением уровня логирования `TRACE` где использует полный путь. - 22. Включает метрики модуля (по умолчанию `true`) - 23. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 24. Включает трассировку модуля (по умолчанию `true`) + tags: #(24)! + key1: value1 + key2: value2 + tracing: + enabled: true #(25)! + attributes: #(26)! + key1: value1 + key2: value2 + ``` + + 1. Порт публичного `HTTP`-сервера (по умолчанию: `8080`) + 2. Порт служебного `HTTP`-сервера (по умолчанию: `8085`) + 3. Путь для получения [метрик](metrics.md) на служебном сервере (по умолчанию: `/metrics`) + 4. Путь для получения статуса [проб готовности](probes.md) на служебном сервере (по умолчанию: `/system/readiness`) + 5. Путь для получения статуса [проб жизнеспособности](probes.md) на служебном сервере (по умолчанию: `/system/liveness`) + 6. Игнорировать ли завершающий `/` в пути: если включено, `/my/path` и `/my/path/` будут считаться одним маршрутом (по умолчанию: `false`) + 7. Количество потоков сетевого ввода-вывода (по умолчанию: количество доступных процессоров, но не меньше `2`) + 8. Количество потоков для блокирующей обработки запросов (по умолчанию: `min(max(доступные процессоры, 2) * 8, 200)`) + 9. Время ожидания обработки перед выключением сервера при [штатном завершении](container.md#component-lifecycle) (по умолчанию: `30s`) + 10. Максимальное время жизни потока обработчика запроса без работы (по умолчанию: `60s`) + 11. Максимальное время ожидания чтения данных из сокета или соединения; `0s` отключает тайм-аут (по умолчанию: `0s`) + 12. Максимальное время ожидания записи данных в сокет или соединение; `0s` отключает тайм-аут (по умолчанию: `0s`) + 13. Включать ли `TCP keep-alive` для сокета или соединения (по умолчанию: `false`) + 14. Включает виртуальные потоки для блокирующей обработки запросов вместо пула `blockingThreads`, требует `Java 21+` (по умолчанию: `false`) + 15. Максимально допустимый размер тела входящего запроса (по умолчанию: `256MiB`) + 16. Включает логирование модуля (по умолчанию: `false`) + 17. Включает логирование стека вызовов при исключении (по умолчанию: `true`) + 18. Маска, которая используется для скрытия указанных заголовков и параметров запроса или ответа (по умолчанию: `***`) + 19. Список параметров запроса, которые следует скрывать (по умолчанию: `[]`) + 20. Список заголовков запроса или ответа, которые следует скрывать (по умолчанию: `[ "authorization", "cookie", "set-cookie" ]`) + 21. Использовать ли шаблон пути запроса при логировании; если не указано, шаблон используется всегда, кроме уровня `TRACE`, где используется полный путь (по умолчанию не указано, необязательно) + 22. Включает метрики модуля (по умолчанию: `true`) + 23. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 24. Настройка тегов для метрик (по умолчанию: `{}`) + 25. Включает трассировку модуля (по умолчанию: `true`) + 26. Настройка атрибутов для трассировки (по умолчанию: `{}`) Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#http-server). -Kora предоставляет тонкую настройку HTTP-сервера Undertow через два специализированных интерфейса конфигурации: `UndertowConfigurer` и `HttpHandlerConfigurer`. -Они позволяют кастомизировать поведение сервера и конвейер обработки запросов, не жертвуя интеграцией с модульной архитектурой Kora. +Kora предоставляет тонкую настройку `HTTP`-сервера `Undertow` через два специализированных интерфейса конфигурации: `UndertowConfigurer` и `HttpHandlerConfigurer`. +Они позволяют настраивать поведение сервера и конвейер обработки запросов, не жертвуя интеграцией с модульной архитектурой Kora. ## Контроллер декларативный { #somecontroller-declarative } @@ -217,7 +260,7 @@ Kora предоставляет тонкую настройку HTTP-серве 1. Указывает что класс является компонентом и его требуется зарегистрировать в контейнере приложения 2. Указывает что класс является контроллером и содержит HTTP-обработчики 3. Указывает что метод является обработчиком пути в контроллере - 4. Указывает тип HTTP метода обработчика + 4. Указывает тип `HTTP`-метода обработчика 5. Указывает путь метода обработчика === ":simple-kotlin: `Kotlin`" @@ -239,18 +282,81 @@ Kora предоставляет тонкую настройку HTTP-серве 1. Указывает что класс является компонентом и его требуется зарегистрировать в контейнере приложения 2. Указывает что класс является контроллером и содержит HTTP-обработчики 3. Указывает что метод является обработчиком пути в контроллере - 4. Указывает тип HTTP метода обработчика + 4. Указывает тип `HTTP`-метода обработчика 5. Указывает путь метода обработчика ### Запрос { #request } -Секция описывает преобразования HTTP-запроса у контроллера. -Предлагается использовать специальные аннотации для указания параметров запроса. +Раздел описывает преобразование `HTTP`-запроса в аргументы метода контроллера. +Для частей запроса используются специальные аннотации, а тело запроса передается аргументом без такой аннотации. + +#### Преобразование параметров из строки { #string-parameter-reader } + +Значения из пути, параметров запроса, заголовков и `cookie` приходят как строки. +Для преобразования строки в нужный тип Kora использует `StringParameterReader`: + +```java +public interface StringParameterReader { + T read(String string); +} +``` + +`StringParameterReader` ищется как компонент графа по точному типу параметра. Если параметр объявлен как `List` или `Set`, +преобразователь применяется к каждому значению отдельно. + +Из коробки поддерживаются `String`, `Boolean`, `Integer`, `Long`, `Float`, `Double`, `UUID`, `BigInteger`, `BigDecimal`, +`Duration`, `LocalDate`, `LocalTime`, `LocalDateTime`, `OffsetTime`, `OffsetDateTime`, `ZonedDateTime` и `enum`. +Для `enum` по умолчанию используется имя значения через `Enum.name()`. Если значение невозможно преобразовать, запрос завершается +ответом `400` через `HttpServerResponseException`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserId(long value) {} + + @Module + public interface UserIdModule { + + default StringParameterReader userIdStringParameterReader() { + return StringParameterReader.of( + value -> new UserId(Long.parseLong(value)), + value -> "Invalid user id: " + value + ); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserId(val value: Long) + + @Module + interface UserIdModule { + + fun userIdStringParameterReader(): StringParameterReader { + return StringParameterReader.of( + { value -> UserId(value.toLong()) }, + { value -> "Invalid user id: $value" } + ) + } + } + ``` + +После регистрации преобразователя пользовательский тип можно использовать в параметрах контроллера: + +```java +@HttpRoute(method = HttpMethod.GET, path = "/users/{id}") +public User get(@Path("id") UserId id) { + return userService.get(id); +} +``` #### Параметр пути { #path-parameter } `@Path` — обозначает значение части пути запроса, сам параметр указывается в `{кавычках}` в пути и имя параметра указывается в `value` либо по умолчанию равно имени аргумента метода. +Значение преобразуется через `StringParameterReader`, поэтому можно использовать как встроенные типы, так и пользовательские. ===! ":fontawesome-brands-java: `Java`" @@ -285,6 +391,8 @@ Kora предоставляет тонкую настройку HTTP-серве #### Параметр запроса { #query-parameter } `@Query` — значение параметра запроса, имя параметра указывается в `value` либо по умолчанию равно имени аргумента метода. +Поддерживаются одиночные значения, `List` и `Set`. Для `List` сохраняются все значения параметра, +для `Set` повторяющиеся значения удаляются с сохранением порядка первого появления. ===! ":fontawesome-brands-java: `Java`" @@ -321,6 +429,8 @@ Kora предоставляет тонкую настройку HTTP-серве #### Заголовок запроса { #request-header } `@Header` — значение [заголовка запроса](https://developer.mozilla.org/ru/docs/Web/HTTP/Headers), имя параметра указывается в `value` либо по умолчанию равно имени аргумента метода. +Поддерживаются одиночные значения, `List` и `Set`. +Для `List` и `Set` используются все значения заголовка. ===! ":fontawesome-brands-java: `Java`" @@ -356,13 +466,13 @@ Kora предоставляет тонкую настройку HTTP-серве #### Тело запроса { #request-body } -Для указания тела запроса требуется использовать аргумент метода без специальных аннотации, -по умолчанию поддерживаются такие типы как `byte[]`, `ByteBuffer`, `String`. +Для указания тела запроса требуется использовать аргумент метода без специальных аннотаций. +По умолчанию поддерживаются `byte[]`, `ByteBuffer`, `String`, `FormUrlEncoded`, `FormMultipart` и пользовательские типы через `HttpServerRequestMapper`. -##### Json { #json } +##### JSON { #json } -Для указания, что тело является Json и ему требуется автоматически создать такой читатель и внедрить его, -требуется использовать аннотацию `@Json`: +Для указания, что тело является `JSON` и для него требуется автоматически создать и внедрить `JsonReader`, +используется аннотация `@Json`: ===! ":fontawesome-brands-java: `Java`" @@ -380,7 +490,7 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` - 1. Указывает что тело должно быть записано как Json + 1. Указывает что тело должно быть прочитано как `JSON` === ":simple-kotlin: `Kotlin`" @@ -398,9 +508,9 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` - 1. Указывает что тело должно быть записано как Json + 1. Указывает что тело должно быть прочитано как `JSON` -Требуется подключить модуль [Json](json.md). +Требуется подключить модуль [JSON](json.md). ##### Текстовая форма { #form-urlencoded } @@ -469,6 +579,7 @@ Kora предоставляет тонкую настройку HTTP-серве #### Куки { #cookie } `@Cookie` — значение [Cookie](https://developer.mozilla.org/ru/docs/Glossary/Cookie), имя параметра указывается в `value` либо по умолчанию равно имени аргумента метода. +Можно получить значение как `String`, как тип `Cookie` с именем, значением и атрибутами, либо как другой тип через `StringParameterReader`. ===! ":fontawesome-brands-java: `Java`" @@ -500,9 +611,10 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` -#### Самописный параметр { #custom-parameter } +#### Пользовательский параметр { #custom-parameter } -В случае если требуется обрабатывать запрос отличным способом, то можно использовать специальный интерфейс `HttpServerRequestMapper`: +Если требуется собрать аргумент метода из запроса вручную, можно использовать специальный интерфейс `HttpServerRequestMapper`. +Такой подход удобен для пользовательского контекста, авторизации, сложной проверки заголовков или нескольких частей запроса сразу: ===! ":fontawesome-brands-java: `Java`" @@ -558,18 +670,20 @@ Kora предоставляет тонкую настройку HTTP-серве ===! ":fontawesome-brands-java: `Java`" - По умолчанию все аргументы объявленные в методе являются **обязательными** (*NotNull*). + По умолчанию все аргументы, объявленные в методе, являются **обязательными**. + Если обязательное значение отсутствует в запросе, Kora вернет ответ `400`. === ":simple-kotlin: `Kotlin`" - По умолчанию все аргументы объявленные в методе которые не используют [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис считаются **обязательными** (*NotNull*). + По умолчанию все аргументы метода, которые не используют синтаксис [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html), + считаются **обязательными**. Если обязательное значение отсутствует в запросе, Kora вернет ответ `400`. #### Необязательные параметры { #optional-parameters } ===! ":fontawesome-brands-java: `Java`" - В случае если аргумент метода является необязательным, то есть может отсутствовать то, - можно использовать аннотацию `@Nullable`: + Если аргумент метода является необязательным, то есть может отсутствовать в запросе, + можно использовать аннотацию `@Nullable` или `Optional` для одиночных значений: ```java @Component @@ -583,11 +697,11 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдет любая аннотация `@Nullable`, например `javax.annotation.Nullable`, `jakarta.annotation.Nullable` или `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - Предполагается использовать [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис и помечать такой параметр как Nullable: + Предполагается использовать синтаксис [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) и помечать такой параметр как необязательный: ```kotlin @Component @@ -603,9 +717,20 @@ Kora предоставляет тонкую настройку HTTP-серве ### Ответ { #response } -По умолчанию можно использовать стандартные типы возвращаемых значений, -такие как `byte[]`, `ByteBuffer`, `String` которые будут обработаны со статус кодом `200` и соответствующим заголовком типа ответа -либо `HttpServerResponse` где надо будет самостоятельно заполнить всю информацию об HTTP ответе. +По умолчанию можно использовать стандартные типы возвращаемых значений: `byte[]`, `ByteBuffer`, `String`. +Они будут обработаны со статусом `200` и соответствующим заголовком типа ответа. + +Если нужно вручную указать статус, заголовки или тело, метод может вернуть `HttpServerResponse`. +Основной контракт `HttpServerResponse` состоит из кода ответа, заголовков и необязательного тела: + +```java +public interface HttpServerResponse { + int code(); + MutableHttpHeaders headers(); + @Nullable + HttpBodyOutput body(); +} +``` ===! ":fontawesome-brands-java: `Java`" @@ -619,13 +744,13 @@ Kora предоставляет тонкую настройку HTTP-серве return HttpServerResponse.of( 200, //(1)! HttpHeaders.of("headerName", "headerValue"), //(2)! - HttpBody.plaintext(body) //(3)! + HttpBody.plaintext("Hello World") //(3)! ); } } ``` - 1. HTTP статус код ответа + 1. Код состояния `HTTP`-ответа 2. Заголовки ответа 3. Тело ответа @@ -641,19 +766,20 @@ Kora предоставляет тонкую настройку HTTP-серве return HttpServerResponse.of( 200, //(1)! HttpHeaders.of("headerName", "headerValue"), //(2)! - HttpBody.plaintext(body) //(3)! + HttpBody.plaintext("Hello World") //(3)! ) } } ``` - 1. HTTP статус код ответа + 1. Код состояния `HTTP`-ответа 2. Заголовки ответа 3. Тело ответа -#### Json { #json-2 } +#### JSON { #json-2 } -В случае если предполагается отвечать в формате Json, то требуется использовать аннотацию `@Json` над методом: +Если предполагается отвечать в формате `JSON`, требуется использовать аннотацию `@Json` над методом. +Для типа ответа Kora найдет или создаст `JsonWriter`: ===! ":fontawesome-brands-java: `Java`" @@ -672,7 +798,7 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` - 1. Указывает что ответ должен быть в формате Json + 1. Указывает что ответ должен быть в формате `JSON` === ":simple-kotlin: `Kotlin`" @@ -691,16 +817,16 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` - 1. Указывает что ответ должен быть в формате Json + 1. Указывает что ответ должен быть в формате `JSON` -Требуется подключить модуль [Json](json.md). +Требуется подключить модуль [JSON](json.md). #### Сущность ответа { #response-entity } -Если предполагается читать тело и получить также заголовки и статус код ответа, -то предполагается использовать `HttpResponseEntity`, это обертка над телом ответа. +Если требуется вернуть тело, заголовки и код состояния ответа вместе, +используется `HttpResponseEntity` — обертка над телом ответа. -Ниже показан пример аналогичный примеру Json вместе с оберткой `HttpResponseEntity`: +Ниже показан пример, аналогичный примеру `JSON`, вместе с оберткой `HttpResponseEntity`: ===! ":fontawesome-brands-java: `Java`" @@ -738,7 +864,11 @@ Kora предоставляет тонкую настройку HTTP-серве #### Ответ исключение { #respond-exception } -Если требуется отвечать ошибкой, то можно использовать `HttpServerResponseException` для того чтобы бросать исключение. +Если требуется прервать обработку и сразу вернуть ошибку, можно бросить `HttpServerResponseException`. +Это одновременно исключение и `HttpServerResponse`, поэтому его можно выбросить из контроллера, сервиса или преобразователя параметра. + +Фабричные методы `HttpServerResponseException.of(...)` позволяют указать код состояния, текст ответа, причину и заголовки. +Тело ответа будет записано как `text/plain; charset=utf-8`. ===! ":fontawesome-brands-java: `Java`" @@ -774,9 +904,10 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` -#### Самописное { #custom-response } +#### Пользовательский ответ { #custom-response } -В случае если требуется чтение ответа отличным способом, то можно использовать специальный интерфейс `HttpServerResponseMapper`: +Если требуется сформировать ответ нестандартным способом, можно использовать специальный интерфейс `HttpServerResponseMapper`. +Он получает `Context`, исходный `HttpServerRequest` и результат метода контроллера, а возвращает готовый `HttpServerResponse`: ===! ":fontawesome-brands-java: `Java`" @@ -847,13 +978,29 @@ Kora предоставляет тонкую настройку HTTP-серве ## Перехватчики { #interceptors } -Можно создавать перехватчики для изменения поведения либо создания дополнительного поведения используя класс `HttpServerInterceptor`. +Можно создавать перехватчики для изменения поведения или добавления общей логики вокруг обработки запроса. +Для этого используется интерфейс `HttpServerInterceptor`: + +```java +public interface HttpServerInterceptor { + CompletionStage intercept(Context context, HttpServerRequest request, InterceptChain chain) throws Exception; + + interface InterceptChain { + CompletionStage process(Context ctx, HttpServerRequest request) throws Exception; + } +} +``` + +Перехватчик получает текущий `Context`, `HttpServerRequest` и цепочку дальнейшей обработки. +Чтобы передать запрос дальше, нужно вызвать `chain.process(context, request)`. Если перехватчик возвращает ответ сам, +обработчик контроллера дальше не вызывается. Перехватчики можно использовать на: - Конкретных методах контроллера - Контроллере целиком -- Всех контроллерах сразу (требуется использовать `@Tag(HttpServerModule.class)` над классом перехватчиком) (такой перехватчик может быть лишь один) +- Всех контроллерах сразу: для этого компонент перехватчика должен быть зарегистрирован с тегом `@Tag(HttpServerModule.class)`; + таких глобальных перехватчиков может быть несколько ===! ":fontawesome-brands-java: `Java`" @@ -908,8 +1055,8 @@ Kora предоставляет тонкую настройку HTTP-серве ### Обработка ошибок { #error-handling } -Обработка ошибок на уровне всех HTTP ответов может быть реализована также посредствам перехватчика, -ниже представлен простой пример такого перехватчика. +Обработка ошибок на уровне всех `HTTP`-ответов может быть реализована через перехватчик. +Ниже представлен простой пример такого перехватчика. ===! ":fontawesome-brands-java: `Java`" @@ -999,7 +1146,7 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` - 1. Указывает тип HTTP метода обработчика + 1. Указывает тип `HTTP`-метода обработчика 2. Указывает путь метода обработчика === ":simple-kotlin: `Kotlin`" @@ -1023,5 +1170,486 @@ Kora предоставляет тонкую настройку HTTP-серве } ``` - 1. Указывает тип HTTP метода обработчика + 1. Указывает тип `HTTP`-метода обработчика 2. Указывает путь метода обработчика + +## Авторизация { #authorization } + +Kora предоставляет механизм извлечения контекста авторизации из HTTP запроса через интерфейс `HttpServerPrincipalExtractor`. +Этот интерфейс позволяет реализовать любую схему аутентификации: [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/). + +### Принцип работы { #how-it-works } + +`HttpServerPrincipalExtractor` извлекает токен из запроса (обычно из заголовка `Authorization`) и возвращает объект `Principal`. +Полученный `Principal` сохраняется в `Context` запроса и может быть получен в любом месте обработки запроса через `Principal.current()`. + +```java +public interface HttpServerPrincipalExtractor { + CompletionStage extract(HttpServerRequest request, @Nullable String value); +} +``` + +Где: + +- `request` — текущий HTTP запрос, из которого можно извлечь дополнительные данные (заголовки, параметры) +- `value` — значение токена, извлеченное из заголовка `Authorization` (или другого источника) +- `T extends Principal` — тип контекста авторизации, который будет сохранен в `Context` + +### Базовый пример { #basic-example } + +Простой пример извлечения API ключа из заголовка `Authorization`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface AuthModule { + + @ConfigSource("auth.apiKey") + interface ApiKeyAuthConfig { + String value(); + } + + default HttpServerPrincipalExtractor apiKeyExtractor(ApiKeyAuthConfig config) { + return (request, value) -> { + if (value == null || !config.value().equals(value)) { + return CompletableFuture.failedFuture( + new IllegalAccessException("Invalid API key") + ); + } + return CompletableFuture.completedFuture( + new Principal() { + @Override + public String name() { + return "api-client"; + } + } + ); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface AuthModule { + + @ConfigSource("auth.apiKey") + interface ApiKeyAuthConfig { + fun value(): String + } + + fun apiKeyExtractor(config: ApiKeyAuthConfig): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || config.value() != value) { + return@HttpServerPrincipalExtractor CompletableFuture.failedFuture( + IllegalAccessException("Invalid API key") + ) + } + CompletableFuture.completedFuture( + object : Principal { + override fun name() = "api-client" + } + ) + } + } + } + ``` + +### Кастомный Principal { #custom-principal } + +Для передачи дополнительной информации об авторизации (userId, роли, scope) создайте собственную реализацию `Principal`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserContext(String userId, List roles) implements Principal {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserContext(val userId: String, val roles: List) : Principal + ``` + +Если требуется работа с scope (областями видимости), используйте интерфейс `PrincipalWithScopes`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record ScopedUser(String userId, Collection scopes) implements PrincipalWithScopes {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class ScopedUser(val userId: String, val scopes: Collection) : PrincipalWithScopes + ``` + +### Bearer токен { #bearer } + +Пример извлечения Bearer токена с кастомной реализацией `Principal`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface BearerAuthModule { + + default HttpServerPrincipalExtractor bearerExtractor(TokenValidator validator) { + return (request, value) -> { + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + new IllegalAccessException("No Bearer token") + ); + } + + String token = value.substring(7); + return validator.validate(token) + .thenApply(userData -> new UserContext(userData.userId(), userData.roles())); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface BearerAuthModule { + + fun bearerExtractor(validator: TokenValidator): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + IllegalAccessException("No Bearer token") + ) + } + + val token = value.substring(7) + validator.validate(token) + .thenApply { userData -> + UserContext(userData.userId, userData.roles) + } + } + } + } + ``` + +### Получение Principal в контроллере { #getting-principal } + +Получить текущий контекст авторизации можно в любом месте обработки запроса: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController + public class SecureController { + + @HttpRoute(method = HttpMethod.GET, path = "/secure") + public String getSecureData() { + Principal principal = Principal.current(); + if (principal instanceof UserContext user) { + return "Hello, user: " + user.userId(); + } + throw new SecurityException("Not authenticated"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController + class SecureController { + + @HttpRoute(method = HttpMethod.GET, path = "/secure") + fun getSecureData(): String { + val principal = Principal.current() + return if (principal is UserContext) { + "Hello, user: ${principal.userId}" + } else { + throw SecurityException("Not authenticated") + } + } + } + ``` + +### OAuth2 { #oauth2 } + +Для OAuth2 авторизации создайте `HttpServerPrincipalExtractor`, который проверяет токен через OAuth2 provider: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface OAuth2Module { + + default HttpServerPrincipalExtractor oauth2Extractor(OAuth2Client oauth2Client) { + return (request, value) -> { + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + new IllegalAccessException("No OAuth2 token") + ); + } + + String token = value.substring(7); + return oauth2Client.introspect(token) + .thenApply(introspection -> + new ScopedUser( + introspection.subject(), + introspection.scopes() + ) + ); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface OAuth2Module { + + fun oauth2Extractor(oauth2Client: OAuth2Client): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || !value.startsWith("Bearer ")) { + return CompletableFuture.failedFuture( + IllegalAccessException("No OAuth2 token") + ) + } + + val token = value.substring(7) + oauth2Client.introspect(token) + .thenApply { introspection -> + ScopedUser(introspection.subject, introspection.scopes) + } + } + } + } + ``` + +### Проверка scope в перехватчике { #scope-check } + +Для проверки scope можно создать перехватчик, который проверяет `PrincipalWithScopes`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ScopeCheckingInterceptor implements HttpServerInterceptor { + + private final String requiredScope; + + public ScopeCheckingInterceptor(@ConfigSource("auth.requiredScope") String requiredScope) { + this.requiredScope = requiredScope; + } + + @Override + public CompletionStage intercept(Context context, + HttpServerRequest request, + InterceptChain chain) { + Principal principal = Principal.current(context); + if (principal instanceof PrincipalWithScopes scoped) { + if (!scoped.scopes().contains(requiredScope)) { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "Insufficient scope") + ); + } + } else { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "No scopes available") + ); + } + + return chain.process(context, request); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ScopeCheckingInterceptor( + @ConfigSource("auth.requiredScope") private val requiredScope: String + ) : HttpServerInterceptor { + + override fun intercept( + context: Context, + request: HttpServerRequest, + chain: HttpServerInterceptor.InterceptChain + ): CompletionStage { + val principal = Principal.current(context) + if (principal is PrincipalWithScopes) { + if (!principal.scopes.contains(requiredScope)) { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "Insufficient scope") + ) + } + } else { + return CompletableFuture.failedFuture( + HttpServerResponseException.of(403, "No scopes available") + ) + } + + return chain.process(context, request) + } + } + ``` + +### OpenAPI интеграция { #openapi } + +При использовании Kora OpenAPI Generator авторизация настраивается автоматически на основе спецификации OpenAPI. +Генератор создает: + +1. Интерфейс `ApiSecurity` с классами-маркерами для каждого типа авторизации +2. `HttpServerInterceptor` для каждого security scheme +3. Требует предоставить `HttpServerPrincipalExtractor` с соответствующим `@Tag` + +Пример из [kora-examples](https://github.com/kora-projects/kora-examples): + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends + HoconConfigModule, + UndertowHttpServerModule, + JsonModule { + + @Tag(ApiSecurity.ApiKeyAuth.class) + default HttpServerPrincipalExtractor apiKeyExtractor(DataApiAuthConfig config) { + return (request, value) -> { + if (value == null || !config.value().equals(value)) { + throw new SecurityException("Invalid API key"); + } + return CompletableFuture.completedFuture( + new DataApiPrincipal("data-api-client") + ); + }; + } + } + ``` + + где `DataApiPrincipal`: + + ```java + public record DataApiPrincipal(String name) implements Principal {} + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : + HoconConfigModule, + UndertowHttpServerModule, + JsonModule { + + @Tag(ApiSecurity.ApiKeyAuth::class) + fun apiKeyExtractor(config: DataApiAuthConfig): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { request, value -> + if (value == null || config.value() != value) { + throw SecurityException("Invalid API key") + } + CompletableFuture.completedFuture( + DataApiPrincipal("data-api-client") + ) + } + } + } + ``` + + где `DataApiPrincipal`: + + ```kotlin + data class DataApiPrincipal(val name: String) : Principal + ``` + +Конфигурация: + +```hocon +auth.apiKey { + value = "secret-api-key-123" +} +``` + +### Обработка ошибок { #error-handling } + +Если `HttpServerPrincipalExtractor` выбрасывает исключение или возвращает `null`, запрос отклоняется с кодом `403 Forbidden`. +Для кастомной обработки ошибок авторизации используйте перехватчик: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(HttpServerModule.class) + @Component + public final class AuthErrorInterceptor implements HttpServerInterceptor { + + @Override + public CompletionStage intercept(Context context, + HttpServerRequest request, + InterceptChain chain) { + return chain.process(context, request).exceptionally(e -> { + if (e instanceof CompletionException) { + e = e.getCause(); + } + if (e instanceof IllegalAccessException) { + return HttpServerResponse.of(401, HttpBody.plaintext("Unauthorized: " + e.getMessage())); + } + if (e instanceof SecurityException) { + return HttpServerResponse.of(403, HttpBody.plaintext("Forbidden: " + e.getMessage())); + } + throw new CompletionException(e); + }); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(HttpServerModule::class) + @Component + class AuthErrorInterceptor : HttpServerInterceptor { + + override fun intercept( + context: Context, + request: HttpServerRequest, + chain: HttpServerInterceptor.InterceptChain + ): CompletionStage { + return chain.process(context, request).exceptionally { e -> + val error = if (e is CompletionException) e.cause!! else e + when (error) { + is IllegalAccessException -> + HttpServerResponse.of(401, HttpBody.plaintext("Unauthorized: ${error.message}")) + is SecurityException -> + HttpServerResponse.of(403, HttpBody.plaintext("Forbidden: ${error.message}")) + else -> throw CompletionException(error) + } + } + } + } + ``` + +## Телеметрия { #telemetry } + +HTTP Server использует контракт телеметрии для логирования, метрик и трассировки запросов. +Конфигурация телеметрии (секция `telemetry { logging / metrics / tracing }`) описана в разделе [Конфигурация](#configuration). +Точки расширения находятся в `ru.tinkoff.kora.http.server.common.telemetry`. + +Для каждого HTTP-запроса создаётся `HttpServerTelemetry.HttpServerTelemetryContext`, который закрывается по завершении обработки запроса. +Запрос описывается через параметры обработчика телеметрии, включая метод, путь, статус ответа и длительность. + +Фабрика по умолчанию `DefaultHttpServerTelemetryFactory` объединяет три фабрики: +- `HttpServerLoggerFactory` строит `HttpServerLogger` для логирования начала/конца обработки запроса; +- `HttpServerMetricsFactory` строит `HttpServerMetrics` для записи метрик запросов; +- `HttpServerTracerFactory` строит `HttpServerTracer` для распределённой трассировки. + +Метрики и трассировка описаны в разделе [Справочник метрик](metrics.md#http-server). diff --git a/mkdocs/docs/ru/documentation/json.md b/mkdocs/docs/ru/documentation/json.md index b93cd92..f13bd8a 100644 --- a/mkdocs/docs/ru/documentation/json.md +++ b/mkdocs/docs/ru/documentation/json.md @@ -1,13 +1,16 @@ --- -description: "Explains Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration. Use when working with @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonIgnore, JsonNullable, JacksonModule." +description: "Explains Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration. Use when working with @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonSkip, JsonNullable, JacksonModule." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration; key triggers include @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonIgnore, JsonNullable, JacksonModule." + use_when: "Use this file for Kora docs or implementation questions about Kora JSON reader and writer generation, field requirements, naming, ignores, serialization levels, JsonNullable, sealed types, and Jackson integration; key triggers include @Json, @JsonReader, @JsonWriter, @JsonInclude, @JsonField, @JsonSkip, JsonNullable, JacksonModule." --- -Модуль Json позволяет создавать производительные и без использования рефлексии -читатели и писатели для классов приложения посредствам разметки классов аннотациями. +Модуль `JSON` создает эффективные реализации `JsonReader` и `JsonWriter` для классов приложения во время компиляции и без использования `Reflection` во время выполнения. +Генерация управляется аннотациями `@Json`, `@JsonReader`, `@JsonWriter` и связанными аннотациями уровня поля. -Если нужен пошаговый разбор перед справочным описанием, смотрите [JSON](../guides/json.md). +`JsonModule` также предоставляет готовые преобразователи для `HTTP`-клиента, `HTTP`-сервера, строковых параметров и `Kafka`. +Это позволяет использовать один и тот же сгенерированный `JsonReader` или `JsonWriter` в разных модулях Kora. + +Пошаговый разбор перед справочным описанием смотрите в разделе [JSON](../guides/json.md). ## Подключение { #dependency } @@ -39,7 +42,8 @@ agent: ## Запись { #writer } -Можно воспользоваться `@JsonWriter` для создания только писателя: +Используйте `@JsonWriter`, чтобы создать только `JsonWriter`. +Этот вариант полезен, когда тип нужно только записывать в `JSON`: ===! ":fontawesome-brands-java: `Java`" @@ -57,7 +61,8 @@ agent: ## Чтение { #reader } -Можно воспользоваться `@JsonReader` для создания только читателя: +Используйте `@JsonReader`, чтобы создать только `JsonReader`. +Этот вариант полезен, когда тип нужно только читать из `JSON`: ===! ":fontawesome-brands-java: `Java`" @@ -73,10 +78,10 @@ agent: data class Dto(val field1: String, val field2: Int) ``` -## Чтение & Запись { #reader-and-writer } +## Чтение и запись { #reader-and-writer } -Можно воспользоваться `@Json` для создания сразу читателя и писателя. -В большинстве случаев предпочтительнее использовать именно аннотацию `@Json`: +Используйте `@Json`, чтобы создать одновременно `JsonReader` и `JsonWriter`. +В большинстве случаев `@Json` — предпочтительная аннотация: ===! ":fontawesome-brands-java: `Java`" @@ -92,11 +97,71 @@ agent: data class Dto(val field1: String, val field2: Int) ``` +## Интерфейсы чтения и записи { #reader-writer-interfaces } + +`JsonReader` и `JsonWriter` — это обычные компоненты графа приложения. +После генерации или ручной регистрации их можно внедрять по сигнатуре, как любую другую зависимость. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MyService { + + private final JsonReader reader; + private final JsonWriter writer; + + public MyService(JsonReader reader, JsonWriter writer) { + this.reader = reader; + this.writer = writer; + } + + public Dto read(String json) throws IOException { + return this.reader.read(json); + } + + public byte[] write(Dto dto) throws IOException { + return this.writer.toByteArray(dto); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyService( + private val reader: JsonReader, + private val writer: JsonWriter + ) { + + fun read(json: String): Dto? { + return reader.read(json) + } + + fun write(dto: Dto): ByteArray { + return writer.toByteArray(dto) + } + } + ``` + +`JsonReader` читает значение из `JsonParser`, `byte[]`, `String` или `InputStream`. +Методы `readUnchecked(...)` делают то же самое, но преобразуют `IOException` в `UncheckedIOException`. + +`JsonWriter` записывает значение через `JsonGenerator` и также может вернуть `byte[]`, строку или форматированную строку через `toByteArray(...)`, `toString(...)` и `toPrettyString(...)`. +Методы `toByteArrayUnchecked(...)`, `toStringUnchecked(...)` и `toPrettyStringUnchecked(...)` преобразуют `IOException` в `UncheckedIOException`. + +Особенности поведения во время выполнения при прямом вызове кодеков: + +- `read(...)` возвращает `null`, когда парсер находится на токене `JSON` `null`, поэтому документ верхнего уровня `null` десериализуется в `null`. +- Некорректный `JSON` или неожиданный токен приводит к `JsonParseException` из `Jackson`, который является подтипом `IOException`. +- Варианты `readUnchecked(...)` и `to...Unchecked(...)` пробрасывают любой `IOException` (включая `JsonParseException`), обернутый в `UncheckedIOException`. + ## Обязательные поля { #required-fields } ===! ":fontawesome-brands-java: `Java`" - По умолчанию все поля объявленные в объекте считаются **обязательными** (*NotNull*). + По умолчанию все поля, объявленные в объекте, считаются **обязательными** (`NotNull`). ```java @Json @@ -105,19 +170,18 @@ agent: === ":simple-kotlin: `Kotlin`" - По умолчанию все поля объявленные в объекте которые не используют [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис считаются **обязательными** (*NotNull*). + По умолчанию все поля, объявленные в объекте без синтаксиса [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html), считаются **обязательными** (`NotNull`). ```kotlin @Json data class Dto(val field1: String, val field2: Int) ``` -## Необязательное поля { #optional-fields } +## Необязательные поля { #optional-fields } ===! ":fontawesome-brands-java: `Java`" - В случае если поле в Json является необязательным, то есть может отсутствовать то, - можно использовать аннотацию `@Nullable` для соответствия поля в Json и DTO: + Если поле `JSON` необязательное и может отсутствовать, используйте аннотацию `@Nullable`: ```java @Json @@ -125,11 +189,11 @@ agent: int field2) { } ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдет любая аннотация `@Nullable`, например `javax.annotation.Nullable`, `jakarta.annotation.Nullable` или `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - Предполагается использовать [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис и помечать такой параметр как Nullable: + Для `Kotlin` используйте синтаксис [Kotlin Nullability](https://kotlinlang.org/docs/null-safety.html) и пометьте параметр как `nullable`: ```kotlin @Json @@ -139,10 +203,10 @@ agent: ) ``` -## Именование поля { #field-naming } +## Именование полей { #field-naming } -В случае если поле в Json называется иначе от того что требуется использовать в классе, -можно использовать аннотацию `@JsonField` для соответствия поля в Json и DTO. +Если поле в `JSON` имеет имя, отличное от имени поля в классе, используйте `@JsonField`. +Она задает имя ключа в `JSON`, а также позволяет указать отдельные реализации `JsonReader` и `JsonWriter` для поля. ===! ":fontawesome-brands-java: `Java`" @@ -162,10 +226,36 @@ agent: ) ``` +Если для поля нужны отдельные преобразователи, укажите их в `reader` и `writer`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record Dto(@JsonField(value = "created_at", + reader = InstantJsonReader.class, + writer = InstantJsonWriter.class) + Instant createdAt) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class Dto( + @field:JsonField( + value = "created_at", + reader = InstantJsonReader::class, + writer = InstantJsonWriter::class + ) + val createdAt: Instant + ) + ``` + ## Игнорирование поля { #field-ignore } -В случае если поле в DTO не хочется читать/писать, -можно использовать аннотацию `@JsonSkip` и проигнорировать такое поле. +Если поле в `DTO` не нужно читать или записывать, используйте `@JsonSkip`. +Такое поле игнорируется при чтении и записи `JSON`. ===! ":fontawesome-brands-java: `Java`" @@ -185,29 +275,29 @@ agent: ) ``` -## Уровни записи { #serialization-levels } +## Уровни сериализации { #serialization-levels } -Поведение по умолчанию не подразумевает запись полей с `null` значениями. (1) +По умолчанию поля со значением `null` не записываются. (1) { .annotate } -1. `IncludeType.NON_NULL` - включать поле в запись если не `null` +1. `IncludeType.NON_NULL` — записывать поле только в том случае, если значение не `null`. -В случае если хочется изменить поведение записи в этих моментах то предлагается использовать аннотацию `@JsonInclude`. -Аннотацию можно использовать не только над полем, но также над классом и тогда правило будет действовать на все поля сразу. +Чтобы изменить это поведение, используйте `@JsonInclude`. +Аннотацию можно разместить не только на поле, но и на классе; в этом случае правило применяется сразу ко всем полям. -Доступны различные варианты использования: +Доступные варианты: -- `IncludeType.ALWAYS` - включать поле в запись всегда -- `IncludeType.NON_NULL` - включать поле в запись если не `null` -- `IncludeType.NON_EMPTY` - включать поле в запись если это не `null` и не пустая коллекция +- `IncludeType.ALWAYS` — всегда записывать поле. +- `IncludeType.NON_NULL` — записывать поле, если значение не `null`. +- `IncludeType.NON_EMPTY` — записывать поле, если значение не `null` и не является пустой коллекцией или ассоциативным массивом. -Пример использования аннотации: +Пример: ===! ":fontawesome-brands-java: `Java`" ```java @Json - @JsonInclude(IncludeType.NOT_NULL) + @JsonInclude(IncludeType.NON_NULL) public record Dto(@JsonInclude(IncludeType.ALWAYS) @Nullable String field1, int field2) { } ``` @@ -222,10 +312,10 @@ agent: ) ``` -## Указание конструктора { #serialization-constructor } +## Конструктор сериализации { #serialization-constructor } -В случае если хочется использовать определенный конструктор для сериализации, -то это можно сделать с указанием над конструктором аннотации `@JsonReader` либо аннотации которая имеет меньший приоритет `@Json`: +Если для чтения `JSON` должен использоваться конкретный конструктор, пометьте его аннотацией `@JsonReader`. +Можно также использовать `@Json`, но `@JsonReader` имеет более высокий приоритет: ===! ":fontawesome-brands-java: `Java`" @@ -251,10 +341,58 @@ agent: } ``` -## JsonNullable обертка { #jsonnullable-wrapper } +`JsonReader` и `JsonWriter` могут быть сгенерированы для классов, `record`, `enum` и `sealed`-типов. +Для чтения класса должен быть один публичный конструктор либо конструктор, явно помеченный `@JsonReader` или `@Json`. + +### Java Bean и обычные классы { #java-bean } + +`@Json`, `@JsonReader` и `@JsonWriter` не ограничиваются `record` и `data class`. +Обычный класс тоже подходит: для чтения требуется единственный публичный конструктор (или конструктор, помеченный `@JsonReader`/`@Json`), а для записи используются методы доступа к полям. +`@JsonField` можно разместить на приватных полях, чтобы переименовать ключ в `JSON`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @JsonWriter + public class DtoJavaBean { + + @JsonField("string_field") + private String field1; + @JsonField("int_field") + private int field2; + + public DtoJavaBean(String field1, int field2) { + this.field1 = field1; + this.field2 = field2; + } + + public String getField1() { return field1; } + + public int getField2() { return field2; } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @JsonWriter + class DtoJavaBean( + @field:JsonField("string_field") val field1: String, + @field:JsonField("int_field") val field2: Int + ) + ``` + +## Обертка JsonNullable { #jsonnullable-wrapper } + +Если при чтении `JSON` необходимо отличать отсутствующее поле от поля со значением `null`, используйте `JsonNullable`. +Основные состояния и фабричные методы: -В случае если во время десериализации, хочется отличать отсутствующее поле от указанного `null` значения, -предполагается использовать специальный тип `JsonNullable`, который позволяет отражать все состояния поля после десериализации. +- `JsonNullable.undefined()` — поле отсутствует в `JSON`. +- `JsonNullable.nullValue()` — поле присутствует и содержит `null`. +- `JsonNullable.of(value)` — поле присутствует и содержит значение. +- `JsonNullable.ofNullable(value)` — создает `nullValue()`, если значение равно `null`, иначе `of(value)`. + +При записи `JSON` `undefined()` пропускается, `nullValue()` записывается как `null`, а `of(value)` записывает само значение. ===! ":fontawesome-brands-java: `Java`" @@ -270,15 +408,82 @@ agent: data class Dto(val field1: String, val field2: JsonNullable) ``` -## Изолированные классы и интерфейсы { #sealed-classes-and-interfaces } +### @Nullable против JsonNullable { #nullable-vs-jsonnullable } + +Обычное [необязательное поле](#optional-fields) (`@Nullable` в `Java` или nullable-тип в `Kotlin`) сводит два разных входных значения `JSON` к одному и тому же результату: и **отсутствующее** поле, и поле с явным значением `null` читаются как `null`. +`JsonNullable` разделяет эти случаи, что и делает его правильным типом для тел `HTTP`-запросов `PATCH`, где клиент отправляет только те поля, которые действительно хочет изменить. + +Три возможных результата чтения для поля `JsonNullable`: + +| Входной `JSON` | Результат чтения | `isDefined()` | `isNull()` | `value()` | +|-------------------------|---------------------------|---------------|------------|---------------| +| `{}` (поле отсутствует) | `JsonNullable.undefined()`| `false` | `false` | выбрасывает | +| `{"field": null}` | `JsonNullable.nullValue()`| `true` | `true` | `null` | +| `{"field": value}` | `JsonNullable.of(value)` | `true` | `false` | `value` | -В случае если требуется писать различные Json объекты в зависимости от значения в конкретном поле, предполагается использовать -[изолированный класс/интерфейс](https://habr.com/ru/companies/otus/articles/720044/) для представления таких объектов. +Поскольку `value()` выбрасывает исключение при `undefined()`, всегда защищайте доступ проверкой `isDefined()` (или проверяйте `isNull()`) перед вызовом. -Для поддержки изолированных классов добавлены две аннотации: +### Частичное обновление PATCH { #jsonnullable-patch } -1. `@JsonDiscriminatorField` - указывает поле дискриминатора в DTO, которым помечается sealed класс/интерфейс -2. `@JsonDiscriminatorValue` - значение для вышеуказанного поля, помечает класс-наследник sealed класса/интерфейса +В запросе `PATCH` отсутствующее поле означает «оставить без изменений», а явный `null` означает «очистить значение». +`JsonNullable` позволяет обработчику различить эти два случая и применить только те поля, которые клиент действительно отправил: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record UserPatch(JsonNullable name, + JsonNullable email) { } + + public void apply(User user, UserPatch patch) { + if (patch.name().isDefined()) { //(1)! + user.setName(patch.name().value()); + } + if (patch.email().isDefined()) { + user.setEmail(patch.email().value()); //(2)! + } + // fields left as undefined() are not touched + } + ``` + + 1. Поле присутствовало в теле запроса, поэтому его необходимо применить (даже если значение — явный `null`). + 2. `value()` возвращает `null`, когда клиент отправил `{"email": null}`, что очищает поле. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class UserPatch( + val name: JsonNullable, + val email: JsonNullable + ) + + fun apply(user: User, patch: UserPatch) { + if (patch.name.isDefined()) { //(1)! + user.name = patch.name.value() + } + if (patch.email.isDefined()) { + user.email = patch.email.value() //(2)! + } + // fields left as undefined() are not touched + } + ``` + + 1. Поле присутствовало в теле запроса, поэтому его необходимо применить (даже если значение — явный `null`). + 2. `value()` возвращает `null`, когда клиент отправил `{"email": null}`, что очищает поле. + +Взаимодействие с [уровнями сериализации](#serialization-levels): `IncludeType.ALWAYS` и `IncludeType.NON_NULL` **не** меняют способ записи `JsonNullable` (применяются его собственные правила `undefined`/`nullValue`/`of`). +Только `IncludeType.NON_EMPTY` влияет на `JsonNullable`, рассматривая поле `undefined()` или `nullValue()` как пустое, так что оно опускается в выводе. + +## Sealed-классы и интерфейсы { #sealed-classes-and-interfaces } + +Если в зависимости от значения конкретного поля нужно читать и записывать разные `JSON`-объекты, используйте +[sealed-класс или интерфейс](https://kotlinlang.org/docs/sealed-classes.html) для представления этих объектов. + +Sealed-типы поддерживаются двумя аннотациями: + +1. `@JsonDiscriminatorField` — задает поле-дискриминатор в `DTO`, помеченном как `sealed`-класс или интерфейс. +2. `@JsonDiscriminatorValue` — задает одно или несколько значений дискриминатора для подкласса. ===! ":fontawesome-brands-java: `Java`" @@ -316,9 +521,11 @@ agent: } ``` -Для классов-наследников будут созданы `JsonReader` и `JsonWriter` по тем же правилам, как если бы на них была аннотация `@Json` и создастся `JsonReader` и `JsonWriter` для самого sealed класса/интерфейса. +Подклассы получают `JsonReader` и `JsonWriter` по тем же правилам, как если бы они были помечены `@Json`. +Сам `sealed`-класс или интерфейс также получает общий `JsonReader` и `JsonWriter`. +Поддерживаются вложенные `sealed`-иерархии, а `@JsonDiscriminatorValue` может принимать несколько значений для одного подкласса. -Json объект ниже будет записан в класс `FirstTypeEvent`: +Приведенный ниже `JSON`-объект записывается в класс `FirstTypeEvent`: ```json { "id": "1", @@ -329,9 +536,139 @@ Json объект ниже будет записан в класс `FirstTypeEve } ``` +Поддерживаются обобщенные (`generic`) типы `DTO`, включая обобщенные `sealed`-иерархии. +Кодек для каждого конкретного аргумента типа разрешается из графа, как и для любого другого типа поля: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + @JsonDiscriminatorField("@type") + public sealed interface Response { + + @JsonDiscriminatorValue("ok") + record Ok(T data) implements Response {} + + @JsonDiscriminatorValue("fail") + record Fail(String error) implements Response {} + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + @JsonDiscriminatorField("@type") + sealed interface Response { + + @JsonDiscriminatorValue("ok") + data class Ok(val data: T) : Response + + @JsonDiscriminatorValue("fail") + data class Fail(val error: String) : Response + } + ``` + +## Перечисления { #enum } + +Для `enum` `JsonReader` и `JsonWriter` можно сгенерировать теми же аннотациями `@Json`, `@JsonReader` и `@JsonWriter`. +По умолчанию значением `enum` в `JSON` является результат `toString()`, поэтому его можно переопределить: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public enum Status { + CREATED, + DELETED; + + @Override + public String toString() { + return this.name().toLowerCase(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + enum class Status { + CREATED, + DELETED; + + override fun toString(): String { + return name.lowercase() + } + } + ``` + +Если требуется значение, отличное от строки из `toString()`, пометьте публичный метод без параметров аннотацией `@Json`. +В этом случае для возвращаемого типа должны быть доступны соответствующие `JsonReader` и `JsonWriter`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public enum Status { + CREATED(1), + DELETED(2); + + private final int code; + + Status(int code) { + this.code = code; + } + + @Json + public int code() { + return this.code; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + enum class Status(private val code: Int) { + CREATED(1), + DELETED(2); + + @Json + fun code(): Int = code + } + ``` + +При чтении значение `JSON`, не совпадающее ни с одной константой `enum`, приводит к `JsonParseException` из `Jackson`, где перечислены допустимые значения. + +## RawJson { #raw-json } + +`RawJson` используется, когда в объект нужно включить уже готовый фрагмент `JSON`, не сериализуя его повторно. +При записи `RawJson` передается в выходной `JSON` как есть, поэтому значение должно быть корректным фрагментом `JSON`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record Dto(String id, RawJson payload) { } + + var dto = new Dto("1", new RawJson("{\"status\":\"ok\"}")); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class Dto(val id: String, val payload: RawJson) + + val dto = Dto("1", RawJson("""{"status":"ok"}""")) + ``` + ## Поддерживаемые типы { #supported-types } -Модуль предоставляет обширный список поддерживаемых из коробки типов которые покрывают большую часть того что может понадобиться. +Модуль предоставляет встроенные типы, которые покрывают большинство распространенных задач. +Для коллекций и ассоциативных массивов Kora использует `JsonReader` или `JsonWriter` типа элемента. ??? abstract "Список поддерживаемых типов" @@ -352,11 +689,17 @@ Json объект ниже будет записан в класс `FirstTypeEve * UUID * BigInteger * BigDecimal - * List - * Set + * RawJson + * Object + * Enum + * List + * Set + * SortedSet + * Map * LocalDate * LocalTime * LocalDateTime + * Instant * OffsetTime * OffsetDateTime * ZonedDateTime @@ -368,11 +711,11 @@ Json объект ниже будет записан в класс `FirstTypeEve * ZoneId * Duration -### Собственные типы { #custom-types } +### Пользовательские типы { #custom-types } -В случае если требуется писать/читать собственный тип, то предлагается зарегистрировать собственную [фабрику](container.md) для `JsonReader` / `JsonWriter`: +Если необходимо читать или записывать пользовательский тип, зарегистрируйте пользовательскую [фабрику](container.md) для `JsonReader` или `JsonWriter`. -Пример регистрации собственно `JsonWriter`: +Пример регистрации пользовательского `JsonWriter`: ===! ":fontawesome-brands-java: `Java`" @@ -382,7 +725,7 @@ Json объект ниже будет записан в класс `FirstTypeEve default JsonWriter zoneOffsetJsonWriter() { return (generator, value) -> { - if(value != null) { + if (value != null) { generator.writeString(value.getId()); } }; @@ -406,10 +749,52 @@ Json объект ниже будет записан в класс `FirstTypeEve } ``` +Пример регистрации пользовательского `JsonReader`. +Reader переключается по текущему токену парсера, возвращает `null` для `JSON` `null`, читает ожидаемый токен и выбрасывает `JsonParseException` для всего остального: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application { + + default JsonReader zoneOffsetJsonReader() { + return parser -> switch (parser.currentToken()) { + case VALUE_NULL -> null; + case VALUE_STRING -> ZoneOffset.of(parser.getValueAsString()); + default -> throw new JsonParseException(parser, + "Expecting VALUE_STRING token, got " + parser.currentToken()); + }; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application { + + fun zoneOffsetJsonReader(): JsonReader = JsonReader { parser -> + when (parser.currentToken()) { + JsonToken.VALUE_NULL -> null + JsonToken.VALUE_STRING -> ZoneOffset.of(parser.valueAsString) + else -> throw JsonParseException(parser, + "Expecting VALUE_STRING token, got ${parser.currentToken()}") + } + } + } + ``` + +Пользовательский `JsonReader` или `JsonWriter` — это обычный компонент графа. +После регистрации сгенерированные кодеки автоматически подхватывают его везде, где встречается поле типа `T`, а также его можно закрепить за отдельным полем через `@JsonField(reader = ..., writer = ...)` (см. [Именование полей](#field-naming)). + ## Jackson { #jackson } -В случае если хочется использовать `Jackson` для записи/чтения, то можно самому зарегистрировать [фабрику](container.md) -предоставляющую `ObjectMapper` и соответствующие `Mappers` которые требуются в других Kora модулях будут предоставлены зависимостью ниже: +Если для чтения и записи `JSON` вместо сгенерированных во время компиляции кодеков нужно использовать `Jackson`, применяйте `JacksonModule`. +Он заменяет преобразователи запросов/ответов `HTTP`-клиента и `HTTP`-сервера на основанные на `Jackson`. + +Каждый преобразователь `JacksonModule` зависит от компонента `ObjectMapper`, поэтому в графе **обязательно** должна присутствовать [фабрика](container.md), предоставляющая `ObjectMapper`. Без нее граф не соберется. ===! ":fontawesome-brands-java: `Java`" @@ -419,12 +804,19 @@ Json объект ниже будет записан в класс `FirstTypeEve implementation "ru.tinkoff.kora:jackson-module" ``` - Модуль: + Модуль и фабрика `ObjectMapper`: ```java @KoraApp - public interface Application extends JacksonModule { } + public interface Application extends JacksonModule { + + default ObjectMapper objectMapper() { //(1)! + return new ObjectMapper(); + } + } ``` + 1. Требуется всем преобразователям `JacksonModule`; настройте его по необходимости (модули, возможности и так далее). + === ":simple-kotlin: `Kotlin`" [Зависимость](general.md#dependencies) `build.gradle.kts`: @@ -433,8 +825,16 @@ Json объект ниже будет записан в класс `FirstTypeEve implementation("ru.tinkoff.kora:jackson-module") ``` - Модуль: + Модуль и фабрика `ObjectMapper`: ```kotlin @KoraApp - interface Application : JacksonModule + interface Application : JacksonModule { + + fun objectMapper(): ObjectMapper = ObjectMapper() //(1)! + } ``` + + 1. Требуется всем преобразователям `JacksonModule`; настройте его по необходимости (модули, возможности и так далее). + +Показанный выше `json-annotation-processor` позволяет `@Json`, `@JsonReader` и `@JsonWriter` по-прежнему генерировать кодеки, так что сгенерированная и `Jackson`-сериализация могут сосуществовать (например, `Jackson` для `HTTP` и сгенерированные кодеки для [Kafka](kafka.md)). +Сами `HTTP`-преобразователи `JacksonModule` зависят только от `ObjectMapper`. diff --git a/mkdocs/docs/ru/documentation/junit5.md b/mkdocs/docs/ru/documentation/junit5.md index 6b7d2ba..06a2000 100644 --- a/mkdocs/docs/ru/documentation/junit5.md +++ b/mkdocs/docs/ru/documentation/junit5.md @@ -1,26 +1,25 @@ --- -description: "Explains Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization. Use when working with @KoraAppTest, @TestComponent, @MockComponent, @Tag, @TestConfig, @TestConfigSource, Graph, Mockito." +description: "Explains Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization. Use when working with @KoraAppTest, @TestComponent, @Tag, KoraAppTestConfigModifier, Graph, Mockito." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization; key triggers include @KoraAppTest, @TestComponent, @MockComponent, @Tag, @TestConfig, @TestConfigSource, Graph, Mockito." + use_when: "Use this file for Kora docs or implementation questions about Kora JUnit 5 testing support, application graph tests, component replacement, mocks, tags, test configuration, and initialization; key triggers include @KoraAppTest, @TestComponent, @Tag, KoraAppTestConfigModifier, Graph, Mockito." --- -Модуль предоставляет `Extension` для [JUnit5](https://junit.org/junit5/docs/current/user-guide/) который позволяет легко тестировать приложение. +Модуль предоставляет расширение для [JUnit 5](https://junit.org/junit5/docs/current/user-guide/), которое позволяет тестировать приложение через тот же граф компонентов, что используется во время работы приложения. -Концепция JUnit 5 расширения Kora предполагает компонентного тестирование именно исходного кода, -который будет в итоге использоваться в реальном приложении. -Это подразумевает, что именно контейнер зависимостей основного приложения и участвует в рамках теста, -он может быть ограничен в рамках теста, либо его части заменены заглушками если этого требуется тест. +Расширение Kora для `JUnit 5` предназначено для компонентного и интеграционного тестирования исходного кода, который впоследствии будет работать в реальном приложении. +В тесте используется контейнер зависимостей основного приложения: его можно ограничить нужными компонентами, +расширить тестовыми компонентами или заменить отдельные части заглушками. Модуль позволяет проводить: -- `Компонентное тестирование` - тестирование одного компонента -- `Межкомпонентное тестирование` - тестирование нескольких компонент и взаимодействие друг с другом -- `Интеграционное тестирование` - тестирование компонент и взаимодействие с внешними системами +- `Компонентные тесты` — тестирование одного компонента. +- `Межкомпонентные тесты` — тестирование нескольких компонентов и их взаимодействия друг с другом. +- `Интеграционные тесты` — тестирование компонентов и взаимодействия с внешними системами. -Настоятельно советуем дополнительно проводить интеграционное тестирование запакованного в финальный образ артефакта сервиса, -по средствам черной коробки с помощью [библиотеки TestContainers](https://java.testcontainers.org/). +Рекомендуется дополнительно тестировать артефакт сервиса, упакованный в итоговый образ, +как черный ящик с помощью [библиотеки Testcontainers](https://java.testcontainers.org/). -Если нужен пошаговый разбор перед справочным описанием, смотрите [Компонентное тестирование](../guides/testing-junit.md), [Интеграционное тестирование](../guides/testing-integration.md) и [Тестирование как черный ящик](../guides/testing-black-box.md). +Пошаговый разбор перед справочным описанием смотрите в разделах [Компонентное тестирование](../guides/testing-junit.md), [Интеграционное тестирование](../guides/testing-integration.md) и [Тестирование черного ящика](../guides/testing-black-box.md). ## Подключение { #dependency } @@ -31,7 +30,7 @@ agent: testImplementation "ru.tinkoff.kora:test-junit5" ``` - Настроить [JUnit платформу](https://docs.gradle.org/current/userguide/java_testing.html#using_junit5) `build.gradle`: + Настройка [платформы JUnit](https://docs.gradle.org/current/userguide/java_testing.html#using_junit5) `build.gradle`: ```groovy test { useJUnitPlatform() @@ -50,10 +49,10 @@ agent: testImplementation("ru.tinkoff.kora:test-junit5") ``` - Настроить [JUnit платформу](https://docs.gradle.org/current/userguide/java_testing.html#using_junit5) `build.gradle.kts`: + Настройка [платформы JUnit](https://docs.gradle.org/current/userguide/java_testing.html#using_junit5) `build.gradle.kts`: ```groovy tasks.test { - useJUnitPlatform() + useJUnitPlatform() testLogging { showStandardStreams = true events("passed", "skipped", "failed") @@ -64,7 +63,7 @@ agent: ## Использование { #usage } -Примеры будут показаны относительно такого приложения: +Примеры будут показаны применительно к такому приложению: ===! ":fontawesome-brands-java: `Java`" @@ -105,21 +104,22 @@ agent: ### Тест { #test } -Предполагается использовать аннотацию `@KoraAppTest` для аннотирования тестового класса. +Чтобы включить расширение Kora, пометьте тестовый класс аннотацией `@KoraAppTest`. +Аннотация подключает расширение `JUnit 5`, находит сгенерированный граф указанного приложения `@KoraApp` и подготавливает контейнер зависимостей для теста. Параметры аннотации `@KoraAppTest`: -- `value` - обязательный параметр который указывает на класс аннотированный `@KoraApp`, представляющий собой граф всех зависимостей которые будут доступны в рамках теста. -- `components` - список `@Root` компонентов, которые надо включить в ограниченный контейнер зависимостей в рамках теста, - подразумевается что будут указаны дополнительные компоненты не объявленные в рамках теста с помощью специальной аннотации `@TestComponent`. -- `modules` - список модулей с компонентами подключенных в приложении, которые надо включить в ограниченный контейнер зависимостей в рамках теста, - подразумевается что будут указаны дополнительные компоненты не объявленные в рамках теста. +- `value` — класс, помеченный `@KoraApp`, граф компонентов которого будет использоваться в тесте (`обязательный`, без значения по умолчанию). +- `components` — дополнительные классы компонентов, которые должны быть включены в тестовый граф в дополнение к компонентам, найденным через `@TestComponent` (по умолчанию: `{}`). +- `modules` — дополнительные модули с фабричными методами компонентов, которые должны быть подключены к тестовому графу (по умолчанию: `{}`). + +В `modules` можно указывать только интерфейсы модулей. Если требуется протестировать весь граф, внедрите `KoraAppGraph` или не ограничивайте граф отдельными компонентами `@TestComponent`. ===! ":fontawesome-brands-java: `Java`" ```java - @KoraAppTest(value = Application.class, - components = { SomeComponent.class }, + @KoraAppTest(value = Application.class, + components = { SomeComponent.class }, modules = { SomeModule.class }) class SomeTests { @@ -128,8 +128,8 @@ agent: === ":simple-kotlin: `Kotlin`" ```kotlin - @KoraAppTest(value = Application::class, - components = [SomeComponent::class], + @KoraAppTest(value = Application::class, + components = [SomeComponent::class], modules = [SomeModule::class]) class SomeTests { @@ -138,13 +138,13 @@ agent: ### Компонент { #component } -Для внедрения и указания компонентов для тестирования предлагается использовать аннотацию `@TestComponent`, -которая позволяет внедрять компоненты в аргументы метода и/или поля тестового класса и ограничивать ими контейнер зависимостей теста. +Для внедрения и выбора компонентов для тестирования используйте аннотацию `@TestComponent`. +Она позволяет внедрять компоненты в аргументы тестовых методов, в конструктор и/или в поля тестового класса, а также ограничивает контейнер зависимостей этими компонентами. -Все компоненты перечисленные в тестовых полях и/или аргументах метода/конструктора с аннотацией `@TestComponent` -будут внедрены как зависимости в рамках теста и весь контейнер зависимостей будет ограничен именно этими компонентами и их зависимостями в рамках теста. +Все компоненты, перечисленные в полях теста и/или в аргументах метода/конструктора и помеченные `@TestComponent`, будут внедрены как зависимости в рамках теста. +Тестовый контейнер зависимостей будет ограничен этими компонентами и их зависимостями. -Важно что компоненты в рамках теста должны использоваться хотя бы одним [@Root компонентом](container.md#root-component) который также указан в рамках теста. +Важно, что компоненты внутри теста должны использоваться хотя бы одним [@Root компонентом](container.md#root-component), который также указан в рамках теста. Пример теста, где компоненты внедряются в поля: @@ -163,7 +163,6 @@ agent: } } ``` - === ":simple-kotlin: `Kotlin`" ```kotlin @@ -242,9 +241,24 @@ agent: } ``` +#### Правила внедрения { #injection-rules } + +Компоненты можно внедрять тремя способами: в поле тестового класса, в конструктор или в параметр тестового метода. +Выбранная форма влияет на то, когда расширение Kora может получить доступ к экземпляру тестового класса и какие дополнительные механизмы доступны. + +- Поля подходят для большинства тестов и совместимы с `KoraAppTestConfigModifier`, `KoraAppTestGraphModifier`, `PER_METHOD` и `PER_CLASS`. +- Внедрение через конструктор удобно для неизменяемых полей, но несовместимо с `KoraAppTestConfigModifier` и `KoraAppTestGraphModifier`, поскольку расширению нужен экземпляр тестового класса для вызова `config()` или `graph()`, тогда как этот экземпляр еще создается во время внедрения через конструктор. +- Параметры метода удобны для зависимостей, локальных для конкретного теста; в режиме `PER_METHOD` граф включает параметры текущего метода, а в режиме `PER_CLASS` расширение заранее собирает параметры `@TestComponent` из всех методов класса. +- Если используется внедрение через конструктор, `@TestComponent`, `@Mock`, `@Spy`, `@MockK` или `@SpyK` нельзя также внедрять в параметры тестового метода. +- В режиме `PER_CLASS` `@Mock` / `@MockK` нельзя внедрять в параметры тестового метода, поскольку заглушки уровня метода живут меньше, чем общий граф тестового класса. +- Один и тот же элемент нельзя одновременно объявить как обычный `@TestComponent`, заглушку (mock) и шпион (spy): расширение завершит тест ошибкой конфигурации. + +Если тесту нужны `KoraAppTestConfigModifier` или `KoraAppTestGraphModifier`, используйте внедрение через поля или параметры метода. +Если требуется внедрение через конструктор, лучше вынести изменение конфигурации и графа в отдельное тестовое `@KoraApp` или подключаемый модуль. + ### Тег { #tag } -Для внедрения зависимости которая имеет `@Tag`, требуется указать соответствующую аннотацию `@Tag` рядом с внедряемым аргументом: +Чтобы внедрить зависимость/заглушку, помеченную `@Tag`, необходимо указать соответствующую аннотацию `@Tag` рядом с аргументом для внедрения: ===! ":fontawesome-brands-java: `Java`" @@ -266,35 +280,80 @@ agent: class SomeTests { @Test - fun example(@Tag(Supplier.class) @TestComponent component1: Supplier) { + fun example(@Tag(Supplier::class) @TestComponent component1: Supplier) { assertEquals("?", component1.get()) } } ``` -### Заглушки { #mock } +### Граф приложения { #application-graph } + +Если тесту нужен прямой доступ к подготовленному графу, внедрите `KoraAppGraph` в поле, конструктор или аргумент тестового метода. +Он может получить один или несколько компонентов по типу, а также учитывать `@Tag`. + +Основные методы `KoraAppGraph`: + +- `getFirst(Type type)` / `getFirst(Class type)` — возвращают первый найденный компонент или `null`. +- `getFirst(Type type, Class... tags)` / `getFirst(Class type, Class... tags)` — возвращают первый компонент с указанными тегами или `null`. +- `findFirst(...)` — возвращает `Optional` вместо `null`. +- `getAll(...)` — возвращает все компоненты указанного типа, при необходимости учитывая теги. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SomeTests { + + @Test + void example(KoraAppGraph graph) { + var component = graph.getFirst(Supplier.class, Supplier.class); + + assertNotNull(component); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SomeTests { + + @Test + fun example(graph: KoraAppGraph) { + val component = graph.getFirst(Supplier::class.java, Supplier::class.java) + + assertNotNull(component) + } + } + ``` + +`KoraAppGraph` нельзя использовать как цель для `@Mock`, `@Spy`, `@MockK` или `@SpyK`, поскольку это служебный объект тестового расширения, а не компонент приложения. + +### Заглушка { #mock } ===! ":fontawesome-brands-java: `Java`" - Для создание заглушек компонент в Java в рамках теста предлагается использовать аннотации предоставляемые библиотекой [Mockito](https://site.mockito.org/) в совокупности с аннотацией `@TestComponent`. + Для создания заглушки компонента в Java в рамках теста предлагается использовать аннотации из библиотеки [Mockito](https://site.mockito.org/) совместно с аннотацией `@TestComponent`. - Требуется подключить библиотеку [Mockito](https://site.mockito.org/) как зависимость `build.gradle`: + Требуется добавить библиотеку [Mockito](https://site.mockito.org/) как зависимость в `build.gradle`: ```groovy testImplementation "org.mockito:mockito-core:5.18.0" ``` - **Важно**, подразумевается что `MockitoExtension` не будет использоваться и будет отключен, нельзя совмещать его работу совместно с `@KoraAppTest`. + **Важно**, предполагается, что `MockitoExtension` использоваться не будет и будет отключено, его нельзя совмещать вместе с `@KoraAppTest`. - Поддерживаются аннотациии [@Mock](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mock.html) и [@Spy](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Spy.html), а также все параметры этих аннотаций. - Рекомендуется подробнее ознакомиться с работой этих аннотацией в [официальной документации библиотеки Mockito](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html). + Поддерживаются аннотации [@Mock](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mock.html) и [@Spy](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Spy.html), а также все параметры этих аннотаций. + Рекомендуется подробнее ознакомиться с тем, как работают эти аннотации, в [официальной документации библиотеки Mockito](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html). - Аннотация [@Mock](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mock.html) позволяет сделать класс заглушку - проаннотированного компонента и контролировать поведение его методов с помощью `Mockito` либо методы будут возвращать значения по-умолчанию: `void`, значения по умолчанию для примитивов, пустые коллекции и `null` для всех остальных объектов. + Аннотация [@Mock](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mock.html) позволяет сделать заглушку класса + помеченного компонента и управлять поведением его методов с помощью `Mockito`, либо методы будут возвращать значения по умолчанию: `void`, значения по умолчанию для примитивов, пустые коллекции и `null` для всех остальных объектов. - Компонент заглушка будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты которые требовали его как зависимость. - Все зависимые компоненты которые больше ни где не требуются в рамках теста будут исключены за ненанобностью. + Компонент-заглушка будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты, которым он требовался как зависимость. + Все зависимые компоненты, которые больше нигде в рамках теста не требуются, будут исключены как ненужные. - Пример теста с использованием `@Mock` компонента и внедрением заглушки в поле: + + Пример теста с использованием компонента `@Mock` и внедрением заглушки в поле: ```java @KoraAppTest(Application.class) @@ -316,13 +375,13 @@ agent: } ``` - Аннотация [@Spy](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Spy.html) позволяет сделать шпион фасад реализации класса - компонента из контейнера зависимостей который по умолчанию будет иметь оригинальное поведение методов компонента, - но как и в случае с заглушками, их поведение можно переопределить. + Аннотация [@Spy](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Spy.html) позволяет сделать шпион-фасад реализации класса + компонента из контейнера зависимостей, который по умолчанию будет иметь исходное поведение методов компонента, + но, как и в случае с заглушками, их поведение можно переопределить. - Компонент шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты которые требовали его как зависимость. + Компонент-шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты, которым он требовался как зависимость. - Пример теста с использованием `@Spy` компонента и внедрением шпиона в аргумент метода: + Пример теста с использованием компонента `@Spy` и внедрением шпиона в аргумент метода: ```java @KoraAppTest(Application.class) @@ -336,12 +395,12 @@ agent: } ``` - Можно также сделать шпиона из значения поля тестового класса. + Также можно сделать шпион из значения поля тестового класса. - Компонент шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты которые требовали его как зависимость. - Все зависимые компоненты которые больше ни где не требуются в рамках теста будут исключены за ненанобностью. + Компонент-шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты, которым он требовался как зависимость. + Все зависимые компоненты, которые больше нигде в рамках теста не требуются, будут исключены как ненужные. - Пример теста с использованием `@Spy` компонента шпиона: + Пример теста с использованием компонента-шпиона `@Spy`: ```java @KoraAppTest(Application.class) @@ -360,33 +419,33 @@ agent: === ":simple-kotlin: `Kotlin`" - Для создание заглушек компонент в Kotlin в рамках теста предлагается использовать аннотации предоставляемые библиотекой [MockK](https://mockk.io/) в совокупности с аннотацией `@TestComponent`. + Для создания заглушек компонентов в Kotlin предлагается использовать аннотации из библиотеки [MockK](https://mockk.io/) совместно с аннотацией `@TestComponent`. - Требуется подключить библиотеку [MockK](https://mockk.io/) как зависимость `build.gradle.kts`: + Требуется подключить библиотеку [MockK](https://mockk.io/) как зависимость в ``build.gradle.kts``: ```groovy testImplementation("io.mockk:mockk:1.13.11") ``` - **Важно**, подразумевается что `MockkExtension` не будет использоваться и будет отключен, нельзя совмещать его работу совместно с `@KoraAppTest`. + **Важно**, предполагается, что `MockkExtension` использоваться не будет и будет отключено, его нельзя совмещать вместе с `@KoraAppTest`. - Поддерживаются аннотациии [@MockK](https://mockk.io/#annotations) и [@SpyK](https://mockk.io/#annotations), а также все параметры этих аннотаций. + Поддерживаются аннотации [@MockK](https://mockk.io/#annotations) и [@SpyK](https://mockk.io/#annotations), а также все параметры этих аннотаций. - Также есть возможность при желании использовать [Mockito](https://site.mockito.org/). - Для более подробного описания работы Kora и [Mockito](https://site.mockito.org/) следует ознакопиться с Java вкладкой этого абзаца. - Чтобы улучшить взаимодействие Mockito и Kotlin можно использовать библиотеку [Mockito Kotlin](https://github.com/mockito/mockito-kotlin). + При желании также можно использовать [Mockito](https://site.mockito.org/). + Для более подробного описания того, как работают Kora и [Mockito](https://site.mockito.org/), следует прочитать вкладку Java этого раздела. + Для улучшения взаимодействия между Mockito и Kotlin можно использовать библиотеку [Mockito Kotlin](https://github.com/mockito/mockito-kotlin). ```groovy testImplementation("org.mockito.kotlin:mockito-kotlin:5.4.0") ``` - **Важно**, подразумевается что `MockitoExtension` не будет использоваться и будет отключен, нельзя совмещать его работу совместно с `@KoraAppTest`. + **Важно**, предполагается, что `MockitoExtension` использоваться не будет и будет отключено, его нельзя совмещать вместе с `@KoraAppTest`. - Аннотация [@MockK](https://mockk.io/#annotations) позволяет сделать класс заглушку - проаннотированного компонента и контролировать поведение его методов с помощью `MockK`. + Аннотация [@MockK](https://mockk.io/#annotations) позволяет сделать заглушку класса + помеченного компонента и управлять поведением его методов с помощью `MockK`. - Компонент заглушка будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты которые требовали его как зависимость. - Все зависимые компоненты которые больше ни где не требуются в рамках теста будут исключены за ненанобностью. + Компонент-заглушка будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты, которым он требовался как зависимость. + Все зависимые компоненты, которые больше нигде в рамках теста не требуются, будут исключены как ненужные. - Пример теста с использованием `@MockK` компонента и внедрением заглушки в поле: + Пример теста с использованием компонента `@MockK` и внедрением заглушки: ```kotlin @KoraAppTest(Application::class) @@ -404,13 +463,13 @@ agent: } ``` - Аннотация [@SpyK](https://mockk.io/#annotations) позволяет сделать шпион фасад реализации класса - компонента из контейнера зависимостей который по умолчанию будет иметь оригинальное поведение методов компонента, - но как и в случае с заглушками, их поведение можно переопределить. + Аннотация [@SpyK](https://mockk.io/#annotations) позволяет сделать шпион-фасад реализации класса + компонента из контейнера зависимостей, который по умолчанию будет иметь исходное поведение методов компонента, + но, как и в случае с заглушками, их поведение можно переопределить. - Компонент шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты которые требовали его как зависимость. + Компонент-шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты, которым он требовался как зависимость. - Пример теста с использованием `@SpyK` компонента и внедрением шпиона в аргумент метода: + Пример теста с использованием компонента `@SpyK` и встраиванием шпиона в аргумент метода: ```kotlin @KoraAppTest(Application::class) @@ -424,12 +483,12 @@ agent: } ``` - Можно также сделать шпиона из значения поля тестового класса. + Также можно сделать шпион из значения поля тестового класса. - Компонент шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты которые требовали его как зависимость. - Все зависимые компоненты которые больше ни где не требуются в рамках теста будут исключены за ненанобностью. + Компонент-шпион будет внедрен как зависимость в аргументы и/или поля тестового класса и во все компоненты, которым он требовался как зависимость. + Все зависимые компоненты, которые больше нигде в рамках теста не требуются, будут исключены как ненужные. - Пример теста с использованием `@SpyK` компонента шпиона: + Пример теста с использованием компонента-шпиона `@SpyK`: ```kotlin @KoraAppTest(Application::class) @@ -446,17 +505,22 @@ agent: } ``` -#### Проверка заглушек { #mock-strictness } +#### Строгость заглушек { #mock-strictness } + +Заглушки `Mockito` можно проверять с помощью аннотации `@MockitoStrictness`. +Она задает уровень проверки для заглушек `Mockito`, созданных расширением Kora в рамках тестового класса. -Возможно проверять использование заглушек `Mockito` в тестах с помощью задания уровня проверки по средствам аннотации `@MockitoStrictness`. +Расширение ведет себя аналогично `MockitoSession`: после завершения теста оно передает созданные заглушки на проверку `Mockito` и сообщает о неиспользованных или подозрительных настройках поведения. +Если `@MockitoStrictness` не указана, Kora использует `Strictness.WARN`: тест не падает, но в лог записываются предупреждения. -Работает аналогично `MockitoSession` и представляет собой имитацию сессии в рамках фреймворка Mockito, -которая обычно включает в себя выполнение одного тестового метода. -Она предоставляет механизм для управления жизненным циклом имитаций и обеспечения надлежащей очистки и проверки. +Поддерживаемые уровни: -Позволяет поддерживать строгие гарантии заглушек с помощью перечисления `Strictness`, -которое помогает выявлять неиспользуемые вызовы и потенциально выбрасывать исключение `UnnecessaryStubbingException` -или писать в лог предупреждения. +- `Strictness.WARN` — значение по умолчанию; записывает предупреждения в лог и не приводит к падению теста. +- `Strictness.STRICT_STUBS` — строгий режим; неиспользованная настройка поведения приводит к падению теста, например с `UnnecessaryStubbingException`. +- `Strictness.LENIENT` — мягкий режим; отключает проверки неиспользованных настроек поведения. + +Если у конкретной `@Mock` есть собственный параметр `strictness`, он применяется к настройкам этой заглушки. +`@MockitoStrictness` удобна как общий уровень для всего тестового класса, чтобы не дублировать настройку на каждой заглушке. ===! ":fontawesome-brands-java: `Java`" @@ -481,6 +545,9 @@ agent: } ``` +В примере выше `Mockito.when(component1.get()).thenReturn("?")` должно быть использовано тестом. +Если убрать вызов `component1.get()` из тестового метода, `Strictness.STRICT_STUBS` приведет к падению теста. + === ":simple-kotlin: `Kotlin`" ```kotlin @@ -500,28 +567,29 @@ agent: } ``` -### Расширенный контейнер { #test-graph } +Для Kotlin с `Mockito Kotlin` действует тот же механизм, поскольку проверку выполняет `Mockito`. +`@MockitoStrictness` не применяется к заглушкам `MockK`. -Иногда может потребоваться использовать расширенный контейнер зависимостей в рамках тестов. -К примеру, тестовый контейнер приложение, расширяющий основное приложение и добавляющий -некоторые компоненты из общих модулей, которые не используются в данном приложении. +### Тестовый граф { #test-graph } -Например, когда у вас есть разные приложения чтения и записи с общими компонентами, -которые могут потребоваться в рамках тестирования одного и другого. -Либо, вам нужны некоторые функции сохранения/удаления/обновления только для тестирования в -качестве быстрой тестовой утилиты. +Иногда в рамках тестов может потребоваться использовать расширенный контейнер зависимостей. +Например, тестовое приложение может расширять основное приложение и добавлять компоненты, которые нужны только в тестах. + +Такой подход полезен, когда у вас есть разные приложения Read API и Write API с общими компонентами, +которые могут потребоваться при тестировании одного и другого. +Или же вам могут понадобиться какие-то функции сохранения/удаления/обновления исключительно для тестирования в качестве быстрой тестовой утилиты. ???+ warning "Рекомендация" - **Настоятельно Рекомендуем Тестировать** приложения как [черный ящик](https://github.com/kora-projects/kora-examples/blob/master/kora-java-crud/src/test/java/ru/tinkoff/kora/example/crud/BlackBoxTests.java) - и полагаться на этот подход в качестве основного источника правды и работоспособности приложения. + **Настоятельно рекомендуется тестировать** приложения как [черный ящик](https://github.com/kora-projects/kora-examples/blob/master/kora-java-crud/src/test/java/ru/tinkoff/kora/example/crud/BlackBoxTests.java) + и полагаться на этот подход как на основной источник истины и корректности приложения. - Приложение может работать по разному в зависимости от флагов JVM, - базового образа и нативных библиотек, отличий частичной конфигурации от полной, - отличий конвертации на точках входа в приложение, использования реестров схем и так далее. - Только готовый образ может гарантировать максимально приближенную среду для тестирования. + Приложение может работать по-разному в зависимости от флагов JVM, + базового образа и нативных библиотек, различий между частичными и полными конфигурациями, + различий в преобразовании на точках входа приложения, использования реестров схем и так далее. + Только prod-ready образ может гарантировать максимально близкое к реальному окружение тестирования. -Представим что приложение выглядит так: +Представим, что приложение выглядит так: ===! ":fontawesome-brands-java: `Java`" @@ -549,12 +617,12 @@ agent: } ``` -В тестах можно создать граф расширяющий основное приложение и использовать уже его в рамках тестах. +В тестах можно создать отдельное тестовое `@KoraApp`, которое расширяет основное приложение, и использовать этот граф. +Для этого сценария требуется сгенерированный субмодуль основного приложения: без него тестовое приложение не сможет унаследовать и подключить компоненты основного графа. ===! ":fontawesome-brands-java: `Java`" - Для этого в первую очередь понадобится включить опцию - для создания сабмодуля основного приложения в `build.gradle`: + Сначала включите параметр, который создает субмодуль основного приложения, в `build.gradle`: ```groovy compileJava { @@ -566,8 +634,7 @@ agent: === ":simple-kotlin: `Kotlin`" - Для этого в первую очередь понадобится включить опцию - для создания сабмодуля основного приложения в `build.gradle.kts`: + Сначала включите параметр, который создает субмодуль основного приложения, в `build.gradle.kts`: ```groovy ksp { @@ -575,9 +642,9 @@ agent: } ``` -Затем требуется создать расширенный тестовый граф приложения в директории для тестовых классов. -Не забывайте помечать компоненты как `@Root` т.к. они скорее всего не используются ни кем кроме тестов -и не будут иначе включены в граф: +Затем требуется создать расширенный тестовый граф приложения в каталоге тестовых исходников. +Не забудьте пометить компоненты как `@Root`, поскольку они, скорее всего, никем не используются, +кроме тестов, и иначе не будут включены в граф: ===! ":fontawesome-brands-java: `Java`" @@ -607,7 +674,7 @@ agent: ===! ":fontawesome-brands-java: `Java`" - Чтобы граф тестового приложения создался, надо подключить процессоры как тестовые зависимости в `build.gradle`: + Чтобы граф тестового приложения был сгенерирован, нужно добавить обработчики как тестовые зависимости в `build.gradle`: ```groovy dependencies { @@ -617,7 +684,7 @@ agent: === ":simple-kotlin: `Kotlin`" - Чтобы граф тестового приложения создался, надо подключить процессоры как тестовые зависимости в `build.gradle.kts`: + Чтобы граф тестового приложения был сгенерирован, нужно добавить обработчики как тестовые зависимости в `build.gradle.kts`: ```groovy dependencies { @@ -625,11 +692,11 @@ agent: } ``` -Возможно, потребуется исключить сканирование созданных Kora классов со стороны JUnit (иногда у JUnit может возникать ошибка при поиске тестов): +Может потребоваться исключить сканирование сгенерированных Kora классов средствами JUnit (иногда возникает ошибка при поиске тестов): ===! ":fontawesome-brands-java: `Java`" - Классы начинаются с символа `$`, надо исключить в `build.gradle`: + Классы начинаются с символа `$`, исключите их в `build.gradle`: ```java test { @@ -639,7 +706,7 @@ agent: === ":simple-kotlin: `Kotlin`" - Классы начинаются с символа `$`, надо исключить в `build.gradle.kts`: + Классы начинаются с символа `$`, исключите их в `build.gradle.kts`: ```kotlin tasks.test { @@ -647,7 +714,7 @@ agent: } ``` -Теперь можно использовать расширенный тестовый граф приложения в тестах: +Теперь вы можете использовать расширенный граф приложения в своих тестах: ===! ":fontawesome-brands-java: `Java`" @@ -680,23 +747,76 @@ agent: } ``` -## Настройка конфигурации { #test-configuration } +Если наследование от основного `@KoraApp` не требуется и нужно добавить только фабричные методы из отдельного модуля, +используйте параметр `modules` аннотации `@KoraAppTest`. +`modules` принимает интерфейсы модулей, а не классы компонентов: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public interface TestModule { -По умолчанию будет использоваться основная конфигурация, как и в случае запуска реального приложения. + @Root + default Integer testOnlyComponent() { + return 1; + } + } -Для изменений/добавления конфига в рамках тестов предполагается чтобы тестовый класс реализовал интерфейс `KoraAppTestConfigModifier`, -где требуется реализовать метод предоставления модификации конфига `KoraConfigModification`. + @KoraAppTest(value = Application.class, modules = TestModule.class) + class SomeTests { -Запрещено использовать `KoraAppTestConfigModifier` и внедрение в конструктор так как в таком случае нельзя получить конфигуацию до внедрения. + @Test + void test(@TestComponent Integer component) { + assertEquals(1, component); + } + } + ``` -### Переменные окружения { #environment-variables } +=== ":simple-kotlin: `Kotlin`" -В случае если в рамках теста надо использовать [конфигурацию по умолчанию](config.md#file) которая использовалась бы во время работы приложения, -и требуется лишь подставить переменные окружения то можно использовать механизм `SystemProperty` в `KoraConfigModification`: + ```kotlin + interface TestModule { + + @Root + fun testOnlyComponent(): Int { + return 1 + } + } + + @KoraAppTest(value = Application::class, modules = [TestModule::class]) + class SomeTests { + + @Test + fun test(@TestComponent component: Int) { + assertEquals(1, component) + } + } + ``` + +Итого: + +- `kora.app.submodule.enabled=true` нужен, когда тестовое `@KoraApp` расширяет основное `@KoraApp`. +- `@KoraAppTest(modules = ...)` подходит для случаев, когда к тестовому графу нужно просто подключить дополнительные модули. +- Компоненты, которые должны появиться в ограниченном тестовом графе, все равно должны быть достижимы из `@TestComponent`, `components` или `KoraAppGraph`. + +## Конфигурация теста { #test-configuration } + +По умолчанию будет использоваться базовая конфигурация, как и в случае запуска реального приложения. + +Чтобы изменить или добавить конфигурацию в рамках тестов, тестовый класс должен реализовать `KoraAppTestConfigModifier`, +а метод `config()` должен возвращать `KoraConfigModification`. + +`KoraAppTestConfigModifier` нельзя использовать вместе с внедрением компонентов в конструктор тестового класса: +расширению нужно получить изменение конфигурации до создания тестового графа, а для этого экземпляр теста должен уже существовать. + +#### Переменные окружения { #environment-variables } + +Если тесту нужно использовать [конфигурацию по умолчанию](config.md#file), которая использовалась бы при запуске приложения, +и требуется лишь подставить переменные окружения, можно воспользоваться механизмом `SystemProperty` в `KoraConfigModification`: ===! ":material-code-json: `Hocon`" - Предположим есть такая конфигурация `application.conf`: + Предположим, есть такая конфигурация `application.conf`: ```javascript db { @@ -710,7 +830,7 @@ agent: === ":simple-yaml: `YAML`" - Предположим есть такая конфигурация `application.yaml`: + Предположим, есть такая конфигурация `application.yaml`: ```yaml db: @@ -721,7 +841,7 @@ agent: poolName: "example" ``` -Тогда, чтобы использовать такой конфиг и передать в него лишь переменные окружения требуется вернуть такой `KoraConfigModification`: +Чтобы использовать такой конфиг и передать только переменные окружения, нужно вернуть такой `KoraConfigModification`: ===! ":fontawesome-brands-java: `Java`" @@ -755,9 +875,49 @@ agent: } ``` +Если нужно передать сразу несколько значений, используйте `withSystemProperties(Map)`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SomeTests implements KoraAppTestConfigModifier { + + @NotNull + @Override + public KoraConfigModification config() { + return KoraConfigModification + .ofSystemProperty("POSTGRES_JDBC_URL", "jdbc:postgresql://localhost:5432/postgres") + .withSystemProperties(Map.of( + "POSTGRES_USER", "postgres", + "POSTGRES_PASS", "postgres" + )); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SomeTests : KoraAppTestConfigModifier { + + override fun config(): KoraConfigModification { + return KoraConfigModification + .ofSystemProperty("POSTGRES_JDBC_URL", "jdbc:postgresql://localhost:5432/postgres") + .withSystemProperties( + mapOf( + "POSTGRES_USER" to "postgres", + "POSTGRES_PASS" to "postgres" + ) + ) + } + } + ``` + ### Файл конфигурации { #configuration-file } -Пример добавления конфигурации в виде файла: +Пример предоставления конфигурации в виде файла: ===! ":fontawesome-brands-java: `Java`" @@ -786,8 +946,8 @@ agent: ### Текст конфигурации { #configuration-text } -Пример добавления конфигурации в виде строки будет выглядеть так, -в таком случае будет использоваться только эта конфигурация без каких либо файлов конфигурации: +Пример добавления конфигурации в виде строки выглядел бы так, +в этом случае будет использоваться только эта конфигурация без каких-либо файлов конфигурации: ===! ":fontawesome-brands-java: `Java`" @@ -824,12 +984,172 @@ agent: } ``` -## Модификация контейнера { #container-modification } +### Подстановка в конфигурации { #configuration-substitution } -Для добавления/замены компонент в рамках контейнера приложения без аннотаций требуется реализовать интерфейс `KoraAppTestGraphModifier` и -реализовать метод предоставления модификатора контейнера. +Подстановка переменных окружения, показанная в разделе [Переменные окружения](#environment-variables), также работает со встроенной конфигурацией: +объявите плейсхолдеры `${ENV}` прямо внутри конфигурации `ofString(...)` и разрешите их через цепочку вызовов `withSystemProperty(...)`. +Это удобно, когда вся конфигурация описана в тесте, но некоторые значения (порты, хосты, учетные данные) известны только во время выполнения: -Запрещено использовать `KoraAppTestGraphModifier` и внедрение в конструктор так как в таком случае нельзя получить граф до внедрения. +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SomeTests implements KoraAppTestConfigModifier { + + @Override + public @Nonnull KoraConfigModification config() { + return KoraConfigModification.ofString(""" + myconfig { + myinnerconfig { + first = ${ENV_FIRST} + second = ${ENV_SECOND} + } + } + """) + .withSystemProperty("ENV_FIRST", "1") + .withSystemProperty("ENV_SECOND", "2"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SomeTests : KoraAppTestConfigModifier { + + override fun config(): KoraConfigModification { + return KoraConfigModification.ofString( + """ + myconfig { + myinnerconfig { + first = \${ENV_FIRST} + second = \${ENV_SECOND} + } + } + """.trimIndent() + ) + .withSystemProperty("ENV_FIRST", "1") + .withSystemProperty("ENV_SECOND", "2") + } + } + ``` + +### Testcontainers { #testcontainers } + +Распространенное применение `KoraAppTestConfigModifier` — интеграция с [Testcontainers](https://java.testcontainers.org/): +тест запускает контейнер и передает его значения подключения времени выполнения в конфигурацию через `config()`. +Testcontainers назначает случайный порт хоста при каждом запуске, поэтому значения нельзя жестко зашивать — они объявляются как плейсхолдеры `${...}` во встроенной конфигурации +и заполняются из геттеров контейнера через `withSystemProperty(...)`. + +Поскольку `config()` выполняется **до** построения тестового графа, конфигурация готова до создания любого компонента. +По той же причине `KoraAppTestConfigModifier` несовместим с [внедрением через конструктор](#injection-rules): используйте внедрение через поле или параметр метода, как показано ниже. + +===! ":fontawesome-brands-java: `Java`" + + Добавьте зависимости [Testcontainers](https://java.testcontainers.org/) в `build.gradle`: + ```groovy + testImplementation "org.testcontainers:junit-jupiter:1.21.4" + testImplementation "org.testcontainers:postgresql:1.21.4" + ``` + + ```java + @Testcontainers + @KoraAppTest(Application.class) + class SomeIntegrationTests implements KoraAppTestConfigModifier { + + @Container + private static final PostgreSQLContainer POSTGRES = new PostgreSQLContainer<>("postgres:16"); + + @TestComponent + private SomeService service; + + @NotNull + @Override + public KoraConfigModification config() { + return KoraConfigModification.ofString(""" + db { + jdbcUrl = ${POSTGRES_JDBC_URL} + username = ${POSTGRES_USER} + password = ${POSTGRES_PASS} + poolName = "kora" + } + """) + .withSystemProperty("POSTGRES_JDBC_URL", POSTGRES.getJdbcUrl()) + .withSystemProperty("POSTGRES_USER", POSTGRES.getUsername()) + .withSystemProperty("POSTGRES_PASS", POSTGRES.getPassword()); + } + + @Test + void example() { + // interact with the service backed by the container + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + Добавьте зависимости [Testcontainers](https://java.testcontainers.org/) в `build.gradle.kts`: + ```groovy + testImplementation("org.testcontainers:junit-jupiter:1.21.4") + testImplementation("org.testcontainers:postgresql:1.21.4") + ``` + + ```kotlin + @Testcontainers + @KoraAppTest(Application::class) + class SomeIntegrationTests : KoraAppTestConfigModifier { + + companion object { + @Container + @JvmStatic + val POSTGRES = PostgreSQLContainer("postgres:16") + } + + @TestComponent + lateinit var service: SomeService + + override fun config(): KoraConfigModification { + return KoraConfigModification.ofString( + """ + db { + jdbcUrl = \${POSTGRES_JDBC_URL} + username = \${POSTGRES_USER} + password = \${POSTGRES_PASS} + poolName = "kora" + } + """.trimIndent() + ) + .withSystemProperty("POSTGRES_JDBC_URL", POSTGRES.jdbcUrl) + .withSystemProperty("POSTGRES_USER", POSTGRES.username) + .withSystemProperty("POSTGRES_PASS", POSTGRES.password) + } + + @Test + fun example() { + // interact with the service backed by the container + } + } + ``` + +Полный разбор — зависимости, тестовое `@KoraApp`, миграции и настройка репозитория — смотрите в руководстве [Интеграционное тестирование](../guides/testing-integration.md). + +## Изменение контейнера { #container-modification } + +Чтобы добавить, заменить или программно создать заглушки в контейнере приложения без аннотаций, реализуйте `KoraAppTestGraphModifier` +и верните `KoraGraphModification` из метода `graph()`. + +`KoraAppTestGraphModifier` нельзя использовать вместе с внедрением компонентов в конструктор тестового класса: +расширению нужно получить изменение графа до создания графа и внедрения компонентов. + +`KoraGraphModification` поддерживает следующие операции: + +- `addComponent(...)` — добавляет новый компонент в тестовый граф. +- `replaceComponent(...)` — заменяет существующий компонент, при этом его зависимости остаются в графе. +- `mockComponent(...)` — заменяет существующий компонент заглушкой и удаляет реальные зависимости заменяемого компонента из графа, если они больше не нужны тесту. + +`addComponent(...)` и `replaceComponent(...)` имеют перегрузки с `Function`, если новый компонент должен быть построен из уже инициализированных компонентов графа. +Для компонентов с `@Tag` используйте перегрузки с `List> tags`. ### Добавление { #adding } @@ -872,7 +1192,7 @@ agent: } ``` -В случае если требуется добавлять компоненты с использованием компонент из контейнера зависимостей, то это также доступно через другую сигнатуру метода: +В случае, когда требуется добавить компоненты с использованием реального компонента из графа, это также доступно через другую сигнатуру метода: ===! ":fontawesome-brands-java: `Java`" @@ -923,7 +1243,7 @@ agent: ### Замена { #replacement } -Пример замены компонента в контейнере, этот механизм также можно использовать для создания собственных заглушек: +Пример замены компонента в контейнере зависимостей, этот механизм также можно использовать для создания собственных заглушек: ===! ":fontawesome-brands-java: `Java`" @@ -962,7 +1282,7 @@ agent: } ``` -В случае если требуется добавлять компоненты с использованием компонент из контейнера зависимостей, то это также доступно через другую сигнатуру метода: +В случае, когда требуется заменить компоненты с использованием реального компонента из графа, это также доступно через другую сигнатуру метода: ===! ":fontawesome-brands-java: `Java`" @@ -1011,9 +1331,56 @@ agent: } ``` +### Программная заглушка { #programmatic-mock } + +Если компонент нужно заменить именно как заглушку, используйте `mockComponent(...)`. +В отличие от `replaceComponent(...)`, этот метод сообщает расширению, что реальные зависимости заменяемого компонента не нужны и могут быть исключены из тестового графа. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(value = Application.class) + class SomeTests implements KoraAppTestGraphModifier { + + @Override + public @Nonnull KoraGraphModification graph() { + return KoraGraphModification.create() + .mockComponent(TypeRef.of(Supplier.class, String.class), () -> Mockito.mock(Supplier.class)); + } + + @Test + void example(@TestComponent Supplier supplier) { + Mockito.when(supplier.get()).thenReturn("?"); + + assertEquals("?", supplier.get()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(value = Application::class) + class SomeTests : KoraAppTestGraphModifier { + + override fun graph(): KoraGraphModification { + return KoraGraphModification.create() + .mockComponent(TypeRef.of(Supplier::class.java, String::class.java), Supplier { mockk>() }) + } + + @Test + fun example(@TestComponent supplier: Supplier) { + every { supplier.get() } returns "?" + + assertEquals("?", supplier.get()) + } + } + ``` + ## Инициализация { #initialization } -В случае если требуется инициализировать контейнер один раз в рамках всего тестового класса, следует проаннотировать тестовый класс с помощью `@TestInstance(TestInstance.Lifecycle.PER_CLASS)`: +По умолчанию `JUnit 5` использует `TestInstance.Lifecycle.PER_METHOD`, поэтому Kora создает и очищает тестовый граф для каждого тестового метода. +Если контейнер должен инициализироваться один раз для всего тестового класса, пометьте тестовый класс аннотацией `@TestInstance(TestInstance.Lifecycle.PER_CLASS)`: ===! ":fontawesome-brands-java: `Java`" @@ -1035,4 +1402,12 @@ agent: } ``` -Поведение по умолчанию - инициализация контейнера с нуля для каждого тестового метода. +С `PER_CLASS` один экземпляр графа используется всеми тестовыми методами класса, а очистка выполняется после завершения всего класса. +Это ускоряет тяжелые интеграционные тесты, но с изменяемым состоянием компонентов и заглушек нужно обращаться аккуратнее. + +Ограничения жизненного цикла: + +- Когда компоненты внедряются в конструктор, `@TestComponent` или заглушки нельзя также внедрять в параметры тестового метода. +- Когда компоненты внедряются в конструктор, нельзя использовать `KoraAppTestConfigModifier` и `KoraAppTestGraphModifier`. +- В режиме `PER_CLASS` `@Mock` / `@MockK` нельзя внедрять в параметры тестового метода; используйте поля или конструктор. +- Для классов `@Nested` нельзя использовать внедрение в поля внутреннего класса, если внешний тестовый класс работает в режиме `PER_CLASS`; используйте параметры метода или отдельный жизненный цикл для вложенного класса. diff --git a/mkdocs/docs/ru/documentation/kafka.md b/mkdocs/docs/ru/documentation/kafka.md index 2dbe35c..6bba916 100644 --- a/mkdocs/docs/ru/documentation/kafka.md +++ b/mkdocs/docs/ru/documentation/kafka.md @@ -4,7 +4,14 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora Kafka consumers and producers, listener and publisher annotations, configuration, serialization, error handling, rebalance events, transactions, and telemetry tags; key triggers include @KafkaListener, @KafkaPublisher, @Topic, @Json, @Tag, KafkaModule, KafkaConsumer, KafkaProducer, KafkaSkipRecordException." --- -Модуль для создания декларативных [Apache Kafka](https://kafka.apache.org/) `Consumer` и `Producer` с помощью аннотаций. +Модуль `Kafka` предоставляет декларативную интеграцию с [Apache Kafka](https://kafka.apache.org/): чтение сообщений через +`@KafkaListener`, отправку сообщений через `@KafkaPublisher`, работу с сериализацией, десериализацией, транзакциями, +ошибками обработки и телеметрией. + +`Apache Kafka` — это распределенная платформа потоковой передачи событий. Приложения записывают события в `topic`, +а другие приложения читают их через `consumer group` или напрямую назначенные разделы. Kora создает нужные `Consumer` +и `Producer` во время компиляции, связывает их с графом зависимостей и позволяет описывать большую часть контракта +через сигнатуры методов. Если нужен пошаговый разбор перед справочным описанием, смотрите [Kafka](../guides/messaging-kafka.md). @@ -38,7 +45,9 @@ agent: ## Потребитель { #consumer } -Описания работы с [Kafka Consumer](https://docs.confluent.io/platform/current/clients/consumer.html) +`Consumer` читает записи из `topic` и передает их в метод приложения. Kora сама создает контейнер потребителя, +вызывает `poll()`, применяет десериализацию, вызывает обработчик и выполняет фиксацию сдвига, если сигнатура метода +не требует ручного управления `Consumer`. Для создания `Consumer` требуется использовать аннотацию `@KafkaListener` над методом: @@ -48,7 +57,7 @@ agent: @Component final class SomeConsumer { - @KafkaListener("kafka.someConsume") + @KafkaListener("kafka.someConsumer") void process(String key, String value) { // my code } @@ -61,17 +70,17 @@ agent: @Component class SomeConsumer { - @KafkaListener("kafka.someConsume") + @KafkaListener("kafka.someConsumer") fun process(key: String, value: String) { // my code } } ``` -Параметр аннотации `@KafkaListener` указывает на путь к конфигурации `Consumer`'а. +Параметр аннотации `@KafkaListener` указывает на путь к конфигурации `Consumer`. -В случае, если нужно разное поведение для разных топиков, существует возможность создавать несколько подобных контейнеров, -каждый со своим индивидуальным конфигом. Выглядит это так: +В случае, если нужно разное поведение для разных `topic`, существует возможность создавать несколько подобных контейнеров, +каждый со своей конфигурацией. Выглядит это так: ===! ":fontawesome-brands-java: `Java`" @@ -109,13 +118,14 @@ agent: } ``` -Значение в аннотации указывает, из какой части файла конфигурации нужно брать настройки. В том, что касается получения конфигурации — работает аналогично `@ConfigSource` +Значение в аннотации указывает, из какой части файла конфигурации нужно брать настройки. +По смыслу это похоже на `@ConfigSource`: путь в аннотации выбирает ветку конфигурации для конкретного контейнера. -### Конфигурация { #configuration } +### Конфигурация { #config-consumer } Конфигурация описывает настройки конкретного `@KafkaListener` и ниже указан пример для конфигурации по пути `kafka.someConsumer`. -Пример полной конфигурации, описанной в классе `KafkaListenerConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации: ===! ":material-code-json: `Hocon`" @@ -123,113 +133,191 @@ agent: kafka { someConsumer { topics = ["topic1", "topic2"] //(1)! - topicsPattern = "topic*" //(2)! - allowEmptyRecords = false //(3)! - offset = "latest" //(4)! - pollTimeout = "5s" //(5)! - backoffTimeout = "15s" //(6)! - partitionRefreshInterval = "1m" //(7)! - threads = 1 //(8)! - shutdownWait = "30s" //(9)! - driverProperties { //(10)! + offset = "latest" //(2)! + pollTimeout = "5s" //(3)! + threads = 1 //(4)! + driverProperties { //(5)! "bootstrap.servers": "localhost:9093" "group.id": "my-group-id" } - telemetry { - logging { - enabled = false //(11)! - } - metrics { - enabled = true //(12)! - slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(12)! - } - tracing { - enabled = true //(14)! - } - } } } ``` - 1. Указываются топики на которые будет подписан Consumer (**обязательный** либо указывается `topicsPattern`) - 2. Указываются паттерн топиков на которые будет подписан Consumer (**обязательный** либо указывается `topics`) - 3. Обрабатывать ли пустые записи в случае если сигнатура принимает `ConsumerRecords` - 4. Работает только если не указан `group.id`. Определяет стратегнию какую позицию в топике должен использовать Consumer. Допустимые значение: - 1. `earliest` - самый ранний доступный offset - 2. `latest` - последний доступный offset - 3. Строка в формате `Duration` (например `5m`) - сдвиг на определённое время назад - 5. Максиимальное время ожидания сообщений из топика в рамках одного вызова - 6. Максимальное время ожидания между неожиданными исключениями во время обработки - 7. Временной интервал в рамках которого требуется делать обновление партиций в случае `assign` метода - 8. Количество потоков на которых будет запущен потребитель для параллельной обработки (если будет равен 0 то ни один потребитель не будет запущен вообще) - 9. Время ожидания обработки перед выключением потребителя в случае [штатного завершения](container.md#component-lifecycle) - 10. *Properties* из официального клиента кафки, документацию по ним можно посмотреть по [ссылке](https://kafka.apache.org/documentation/#consumerconfigs) (**обязательный**) - 11. Включает логгирование модуля (по умолчанию `false`) - 12. Включает метрики модуля (по умолчанию `true`) - 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 14. Включает трассировку модуля (по умолчанию `true`) + 1. Список `topic` для подписки (`обязательно` указать `topics` или `topicsPattern`) + 2. Начальная позиция чтения (по умолчанию: `latest`). Допустимые значения: `earliest`, `latest`, или сдвиг времени (например `5m`) + 3. Максимальное время ожидания сообщений (по умолчанию: `5s`) + 4. Количество потоков для потребителя (по умолчанию: `1`) + 5. `Properties` официального `Kafka Consumer` (`обязательные`, по умолчанию не указано) === ":simple-yaml: `YAML`" ```yaml kafka: someConsumer: - topics: #(1)! + topics: - "topic1" - - "topic2" - topicsPattern: "topic*" #(2)! - allowEmptyRecords: false #(3)! - offset: "latest" #(4)! - pollTimeout: "5s" #(5)! - backoffTimeout: "15s" #(6)! - partitionRefreshInterval: "1m" #(7)! - threads: 1 #(8)! - shutdownWait: "30s" #(9)! - driverProperties: #(10)! - bootstrap.servers: "localhost:9093" - group.id: "my-group-id" - telemetry: - logging: - enabled: false #(11)! - metrics: - enabled: true #(12)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! - tags: #(14)! - key1: value1 - key2: value2 - tracing: - enabled: true #(15)! - attributes: #(16)! - key1: value1 - key2: value2 - ``` - - 1. Указываются топики на которые будет подписан Consumer (**обязательный** либо указывается `topicsPattern`) - 2. Указываются паттерн топиков на которые будет подписан Consumer (**обязательный** либо указывается `topics`) - 3. Обрабатывать ли пустые записи в случае если сигнатура принимает `ConsumerRecords` - 4. Работает только если не указан `group.id`. Определяет стратегнию какую позицию в топике должен использовать Consumer. Допустимые значение: - 1. `earliest` - самый ранний доступный offset - 2. `latest` - последний доступный offset - 3. Строка в формате `Duration` (например `5m`) - сдвиг на определённое время назад - 5. Максиимальное время ожидания сообщений из топика в рамках одного вызова - 6. Максимальное время ожидания между неожиданными исключениями во время обработки - 7. Временной интервал в рамках которого требуется делать обновление партиций в случае `assign` метода - 8. Количество потоков на которых будет запущен потребитель для параллельной обработки (если будет равен 0 то ни один потребитель не будет запущен вообще) - 9. Время ожидания обработки перед выключением потребителя в случае [штатного завершения](container.md#component-lifecycle) - 10. *Properties* из официального клиента кафки, документацию по ним можно посмотреть по [ссылке](https://kafka.apache.org/documentation/#consumerconfigs) (**обязательный**) - 11. Включает логгирование модуля (по умолчанию `false`) - 12. Включает метрики модуля (по умолчанию `true`) - 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 14. Настройка тегов для метрик (опционально) - 15. Включает трассировку модуля (по умолчанию `true`) - 16. Настройка атрибутов для трассировки (опционально) + - "topic2" #(1)! + offset: "latest" #(2)! + pollTimeout: "5s" #(3)! + threads: 1 #(4)! + driverProperties: #(5)! + "bootstrap.servers": "localhost:9093" + "group.id": "my-group-id" + ``` + + 1. Список `topic` для подписки (`обязательно` указать `topics` или `topicsPattern`) + 2. Начальная позиция чтения (по умолчанию: `latest`). Допустимые значения: `earliest`, `latest`, или сдвиг времени (например `5m`) + 3. Максимальное время ожидания сообщений (по умолчанию: `5s`) + 4. Количество потоков для потребителя (по умолчанию: `1`) + 5. `Properties` официального `Kafka Consumer` (`обязательные`, по умолчанию не указано) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `KafkaListenerConfig` (указаны примеры значений или значения по умолчанию): + + В реальной конфигурации обычно указывается либо `topics`, либо `topicsPattern`. + + ===! ":material-code-json: `Hocon`" + + ```javascript + kafka { + someConsumer { + topics = ["topic1", "topic2"] //(1)! + topicsPattern = "topic*" //(2)! + partitions = ["0", "1"] //(3)! + allowEmptyRecords = false //(4)! + offset = "latest" //(5)! + pollTimeout = "5s" //(6)! + backoffTimeout = "15s" //(7)! + partitionRefreshInterval = "1m" //(8)! + threads = 1 //(9)! + shutdownWait = "30s" //(10)! + driverProperties { //(11)! + "bootstrap.servers": "localhost:9093" + "group.id": "my-group-id" + } + telemetry { + logging { + enabled = false //(12)! + } + metrics { + enabled = true //(13)! + slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(14)! + tags = { // (15)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(16)! + attributes = { // (17)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + ``` + + 1. Список `topic`, на которые будет подписан `Consumer` (по умолчанию не указано, необязательно; требуется указать `topics` или `topicsPattern`) + 2. Шаблон `topic`, на которые будет подписан `Consumer` (по умолчанию не указано, необязательно; требуется указать `topics` или `topicsPattern`) + 3. Список разделов, который используется только при формировании имени потребителя, если не указаны `group.id`, `topics` и `topicsPattern`; назначением разделов управляет контейнер `assign` (по умолчанию не указано, необязательно) + Если `false` и `ConsumerRecords` пустой (нет сообщений), метод потребителя не будет вызван. + Если `true`, метод будет вызван с пустым `ConsumerRecords` (полезно для периодических проверок). + 4. Обрабатывать ли пустые пачки записей, если сигнатура принимает `ConsumerRecords` (по умолчанию: `false`) + 5. Начальная позиция чтения для стратегии `assign`, когда не указан `group.id` (по умолчанию: `latest`). Допустимые значения: + 1. `earliest` - самый ранний доступный `offset` + 2. `latest` - последний доступный `offset` + 3. строка в формате `Duration`, например `5m`, - сдвиг на указанное время назад + Формат: число + единица (ms, s, m, h, d). Примеры: `5m` = 5 минут назад, `1h` = 1 час назад. + 6. Максимальное время ожидания сообщений из `topic` в рамках одного вызова `poll()` (по умолчанию: `5s`) + 7. Начальное время ожидания между неожиданными исключениями во время обработки; при повторных ошибках задержка увеличивается до `60s` (по умолчанию: `15s`) + Если потребитель выбрасывает непредусмотренное исключение (не `KafkaSkipRecordException`), + Kora перезапустит потребителя с задержкой `backoffTimeout` для предотвращения циклических ошибок. + 8. Период обновления списка разделов для стратегии `assign` (по умолчанию: `1m`) + 9. Количество потоков, на которых будет запущен потребитель; если указать `0`, потребитель не будет запущен (по умолчанию: `1`) + 10. Время ожидания обработки перед выключением потребителя при [штатном завершении](container.md#component-lifecycle) (по умолчанию: `30s`) + 11. `Properties` официального `Kafka Consumer`; документация по ним доступна в [Apache Kafka Consumer Configs](https://kafka.apache.org/documentation/#consumerconfigs) (`обязательная`, по умолчанию не указано) + 12. Включает логирование модуля (по умолчанию: `false`) + 13. Включает метрики модуля (по умолчанию: `true`) + 14. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Настройка тегов для метрик (по умолчанию: `{}`) + 16. Включает трассировку модуля (по умолчанию: `true`) + 17. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + kafka: + someConsumer: + topics: #(1)! + - "topic1" + - "topic2" + topicsPattern: "topic*" #(2)! + partitions: #(3)! + - "0" + - "1" + allowEmptyRecords: false #(4)! + offset: "latest" #(5)! + pollTimeout: "5s" #(6)! + backoffTimeout: "15s" #(7)! + partitionRefreshInterval: "1m" #(8)! + threads: 1 #(9)! + shutdownWait: "30s" #(10)! + driverProperties: #(11)! + bootstrap.servers: "localhost:9093" + group.id: "my-group-id" + telemetry: + logging: + enabled: false #(12)! + metrics: + enabled: true #(13)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(14)! + tags: #(15)! + key1: value1 + key2: value2 + tracing: + enabled: true #(16)! + attributes: #(17)! + key1: value1 + key2: value2 + ``` + + 1. Список `topic`, на которые будет подписан `Consumer` (по умолчанию не указано, необязательно; требуется указать `topics` или `topicsPattern`) + 2. Шаблон `topic`, на которые будет подписан `Consumer` (по умолчанию не указано, необязательно; требуется указать `topics` или `topicsPattern`) + 3. Список разделов, который используется только при формировании имени потребителя, если не указаны `group.id`, `topics` и `topicsPattern`; назначением разделов управляет контейнер `assign` (по умолчанию не указано, необязательно) + Если `false` и `ConsumerRecords` пустой (нет сообщений), метод потребителя не будет вызван. + Если `true`, метод будет вызван с пустым `ConsumerRecords` (полезно для периодических проверок). + 4. Обрабатывать ли пустые пачки записей, если сигнатура принимает `ConsumerRecords` (по умолчанию: `false`) + 5. Начальная позиция чтения для стратегии `assign`, когда не указан `group.id` (по умолчанию: `latest`). Допустимые значения: + 1. `earliest` - самый ранний доступный `offset` + 2. `latest` - последний доступный `offset` + 3. строка в формате `Duration`, например `5m`, - сдвиг на указанное время назад + Формат: число + единица (ms, s, m, h, d). Примеры: `5m` = 5 минут назад, `1h` = 1 час назад. + 6. Максимальное время ожидания сообщений из `topic` в рамках одного вызова `poll()` (по умолчанию: `5s`) + 7. Начальное время ожидания между неожиданными исключениями во время обработки; при повторных ошибках задержка увеличивается до `60s` (по умолчанию: `15s`) + Если потребитель выбрасывает непредусмотренное исключение (не `KafkaSkipRecordException`), + Kora перезапустит потребителя с задержкой `backoffTimeout` для предотвращения циклических ошибок. + 8. Период обновления списка разделов для стратегии `assign` (по умолчанию: `1m`) + 9. Количество потоков, на которых будет запущен потребитель; если указать `0`, потребитель не будет запущен (по умолчанию: `1`) + 10. Время ожидания обработки перед выключением потребителя при [штатном завершении](container.md#component-lifecycle) (по умолчанию: `30s`) + 11. `Properties` официального `Kafka Consumer`; документация по ним доступна в [Apache Kafka Consumer Configs](https://kafka.apache.org/documentation/#consumerconfigs) (`обязательная`, по умолчанию не указано) + 12. Включает логирование модуля (по умолчанию: `false`) + 13. Включает метрики модуля (по умолчанию: `true`) + 14. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 15. Настройка тегов для метрик (по умолчанию: `{}`) + 16. Включает трассировку модуля (по умолчанию: `true`) + 17. Настройка атрибутов для трассировки (по умолчанию: `{}`) Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#kafka). ### Стратегия подключения { #consume-strategy } -`subscribe` стратегия подразумевает использование [group.id](https://medium.com/@kirill.sereda/kafka-%D0%B4%D0%BB%D1%8F-%D1%81%D0%B0%D0%BC%D1%8B%D1%85-%D0%BC%D0%B0%D0%BB%D0%B5%D0%BD%D1%8C%D0%BA%D0%B8%D1%85-f42864cb1bfb), -чтобы объединить исполнителей в группы и они не дублировали вычитывание записей из своей очереди в рамках нескольких экземпляров приложений. +Стратегия `subscribe` используется, когда в `driverProperties` указан `group.id`. +В этом режиме экземпляры приложения входят в одну `consumer group`, а `Kafka` распределяет разделы между ними так, +чтобы разные экземпляры не обрабатывали одни и те же записи одновременно. Пример конфигурации `subscribe` стратегии: @@ -238,7 +326,7 @@ agent: ```javascript kafka { someConsumer { - topics: "first" + topics = ["first"] driverProperties { "group.id": "my-group-id" "bootstrap.servers": "localhost:9093" @@ -252,16 +340,20 @@ agent: ```yaml kafka: someConsumer: - topics: "first" + topics: + - "first" driverProperties: "group.id": "my-group-id" "bootstrap.servers": "localhost:9093" ``` -`assign` стратегия подключения подразумевает, что каждый экземпляр приложения читает сообщения из топика одновременно с другими, -то есть сообщения дублируются между всеми экземплярами приложения в рамках топика. -Для использования такой стратегии надо просто **не указывать** `group.id` в конфигурации потребителя, -но в такой стратегии можно указать одновременно лишь 1 топик. +Стратегия `assign` используется, когда в `driverProperties` не указан `group.id`. +В этом режиме каждый экземпляр приложения сам назначает себе разделы выбранного `topic`, поэтому сообщения могут читаться +каждым экземпляром приложения независимо. В такой стратегии можно указать только один `topic`, а начальная позиция чтения +управляется параметром `offset`. + +Такая стратегия полезна, когда одно и то же сообщение должны получить все реплики приложения: например, для сброса локального +кеша, обновления справочников в памяти или доставки служебного события каждому экземпляру приложения. Пример конфигурации `assign` стратегии: @@ -270,7 +362,7 @@ agent: ```javascript kafka { someConsumer { - topics: "first" + topics = ["first"] driverProperties { "bootstrap.servers": "localhost:9093" } @@ -283,14 +375,17 @@ agent: ```yaml kafka: someConsumer: - topics: "first" + topics: + - "first" driverProperties: "bootstrap.servers": "localhost:9093" ``` ### Десериализация { #deserialization } -`Deserializer` - используется для десериализации ключей и значений `ConsumerRecord`. +`Deserializer` используется для десериализации ключей и значений `ConsumerRecord`. +Kora предоставляет компоненты `Deserializer` для базовых типов: `String`, `UUID`, `byte[]`, `Bytes`, `ByteBuffer`, +`Double`, `Float`, `Integer`, `Long`, `Short` и `Void`. Для более точной настройки `Deserializer` поддерживаются теги. Теги можно установить на параметре-ключе, параметре-значении, а так же на параметрах типа `ConsumerRecord` и `ConsumerRecords`. @@ -331,7 +426,8 @@ agent: } ``` -В случае если требуется десериализация из `Json`, то можно использовать тег `@Json`: +Если требуется десериализация из `JSON`, можно использовать тег `@Json`. +В таком случае Kora использует `JsonReader` и `JsonKafkaDeserializer` из модуля [JSON](json.md): ===! ":fontawesome-brands-java: `Java`" @@ -375,9 +471,147 @@ agent: } ``` -Для потребителей, не использующих ключ, по умолчанию используется `Deserializer` т.к. он просто возвращает не обработанные байты. +Для потребителей, не использующих ключ, по умолчанию используется `Deserializer`, так как он возвращает необработанные байты. + +### Кастомный десериализатор { #custom-deserializer } + +В случае если требуется кастомная десериализация, можно реализовать собственный `Deserializer`. + +**Вариант 1: Десериализатор по умолчанию для типа** + +Если предоставить `Deserializer` как компонент без тега, он будет использоваться для всех потребителей этого типа: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public static class MyEventDeserializer implements Deserializer { + + private final JsonReader reader; + + public MyEventDeserializer(JsonReader reader) { + this.reader = reader; + } + + @Override + public MyEvent deserialize(String topic, byte[] data) { + try { + return reader.read(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + @Component + final class SomeConsumer { + + @KafkaListener("kafka.someConsumer") + void process(MyEvent value) { // Используется MyEventDeserializer + // обработка события + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyEventDeserializer( + private val reader: JsonReader + ) : Deserializer { + + override fun deserialize(topic: String, data: ByteArray): MyEvent { + return try { + reader.read(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + @Component + class SomeConsumer { + + @KafkaListener("kafka.someConsumer") + fun process(value: MyEvent) { // Используется MyEventDeserializer + // обработка события + } + } + ``` + +**Вариант 2: Точечный десериализатор для конкретного потребителя** + +Если требуется использовать разную десериализацию для разных потребителей одного типа, можно использовать теги: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + final class SomeConsumer { + + @Json + public record MyEvent(String username, int code) {} + + @Tag(MyEvent.class) + @Component + public static class MyDeserializer implements Deserializer { + + private final JsonReader reader; + + public MyDeserializer(JsonReader reader) { + this.reader = reader; + } + + @Override + public MyEvent deserialize(String topic, byte[] data) { + try { + return reader.read(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + @KafkaListener("kafka.someConsumer") + void process(@Tag(MyEvent.class) MyEvent value) { + // обработка события + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeConsumer { + + @Json + data class MyEvent(val username: String, val code: Int) + + @Tag(MyEvent::class) + @Component + class MyDeserializer( + private val reader: JsonReader + ) : Deserializer { + + override fun deserialize(topic: String, data: ByteArray): MyEvent { + return try { + reader.read(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + @KafkaListener("kafka.someConsumer") + fun process(@Tag(MyEvent::class) value: MyEvent) { + // обработка события + } + } + ``` -### Обработка исключений { #exception-handling } +### Обработка исключений { #exception-handling-consumer } Если метод помеченный `@KafkaListener` выбросит исключение, то Consumer будет перезапущен, потому что нет общего решения, как реагировать на это и разработчик **должен** сам решить как эту ситуацию обрабатывать. @@ -448,7 +682,49 @@ agent: * `ru.tinkoff.kora.kafka.common.exceptions.RecordKeyDeserializationException` * `ru.tinkoff.kora.kafka.common.exceptions.RecordValueDeserializationException` -Из этих исключений можно получить сырой `ConsumerRecord`. +Из этих исключений можно получить сырой `ConsumerRecord` через метод `getRecord()`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaListener("kafka.someConsumer") + void process(ConsumerRecord record) { + try { + var key = record.key(); + var value = record.value(); + // обработка + } catch (RecordKeyDeserializationException e) { + ConsumerRecord rawRecord = e.getRecord(); + // Логирование сырых данных для отладки + logger.error("Failed to deserialize key for record: {}", rawRecord); + } catch (RecordValueDeserializationException e) { + ConsumerRecord rawRecord = e.getRecord(); + // Логирование сырых данных для отладки + logger.error("Failed to deserialize value for record: {}", rawRecord); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaListener("kafka.someConsumer") + fun process(record: ConsumerRecord) { + try { + val key = record.key() + val value = record.value() + // обработка + } catch (e: RecordKeyDeserializationException) { + val rawRecord = e.record + // Логирование сырых данных для отладки + logger.error("Failed to deserialize key for record: {}", rawRecord) + } catch (e: RecordValueDeserializationException) { + val rawRecord = e.record + // Логирование сырых данных для отладки + logger.error("Failed to deserialize value for record: {}", rawRecord) + } + } + ``` Если вы используете сигнатуру с распакованными `key`/`value`/`headers`, то можно добавить последним аргументом `Exception`, `Throwable`, `RecordKeyDeserializationException` или `RecordValueDeserializationException`, @@ -524,7 +800,7 @@ agent: ### События ребалансировки { #rebalance-events } -Можно слушать и реагировать на события ребалансировки с помощью свой реализации интерфейса `ConsumerAwareRebalanceListener`, +Можно слушать и реагировать на события ребалансировки с помощью своей реализации интерфейса `ConsumerAwareRebalanceListener`, его следует предоставить как компонент по тегу потребителя: ===! ":fontawesome-brands-java: `Java`" @@ -536,12 +812,21 @@ agent: @Override public void onPartitionsRevoked(Consumer consumer, Collection partitions) { - + // Вызывается когда партиции были отобраны у потребителя (перед коммитом offset'ов) + // Можно использовать для сохранения состояния или коммита offset'ов } @Override public void onPartitionsAssigned(Consumer consumer, Collection partitions) { + // Вызывается когда партиции были назначены потребителю + // Можно использовать для инициализации состояния + } + @Override + public void onPartitionsLost(Consumer consumer, Collection partitions) { + // Вызывается когда партиции были потеряны (например, при ребалансировке группы) + // В отличие от onPartitionsRevoked, коммит offset'ов уже не возможен + // По умолчанию вызывает onPartitionsRevoked } } ``` @@ -554,11 +839,19 @@ agent: class SomeListener : ConsumerAwareRebalanceListener { override fun onPartitionsRevoked(consumer: Consumer<*, *>, partitions: Collection) { - + // Вызывается когда партиции были отобраны у потребителя (перед коммитом offset'ов) + // Можно использовать для сохранения состояния или коммита offset'ов } - + override fun onPartitionsAssigned(consumer: Consumer<*, *>, partitions: Collection) { - + // Вызывается когда партиции были назначены потребителю + // Можно использовать для инициализации состояния + } + + override fun onPartitionsLost(consumer: Consumer<*, *>, partitions: Collection) { + // Вызывается когда партиции были потеряны (например, при ребалансировке группы) + // В отличие от onPartitionsRevoked, коммит offset'ов уже не возможен + // По умолчанию вызывает onPartitionsRevoked } } ``` @@ -586,14 +879,19 @@ public interface BaseKafkaRecordsHandler { ### Сигнатуры { #signatures } -Доступные сигнатуры для методов Kafka потребителя из коробки, где под `K` подразумевается тип ключа и под `V` тип значения сообщения. +Доступные сигнатуры для методов `Kafka Consumer` из коробки, где под `K` подразумевается тип ключа, а под `V` тип значения сообщения. +Генератор поддерживает три семейства сигнатур: отдельные `key`/`value`, один `ConsumerRecord` или всю пачку `ConsumerRecords`. +Эти семейства нельзя смешивать между собой в одном методе. + +#### Ключ и значение { #key-value-signature } -Работа потребителя начинается с вызова `poll()` для пачки `ConsumerRecords`, каждое событие по аргументам передается в потребителя. -Потребитель может принимать `value` (обязательный), `key` (опциональный), `Headers` (опциональный) аргумент от `ConsumerRecord`, -после обработки **каждого** события вызывается `commitSync()`. +Сигнатура с отдельными аргументами принимает `value`, необязательный `key`, необязательные `Headers`, необязательный `Consumer` и необязательные ошибки чтения. +Один пользовательский аргумент считается `value`, два пользовательских аргумента считаются `key` и `value` именно в таком порядке. +Если `key` не указан, тип ключа для десериализации считается `byte[]`. -Учитывайте что при использовании такой сигнатуры в случае ошибки чтения ключа/значения потребитель уйдет -в бесконечный цикл повторных вычитываний без фиксации текущего сдвига по топику. +Для обработки ошибки чтения можно добавить `Exception`, `RecordKeyDeserializationException` или `RecordValueDeserializationException`. +Если такой аргумент есть, Kora передаст в него ошибку чтения, а значение соответствующего `key` или `value` будет `null`. +Без такого аргумента ошибка чтения будет выброшена из обработчика, и событие будет вычитано повторно без фиксации текущего сдвига. ===! ":fontawesome-brands-java: `Java`" @@ -613,11 +911,6 @@ public interface BaseKafkaRecordsHandler { } ``` -Рекомендуется дополнительно использовать в потребителе аргумент `Exception` (опциональный), -который будет сигнализировать о случаях ошибки чтения ключа/значения. - -Учитывайте что в таком случае все значения становятся опциональными, т.к. может вернуться либо результат, либо ошибка. - ===! ":fontawesome-brands-java: `Java`" ```java @@ -644,12 +937,14 @@ public interface BaseKafkaRecordsHandler { } ``` -Возможно также принимать аргументом `ConsumerRecord`, в таком случае ошибка чтения ключа/значения -может быть выброшена при обращении к методам `key()`/`value()` у события. +#### Событие целиком { #record-signature } + +Сигнатура с `ConsumerRecord` принимает одно событие целиком, необязательный `Consumer` и необязательные ошибки чтения: +`Exception`, `RecordKeyDeserializationException` или `RecordValueDeserializationException`. +`Headers`, отдельные `key`/`value` и контекст телеметрии в такой сигнатуре не поддерживаются. -Работа потребителя начинается с вызова `poll()` для пачки `ConsumerRecords` и передаёт каждое событие `ConsumerRecord` по отдельности в потребителя. -Потребитель принимает `ConsumerRecord` и `KafkaConsumerRecordsTelemetryContext`/`KafkaConsumerRecordTelemetryContext` (опционально) -и после обработки **каждого** события вызывается `commitSync()`: +Если аргументы ошибок не указаны, ошибка чтения может быть выброшена при обращении к `record.key()` или `record.value()`. +Если аргументы ошибок указаны, Kora заранее вызовет `key()` и/или `value()`, поймает ошибку чтения и передаст ее в метод. ===! ":fontawesome-brands-java: `Java`" @@ -687,17 +982,111 @@ public interface BaseKafkaRecordsHandler { } ``` -Возможно также принимать аргументом `ConsumerRecords` и обрабатывать всю пачку самостоятельно. +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaListener("kafka.someConsumer") + void process(ConsumerRecord record, + @Nullable RecordKeyDeserializationException keyException, + @Nullable RecordValueDeserializationException valueException) { + if (keyException != null || valueException != null) { + // do deserialization handling work + return; + } + + var key = record.key(); + var value = record.value(); + // some value handling work + } + ``` -Вызывается `poll()` для пачки `ConsumerRecords` и передаёт всю пачку событий в потребитель. -Потребитель принимает `ConsumerRecords` и `KafkaConsumerRecordsTelemetryContext`/`KafkaConsumerRecordTelemetryContext` (опционально) -и после обработки **всей пачки** событий вызывается `commitSync()`: +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaListener("kafka.someConsumer") + fun process( + record: ConsumerRecord, + keyException: RecordKeyDeserializationException?, + valueException: RecordValueDeserializationException?, + ) { + if (keyException != null || valueException != null) { + // do deserialization handling work + return + } + + val key = record.key() + val value = record.value() + // some value handling work + } + ``` + +#### Пачка событий { #records-signature } + +Сигнатура с `ConsumerRecords` принимает всю пачку событий из одного `poll()`. +Вместе с ней можно указать только `Consumer` и `KafkaConsumerRecordsTelemetryContext`. +Отдельные `key`/`value`, `Headers` и аргументы ошибок чтения в такой сигнатуре не поддерживаются; ошибки чтения нужно обрабатывать при обходе событий. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaListener("kafka.someConsumer") + void process(ConsumerRecords records, + KafkaConsumerTelemetry.KafkaConsumerRecordsTelemetryContext ctx) { + for (ConsumerRecord record : records) { + var telemetryContext = ctx.get(record); + // обработка события + telemetryContext.close(null); // закрыть с результатом (null = успех) + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaListener("kafka.someConsumer") + fun process(records: ConsumerRecords, + ctx: KafkaConsumerTelemetry.KafkaConsumerRecordsTelemetryContext) { + for (record in records) { + val telemetryContext = ctx.get(record) + // обработка события + telemetryContext.close(null) // закрыть с результатом (null = успех) + } + } + ``` + +`KafkaConsumerRecordsTelemetryContext` позволяет вручную управлять телеметрией для каждого сообщения. +Используйте `ctx.get(record)` для получения контекста, и `close(exception)` для закрытия с результатом. +Если не передавать контекст явно, Kora автоматически закроет его после обработки. + +Для обработки единичных событий с ручным управлением телеметрией используйте `KafkaConsumerRecordTelemetryContext`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaListener("kafka.someConsumer") + void process(ConsumerRecord record, + KafkaConsumerTelemetry.KafkaConsumerRecordTelemetryContext ctx) { + // обработка события + ctx.close(null); // закрыть с результатом + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaListener("kafka.someConsumer") + fun process(record: ConsumerRecord, + ctx: KafkaConsumerTelemetry.KafkaConsumerRecordTelemetryContext) { + // обработка события + ctx.close(null) // закрыть с результатом + } + ``` ===! ":fontawesome-brands-java: `Java`" ```java @KafkaListener("kafka.someConsumer") - void process(ConsumerRecords record) { + void process(ConsumerRecords records) { for (var record : records) { try { var key = record.key(); @@ -717,7 +1106,7 @@ public interface BaseKafkaRecordsHandler { ```kotlin @KafkaListener("kafka.someConsumer") - fun process(record: ConsumerRecords) { + fun process(records: ConsumerRecords) { for (record in records) { try { val key = record.key() @@ -733,10 +1122,17 @@ public interface BaseKafkaRecordsHandler { } ``` -Можно также контролировать самостоятельно, как и когда вызывать фиксацию сдвига по топику у потребителя. -Такая сигнатура доступна как для одного события, так и для пачки событий. +#### Фиксация сдвига { #manual-commit } -В случае если аргументом принимается `Consumer`, то `commit` всегда нужно **вызывать самостоятельно**. +Если в сигнатуре нет аргумента `Consumer`, Kora фиксирует сдвиг самостоятельно: после каждого события для сигнатур `key`/`value` и `ConsumerRecord`, либо после всей пачки для `ConsumerRecords`. +Для этого вызывается `commitSync()`. + +Если в сигнатуре есть аргумент `Consumer`, автоматическая фиксация сдвига отключается, и обработчик полностью отвечает за вызов `commitSync()` или `commitAsync()`. +Такой режим нужен, когда нужно зафиксировать сдвиг только после внешней операции, зафиксировать несколько событий вместе или вручную управлять позицией чтения. + +В режиме `subscribe` ручной `commit` фиксирует сдвиг внутри группы потребителей. +В режиме `assign` нет распределения разделов через группу потребителей, поэтому обычно важнее вручную управлять позицией через `seek()`, `pause()` и `resume()`, а не рассчитывать на групповую фиксацию сдвига. +Если обработчик завершился с ошибкой до ручной фиксации, событие или пачка будут вычитаны повторно согласно текущей позиции потребителя. ===! ":fontawesome-brands-java: `Java`" @@ -753,7 +1149,7 @@ public interface BaseKafkaRecordsHandler { } catch (RecordValueDeserializationException e) { // do deserialization handling work } finally { - consumer.commitSync() + consumer.commitSync(); } } ``` @@ -778,12 +1174,27 @@ public interface BaseKafkaRecordsHandler { } ``` +### Телеметрия { #telemetry } + +Kafka использует контракт телеметрии для логирования, метрик и трассировки сообщений. +Конфигурация телеметрии (секция `telemetry { logging / metrics / tracing }`) описана в разделе [Конфигурация](#config-consumer). + +Для каждого события и пачки-событий `KafkaListener` создаётся отдельный контекст телеметрии, который закрывается по завершении обработки. + +Фабрика по умолчанию `DefaultKafkaListenerTelemetryFactory` объединяет три фабрики: +- `KafkaListenerLoggerFactory` строит `KafkaListenerLogger` для логирования начала/конца обработки сообщения; +- `KafkaListenerMetricsFactory` строит `KafkaListenerMetrics` для записи метрик сообщений; +- `KafkaListenerTracerFactory` строит `KafkaListenerTracer` для распределённой трассировки. + +Метрики и трассировка описаны в разделе [Справочник метрик](metrics.md#kafka). + ## Продюсер { #producer } -Описания работы с [Kafka Producer](https://docs.confluent.io/platform/current/clients/producer.html) +`Producer` отправляет записи в `topic`. Kora создает реализацию интерфейса, помеченного `@KafkaPublisher`, +подбирает `Serializer` для ключа и значения, вызывает `KafkaProducer#send` и связывает отправку с телеметрией. -Предполагается использовать аннотацию `@KafkaPublisher` на интерфейсе для создания `Kafka Producer`, -для того чтобы отправлять сообщения в любой топик предполагается создание метода с сигнатурой `ProducerRecord`: +Для создания `Producer` используется аннотация `@KafkaPublisher` на интерфейсе. +Чтобы отправлять сообщения в произвольный `topic`, можно объявить метод с параметром `ProducerRecord`: ===! ":fontawesome-brands-java: `Java`" @@ -807,8 +1218,7 @@ public interface BaseKafkaRecordsHandler { ### Топик { #topic } -В случае если требуется использовать типизированные контракты на определенные топики -то предполагается использование аннотации `@KafkaPublisher.Topic` для создания таких контрактов: +Если требуется использовать типизированные методы для конкретных `topic`, используется аннотация `@KafkaPublisher.Topic`: ===! ":fontawesome-brands-java: `Java`" @@ -832,13 +1242,13 @@ public interface BaseKafkaRecordsHandler { } ``` -Параметр аннотации указывает на путь для конфигурации топика. +Параметр аннотации указывает на путь для конфигурации `topic`. -### Конфигурация { #configuration-2 } +### Конфигурация { #config-producer } -Конфигурация описывает настройки конкретного `@KafkaPublisher` и ниже указан пример для конфигурации по пути `kafka.someConsumer`. +Конфигурация описывает настройки конкретного `@KafkaPublisher`; ниже указан пример для конфигурации по пути `kafka.someProducer`. -Пример полной конфигурации, описанной в классе `KafkaPublisherConfig` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации: ===! ":material-code-json: `Hocon`" @@ -848,27 +1258,11 @@ public interface BaseKafkaRecordsHandler { driverProperties { //(1)! "bootstrap.servers": "localhost:9093" } - telemetry { - logging { - enabled = false //(2)! - } - metrics { - enabled = true //(3)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(4)! - } - tracing { - enabled = true //(5)! - } - } } } ``` - 1. *Properties* из официального клиента кафки, документацию по ним можно посмотреть по [ссылке](https://kafka.apache.org/documentation/#producerconfigs) (**обязательный**) - 2. Включает логгирование модуля (по умолчанию `false`) - 3. Включает метрики модуля (по умолчанию `true`) - 4. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 5. Включает трассировку модуля (по умолчанию `true`) + 1. `Properties` официального `Kafka Producer` (`обязательные`, по умолчанию не указано) === ":simple-yaml: `YAML`" @@ -876,24 +1270,87 @@ public interface BaseKafkaRecordsHandler { kafka: someProducer: driverProperties: #(1)! - bootstrap.servers: "localhost:9093" - telemetry: - logging: - enabled: true #(2)! - metrics: - enabled: true #(3)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(4)! - telemetry: - enabled: true #(5)! + "bootstrap.servers": "localhost:9093" ``` - 1. *Properties* из официального клиента кафки, документацию по ним можно посмотреть по [ссылке](https://kafka.apache.org/documentation/#producerconfigs) (**обязательный**) - 2. Включает логгирование модуля (по умолчанию `false`) - 3. Включает метрики модуля (по умолчанию `true`) - 4. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 5. Включает трассировку модуля (по умолчанию `true`) + 1. `Properties` официального `Kafka Producer` (`обязательные`, по умолчанию не указано) -Конфигурация топика описывает настройки конкретного `@KafkaPublisher.Topic` и ниже указан пример для конфигурации по пути `kafka.someProducer.someTopic`. +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классе `KafkaPublisherConfig` (указаны примеры значений или значения по умолчанию): + + ===! ":material-code-json: `Hocon`" + + ```javascript + kafka { + someProducer { + driverProperties { //(1)! + "bootstrap.servers": "localhost:9093" + } + telemetry { + logging { + enabled = false //(2)! + } + metrics { + enabled = true //(3)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(4)! + tags = { // (5)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(6)! + attributes = { // (7)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + ``` + + 1. `Properties` официального `Kafka Producer`; документация по ним доступна в [Apache Kafka Producer Configs](https://kafka.apache.org/documentation/#producerconfigs) (`обязательная`, по умолчанию не указано) + 2. Включает логирование модуля (по умолчанию: `false`) + 3. Включает метрики модуля (по умолчанию: `true`) + 4. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 5. Настройка тегов для метрик (по умолчанию: `{}`) + 6. Включает трассировку модуля (по умолчанию: `true`) + 7. Настройка атрибутов для трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + kafka: + someProducer: + driverProperties: #(1)! + bootstrap.servers: "localhost:9093" + telemetry: + logging: + enabled: false #(2)! + metrics: + enabled: true #(3)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(4)! + tags: #(5)! + key1: value1 + key2: value2 + tracing: + enabled: true #(6)! + attributes: #(7)! + key1: value1 + key2: value2 + ``` + + 1. `Properties` официального `Kafka Producer`; документация по ним доступна в [Apache Kafka Producer Configs](https://kafka.apache.org/documentation/#producerconfigs) (`обязательная`, по умолчанию не указано) + 2. Включает логирование модуля (по умолчанию: `false`) + 3. Включает метрики модуля (по умолчанию: `true`) + 4. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 5. Настройка тегов для метрик (по умолчанию: `{}`) + 6. Включает трассировку модуля (по умолчанию: `true`) + 7. Настройка атрибутов для трассировки (по умолчанию: `{}`) + +Конфигурация `topic` описывает настройки конкретного `@KafkaPublisher.Topic`; ниже указан пример для конфигурации по пути `kafka.someProducer.someTopic`. Пример полной конфигурации, описанной в классе `KafkaPublisherConfig.TopicConfig` (указаны примеры значений или значения по умолчанию): @@ -910,8 +1367,10 @@ public interface BaseKafkaRecordsHandler { } ``` - 1. В какой топик метод будет отправлять данные (**обязательный**) - 2. В какой partition топика метод будет отправлять данные (по умолчанию отсутвует) + 1. `topic`, в который метод будет отправлять данные (`обязательная`, по умолчанию не указано) + 2. Раздел `topic`, в который метод будет отправлять данные (по умолчанию не указано, необязательно) + Если указан, все сообщения будут отправляться в указанную партицию. + Если не указан, используется стандартное партиционирование Kafka (по ключу или random). === ":simple-yaml: `YAML`" @@ -923,12 +1382,18 @@ public interface BaseKafkaRecordsHandler { partition: 1 #(2)! ``` - 1. В какой топик метод будет отправлять данные (**обязательный**) - 2. В какой partition топика метод будет отправлять данные (по умолчанию отсутвует) + 1. `topic`, в который метод будет отправлять данные (`обязательная`, по умолчанию не указано) + 2. Раздел `topic`, в который метод будет отправлять данные (по умолчанию не указано, необязательно) + Если указан, все сообщения будут отправляться в указанную партицию. + Если не указан, используется стандартное партиционирование Kafka (по ключу или random). ### Сериализация { #serialization } -Для уточнения какой `Serializer` взять из контейнера есть возможность использовать теги. +`Serializer` используется для сериализации ключей и значений `ProducerRecord`. +Kora предоставляет компоненты `Serializer` для базовых типов: `String`, `UUID`, `byte[]`, `Bytes`, `ByteBuffer`, +`Double`, `Float`, `Integer`, `Long`, `Short` и `Void`. + +Для уточнения, какой `Serializer` взять из контейнера, можно использовать теги. Теги необходимо устанавливать на параметры `ProducerRecord` или `key`/`value` методов: ===! ":fontawesome-brands-java: `Java`" @@ -957,7 +1422,8 @@ public interface BaseKafkaRecordsHandler { } ``` -В случае если хочется сериализовать как Json то следует использовать `@Json` аннотацию: +Если требуется сериализация в `JSON`, используется тег `@Json`. +В таком случае Kora использует `JsonWriter` и `JsonKafkaSerializer` из модуля [JSON](json.md): ===! ":fontawesome-brands-java: `Java`" @@ -991,22 +1457,175 @@ public interface BaseKafkaRecordsHandler { } ``` -### Обработка исключений { #exception-handling-2 } +### Кастомный сериализатор { #custom-serializer } + +В случае если требуется кастомная сериализация, можно реализовать собственный `Serializer`. + +**Вариант 1: Сериализатор по умолчанию для типа** + +Если предоставить `Serializer` как компонент без тега, он будет использоваться для всех продюсеров этого типа: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public static class MyEventSerializer implements Serializer { + + private final JsonWriter writer; + + public MyEventSerializer(JsonWriter writer) { + this.writer = writer; + } + + @Override + public byte[] serialize(String topic, MyEvent data) { + try { + return writer.toByteArray(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + @KafkaPublisher("kafka.someProducer") + public interface MyPublisher { + + @KafkaPublisher.Topic("kafka.someProducer.topic") + void send(MyEvent value); // Используется MyEventSerializer + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyEventSerializer( + private val writer: JsonWriter + ) : Serializer { + + override fun serialize(topic: String, data: MyEvent): ByteArray { + return try { + writer.toByteArray(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + @KafkaPublisher("kafka.someProducer") + interface MyPublisher { + + @KafkaPublisher.Topic("kafka.someProducer.topic") + fun send(value: MyEvent) // Используется MyEventSerializer + } + ``` -В случае ошибки отправки методе проаннотированным `@Topic` и который не возвращает `Future` будет выброшено `ru.tinkoff.kora.kafka.common.exceptions.KafkaPublishException` -где в `cause` будет лежать реальная ошибка из `KafkaProducer`. +**Вариант 2: Точечный сериализатор для конкретного продюсера** + +Если требуется использовать разную сериализацию для разных продюсеров одного типа, можно использовать теги: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KafkaPublisher("kafka.someProducer") + public interface MyKafkaProducer { + + @Json + record MyEvent(String username, int code) {} + + @Tag(MyEvent.class) + @Component + class MySerializer implements Serializer { + + private final JsonWriter writer; + + public MySerializer(JsonWriter writer) { + this.writer = writer; + } + + @Override + public byte[] serialize(String topic, MyEvent data) { + try { + return writer.toByteArray(data); + } catch (IOException e) { + throw new IllegalArgumentException(e); + } + } + } + + void send(ProducerRecord record); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KafkaPublisher("kafka.someProducer") + interface MyKafkaProducer { + + @Json + data class MyEvent(val username: String, val code: Int) + + @Tag(MyEvent::class) + @Component + class MySerializer( + private val writer: JsonWriter + ) : Serializer { + + override fun serialize(topic: String, data: MyEvent): ByteArray { + return try { + writer.toByteArray(data) + } catch (e: IOException) { + throw IllegalArgumentException(e) + } + } + } + + fun send(record: ProducerRecord) + } + ``` + +### Обработка исключений { #exception-handling-producer } + +В случае ошибки отправки в методе, помеченном `@KafkaPublisher.Topic`, который не возвращает `Future`, +будет выброшено `ru.tinkoff.kora.kafka.common.exceptions.KafkaPublishException`. +Исходная ошибка из `KafkaProducer` будет доступна в `cause`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + try { + myPublisher.send("key", "value"); + } catch (KafkaPublishException e) { + // Обработка ошибки публикации + Throwable cause = e.getCause(); // Реальная ошибка от KafkaProducer + // ... + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + try { + myPublisher.send("key", "value") + } catch (e: KafkaPublishException) { + // Обработка ошибки публикации + val cause = e.cause // Реальная ошибка от KafkaProducer + // ... + } + ``` #### Ошибки сериализации { #serialization-errors } -В случае ошибки сериализации ключа/значения в методе проаннотированным `@Topic` будет выброшено `org.apache.kafka.common.errors.SerializationException` -аналогично как это было бы в случае `org.apache.kafka.clients.producer.Producer#send` +В случае ошибки сериализации ключа или значения в методе, помеченном `@KafkaPublisher.Topic`, +будет выброшено `org.apache.kafka.common.errors.SerializationException`, как и при прямом вызове `org.apache.kafka.clients.producer.Producer#send`. ### Транзакции { #transactions } -Возможно отправлять сообщение в Kafka в [рамках транзакции](https://www.confluent.io/blog/transactions-apache-kafka/), для этого предполагается использовать -аннотацию `@KafkaPublisher` и наследование интерфейса `TransactionalPublisher` для создания такого `KafkaProducer`. +Можно отправлять сообщения в `Kafka` в [рамках транзакции](https://www.confluent.io/blog/transactions-apache-kafka/). +Для этого используется аннотация `@KafkaPublisher` и наследование от `TransactionalPublisher`. -Требуется сначала создать обычного `KafkaProducer` а затем его использовать для создания транзакционного Producer'а: +Сначала требуется описать обычный `KafkaProducer`, а затем использовать его тип для создания транзакционного `Producer`: ===! ":fontawesome-brands-java: `Java`" @@ -1040,7 +1659,8 @@ public interface BaseKafkaRecordsHandler { ``` -Предполагается использовать методы `inTx` для отправки таких сообщений, все сообщения в рамках Lambda будут применены в случае успешного ее выполнения и отменены в случае ошибки. +Для отправки в транзакции используются методы `inTx`: все сообщения внутри `lambda` будут подтверждены при успешном выполнении +и отменены при ошибке. ===! ":fontawesome-brands-java: `Java`" @@ -1060,7 +1680,7 @@ public interface BaseKafkaRecordsHandler { }) ``` -Также возможно вручную произвести все манипуляции с `KafkaProducer`: +Также можно вручную управлять транзакцией через `begin()`: ===! ":fontawesome-brands-java: `Java`" @@ -1086,7 +1706,7 @@ public interface BaseKafkaRecordsHandler { } ``` -#### Конфигурация { #configuration-3 } +#### Конфигурация { #config-producer-tx } `KafkaPublisherConfig.TransactionConfig` используется для конфигурации `@KafkaPublisher` с интерфейсом `TransactionalPublisher`: @@ -1095,36 +1715,171 @@ public interface BaseKafkaRecordsHandler { ```javascript kafka { someTransactionalProducer { - idPrefix = "kafka-app-" //(1)! + idPrefix = "kora-app-" //(1)! maxPoolSize = 10 //(2)! maxWaitTime = "10s" //(3)! } } ``` - 1. Префикс индетификатора транзакций - 2. Размер набора соединений для транзакций - 3. Максимальное время ожидания транзакции + 1. Префикс идентификатора транзакций. Используется для генерации уникального `transactional.id`. + Формат: `{idPrefix}-{uuid}`. Пример: `kafka-app-550e8400-e29b-41d4-a716-446655440000`. + 2. Размер пула транзакционных продюсеров. Определяет максимальное количество параллельных транзакций. + 3. Максимальное время ожидания получения транзакции из пула. Если превышено, будет выброшено исключение. === ":simple-yaml: `YAML`" ```yaml kafka: someTransactionalProducer: - idPrefix: "kafka-app-" #(1)! + idPrefix: "kora-app-" #(1)! maxPoolSize: 10 #(2)! maxWaitTime: "10s" #(3)! ``` - 1. Префикс индетификатора транзакций - 2. Размер набора соединений для транзакций - 3. Максимальное время ожидания транзакции + 1. Префикс идентификатора транзакций. Используется для генерации уникального `transactional.id`. + Формат: `{idPrefix}-{uuid}`. Пример: `kafka-app-550e8400-e29b-41d4-a716-446655440000`. + 2. Размер пула транзакционных продюсеров. Определяет максимальное количество параллельных транзакций. + 3. Максимальное время ожидания получения транзакции из пула. Если превышено, будет выброшено исключение. + +### Продвинутое использование транзакций { #advanced-transactions } + +#### Интерфейс Transaction { #transaction-interface } + +Метод `begin()` возвращает объект `Transaction

`, который предоставляет расширенные возможности управления транзакцией: + +===! ":fontawesome-brands-java: `Java`" + + ```java + try (var tx = transactionalPublisher.begin()) { + // Отправка сообщений + tx.publisher().send("key1", "value1"); + tx.publisher().send("key2", "value2"); + + // Коммит offset'ов потребителя в транзакции (exactly-once семантика) + Map offsets = ...; + ConsumerGroupMetadata groupMetadata = ...; + tx.sendOffsetsToTransaction(offsets, groupMetadata); + + // Явный flush для гарантии отправки перед коммитом + tx.flush(); + + // commit() вызывается автоматически при закрытии try-with-resources + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + transactionalPublisher.begin().use { tx -> + // Отправка сообщений + tx.publisher().send("key1", "value1") + tx.publisher().send("key2", "value2") + + // Коммит offset'ов потребителя в транзакции (exactly-once семантика) + val offsets: Map = ... + val groupMetadata: ConsumerGroupMetadata = ... + tx.sendOffsetsToTransaction(offsets, groupMetadata) + + // Явный flush для гарантии отправки перед коммитом + tx.flush() + + // commit() вызывается автоматически при закрытии use + } + ``` + +**Методы `Transaction

`:** + +| Метод | Описание | +|-------|----------| +| `publisher()` | Возвращает типизированный publisher для отправки сообщений | +| `producer()` | Возвращает raw `Producer` для низкоуровневых операций | +| `sendOffsetsToTransaction(offsets, groupMetadata)` | Коммитит offset'ы потребителя в рамках той же транзакции | +| `flush()` | Гарантирует отправку всех сообщений перед коммитом | +| `abort()` | Откатывает транзакцию | +| `abort(cause)` | Откатывает транзакцию с указанием причины | +| `close()` | Закрывает транзакцию (коммит если не было abort) | + +#### Методы транзакций { #tx-methods } + +`TransactionalPublisher` предоставляет 4 метода для работы с транзакциями: + +| Метод | Что передаёт в callback | Возвращает значение | +|-------|------------------------|---------------------| +| `inTx(TransactionalConsumer)` | `P publisher` | `void` | +| `inTx(TransactionalFunction)` | `P publisher` | `R` | +| `withTx(TransactionConsumer)` | `Transaction

tx` | `void` | +| `withTx(TransactionFunction)` | `Transaction

tx` | `R` | + +**Пример с возвратом значения:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + // inTx с возвратом значения + Long messageId = transactionalPublisher.inTx(producer -> { + producer.send("key", "value"); + return System.currentTimeMillis(); + }); + + // withTx с доступом к Transaction + transactionalPublisher.withTx(tx -> { + tx.publisher().send("key", "value"); + tx.sendOffsetsToTransaction(offsets, groupMetadata); + tx.flush(); // Явный flush + }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // inTx с возвратом значения + val messageId = transactionalPublisher.inTx { producer -> + producer.send("key", "value") + System.currentTimeMillis() + } + + // withTx с доступом к Transaction + transactionalPublisher.withTx { tx -> + tx.publisher().send("key", "value") + tx.sendOffsetsToTransaction(offsets, groupMetadata) + tx.flush() // Явный flush + } + ``` + +#### Сериализаторы и десериализаторы по умолчанию { #default-serializers } + +`KafkaModule` автоматически предоставляет сериализаторы и десериализаторы для базовых типов через `KafkaSerializersModule` и `KafkaDeserializersModule`. + +Эти сериализаторы/десериализаторы предоставляются как компоненты **без тегов** и используются по умолчанию для всех потребителей/продюсеров соответствующих типов. + +**Поддерживаемые типы из коробки:** + +| Тип | Serializer | Deserializer | +|-----|------------|--------------| +| `String` | `StringSerializer` | `StringDeserializer` | +| `byte[]` | `ByteArraySerializer` | `ByteArrayDeserializer` | +| `ByteBuffer` | `ByteBufferSerializer` | `ByteBufferDeserializer` | +| `Bytes` | `BytesSerializer` | `BytesDeserializer` | +| `UUID` | `UUIDSerializer` | `UUIDDeserializer` | +| `Integer` | `IntegerSerializer` | `IntegerDeserializer` | +| `Long` | `LongSerializer` | `LongDeserializer` | +| `Short` | `ShortSerializer` | `ShortDeserializer` | +| `Double` | `DoubleSerializer` | `DoubleDeserializer` | +| `Float` | `FloatSerializer` | `FloatDeserializer` | +| `Void` | `VoidSerializer` | `VoidDeserializer` | ### Сигнатуры { #signatures-3 } -Доступные сигнатуры для методов Kafka продюсера из коробки, где под `K` подразумевается тип ключа и под `V` тип значения сообщения. +Доступные сигнатуры для методов `Kafka Producer` из коробки, где под `K` подразумевается тип ключа, а под `V` тип значения сообщения. +Генератор поддерживает два семейства сигнатур: отправку готового `ProducerRecord` и отправку через метод с `@KafkaPublisher.Topic`. +Эти семейства нельзя смешивать между собой в одном методе. -Позволяет отправлять `value` (обязательный) и `key` (опциональный) и `headers` (опциональный) от `ProducerRecord`: +#### Готовое событие { #producer-record-signature } + +Метод с `ProducerRecord` используется, когда `topic`, раздел, время создания или `Headers` нужно задать на стороне вызывающего кода. +Такой метод нельзя помечать `@KafkaPublisher.Topic`, потому что все сведения об отправке уже находятся в самом `ProducerRecord`. +Дополнительно можно передать один `Callback`. ===! ":fontawesome-brands-java: `Java`" @@ -1132,8 +1887,9 @@ public interface BaseKafkaRecordsHandler { @KafkaPublisher("kafka.someProducer") public interface MyPublisher { - @KafkaPublisher.Topic("kafka.someProducer.someTopic") - void send(K key, V value, Headers headers); + void send(ProducerRecord record); + + void send(ProducerRecord record, Callback callback); } ``` @@ -1143,56 +1899,67 @@ public interface BaseKafkaRecordsHandler { @KafkaPublisher("kafka.someProducer") interface MyPublisher { - @KafkaPublisher.Topic("kafka.someProducer.someTopic") - fun send(key: K, value: V, headers: Headers) + fun send(record: ProducerRecord) + + fun send(record: ProducerRecord, callback: Callback) } ``` -===! ":fontawesome-brands-java: `Java`" +#### Методы по топику { #topic-signature } - Можно получать как результат операции `RecordMetadata` либо `Future` либо `CompletionStage`: +Метод с `key`, `value` и `Headers` должен быть помечен `@KafkaPublisher.Topic`. +Один пользовательский аргумент считается `value`, два пользовательских аргумента считаются `key` и `value` именно в таком порядке. +`Headers` и `Callback` можно указать дополнительно, но не больше одного аргумента каждого типа. +Если `Headers` не переданы, Kora создаст пустые заголовки. + +===! ":fontawesome-brands-java: `Java`" ```java @KafkaPublisher("kafka.someProducer") public interface MyPublisher { @KafkaPublisher.Topic("kafka.someProducer.someTopic") - RecordMetadata send(V value); + void send(V value); @KafkaPublisher.Topic("kafka.someProducer.someTopic") - Future sendFuture(V value); + void send(K key, V value); @KafkaPublisher.Topic("kafka.someProducer.someTopic") - CompletionStage sendStage(V value); + void send(K key, V value, Headers headers); + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + void send(K key, V value, Headers headers, Callback callback); } ``` === ":simple-kotlin: `Kotlin`" - Можно получать как результат операции `RecordMetadata` либо иметь модификатор `suspend` либо `Future` либо `CompletionStage` либо `Deferred`: - ```kotlin @KafkaPublisher("kafka.someProducer") interface MyPublisher { @KafkaPublisher.Topic("kafka.someProducer.someTopic") - fun send(value: V): RecordMetadata - - @KafkaPublisher.Topic("kafka.someProducer.someTopic") - suspend fun sendSuspend(value: V): RecordMetadata + fun send(value: V) @KafkaPublisher.Topic("kafka.someProducer.someTopic") - fun send(value: String): Future + fun send(key: K, value: V) @KafkaPublisher.Topic("kafka.someProducer.someTopic") - fun send(value: String): CompletionStage + fun send(key: K, value: V, headers: Headers) @KafkaPublisher.Topic("kafka.someProducer.someTopic") - fun send(value: String): Deferred - } + fun send(key: K, value: V, headers: Headers, callback: Callback) + } ``` -Возможна отправка `ProducerRecord` и `Callback` (опционально) и комбинировать сигнатуры ответа: +#### Результат отправки { #publisher-result } + +Для синхронного метода можно вернуть `void`/`Unit` или `RecordMetadata`. +В таком случае Kora вызывает `KafkaProducer#send`, ожидает завершения отправки через `Future#get()` и только после этого возвращает управление вызывающему коду. + +Для асинхронной отправки можно вернуть `Future`, `CompletionStage` или `CompletableFuture`. +В `Kotlin` дополнительно поддерживаются `suspend`-методы и `Deferred`. +Если в сигнатуре есть `Callback`, Kora сначала завершает собственную телеметрию отправки, а затем вызывает пользовательский `Callback`. ===! ":fontawesome-brands-java: `Java`" @@ -1200,7 +1967,17 @@ public interface BaseKafkaRecordsHandler { @KafkaPublisher("kafka.someProducer") public interface MyPublisher { - void send(ProducerRecord record, Callback callback); + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + RecordMetadata send(V value); // Синхронный, ждёт подтверждения от брокера + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + Future sendFuture(V value); // Асинхронный через Java Future + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + CompletionStage sendStage(V value); + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + CompletableFuture sendCompletableFuture(V value); } ``` @@ -1210,6 +1987,38 @@ public interface BaseKafkaRecordsHandler { @KafkaPublisher("kafka.someProducer") interface MyPublisher { - fun send(record: ProducerRecord, callback: Callback) - } + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: V): RecordMetadata // Синхронный, ждёт подтверждения от брокера + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + suspend fun sendSuspend(value: V): RecordMetadata // Kotlin Coroutines + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: V): Future // Java Future + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: V): CompletionStage // Java CompletableFuture + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: String): CompletableFuture + + @KafkaPublisher.Topic("kafka.someProducer.someTopic") + fun send(value: V): Deferred // Kotlin Deferred + } ``` + +Недопустимые сочетания: `ProducerRecord` вместе с `@KafkaPublisher.Topic`, `ProducerRecord` вместе с отдельными `key`/`value`/`Headers`, больше одного `Headers`, больше одного `Callback`, а также метод с отдельными `key`/`value` без `@KafkaPublisher.Topic`. + +### Телеметрия { #telemetry } + +Kafka использует контракт телеметрии для логирования, метрик и трассировки сообщений. +Конфигурация телеметрии (секция `telemetry { logging / metrics / tracing }`) описана в разделе [Конфигурация](#config-producer). + +Для каждого сообщения KafkaPublisher создаётся отдельный контекст телеметрии, который закрывается по завершении обработки. + +Фабрика по умолчанию `DefaultKafkaPublisherTelemetryFactory` объединяет три фабрики: +- `KafkaPublisherLoggerFactory` строит `KafkaPublisherLogger` для логирования начала/конца обработки сообщения; +- `KafkaPublisherMetricsFactory` строит `KafkaPublisherMetrics` для записи метрик сообщений; +- `KafkaPublisherTracerFactory` строит `KafkaPublisherTracer` для распределённой трассировки. + +Метрики и трассировка описаны в разделе [Справочник метрик](metrics.md#kafka). diff --git a/mkdocs/docs/ru/documentation/logging-aspect.md b/mkdocs/docs/ru/documentation/logging-aspect.md index dcc3b1c..8d18e14 100644 --- a/mkdocs/docs/ru/documentation/logging-aspect.md +++ b/mkdocs/docs/ru/documentation/logging-aspect.md @@ -4,13 +4,16 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora logging aspects for argument and result logging, selective logging, MDC enrichment, structured parameters, conversion, and signatures; key triggers include @Log, @Log.in, @Log.out, @Log.off, @Mdc, @StructuredArgument, MDC, LogAspect." --- -Модуль для декларативного логирования аргументов и результата методов с помощью аннотаций аспектов. +Модуль декларативного логирования позволяет описывать логирование метода с помощью аннотаций `@Log` и `@Mdc`. +На этапе компиляции Kora создает аспект-обертку для метода; обертка логирует вход в метод, выход из метода, результат, ошибку и значения `MDC` без ручного кода в бизнес-логике. +Это удобно для единообразной диагностики вызовов, особенно когда нужно быстро понять, какой метод был вызван, с какими аргументами и как он завершился. -Если нужен пошаговый разбор перед справочным описанием, смотрите [Наблюдаемость](../guides/observability.md). +Пошаговый разбор перед справочным описанием смотрите в разделе [Наблюдаемость](../guides/observability.md). ## Подключение { #dependency } -Скорее всего уже транзитивно подключен из других зависимостей либо из [Logback](logging-slf4j.md#logback), в противном случае требуется подключить: +Аннотации и вспомогательные классы предоставляются зависимостью `logging-common`. +Обычно она уже приходит через другие модули Kora или через [Logback](logging-slf4j.md#logback), но при использовании аннотаций напрямую зависимость можно добавить явно: ===! ":fontawesome-brands-java: `Java`" @@ -38,9 +41,23 @@ agent: interface Application : LoggingModule ``` +Для генерации аспектов также должны быть подключены общие [обработчики аннотаций](general.md#annotation-processor) или [`KSP`-обработчики](general.md#ksp). +В обычном приложении Kora они уже подключены как часть базовой настройки проекта. + ## Логирование { #logging } -Предполагается использовать специальные комбинации аннотаций для настройки логирование методов. +Логирование метода настраивается комбинациями аннотаций: + +- `@Log` - логирует вход и выход метода (по умолчанию: `INFO`). +- `@Log.in` - логирует только вход в метод (по умолчанию: `INFO`). +- `@Log.out` - логирует только выход из метода (по умолчанию: `INFO`). +- `@Log.result` - задает уровень, начиная с которого в лог добавляется значение результата (по умолчанию: `DEBUG`). +- `@Log.off` - отключает логирование результата метода или отдельного параметра. +- `@Log(Level)` на параметре - задает уровень, начиная с которого параметр попадает в структурированные данные (по умолчанию: `DEBUG` для параметра без отдельной аннотации). + +Само событие входа или выхода пишется на уровне, указанном в `@Log`, `@Log.in` или `@Log.out`. +Значения аргументов и результата добавляются в структурированные данные только если включен соответствующий уровень детализации. +Какой уровень детализации активен, зависит от эффективного уровня логгера, настроенного через `logging.level` / `logging.levels` — смотрите [настройку уровней логирования](logging-slf4j.md#configuration). ### Аргументов { #argument } @@ -64,13 +81,19 @@ agent: - + - + + + + + @@ -103,13 +126,19 @@ agent:
Уровень логгированияУровень логирования Лог
TRACE, DEBUGDEBUG -

DEBUG [main] r.t.e.e.Example.doWork: > {data: {numParam: "4"}}

+

INFO [main] r.t.e.e.Example.doWork: > {data: {numParam: "4"}}

+
TRACE +

INFO [main] r.t.e.e.Example.doWork: > {data: {numParam: "4"}}
- + - + + + + + @@ -142,14 +171,21 @@ agent:
Уровень логгированияУровень логирования Лог
TRACE, DEBUGDEBUG +

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}

+
TRACE -

DEBUG [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}

+

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}
- + - + + + + + @@ -161,6 +197,9 @@ agent:
Уровень логгированияУровень логирования Лог
TRACE, DEBUGDEBUG +

INFO [main] r.t.e.e.Example.doWork: > {data: {strParam: "s", numParam: "4"}}

+

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}

+
TRACE -

DEBUG [main] r.t.e.e.Example.doWork: > {data: {strParam: "s", numParam: "4"}}

-

DEBUG [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}

+

INFO [main] r.t.e.e.Example.doWork: > {data: {strParam: "s", numParam: "4"}}

+

INFO [main] r.t.e.e.Example.doWork: < {data: {out: "testResult"}}
+Если метод завершается ошибкой, аспект записывает выход из метода с данными об ошибке: `errorType` и `errorMessage`. +При включенном `DEBUG` в лог также передается объект исключения. + ### Выборочное логирование { #selective-logging } ===! ":fontawesome-brands-java: `Java`" @@ -185,7 +224,7 @@ agent: - + @@ -202,10 +241,53 @@ agent:
Уровень логгированияУровень логирования Лог
+В этом примере `@Log.off` на методе отключает запись значения результата, но не отключает само событие выхода из метода. +Чтобы исключить из лога отдельный аргумент, `@Log.off` ставится на параметр. + +Уровень детализации параметров можно задавать отдельно: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log.in + public void doWork(@Log(Level.INFO) String id, @Log(Level.TRACE) String payload) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log.`in` + fun doWork(@Log(Level.INFO) id: String, @Log(Level.TRACE) payload: String) { } + ``` + +При уровне `INFO` в структурированные данные попадет только `id`, а `payload` появится только при включенном `TRACE`. + +Значение результата можно вывести уже на уровне `INFO`, если явно указать `@Log.result(Level.INFO)`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Log.out + @Log.result(Level.INFO) + public String doWork() { + return "testResult"; + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Log.out + @Log.result(Level.INFO) + fun doWork(): String { + return "testResult" + } + ``` + ### Структурированный параметр { #structured-parameter } -В случае если представление параметра как строкой не является желаемым поведением, -его можно реализовать интерфейс `StructuredArgument` и параметр научится логировать себя сам: +Если строковое представление параметра не подходит для лога, тип параметра может реализовать интерфейс `StructuredArgument`. +В этом случае объект сам задает имя поля через `fieldName()` и записывает значение в `JsonGenerator` через `writeTo(...)`. ===! ":fontawesome-brands-java: `Java`" @@ -247,11 +329,11 @@ agent: - + - +
Уровень логгированияУровень логирования Лог
TRACE, DEBUGDEBUG, TRACE

INFO [main] r.t.e.e.Example.doWork: >

     data={"entity":"Bob"}

@@ -261,15 +343,38 @@ agent: 		INFO

INFO [main] r.t.e.e.Example.doWork: >

-

     data={"entity":"Bob"}
+Когда нужно структурированное значение без введения отдельного типа, интерфейс `StructuredArgument` предоставляет статические фабричные методы: +`arg(fieldName, value)` / `arg(fieldName, value, JsonWriter)` создают структурированный аргумент (перегрузки принимают `String`, `Integer`, `Long`, `Boolean`, `Map`, `JsonWriter` или сырой `StructuredArgumentWriter`), +а `marker(fieldName, value)` создает `org.slf4j.Marker` для одного вызова лога. Полученный `StructuredArgument` можно также передать напрямую в `MDC.put`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + // ad-hoc structured value fed into MDC + MDC.put("order", StructuredArgument.arg("orderId", orderId)); + + // or as an SLF4J marker on a single log line + log.info(StructuredArgument.marker("orderId", orderId), "order accepted"); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // ad-hoc structured value fed into MDC + MDC.put("order", StructuredArgument.arg("orderId", orderId)) + + // or as an SLF4J marker on a single log line + log.info(StructuredArgument.marker("orderId", orderId), "order accepted") + ``` + ### Конвертация параметров { #parameter-conversion } -В случае если представление параметра как строкой не является желаемым поведением, -его можно переназначить через указание `StructuredArgumentMapper` напротив желаемого аргумента: +Если менять сам тип параметра нельзя, можно описать внешний преобразователь `StructuredArgumentMapper` и указать его через `@Mapping` на нужном аргументе. +Такой преобразователь получает исходное значение параметра и записывает структурированное значение в `JsonGenerator`. ===! ":fontawesome-brands-java: `Java`" @@ -307,11 +412,11 @@ agent: - + - +
Уровень логгированияУровень логирования Лог
TRACE, DEBUGDEBUG, TRACE

INFO [main] r.t.e.e.Example.doWork: >

     data={"entity":"Bob"}

@@ -321,23 +426,28 @@ agent: 		INFO

INFO [main] r.t.e.e.Example.doWork: >

-

     data={"entity":"Bob"}
### MDC (Mapped Diagnostic Context) { #mdc-mapped-diagnostic-context } -Аннотация `@Mdc` позволяет добавлять пары ключ-значение в MDC (Mapped Diagnostic Context) для структурированного логирования. -MDC позволяет добавлять контекстную информацию к каждому лог-сообщению. +Аннотация `@Mdc` добавляет пары ключ-значение в `MDC` (`Mapped Diagnostic Context`). +`MDC` хранит контекст выполнения и позволяет добавлять его к лог-сообщениям: например, идентификатор запроса, пользователя или операции. -Аннотация может применяться к методам и параметрам методов. Поддерживается множественное применение. +Аннотация может применяться к методам и параметрам методов. +На методе поддерживается множественное применение `@Mdc`. +Значения, добавленные без `global = true`, восстанавливаются после выполнения метода. **Параметры аннотации `@Mdc`:** -- `key()` - Ключ для MDC записи. Если не указано, используется имя аннотированного параметра. -- `value()` - Значение для MDC записи. Если не указано, используется значение аннотированного параметра. -- `global()` - Если true, MDC значение будет доступно глобально в рамках потока, а не только во время выполнения метода. +- `key()` - ключ записи `MDC` (по умолчанию: `""`). +- `value()` - значение записи `MDC` (по умолчанию: `""`). +- `global()` - оставлять значение в `MDC` после выхода из метода (по умолчанию: `false`). + +Для `@Mdc` на методе обязательны непустые `key` и `value`. +Для `@Mdc` на параметре ключ берется из `key`, затем из `value`, а если оба значения пустые - из имени параметра. +Значением записи становится значение параметра. #### Аннотация параметра { #parameter-annotation } @@ -357,7 +467,7 @@ MDC позволяет добавлять контекстную информа } ``` -В этом случае ключ MDC будет совпадать с именем параметра ("s"), а значением будет значение параметра. +В этом случае ключ `MDC` будет совпадать с именем параметра `s`, а значением будет значение параметра. #### Аннотация параметра с ключом { #parameter-annotation-with-key } @@ -377,7 +487,7 @@ MDC позволяет добавлять контекстную информа } ``` -Здесь ключ MDC будет "123", а значением - значение параметра "s". +Здесь ключом `MDC` будет `123`, а значением - значение параметра `s`. #### Аннотация метода { #method-use } @@ -399,8 +509,8 @@ MDC позволяет добавлять контекстную информа } ``` -В этом примере демонстрируется: -- Аннотация метода с локальным MDC значением +В этом примере перед вызовом метода в `MDC` будет добавлена запись `key1=value2`. +После завершения метода предыдущее значение `key1` будет восстановлено. #### Комбинированное { #combined } @@ -424,7 +534,11 @@ MDC позволяет добавлять контекстную информа } ``` -В этом примере к методу применены две аннотации MDC, а к параметру одна аннотация. +В этом примере к методу применены две аннотации `@Mdc`, а к параметру - одна. +Запись `key=value` останется в `MDC` после выполнения метода из-за `global = true`, остальные записи будут восстановлены или удалены. + +Под капотом неглобальные записи сохраняются в виде снимка до вызова и восстанавливаются в блоке `finally` после возврата из метода, поэтому они никогда не выходят за пределы области видимости метода. +Записи, добавленные с `global = true` (а также любое значение, установленное через императивный `MDC.put`, смотрите ниже), остаются в `Context` на протяжении всей области видимости запроса/потока и потому видны в каждой последующей строке лога. #### Генерация значения из кода { #generated-value-for-mdc-value } @@ -446,35 +560,101 @@ MDC позволяет добавлять контекстную информа } ``` -При вызове метода в MDC будет добавлена запись с ключом "key" и в данном случае значением будет случайный UUID. +При вызове метода в `MDC` будет добавлена запись с ключом `key`, а значением будет случайный `UUID`. +Для `Java` значение в формате `${...}` вставляется в сгенерированный код как выражение. -**Пример лога с MDC:** +**Пример лога с `MDC`:** ``` INFO [main] r.t.e.e.Example.test: > {data: {s: "testValue"}} key=some-uuid-value key1=value2 123=testValue ``` +`@Mdc` не поддерживается для методов, которые возвращают `CompletionStage`, `Mono` или `Flux`. +Для `Kotlin` поддерживаются обычные методы и `suspend`-методы, но `global = true` нельзя использовать в `suspend`-методах. + +### Императивный MDC { #imperative-mdc } + +Там, где аннотация не подходит — внутри перехватчиков, фильтров или обычного кода сервиса — используйте императивный API `ru.tinkoff.kora.logging.common.MDC`. +Это программный аналог `@Mdc`: записи привязываются к `Context` Kora, поэтому они распространяются через асинхронные границы точно так же, как записи `@Mdc(global = true)`, и появляются в каждой строке лога, выводимой на протяжении оставшейся области видимости текущего `Context`. + +Статический метод `put` имеет перегрузки для значений `String`, `Integer`, `Long` и `Boolean`, а также перегрузку с `StructuredArgumentWriter` для структурированных значений. +`remove(key)` удаляет одну запись, а `get().values()` возвращает текущие записи как неизменяемую `Map`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.logging.common.MDC; + + @Component + public final class OrderService { + + public void process(String orderId) { + MDC.put("orderId", orderId); // String + MDC.put("attempt", 1); // Integer + MDC.put("bytes", 1024L); // Long + MDC.put("retryable", true); // Boolean + MDC.put("payload", gen -> gen.writeString(orderId)); // StructuredArgumentWriter + + // ... business logic; every log line in this Context now carries the keys + + MDC.remove("attempt"); // drop a single key + var current = MDC.get().values(); // read current entries + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.logging.common.MDC + + @Component + class OrderService { + + fun process(orderId: String) { + MDC.put("orderId", orderId) // String + MDC.put("attempt", 1) // Integer + MDC.put("bytes", 1024L) // Long + MDC.put("retryable", true) // Boolean + MDC.put("payload") { gen -> gen.writeString(orderId) } // StructuredArgumentWriter + + // ... business logic; every log line in this Context now carries the keys + + MDC.remove("attempt") // drop a single key + val current = MDC.get().values() // read current entries + } + } + ``` + +Когда у вас уже есть `Context` (например, внутри перехватчика), обращайтесь к нему явно через `MDC.get(ctx)` и `MDC.put(ctx, key, writer)` вместо сокращений для текущего `Context`. +В отличие от `@Mdc`, у императивного API нет ограничения на реактивные/`suspend`-методы, поскольку он пишет напрямую в `Context`, а не оборачивает вызов метода. + +!!! warning "Используйте `MDC` из Kora, а не из SLF4J" + + Всегда импортируйте `ru.tinkoff.kora.logging.common.MDC` — никогда `org.slf4j.MDC`. + Класс SLF4J пишет в отдельный `ThreadLocal`, не связанный с `Context` Kora: помещенные туда значения не появятся в структурированных логах Kora и не будут распространяться через асинхронные границы (реактивные операторы, `suspend`-функции, передача между потоками). + ## Сигнатуры { #signatures } -Доступные сигнатуры для методов которые поддерживают аннотации из коробки: +Сигнатуры методов, поддерживаемые для аспектов логирования: ===! ":fontawesome-brands-java: `Java`" - Класс не должен быть `final`, чтобы аспекты работали. + Класс не должен быть `final`, чтобы аспекты могли создать наследника. Под `T` подразумевается тип возвращаемого значения, либо `Void`. - `T myMethod()` - `Optional myMethod()` - - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) (только для `@Log`) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (только для `@Log`, требует [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (только для `@Log`, требует [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - Класс должен быть `open`, чтобы аспекты работали. + Класс должен быть `open`, чтобы аспекты могли создать наследника. Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требует [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `myMethod(): Flow` [Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требует [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) diff --git a/mkdocs/docs/ru/documentation/logging-slf4j.md b/mkdocs/docs/ru/documentation/logging-slf4j.md index d4deac6..4d48e3a 100644 --- a/mkdocs/docs/ru/documentation/logging-slf4j.md +++ b/mkdocs/docs/ru/documentation/logging-slf4j.md @@ -1,66 +1,97 @@ --- -description: "Explains Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC. Use when working with Slf4jModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." +description: "Explains Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC. Use when working with LoggingModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC; key triggers include Slf4jModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." + use_when: "Use this file for Kora docs or implementation questions about Kora SLF4J logging setup, module log configuration, Logback integration, alternative implementations, structured logs, markers, parameters, and MDC; key triggers include LoggingModule, LogbackModule, LoggerFactory, StructuredArgument, Marker, MDC, loggingConfig." --- -Kora использует [slf4j-api](https://www.slf4j.org/) как движок для логирования в рамках всего фреймворка, -предполагается что будет использоваться реализация на основе [Logback](#logback). +Kora использует [`slf4j-api`](https://www.slf4j.org/) как общий фасад логирования во всем фреймворке. +`SLF4J` отделяет код приложения от конкретной реализации логирования, а в качестве основной реализации Kora предполагает использование [`Logback`](#logback). -Если нужен пошаговый разбор перед справочным описанием, смотрите [Наблюдаемость](../guides/observability.md). +Модуль логирования отвечает за получение `Logger` через стандартную фабрику `SLF4J`, управление уровнями логирования через конфигурацию Kora и передачу структурированных данных в записи логов. +Структурированные данные можно добавлять через `StructuredArgument`, `Marker` и `MDC`, чтобы они выводились вместе с обычным текстовым сообщением. + +Пошаговый разбор перед справочным описанием смотрите в разделе [Наблюдаемость](../guides/observability.md). ## Использование { #usage } -Логеры требуется предоставлять посредствам фабрики [SLF4J](https://www.slf4j.org/manual.html#hello_world). +`Logger` создается через фабрику [`SLF4J`](https://www.slf4j.org/manual.html#hello_world): ===! ":fontawesome-brands-java: `Java`" ```java - Logger logger = LoggerFactory.getLogger(SomeService.class) + Logger logger = LoggerFactory.getLogger(SomeService.class); ``` === ":simple-kotlin: `Kotlin`" ```kotlin - val logger = LoggerFactory.getLogger(SomeService::class.java); + val logger = LoggerFactory.getLogger(SomeService::class.java) ``` ## Конфигурация { #configuration } -Уровни логирования описанные в классе `LoggingConfig`: +Уровни логирования описываются классом `LoggingConfig`. +Конфигурация задает уровень для `ROOT`, пакета или конкретного класса: ===! ":material-code-json: `Hocon`" ```javascript logging { levels { //(1)! + "ROOT": "WARN" + "ru.tinkoff.kora": "INFO" "ru.tinkoff.kora.http.server.common.telemetry": "INFO" "ru.tinkoff.kora.http.client.common.telemetry.DefaultHttpClientTelemetry": "INFO" } } ``` - 1. Указываются уровни логгирования для классов и пакетов + 1. Уровни логирования для `ROOT`, классов и пакетов (по умолчанию: не указано, необязательно). === ":simple-yaml: `YAML`" ```yaml logging: levels: #(1)! + ROOT: "WARN" + ru.tinkoff.kora: "INFO" ru.tinkoff.kora.http.server.common.telemetry: "INFO" ru.tinkoff.kora.http.client.common.telemetry.DefaultHttpClientTelemetry: "INFO" - } ``` - 1. Указываются уровни логгирования для классов и пакетов + 1. Уровни логирования для `ROOT`, классов и пакетов (по умолчанию: не указано, необязательно). + +Ключ секции можно записать как `levels` или как `level` — оба варианта принимаются как псевдонимы. +Имена логгеров можно перечислять плоскими строками с точками (как выше) или как вложенный объект; Kora разворачивает вложенные объекты в имена логгеров с точками. +Имя логгера `ROOT` сопоставляется без учета регистра, поэтому `ROOT` и `root` эквивалентны. +Например, в поставляемых [примерах](https://github.com/kora-projects/kora-examples) используется псевдоним `level` в единственном числе с вложенным `root` в нижнем регистре: + +```javascript +logging.level { + "root": "WARN" + "ru.tinkoff.kora": "INFO" + "ru.tinkoff.kora.example": "INFO" +} +``` + +!!! note + + Когда секция `logging` отсутствует, Kora не применяет собственную карту уровней. + Однако реализация [Logback](#logback) сбрасывает все логгеры при каждом (повторном) применении: `ROOT` нормализуется к `INFO`, а уровень каждого остального логгера очищается, чтобы он наследовался от родителя, после чего сверху применяются настроенные уровни. + В результате значение `` из `logback.xml` при запуске фактически заменяется на `INFO`, если только уровень `ROOT` не задан в конфигурации. + +### Обновление уровней во время работы { #levels-refresh } -Параметры конфигурации сбора логов описываются в модулях в которых присутствует сбор логов, например [HTTP сервер](http-server.md), [HTTP клиент](http-client.md) и т.д. +Настроенные уровни применяются компонентом `LoggingLevelRefresher` — корневым компонентом, который при запуске сбрасывает все логгеры и заново применяет уровни из секции `logging` через `LoggingLevelApplier`. +Он повторно запускается при каждом обновлении конфигурации, поэтому когда активен [наблюдатель конфигурации](config.md#config-watcher), изменение уровня в файле конфигурации вступает в силу во время работы без перезапуска приложения. + +Параметры логирования конкретных модулей описываются в документации этих модулей, например [HTTP сервер](http-server.md), [HTTP клиент](http-client.md), [gRPC-клиент](grpc-client.md). ### Модули { #module } -Включение и выключение логирования определенных модулей указывается в конфигурации самих модулей. +Включение и выключение логирования конкретных модулей задается в конфигурации самих модулей через `telemetry.logging.enabled`. -По умолчанию логирование **всех модулей выключено**, по этому для удобства тут указана отдельно конфигурация для включения логирования большинства модулей. +По умолчанию логирование **выключено для всех модулей**, поэтому ниже приведена конфигурация для включения логирования большинства модулей: ===! ":material-code-json: `Hocon`" @@ -70,23 +101,23 @@ Kora использует [slf4j-api](https://www.slf4j.org/) как движо grpcServer.telemetry.logging.enabled = true //(3)! httpServer.telemetry.logging.enabled = true //(4)! scheduling.telemetry.logging.enabled = true //(5)! - grpcClient.ИмяСервисаGrpc.telemetry.logging.enabled = true //(6)! - soapClient.ИмяСервисаSoap.telemetry.logging.enabled = true //(7)! - ПутьДоКонфигурацииHttpКлиента.telemetry.logging.enabled = true //(8)! - ПутьДоКонфигурацииKafkaПотребителя.telemetry.logging.enabled = true //(9)! - ПутьДоКонфигурацииKafkaПродюсера.telemetry.logging.enabled = true //(10)! + grpcClient.SomeGrpcServiceName.telemetry.logging.enabled = true //(6)! + soapClient.SomeSoapServiceName.telemetry.logging.enabled = true //(7)! + SomePathToConfigHttpClient.telemetry.logging.enabled = true //(8)! + SomePathToConfigKafkaConsumer.telemetry.logging.enabled = true //(9)! + SomePathToConfigKafkaProducer.telemetry.logging.enabled = true //(10)! ``` - 1. База данных [JDBC](database-jdbc.md) / [R2DBC](database-jdbc.md) / [Vertx](database-vertx.md) - 2. База данных [Cassandra](database-cassandra.md) - 3. [gRPC сервер](grpc-server.md) - 4. [HTTP сервер](http-server.md) - 5. [Планировщик](scheduling.md) - 6. [gRPC клиент](grpc-client.md) (Указывается для конкретного сервиса) - 7. [SOAP клиент](soap-client.md) (Указывается для конкретного сервиса) - 8. [HTTP клиент](http-client.md) (Указывается для конкретного клиента) - 9. Kafka [потребитель](kafka.md#configuration) (Указывается для конкретного потребителя) - 10. Kafka [продюсер](kafka.md#manual-override) (Указывается для конкретного продюсера) + 1. Логирование запросов к базе данных [JDBC](database-jdbc.md), `R2DBC` или `Vertx` (по умолчанию: `false`). + 2. Логирование запросов к базе данных [Cassandra](database-cassandra.md) (по умолчанию: `false`). + 3. Логирование запросов [gRPC-сервера](grpc-server.md) (по умолчанию: `false`). + 4. Логирование запросов [HTTP-сервера](http-server.md) (по умолчанию: `false`). + 5. Логирование запусков [планировщика](scheduling.md) (по умолчанию: `false`). + 6. Логирование запросов [gRPC-клиента](grpc-client.md), указывается для конкретного сервиса (по умолчанию: `false`). + 7. Логирование запросов [SOAP-клиента](soap-client.md), указывается для конкретного сервиса (по умолчанию: `false`). + 8. Логирование запросов [HTTP-клиента](http-client.md), указывается для конкретного клиента (по умолчанию: `false`). + 9. Логирование Kafka-[потребителя](kafka.md#config-consumer), указывается для конкретного потребителя (по умолчанию: `false`). + 10. Логирование Kafka-[производителя](kafka.md#config-producer), указывается для конкретного производителя (по умолчанию: `false`). === ":simple-yaml: `YAML`" @@ -96,27 +127,27 @@ Kora использует [slf4j-api](https://www.slf4j.org/) как движо grpcServer.telemetry.logging.enabled: true #(3)! httpServer.telemetry.logging.enabled: true #(4)! scheduling.telemetry.logging.enabled: true #(5)! - grpcClient.ИмяСервисаGrpc.telemetry.logging.enabled: true #(6)! - soapClient.ИмяСервисаSoap.telemetry.logging.enabled: true #(7)! - ПутьДоКонфигурацииHttpКлиента.telemetry.logging.enabled: true #(8)! - ПутьДоКонфигурацииKafkaПотребителя.telemetry.logging.enabled: true #(9)! - ПутьДоКонфигурацииKafkaПродюсера.telemetry.logging.enabled: true #(10)! + grpcClient.SomeGrpcServiceName.telemetry.logging.enabled: true #(6)! + soapClient.SomeSoapServiceName.telemetry.logging.enabled: true #(7)! + SomePathToConfigHttpClient.telemetry.logging.enabled: true #(8)! + SomePathToConfigKafkaConsumer.telemetry.logging.enabled: true #(9)! + SomePathToConfigKafkaProducer.telemetry.logging.enabled: true #(10)! ``` - 1. База данных [JDBC](database-jdbc.md) / [R2DBC](database-jdbc.md) / [Vertx](database-vertx.md) - 2. База данных [Cassandra](database-cassandra.md) - 3. [gRPC сервер](grpc-server.md) - 4. [HTTP сервер](http-server.md) - 5. [Планировщик](scheduling.md) - 6. [gRPC клиент](grpc-client.md) (Указывается для конкретного сервиса) - 7. [SOAP клиент](soap-client.md) (Указывается для конкретного сервиса) - 8. [HTTP клиент](http-client.md) (Указывается для конкретного клиента) - 9. Kafka [потребитель](kafka.md#configuration) (Указывается для конкретного потребителя) - 10. Kafka [продюсер](kafka.md#manual-override) (Указывается для конкретного продюсера) + 1. Логирование запросов к базе данных [JDBC](database-jdbc.md), `R2DBC` или `Vertx` (по умолчанию: `false`). + 2. Логирование запросов к базе данных [Cassandra](database-cassandra.md) (по умолчанию: `false`). + 3. Логирование запросов [gRPC-сервера](grpc-server.md) (по умолчанию: `false`). + 4. Логирование запросов [HTTP-сервера](http-server.md) (по умолчанию: `false`). + 5. Логирование запусков [планировщика](scheduling.md) (по умолчанию: `false`). + 6. Логирование запросов [gRPC-клиента](grpc-client.md), указывается для конкретного сервиса (по умолчанию: `false`). + 7. Логирование запросов [SOAP-клиента](soap-client.md), указывается для конкретного сервиса (по умолчанию: `false`). + 8. Логирование запросов [HTTP-клиента](http-client.md), указывается для конкретного клиента (по умолчанию: `false`). + 9. Логирование Kafka-[потребителя](kafka.md#config-consumer), указывается для конкретного потребителя (по умолчанию: `false`). + 10. Логирование Kafka-[производителя](kafka.md#config-producer), указывается для конкретного производителя (по умолчанию: `false`). ## Logback { #logback } -Модуль предоставляет реализацию логирования на основе [Logback](https://www.baeldung.com/logback), добавляет поддержку структурированных логов и возможность конфигурации уровней логирования через [файл конфигурации](config.md). +Модуль предоставляет реализацию логирования на основе [`Logback`](https://www.baeldung.com/logback), добавляет поддержку структурированных логов и позволяет управлять уровнями логирования через [файл конфигурации](config.md). ### Подключение { #dependency } @@ -148,11 +179,13 @@ Kora использует [slf4j-api](https://www.slf4j.org/) как движо ### Конфигурация { #configuration-2 } -Предполагается что [настраиваться Logback](https://logback.qos.ch/manual/configuration.html) будет через `logback.xml`, а в конфигурации Kora указываться будут лишь уровни логирования, пример `logback.xml`: +`Logback` настраивается через `logback.xml`, а в конфигурации Kora обычно указываются только уровни логирования. +Пример `logback.xml`: ```xml - + + @@ -167,14 +200,47 @@ Kora использует [slf4j-api](https://www.slf4j.org/) как движо ``` +`ConsoleTextRecordEncoder` выводит текстовую запись лога и добавляет к ней структурированные данные из `StructuredArgument`, `Marker`, пар ключ-значение `SLF4J` и `MDC`. +Это единственный энкодер, поставляемый модулем: он формирует текст с добавленными структурированными полями, а не единый JSON-документ. +Запись выводится как обычная текстовая строка — `timestamp level [thread] logger - message` — за которой, при наличии структурированных полей, следуют строки `fieldName={json}` с отступом табуляцией: + +```text +2026-07-02 10:15:30.123 INFO [main] r.t.k.example.SomeService - userId=42 user logged in + role="admin" +``` + +`KoraAsyncAppender` используется для асинхронной записи логов: он сохраняет значения `MDC` из текущего контекста в `KoraLoggingEvent`, чтобы они не терялись при передаче записи в другой поток. + +### Собственный шаблон { #custom-pattern } + +Вместо `ConsoleTextRecordEncoder` можно использовать стандартный `PatternLayoutEncoder` вместе с конвертерами, которые отображают структурированные данные Kora. +`KoraMdcConverter` отображает `MDC` контекста Kora, а `KoraLoggingMarkerConverter` отображает маркер `StructuredArgument`; зарегистрируйте их как слова преобразования и сошлитесь на них в шаблоне: + +```xml + + + + + + + %d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %logger - %koraMdc%msg %koraMarker%n + + + + + + + +``` + ## Другая реализация { #other-implementation } -Kora использует [slf4j-api](https://www.slf4j.org/) как движок для логирования, можно подключить свою любую совместимую реализацию. -Базовый модуль добавляет поддержку структурированных логов и возможность конфигурации уровней логирования через [файл конфигурации](config.md). +Kora использует [`slf4j-api`](https://www.slf4j.org/) как фасад логирования, поэтому можно подключить любую совместимую реализацию. +Базовый модуль добавляет общие компоненты для структурированных логов и управления уровнями логирования через [файл конфигурации](config.md). ### Подключение { #dependency-2 } -Потребуется подключить общую реализацию логирования: +Требуется подключить общий модуль логирования: ===! ":fontawesome-brands-java: `Java`" @@ -204,31 +270,32 @@ Kora использует [slf4j-api](https://www.slf4j.org/) как движо ### Использование { #usage-2 } -При использовании собственной реализации потребуется предоставить реализацию `LoggingLevelApplier` который бы реализовывал -установление уровня логирование и его сброс. +При использовании собственной реализации предоставьте компонент `LoggingLevelApplier`, который умеет применять уровень логирования для указанного `Logger` и сбрасывать уровни к их исходному состоянию. -Также потребуется в реализации самостоятельно поддержать запись `StructuredArgument`, `StructuredArgumentWriter` и `MDC` если они будут использоваться. +Если приложение использует структурированные данные, собственная реализация также должна поддерживать запись `StructuredArgument`, `StructuredArgumentWriter` и `MDC`. ## Структурированные логи { #structured-logs } -Передать структурированные данные в запись лога можно двумя способами через: +Структурированные логи позволяют передавать в запись лога не только текст, но и именованные поля. +Такие поля удобны для средств сбора логов и могут использоваться для поиска, фильтрации и построения представлений. -- Маркер -- Параметр +Передать структурированные данные в запись лога можно двумя способами: -Методы маркера и параметра также принимают в качестве аргументов `Long`, `Integer`, `String`, `Boolean` и `Map`. +- через `Marker`; +- через параметр сообщения. -### Маркер { #marker } +Методы `marker` и `arg` также принимают значения `Long`, `Integer`, `String`, `Boolean` и `Map`. +Для более сложных объектов передайте свой `StructuredArgumentWriter` или `JsonWriter`. -Передать структурированные данные в лог можно через маркер: +### Marker { #marker } + +`Marker` добавляет структурированное поле к записи лога и не занимает место параметра в текстовом сообщении: ===! ":fontawesome-brands-java: `Java`" ```java var logger = LoggerFactory.getLogger(getClass()); - var marker = StructuredArgument.marker("key", gen -> { - gen.writeString("value"); - }); + var marker = StructuredArgument.marker("key", "value"); logger.info(marker, "message"); ``` @@ -236,47 +303,114 @@ Kora использует [slf4j-api](https://www.slf4j.org/) как движо ```kotlin val logger = LoggerFactory.getLogger(javaClass) - val marker = StructuredArgument.marker("key") { it.writeString("value") } + val marker = StructuredArgument.marker("key", "value") logger.info(marker, "message") ``` ### Параметр { #parameter } -Передать структурированные данные в лог можно через параметры: +Параметр сообщения добавляет структурированное поле через обычный массив аргументов `SLF4J`: ===! ":fontawesome-brands-java: `Java`" ```java var logger = LoggerFactory.getLogger(getClass()); - var parameter = StructuredArgument.arg("key", gen -> { - gen.writeString("value"); - }); - log.info("message", parameter); + var parameter = StructuredArgument.arg("key", "value"); + logger.info("message", parameter); ``` === ":simple-kotlin: `Kotlin`" ```kotlin val logger = LoggerFactory.getLogger(javaClass) - val parameter = StructuredArgument.arg("key") { it.writeString("value") } + val parameter = StructuredArgument.arg("key", "value") logger.info("message", parameter) ``` +### Сложный объект { #complex-object } + +Для значений, которые не являются `String`, числом, `Boolean` или `Map`, передайте `JsonWriter` (тот же генерируемый для типа писатель [`@Json`](json.md)) или необработанную лямбду `StructuredArgumentWriter`, которая пишет значение поля напрямую в `JsonGenerator`. +Обе перегрузки предоставляют и `arg`, и `marker`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var logger = LoggerFactory.getLogger(getClass()); + var parameter = StructuredArgument.arg("user", gen -> { + gen.writeStartObject(); + gen.writeStringField("id", "42"); + gen.writeStringField("role", "admin"); + gen.writeEndObject(); + }); + logger.info("user logged in", parameter); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val logger = LoggerFactory.getLogger(javaClass) + val parameter = StructuredArgument.arg("user") { gen -> + gen.writeStartObject() + gen.writeStringField("id", "42") + gen.writeStringField("role", "admin") + gen.writeEndObject() + } + logger.info("user logged in", parameter) + ``` + ### MDC { #mdc } -Структурные данные можно прикреплять ко всем записям в рамках контекста с помощью класса `ru.tinkoff.kora.logging.common.MDC`: +Структурированные данные можно прикрепить ко всем записям в рамках текущего контекста с помощью класса `ru.tinkoff.kora.logging.common.MDC`. +Значение будет добавляться в каждую запись лога, пока оно не будет удалено из `MDC`: + +!!! warning "Импорт" + + Используйте `ru.tinkoff.kora.logging.common.MDC`, а не `org.slf4j.MDC`. Kora хранит свой `MDC` внутри контекста Kora, а не в thread-local, поэтому значения, помещенные в `org.slf4j.MDC`, не отображаются энкодерами Kora и не распространяются через асинхронные границы. Декларативную альтернативу смотрите в [`@Mdc`](logging-aspect.md). ===! ":fontawesome-brands-java: `Java`" ```java - MDC.put("key", gen -> gen.writeString("value")); + MDC.put("key", "value"); + try { + logger.info("message"); + } finally { + MDC.remove("key"); + } ``` === ":simple-kotlin: `Kotlin`" ```kotlin - MDC.put("key") { it.writeString("value") } + MDC.put("key", "value") + try { + logger.info("message") + } finally { + MDC.remove("key") + } + ``` + +`put` принимает значения `String`, `Integer`, `Long` и `Boolean`, а также необработанный `StructuredArgumentWriter` для произвольного JSON; типизированные значения отображаются как их JSON-тип, а не как текст. +Также есть перегрузка `put(Context, key, value)` для записи в явно переданный контекст вместо текущего: + +===! ":fontawesome-brands-java: `Java`" + + ```java + MDC.put("userId", 42); //(1)! + logger.info("user resolved"); ``` -Если вы используете `AsyncAppender` для отправки логов, то для корректной передачи MDC параметров нужно воспользоваться `ru.tinkoff.kora.logging.logback.KoraAsyncAppender`, -который передаст делегату `ru.tinkoff.kora.logging.logback.KoraLoggingEvent`, содержащий, в том числе структурный MDC. + 1. Отображается как JSON-число (`userId=42`), а не как строка. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + MDC.put("userId", 42) //(1)! + logger.info("user resolved") + ``` + + 1. Отображается как JSON-число (`userId=42`), а не как строка. + +Поскольку `MDC` находится в контексте Kora, он распространяется через асинхронные и реактивные границы вместе с контекстом. + +Если используется `AsyncAppender`, для корректной передачи параметров `MDC` используйте `ru.tinkoff.kora.logging.logback.KoraAsyncAppender`. +Он делает снимок `MDC` текущего контекста в момент добавления и передает делегату `ru.tinkoff.kora.logging.logback.KoraLoggingEvent`, поэтому структурированный `MDC` сохраняется при передаче записи в асинхронный рабочий поток. diff --git a/mkdocs/docs/ru/documentation/mapstruct.md b/mkdocs/docs/ru/documentation/mapstruct.md index d660fe7..9153fd5 100644 --- a/mkdocs/docs/ru/documentation/mapstruct.md +++ b/mkdocs/docs/ru/documentation/mapstruct.md @@ -1,13 +1,16 @@ --- -description: "Explains Kora MapStruct integration for generated mappers and dependency injection of mapper components. Use when working with @Mapper, MapStruct, MapStructModule, @Component, annotation processor." +description: "Explains Kora MapStruct integration: MapStruct-generated @Mapper implementations become injectable Kora components, dependency injection of mapper helpers via uses, tags, and the compile-time extension. Use when working with @Mapper, @Mapping, MapStruct, generated Impl, uses, injectionStrategy, componentModel, @Tag, annotation processor, KSP." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora MapStruct integration for generated mappers and dependency injection of mapper components; key triggers include @Mapper, MapStruct, MapStructModule, @Component, annotation processor." + use_when: "Use this file for Kora docs or implementation questions about Kora MapStruct integration where MapStruct-generated @Mapper implementations become injectable Kora components; key triggers include @Mapper, @Mapping, MapStruct, generated Impl, uses, injectionStrategy, componentModel, @Tag, annotation processor, KSP." --- Модуль позволяет интегрировать библиотеку [MapStruct](https://mapstruct.org/) для преобразования классов между собой. ## Подключение { #dependency } +Интеграция с Kora — это расширение времени компиляции, которое активируется автоматически, как только `mapstruct-processor` +оказывается в classpath обработчиков аннотаций — никакой дополнительный артефакт Kora или подключение модуля не требуется. + ===! ":fontawesome-brands-java: `Java`" [Зависимость](general.md#dependencies) `build.gradle`: @@ -15,19 +18,19 @@ agent: annotationProcessor "org.mapstruct:mapstruct-processor:1.5.5.Final" implementation "org.mapstruct:mapstruct:1.5.5.Final" ``` - + === ":simple-kotlin: `Kotlin`" - [MapStruct](https://mapstruct.org/) в Kotlin работает с помощью [kapt](https://kotlinlang.org/docs/kapt.html), по этому требуется также настроить плагин `build.gradle.kts`: + [MapStruct](https://mapstruct.org/) в Kotlin работает через [kapt](https://kotlinlang.org/docs/kapt.html), поэтому требуется настроить плагин kapt в `build.gradle.kts`: ```groovy plugins { kotlin("kapt") version ("1.9.10") } ``` - Последняя рабочая версия для `kapt` + `ksp` является `1.9.10-1.0.13`, в последующих версиях KSP совместимость между этими двумя инструментами сломали на уровне Gradle Plugin. + Последняя рабочая версия для `kapt` + `ksp` — это `1.9.10-1.0.13`, в более поздних версиях KSP совместимость между двумя инструментами нарушена на уровне Gradle-плагина. - Надо разрешить использовать выходные данные [kapt](https://kotlinlang.org/docs/kapt.html) как входные для [KSP](https://kotlinlang.org/docs/ksp-overview.html) `build.gradle.kts`: + Необходимо разрешить использование выходных данных [kapt](https://kotlinlang.org/docs/kapt.html) в качестве входных для [KSP](https://kotlinlang.org/docs/ksp-overview.html) в `build.gradle.kts`: ```groovy ksp { allowSourcesFromOtherPlugins = true @@ -38,9 +41,9 @@ agent: } ``` - Успешная сборка приложения может быть только со второго раза, это особенности KSP. + Успешная сборка приложения возможна только со второй попытки, это особенность поведения KSP. - [Зависимость](general.md#dependencies) `build.gradle.kts`: + [Зависимость](general.md#dependencies) `build.gradle.kts`: ```groovy kapt("org.mapstruct:mapstruct-processor:1.5.5.Final") implementation("org.mapstruct:mapstruct:1.5.5.Final") @@ -48,30 +51,74 @@ agent: ## Использование { #usage } -Создание самих преобразователей ложится на библиотеку [MapStruct](https://mapstruct.org/), -Kora в данном случае лишь предоставляет созданные библиотекой классы как зависимости в контейнер зависимостей. +Создание самих мапперов возлагается на библиотеку [MapStruct](https://mapstruct.org/); Kora лишь добавляет +расширение времени компиляции, которое делает сгенерированные мапперы доступными в контейнере зависимостей. + +Расширение регистрируется автоматически через `ServiceLoader` и активируется, как только аннотация `org.mapstruct.Mapper` +присутствует в classpath (расширение обработчика аннотаций для Java, расширение KSP для Kotlin). Для каждого +запрошенного интерфейса или абстрактного класса `@Mapper` оно находит сгенерированную MapStruct реализацию `Impl` в том же пакете +и предоставляет её публичный конструктор как компонент. Благодаря этому вам **не** нужен ни модуль Kora, ни какая-либо конфигурация, и вам +**не** нужен `componentModel = "kora"` — стандартный `componentModel` работает из коробки. + +Объявите маппер стандартным для MapStruct способом, и он станет доступным для внедрения: ===! ":fontawesome-brands-java: `Java`" ```java - @KoraApp - public interface Application { + public enum CarType { TYPE1, TYPE2 } + + public record Car(String make, int numberOfSeats, CarType type) { } - public enum CarType {TYPE1, TYPE2} + public record CarDto(String make, int seatCount, String type) { } - public record Car(String make, int numberOfSeats, CarType type) { } + @Mapper + public interface CarMapper { - public record CarTO(String make, int seatCount, String type) { } + @Mapping(source = "numberOfSeats", target = "seatCount") + CarDto map(Car car); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + enum class CarType { TYPE1, TYPE2 } + + data class Car(val make: String, val numberOfSeats: Int, val type: CarType) + + data class CarDto(val make: String, val seatCount: Int, val type: String) + + @Mapper + interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + fun map(car: Car): CarDto + } + ``` - @Mapper - public interface CarMapper { +`@Mapper` поддерживается как на интерфейсах, так и на абстрактных классах, а также на мапперах, вложенных внутрь внешнего типа — в +случае вложенного типа расширение определяет сгенерированную реализацию, соединяя имена внешних типов через `$` +(например, `SomeInterface.CarMapper` становится `SomeInterface$CarMapperImpl`). - @Mapping(source = "numberOfSeats", target = "seatCount") - CarTO map(Car car); +### Использование в сервисе { #service } + +Внедрённый маппер — это обычный компонент Kora, поэтому вы внедряете его через конструктор в сервис +[@Component](container.md#components), как и любую другую зависимость: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class CarService { + + private final CarMapper carMapper; + + public CarService(CarMapper carMapper) { + this.carMapper = carMapper; } - - default SomeService someService(CarMapper carMapper) { - return new SomeService(carMapper); + + public CarDto convert(Car car) { + return carMapper.map(car); } } ``` @@ -79,24 +126,109 @@ Kora в данном случае лишь предоставляет созда === ":simple-kotlin: `Kotlin`" ```kotlin - @KoraApp - interface Application { + @Component + class CarService(private val carMapper: CarMapper) { - enum class CarType { TYPE1, TYPE2 } + fun convert(car: Car): CarDto { + return carMapper.map(car) + } + } + ``` + +### Зависимости маппера { #dependencies } - data class Car(val make: String, val numberOfSeats: Int, val type: CarType) +Маппер часто делегирует работу вспомогательным мапперам или сервисам. MapStruct связывает такие вспомогательные компоненты через атрибут `uses` +аннотации `@Mapper`. Чтобы Kora предоставляла их из контейнера зависимостей (вместо того, чтобы MapStruct создавала их сам), сгенерируйте +реализацию с внедрением через конструктор: задайте `injectionStrategy = InjectionStrategy.CONSTRUCTOR` и +`componentModel = "jakarta"`. Тогда сгенерированный `Impl` получает каждый тип из `uses` через свой публичный конструктор, и +Kora разрешает каждый из них из графа — поэтому вспомогательный компонент должен быть доступен как компонент (например, помеченный аннотацией +[@Component](container.md#components) или предоставленный фабрикой). - data class CarTO(val make: String, val seatCount: Int, val type: String) +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class DateMapper { - @Mapper - interface CarMapper { + public String asString(Date date) { + return date != null ? new SimpleDateFormat("yyyy-MM-dd").format(date) : null; + } - @Mapping(source = "numberOfSeats", target = "seatCount") - fun map(car: Car): CarTO + public Date asDate(String date) throws ParseException { + return date != null ? new SimpleDateFormat("yyyy-MM-dd").parse(date) : null; } + } + + @Mapper(uses = DateMapper.class, + injectionStrategy = InjectionStrategy.CONSTRUCTOR, + componentModel = "jakarta") + public interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + CarDto map(Car car); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class DateMapper { + + fun asString(date: Date?): String? = + date?.let { SimpleDateFormat("yyyy-MM-dd").format(it) } + + fun asDate(date: String?): Date? = + date?.let { SimpleDateFormat("yyyy-MM-dd").parse(it) } + } + + @Mapper(uses = [DateMapper::class], + injectionStrategy = InjectionStrategy.CONSTRUCTOR, + componentModel = "jakarta") + interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + fun map(car: Car): CarDto + } + ``` + +### Тег { #tag } - fun someService(carMapper: CarMapper): SomeService { - return SomeService(carMapper) +`@Mapper` может быть уточнён тегом [@Tag](container.md#tags), и расширение предоставляет маппер только тогда, когда запрошенные +теги совпадают с тегами, объявленными на типе маппера. Это позволяет зарегистрировать несколько мапперов одного типа и различать их +в точке внедрения: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Tag(MyTag.class) + @Mapper + public interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + CarDto map(Car car); + } + + @Component + public final class CarService { + + public CarService(@Tag(MyTag.class) CarMapper carMapper) { + // ... } } ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Tag(MyTag::class) + @Mapper + interface CarMapper { + + @Mapping(source = "numberOfSeats", target = "seatCount") + fun map(car: Car): CarDto + } + + @Component + class CarService(@Tag(MyTag::class) private val carMapper: CarMapper) + ``` diff --git a/mkdocs/docs/ru/documentation/metrics.md b/mkdocs/docs/ru/documentation/metrics.md index 4d84c25..c2db0a1 100644 --- a/mkdocs/docs/ru/documentation/metrics.md +++ b/mkdocs/docs/ru/documentation/metrics.md @@ -4,11 +4,13 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora metrics with Micrometer, Prometheus export, OpenTelemetry metric standards, registry customization, and module-specific metric references; key triggers include MetricsModule, Micrometer, PrometheusMeterRegistry, MetricsConfig, PrometheusMeterRegistryInitializer, OpenTelemetry, Metrics Reference." --- -Модуль для сбора метрик приложения с использованием [Micrometer](https://micrometer.io/docs/concepts#_purpose). +Модуль для сбора метрик приложения с помощью [Micrometer](https://micrometer.io/docs/concepts#_purpose). +Он создает `PrometheusMeterRegistry`, подключает к нему метрики компонентов Kora и отдает результат в формате `Prometheus` через приватный `HTTP`-сервер. +Это позволяет собирать метрики приложения, `JVM`, процесса и встроенных интеграций в одном месте и опрашивать их внешней системой наблюдаемости. -Требует подключения [служебного HTTP сервера](http-server.md) для предоставления метрик в формате [prometheus](https://prometheus.io/docs/concepts/data_model/). +Для публикации метрик требуется [приватный HTTP-сервер](http-server.md), который отдает их в формате [Prometheus](https://prometheus.io/docs/concepts/data_model/). -Если нужен пошаговый разбор перед справочным описанием, смотрите [Наблюдаемость](../guides/observability.md). +Для пошагового разбора перед справочным описанием смотрите [Наблюдаемость](../guides/observability.md). ## Подключение { #dependency } @@ -40,7 +42,7 @@ agent: ## Конфигурация { #configuration } -Пример конфигурации пути HTTP сервера для получения метрик, описанной в классе `HttpServerConfig` (указаны значения по умолчанию): +Пример конфигурации пути приватного `HTTP`-сервера для получения метрик, описанной в классе `HttpServerConfig` (указаны значения по умолчанию): ===! ":material-code-json: `Hocon`" @@ -50,7 +52,7 @@ agent: } ``` - 1. Путь для получения метрик в формате `prometheus` (если подключен модуль [HTTP сервера](http-server.md)): + 1. Путь для получения метрик в формате `Prometheus` (по умолчанию: `"/metrics"`). === ":simple-yaml: `YAML`" @@ -59,7 +61,7 @@ agent: privateApiHttpMetricsPath: "/metrics" #(1)! ``` - 1. Путь для получения метрик в формате `prometheus` (если подключен модуль [HTTP сервера](http-server.md)): + 1. Путь для получения метрик в формате `Prometheus` (по умолчанию: `"/metrics"`). Пример полной конфигурации, описанной в классе `MetricsConfig` (указаны значения по умолчанию): @@ -71,7 +73,7 @@ agent: } ``` - 1. Формат метрик по стандарту OpenTelemetry (доступные значения: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/)) + 1. Формат метрик согласно стандарту `OpenTelemetry` (доступные значения: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/), по умолчанию: `V120`). === ":simple-yaml: `YAML`" @@ -80,19 +82,181 @@ agent: opentelemetrySpec: "V120" #(1)! ``` - 1. Формат метрик по стандарту OpenTelemetry (доступные значения: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/)) + 1. Формат метрик согласно стандарту `OpenTelemetry` (доступные значения: [V120](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/#migrating-from-a-version-prior-to-v1200) / [V123](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/), по умолчанию: `V120`). -Параметры конфигурации сбора метрик описываются в модулях в которых присутствует сбор метрик, например [HTTP сервер](http-server.md), [HTTP клиент](http-client.md) и т.д. +### Метрики модуля { #module-metrics } + +Блок `metrics` выше настраивает реестр глобально. Каждый модуль, собирающий метрики, дополнительно предоставляет собственный +блок `telemetry.metrics`, описанный в `TelemetryConfig.MetricsConfig`, который позволяет включать и отключать метрики, настраивать корзины гистограммы +и добавлять дополнительные теги только для этого модуля. В примере ниже в качестве носителя используется модуль [HTTP-сервер](http-server.md), но +те же поля `telemetry.metrics` применяются дословно к [HTTP-клиенту](http-client.md), [Базе данных](database-common.md), +[Kafka](kafka.md), [gRPC-серверу](grpc-server.md), [gRPC-клиенту](grpc-client.md), [Планировщику](scheduling.md), +[Кэшу](cache.md) и любой другой интеграции, которая сообщает метрики: + +===! ":material-code-json: `Hocon`" + + ```javascript + httpServer { + telemetry { + metrics { + enabled = true //(1)! + slo = [1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000] //(2)! + tags { //(3)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Включает сбор метрик для модуля (по умолчанию: `true`) + 2. Корзины гистограммы [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик `DistributionSummary`/`Timer` (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO` в миллисекундах для `V120` / `#DEFAULT_SLO_V123` в секундах для `V123`) + 3. Дополнительные общие теги, добавляемые к каждой метрике, которую сообщает модуль (по умолчанию: `{}`) + +=== ":simple-yaml: `YAML`" + + ```yaml + httpServer: + telemetry: + metrics: + enabled: true #(1)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(2)! + tags: #(3)! + key1: value1 + key2: value2 + ``` + + 1. Включает сбор метрик для модуля (по умолчанию: `true`) + 2. Корзины гистограммы [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик `DistributionSummary`/`Timer` (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO` в миллисекундах для `V120` / `#DEFAULT_SLO_V123` в секундах для `V123`) + 3. Дополнительные общие теги, добавляемые к каждой метрике, которую сообщает модуль (по умолчанию: `{}`) + +Установка `enabled = false` полностью отключает создание метрик для этого модуля (`MetricsFactory` модуля не возвращает +метрик), что является рекомендованным способом заглушить шумную интеграцию. Значения корзин `slo` по умолчанию для каждого стандарта +перечислены в разделе [Персонализация](#personalization). + +Параметры конфигурации сбора метрик также описаны в модулях, которые собирают метрики: [HTTP-сервер](http-server.md), [HTTP-клиент](http-client.md), [gRPC-сервер](grpc-server.md), [gRPC-клиент](grpc-client.md), [Планировщик](scheduling.md), [Кэш](cache.md) и другие интеграции. ## Использование { #usage } -Мы следуем и вам советуем использовать нотацию, описанную в [спецификации](https://prometheus.io/docs/concepts/data_model/). +Kora следует нотации, описанной в [спецификации `Prometheus`](https://prometheus.io/docs/concepts/data_model/). + +После подключения модуля `PrometheusMeterRegistry` регистрируется в `Metrics.globalRegistry` и используется всеми компонентами, которые собирают метрики. +При остановке приложения этот реестр удаляется из `Metrics.globalRegistry` и закрывается. + +Компонент `PrometheusMeterRegistryWrapper` является `Root`-компонентом и реализует `Wrapped`, поэтому пользовательский код может внедрять как обобщенный `MeterRegistry`, так и конкретный `PrometheusMeterRegistry`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + private final MeterRegistry meterRegistry; + + public SomeService(MeterRegistry meterRegistry) { + this.meterRegistry = meterRegistry; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val meterRegistry: MeterRegistry + ) + ``` + +Реестр автоматически получает стандартные привязки `Micrometer`: `ClassLoaderMetrics`, `JvmMemoryMetrics`, `JvmGcMetrics`, `JvmThreadMetrics`, `ProcessorMetrics`, `FileDescriptorMetrics`, `UptimeMetrics`. +Kora также регистрирует метрику `kora.up` со значением `1` и тегом `version`. + +Kora дополнительно связывает реестр `Micrometer` с `MeterProvider` из `OpenTelemetry` (`MicrometerMeterProvider` из `io.opentelemetry.contrib.metrics.micrometer`), поэтому библиотеки, инструментированные с помощью API метрик `OpenTelemetry`, публикуют данные через тот же `PrometheusMeterRegistry`. -После подключения модуля `Metrics.globalRegistry` будет зарегистрирован `PrometheusMeterRegistry`, который будет использоваться во всех компонентах, собирающих метрики. +Готовый к запуску базовый пример, который связывает `MetricsModule` вместе с `HoconConfigModule`, `LogbackModule`, `UndertowHttpServerModule` и экспортером `OpenTelemetry`, доступен в примере [kora-java-telemetry](https://github.com/kora-projects/kora-examples/tree/master/examples/java/kora-java-telemetry). + +### Экспорт в Prometheus { #prometheus-export } + +Метрики отдаются в текстовом формате [Prometheus](https://prometheus.io/docs/concepts/data_model/) [приватным HTTP-сервером](http-server.md) по пути `privateApiHttpMetricsPath` (по умолчанию `/metrics`), обслуживаемому на порту `privateApiHttpPort`. +У приватного сервера должен быть настроен порт, чтобы маршрут был доступен. +При примерной конфигурации (`privateApiHttpPort = 8085`) текущий снимок метрик можно получить так: + +```shell +curl http://localhost:8085/metrics +``` + +Направьте цель опроса вашего `Prometheus` (или любого совместимого сборщика) на тот же хост, порт и путь. + +### Пользовательская метрика { #custom-metric } + +Для пользовательской метрики лучше создать отдельный компонент, внедрить `MeterRegistry` и переиспользовать созданные экземпляры `Meter`. +Не создавайте новую метрику при каждом вызове метода: если набор тегов зависит от операции, используйте ключ с ограниченной кардинальностью и кэшируйте метрику в `ConcurrentHashMap`. +Вызов `register(...)` нужен для первоначальной регистрации метрики в `MeterRegistry`; на горячем пути предпочтительнее использовать уже созданный `Timer` / `Counter` / `Gauge` и вызывать только `record(...)` или `increment(...)`. +Kora использует такой же подход для своих внутренних метрик. + +Например, метрика длительности внешней операции: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class ExternalOperationMetrics { + private record Key(String operation, String status) {} + + private final MeterRegistry meterRegistry; + private final ConcurrentHashMap timers = new ConcurrentHashMap<>(); + + public ExternalOperationMetrics(MeterRegistry meterRegistry) { + this.meterRegistry = meterRegistry; + } + + public void record(String operation, String status, long durationNanos) { + var key = new Key(operation, status); + var timer = this.timers.computeIfAbsent(key, k -> Timer.builder("external.operation.duration") + .tag("operation", k.operation()) + .tag("status", k.status()) + .register(this.meterRegistry)); + + timer.record(durationNanos, TimeUnit.NANOSECONDS); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class ExternalOperationMetrics( + private val meterRegistry: MeterRegistry + ) { + private data class Key( + val operation: String, + val status: String + ) + + private val timers = ConcurrentHashMap() + + fun record(operation: String, status: String, durationNanos: Long) { + val key = Key(operation, status) + val timer = timers.computeIfAbsent(key) { + Timer.builder("external.operation.duration") + .tag("operation", it.operation) + .tag("status", it.status) + .register(meterRegistry) + } + + timer.record(durationNanos, TimeUnit.NANOSECONDS) + } + } + ``` + +Значения тегов должны иметь ограниченное число вариантов. +Не используйте в качестве тегов идентификаторы пользователей, номера запросов, полный текст ошибки или другие значения с высокой кардинальностью. ## Персонализация { #personalization } -Для внесения изменений в конфигурацию `PrometheusMeterRegistry` нужно добавить в контейнер `PrometheusMeterRegistryInitializer`. +Чтобы изменить конфигурацию `PrometheusMeterRegistry`, добавьте в контейнер `PrometheusMeterRegistryInitializer`. +Инициализатор получает созданный реестр до регистрации стандартных системных метрик, поэтому он может добавить общие теги, `MeterFilter`, правила переименования или пользовательские настройки `PrometheusMeterRegistry`. **Важно**, `PrometheusMeterRegistryInitializer` применяется только один раз при инициализации приложения. @@ -126,204 +290,226 @@ agent: } ``` -Так же стандартные метрики имеют некоторые конфигурации, такие как `ServiceLayerObjectives` для Distribution summary метрик. -Имена полей конфигурации можно посмотреть в `ru.tinkoff.kora.micrometer.module.MetricsConfig`. +У стандартных метрик также есть собственные настройки, например корзины гистограммы `slo` для метрик `DistributionSummary`/`Timer`, настраиваемые для каждого модуля в блоке [`telemetry.metrics`](#module-metrics). +Когда `slo` не переопределено, значения по умолчанию зависят от выбранного стандарта `OpenTelemetry`: + +- `V120` — `DEFAULT_SLO` в **миллисекундах**: `1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000` +- `V123` — `DEFAULT_SLO_V123` в **секундах**: `0.001, 0.010, 0.050, 0.100, 0.200, 0.500, 1, 2, 5, 10, 20, 30, 60, 90` + +Оба массива объявлены в `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig`; имена полей глобального реестра находятся в `ru.tinkoff.kora.micrometer.module.MetricsConfig`. + +### Поставщики тегов { #tag-providers } + +Набор тегов, прикрепляемых к метрикам фреймворка, формируется поставщиками тегов для каждого модуля, зарегистрированными как `@DefaultComponent`. +Чтобы изменить, какие теги выдаются для конкретной интеграции, предоставьте собственную реализацию соответствующего интерфейса в качестве переопределения `@DefaultComponent`: + +- `MicrometerHttpServerTagsProvider` (пакет `ru.tinkoff.kora.micrometer.module.http.server.tag`) — метрики HTTP-сервера +- `MicrometerHttpClientTagsProvider` (пакет `ru.tinkoff.kora.micrometer.module.http.client.tag`) — метрики HTTP-клиента +- `MicrometerGrpcServerTagsProvider` / `MicrometerGrpcClientTagsProvider` (пакеты `...grpc.server.tag` / `...grpc.client.tag`) — метрики gRPC +- `MicrometerKafkaConsumerTagsProvider` / `MicrometerKafkaProducerTagsProvider` (пакеты `...kafka.consumer.tag` / `...kafka.producer.tag`) — метрики Kafka + +Поставщик по умолчанию выбирается по значению `metrics.opentelemetrySpec`, поэтому переопределение заменяет сопоставление тегов для обоих стандартов. + +## Стандарт { #standard } -## Стандарты { #standard } +Изначально формат метрик использовал стандарт `V120` из `OpenTelemetry`; начиная с Kora `1.1.0` метрики также могут предоставляться +в стандарте `V123` из `OpenTelemetry`. Частичный список изменений доступен в [документации OpenTelemetry](https://opentelemetry.io/blog/2023/http-conventions-declared-stable/) +и в [руководстве по миграции OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/). -Изначальный формат метрик использовал стандарт OpenTelemetry `V120`, после Kora `1.1.0` появилась возможность предоставления метрик -в стандарте OpenTelemetry `V123`, частичный список изменений можно посмотреть [в документации OpenTelemetry](https://opentelemetry.io/blog/2023/http-conventions-declared-stable/) -и [рекомендациях миграции OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/http/migration-guide/) +Параметр `metrics.opentelemetrySpec` влияет на некоторые имена метрик, единицы измерения и наборы тегов. +Справочник ниже перечисляет варианты как `V120`, так и `V123` для таких метрик; если вариант не указан, имя одинаково для обоих стандартов. ## Справочник метрик { #metrics-reference } -Все метрики Kora используют [OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/semconv/) для именования и тегов. +Все метрики Kora используют [семантические соглашения OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/) для именования и тегов. Используемые типы метрик [Micrometer](https://docs.micrometer.io/micrometer/reference/concepts.html): -- [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) — применяется как инструмент для сбора распределения произвольных величин. -Тип метрики позволяет эффективно визуализировать данные по бакетам и рассчитывать персентиль. -- [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) — монотонно возрастающий счётчик +- [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) — используется для сбора распределений произвольных значений. +Этот тип метрики обеспечивает эффективную визуализацию данных по корзинам и вычисление процентилей. +- [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) — монотонно возрастающий счетчик - [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) — текущее значение метрики +- [Timer](https://docs.micrometer.io/micrometer/reference/concepts/timers.html) — длительность операции с поддержкой count, sum, max и корзин -### HTTP сервер { #http-server } +### HTTP-сервер { #http-server } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `http.server.request.duration` | `http_server_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки HTTP-запроса на сервере | `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | -| `http.server.active_requests` | `http_server_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество активных HTTP-запросов | `http.request.method`, `http.route`, `server.address`, `url.scheme` | +|--------|------------|------|-------------|------| +| `http.server.duration` (`V120`), `http.server.request.duration` (`V123`) | `http_server_duration_milliseconds` (`V120`) / `http_server_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки запроса `HTTP`-сервером | `V120`: `http.request.method`, `http.response.status_code`, `http.route`, `server.address`, `url.scheme`, `http.target`, `http.method`, `http.status_code`; `V123`: `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | +| `http.server.active_requests` | `http_server_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество активных `HTTP`-запросов | `V120`: `http.route`, `http.request.method`, `server.address`, `url.scheme`, `http.target`, `http.method`; `V123`: `http.route`, `http.request.method`, `server.address`, `url.scheme` | -Подробнее о модуле в документации [HTTP сервер](http-server.md). +Подробнее смотрите в документации модуля [HTTP-сервер](http-server.md). -### HTTP клиент { #http-client } +### HTTP-клиент { #http-client } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `http.client.request.duration` | `http_client_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность HTTP-запроса клиента | `http.request.method`, `http.response.status_code`, `server.address`, `url.scheme`, `http.route`, `error.type` | +|--------|------------|------|-------------|------| +| `http.client.duration` (`V120`), `http.client.request.duration` (`V123`) | `http_client_duration_milliseconds` (`V120`) / `http_client_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность запроса `HTTP`-клиента | `V120`: `http.request.method`, `http.response.status_code`, `server.address`, `url.scheme`, `http.route`, `http.status_code`, `http.method`, `http.target`, `error.type`; `V123`: `http.request.method`, `http.response.status_code`, `server.address`, `url.scheme`, `http.route`, `http.status_code`, `error.type` | -Подробнее о модуле в документации [HTTP клиент](http-client.md). +Подробнее смотрите в документации модуля [HTTP-клиент](http-client.md). ### База данных { #database } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `db.client.request.duration` | `db_client_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность операции/запроса к БД | `db.pool.name`, `db.statement`, `db.operation`, `error.type` | +|--------|------------|------|-------------|------| +| `database.client.request.duration` (`V120`), `db.client.request.duration` (`V123`) | `database_client_request_duration_milliseconds` (`V120`) / `db_client_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность операции/запроса к базе данных | `V120`: `pool`, `query.id`, `query.operation`, `error`; `V123`: `db.pool.name`, `db.statement`, `db.operation`, `error.type` | -Подробнее о модуле в документации [Базы данных](database-common.md). +Подробнее смотрите в документации модуля [База данных](database-common.md). ### Kafka { #kafka } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| +|--------|------------|------|-------------|------| | `messaging.receive.duration` | `messaging_receive_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки одного сообщения | `messaging.system`, `messaging.destination`, `messaging.operation`, `error.type` | | `messaging.publish.duration` | `messaging_publish_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность отправки сообщения | `messaging.system`, `messaging.destination`, `messaging.partition_id`, `error.type` | -| `messaging.process.batch.duration` | `messaging_process_batch_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки батча сообщений | `messaging.system`, `messaging.destination`, `error.type` | -| `messaging.kafka.consumer.lag` | `messaging_kafka_consumer_lag` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Лаг консьюмера по партициям | `messaging.system`, `messaging.destination`, `messaging.partition_id`, `messaging.consumer_group` | +| `messaging.process.batch.duration` | `messaging_process_batch_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки пакета сообщений | `messaging.system`, `messaging.destination`, `error.type` | +| `messaging.kafka.consumer.lag` | `messaging_kafka_consumer_lag` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Отставание потребителя по разделу | `messaging.system`, `messaging.destination`, `messaging.partition_id`, `messaging.consumer_group` | -Подробнее о модуле в документации [Kafka](kafka.md). +Подробнее смотрите в документации модуля [Kafka](kafka.md). -### gRPC сервер { #grpc-server } +### gRPC-сервер { #grpc-server } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `rpc.server.duration` | `rpc_server_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки gRPC-вызова на сервере | `rpc.service`, `rpc.method`, `rpc.status`, `error.type` | +|--------|------------|------|-------------|------| +| `rpc.server.duration` | `rpc_server_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки вызова gRPC-сервером | `rpc.service`, `rpc.method`, `rpc.status`, `error.type` | | `rpc.server.requests_per_rpc` | `rpc_server_requests_per_rpc_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество запросов, полученных за один RPC | `rpc.service`, `rpc.method` | | `rpc.server.responses_per_rpc` | `rpc_server_responses_per_rpc_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество ответов, отправленных за один RPC | `rpc.service`, `rpc.method` | -Подробнее о модуле в документации [gRPC сервер](grpc-server.md). +Подробнее смотрите в документации модуля [gRPC-сервер](grpc-server.md). -### gRPC клиент { #grpc-client } +### gRPC-клиент { #grpc-client } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `rpc.client.duration` | `rpc_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность gRPC-вызова клиента | `rpc.service`, `rpc.method`, `rpc.status`, `error.type`, `server.address` | +|--------|------------|------|-------------|------| +| `rpc.client.duration` | `rpc_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность вызова gRPC-клиента | `rpc.service`, `rpc.method`, `rpc.status`, `error.type`, `server.address` | | `rpc.client.requests_per_rpc` | `rpc_client_requests_per_rpc_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество запросов, отправленных за один RPC | `rpc.service`, `rpc.method`, `server.address` | | `rpc.client.responses_per_rpc` | `rpc_client_responses_per_rpc_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество ответов, полученных за один RPC | `rpc.service`, `rpc.method`, `server.address` | -Подробнее о модуле в документации [gRPC клиент](grpc-client.md). +Подробнее смотрите в документации модуля [gRPC-клиент](grpc-client.md). -### SOAP клиент { #soap-client } +### SOAP-клиент { #soap-client } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `rpc.client.duration` | `rpc_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность SOAP-вызова клиента | `rpc.system`, `rpc.service`, `rpc.method`, `rpc.result`, `server.address`, `server.port` | +|--------|------------|------|-------------|------| +| `rpc.client.duration` | `rpc_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность вызова SOAP-клиента | `rpc.system`, `rpc.service`, `rpc.method`, `rpc.result`, `server.address`, `server.port` | -Подробнее о модуле в документации [SOAP клиент](soap-client.md). +Подробнее смотрите в документации модуля [SOAP-клиент](soap-client.md). ### Планировщик { #scheduling } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| +|--------|------------|------|-------------|------| | `scheduling.job.duration` | `scheduling_job_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность выполнения запланированной задачи | `code.class`, `code.function`, `error.type` | -Подробнее о модуле в документации [Планировщик](scheduling.md). +Подробнее смотрите в документации модуля [Планировщик](scheduling.md). ### Кэш { #cache } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `cache.duration` | `cache_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность операции кэша (GET, SET, DELETE и т.д.) | `cache`, `operation`, `origin`, `status` | -| `cache.ratio` | `cache_ratio_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Счётчик попаданий/промахов кэша | `cache`, `origin`, `type` | +|--------|------------|------|-------------|------| +| `cache.duration` | `cache_duration_seconds` / `_count` / `_sum` / `_bucket` / `_max` | [Timer](https://docs.micrometer.io/micrometer/reference/concepts/timers.html) | Длительность операции с кэшем (`GET`, `SET`, `DELETE` и другие) | `cache`, `operation`, `origin`, `status` | +| `cache.ratio` | `cache_ratio_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Счетчик попаданий/промахов кэша | `cache`, `origin`, `type` | +| `cache.hit`, `cache.miss` | `cache_hit_total`, `cache_miss_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Устаревшие счетчики попаданий/промахов, сохраненные для совместимости | `cache`, `origin` | -При использовании Caffeine автоматически регистрируются стандартные метрики Micrometer: +Стандартные метрики `Micrometer` регистрируются автоматически при использовании `Caffeine`: | Метрика | Prometheus | Тип | Описание | -|---------|------------|-----|----------| -| `cache.gets` | `cache_gets_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество запросов к кэшу | +|--------|------------|------|-------------| +| `cache.gets` | `cache_gets_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество обращений к кэшу | | `cache.puts` | `cache_puts_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество записей в кэш | | `cache.evictions` | `cache_evictions_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество вытеснений из кэша | | `cache.size` | `cache_size` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Текущий размер кэша | -Подробнее о модуле в документации [Кэш](cache.md). +Подробнее смотрите в документации модуля [Кэш](cache.md). ### Redis / Lettuce { #redis-lettuce } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `lettuce.command.completion.duration` | `lettuce_command_completion_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность выполнения Redis-команды | `type`, `remote`, `local`, `command`, `error.type` | -| `lettuce.command.firstresponse.duration` | `lettuce_command_firstresponse_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Время до первого ответа Redis-команды | `type`, `remote`, `local`, `command`, `error.type` | +|--------|------------|------|-------------|------| +| `lettuce.command.completion.duration` | `lettuce_command_completion_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность завершения команды Redis | `type`, `remote`, `local`, `command`, `error.type` | +| `lettuce.command.firstresponse.duration` | `lettuce_command_firstresponse_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность первого ответа на команду Redis | `type`, `remote`, `local`, `command`, `error.type` | ### Отказоустойчивость { #resilience } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `resilient.circuitbreaker.state` | `resilient_circuitbreaker_state` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Состояние circuit breaker (0=CLOSED, 1=HALF_OPEN, 2=OPEN) | `name` | -| `resilient.circuitbreaker.transition` | `resilient_circuitbreaker_transition_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Переходы состояния circuit breaker | `name`, `state` | -| `resilient.circuitbreaker.call.acquire` | `resilient_circuitbreaker_call_acquire_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Попытки/отказы вызова через circuit breaker | `name`, `state`, `status` | -| `resilient.retry.attempts` | `resilient_retry_attempts_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество попыток ретрая | `name` | -| `resilient.retry.exhausted` | `resilient_retry_exhausted_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество исчерпанных ретраев | `name` | +|--------|------------|------|-------------|------| +| `resilient.circuitbreaker.state` | `resilient_circuitbreaker_state` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Состояние предохранителя (0=CLOSED, 1=HALF_OPEN, 2=OPEN) | `name` | +| `resilient.circuitbreaker.transition` | `resilient_circuitbreaker_transition_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Переходы состояний предохранителя | `name`, `state` | +| `resilient.circuitbreaker.call.acquire` | `resilient_circuitbreaker_call_acquire_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Попытки/отклонения захвата вызова предохранителем | `name`, `state`, `status` | +| `resilient.retry.attempts` | `resilient_retry_attempts_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество повторных попыток | `name` | +| `resilient.retry.exhausted` | `resilient_retry_exhausted_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество исчерпанных повторов | `name` | | `resilient.timeout.exhausted` | `resilient_timeout_exhausted_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество таймаутов | `name` | -| `resilient.fallback.attempts` | `resilient_fallback_attempts_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество вызовов фолбэка | `name`, `type` | +| `resilient.fallback.attempts` | `resilient_fallback_attempts_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество вызовов резервного варианта | `name`, `type` | -Подробнее о модуле в документации [Отказоустойчивость](resilient.md). +Подробнее смотрите в документации модуля [Отказоустойчивость](resilient.md). ### JMS { #jms } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `messaging.receive.duration` | `messaging_receive_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность получения JMS-сообщения | `messaging.system`, `messaging.destination.name`, `error.type` | +|--------|------------|------|-------------|------| +| `messaging.receive.duration` | `messaging_receive_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность получения сообщения JMS | `messaging.system`, `messaging.destination.name`, `error.type` | -### S3 клиент { #s3-client } +### S3-клиент { #s3-client } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `s3.client.duration` | `s3_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность S3 HTTP-запроса | `aws.s3.bucket`, `aws.operation.name`, `error.type` | -| `s3.kora.client.duration` | `s3_kora_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность операции Kora S3-клиента | `aws.client.name`, `aws.s3.bucket`, `aws.operation.name`, `error.type` | +|--------|------------|------|-------------|------| +| `s3.client.duration` | `s3_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность HTTP-запроса к S3 | `aws.s3.bucket`, `aws.operation.name`, `error.type` | +| `s3.kora.client.duration` | `s3_kora_client_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность операции S3-клиента Kora | `aws.client.name`, `aws.s3.bucket`, `aws.operation.name`, `error.type` | -Подробнее о модуле в документации [S3 клиент](s3-client.md). +Подробнее смотрите в документации модуля [S3-клиент](s3-client.md). ### Camunda 7 BPMN { #camunda-7-bpmn } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `camunda.engine.delegate.duration` | `camunda_engine_delegate_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность выполнения Camunda BPMN Java delegate | `delegate`, `business.key`, `error.type` | -| `camunda.engine.delegate.active_requests` | `camunda_engine_delegate_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество активных executions делегата | `delegate`, `business.key` | +|--------|------------|------|-------------|------| +| `camunda.engine.delegate.duration` | `camunda_engine_delegate_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность выполнения Java-делегата Camunda BPMN | `delegate`, `business.key`, `error.type` | +| `camunda.engine.delegate.active_requests` | `camunda_engine_delegate_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество активных выполнений делегата | `delegate`, `business.key` | -Подробнее о модуле в документации [Camunda 7 BPMN](camunda7-bpmn.md). +Подробнее смотрите в документации модуля [Camunda 7 BPMN](camunda7-bpmn.md). ### Camunda REST { #camunda-rest } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `camunda.rest.server.request.duration` | `camunda_rest_server_request_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность Camunda REST-запроса | `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | -| `camunda.rest.server.active_requests` | `camunda_rest_server_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество активных Camunda REST-запросов | `http.route`, `http.request.method`, `server.address`, `url.scheme` | +|--------|------------|------|-------------|------| +| `camunda.rest.server.duration` (`V120`), `camunda.rest.server.request.duration` (`V123`) | `camunda_rest_server_duration_milliseconds` (`V120`) / `camunda_rest_server_request_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность запроса `Camunda REST` | `V120`: `http.request.method`, `http.response.status_code`, `http.route`, `server.address`, `url.scheme`, `http.target`, `http.method`, `http.status_code`; `V123`: `http.request.method`, `http.response.status_code`, `http.route`, `url.scheme`, `server.address`, `error.type` | +| `camunda.rest.server.active_requests` | `camunda_rest_server_active_requests` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество активных запросов Camunda REST | `http.route`, `http.request.method`, `server.address`, `url.scheme` | -Подробнее о модуле в документации [Camunda 7 REST](camunda7-rest.md). +Подробнее смотрите в документации модуля [Camunda 7 REST](camunda7-rest.md). ### Camunda 8 Worker { #camunda-8-worker } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `zeebe.worker.handler.duration` | `zeebe_worker_handler_duration_milliseconds` / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработки задачи Zeebe worker | `job.name`, `job.type`, `status`, `error`, `error.code` | -| `zeebe.worker.handler` | `zeebe_worker_handler_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Счётчик ошибок Zeebe worker | `job.name`, `job.type`, `status`, `error.code` | -| `zeebe.client.worker.job` | `zeebe_client_worker_job_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество активированных/обработанных задач Zeebe | `action`, `type` | +|--------|------------|------|-------------|------| +| `zeebe.worker.handler` (`V120`), `zeebe.worker.handler.duration` (`V123`) | `zeebe_worker_handler_seconds` (`V120`) / `zeebe_worker_handler_duration_seconds` (`V123`) / `_count` / `_sum` / `_bucket` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность обработчика задачи `Zeebe Worker` | `job.name`, `job.type`, `status`, `error`, `error.code` | +| `zeebe.worker.handler` | `zeebe_worker_handler_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Счетчик ошибок `Zeebe Worker` | `job.name`, `job.type`, `status`, `error.code` | +| `zeebe.client.worker.job` | `zeebe_client_worker_job_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Количество активированных и обработанных задач `Zeebe` | `action`, `type` | -Подробнее о модуле в документации [Camunda 8 Worker](camunda8-worker.md). +Подробнее смотрите в документации модуля [Camunda 8 Worker](camunda8-worker.md). ### Система { #system } | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `kora.up` | `kora_up` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Индикатор статуса фреймворка (значение = 1) | `version` | +|--------|------------|------|-------------|------| +| `kora.up` | `kora_up` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Индикатор состояния фреймворка (значение = 1) | `version` | ### JVM { #jvm } -Стандартные JVM-метрики собираются автоматически через [Micrometer](https://docs.micrometer.io/micrometer/reference/concepts.html): +Стандартные метрики JVM собираются автоматически через [Micrometer](https://docs.micrometer.io/micrometer/reference/concepts.html): | Метрика | Prometheus | Тип | Описание | Теги | -|---------|------------|-----|----------|------| -| `jvm.gc.pause` | `jvm_gc_pause_milliseconds` / `_count` / `_sum` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность пауз сборщика мусора | `action`, `cause` | -| `jvm.gc.memory.allocated` | `jvm_gc_memory_allocated_bytes_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Объём выделенной памяти | — | -| `jvm.gc.memory.promoted` | `jvm_gc_memory_promoted_bytes_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Объём памяти, перемещённой в old gen | — | -| `jvm.gc.max.data.size` | `jvm_gc_max_data_size_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Максимальный размер old gen | — | -| `jvm.gc.live.data.size` | `jvm_gc_live_data_size_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Размер old gen после полной сборки мусора | — | -| `jvm.memory.used` | `jvm_memory_used_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Использованная память | `area`, `id` | -| `jvm.memory.committed` | `jvm_memory_committed_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Выделенная JVM память | `area`, `id` | +|--------|------------|------|-------------|------| +| `jvm.gc.pause` | `jvm_gc_pause_milliseconds` / `_count` / `_sum` / `_max` | [DistributionSummary](https://docs.micrometer.io/micrometer/reference/concepts/distribution-summaries.html) | Длительность паузы GC | `action`, `cause` | +| `jvm.gc.memory.allocated` | `jvm_gc_memory_allocated_bytes_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Размер выделенной памяти | — | +| `jvm.gc.memory.promoted` | `jvm_gc_memory_promoted_bytes_total` | [Counter](https://docs.micrometer.io/micrometer/reference/concepts/counters.html) | Память, повышенная в старое поколение | — | +| `jvm.gc.max.data.size` | `jvm_gc_max_data_size_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Максимальный размер старого поколения | — | +| `jvm.gc.live.data.size` | `jvm_gc_live_data_size_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Размер старого поколения после полной сборки GC | — | +| `jvm.memory.used` | `jvm_memory_used_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Используемая память | `area`, `id` | +| `jvm.memory.committed` | `jvm_memory_committed_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Зарезервированная память JVM | `area`, `id` | | `jvm.memory.max` | `jvm_memory_max_bytes` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Максимально доступная память | `area`, `id` | -| `jvm.threads.live` | `jvm_threads_live_threads` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество активных потоков | — | -| `jvm.threads.daemon` | `jvm_threads_daemon_threads` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество daemon-потоков | — | +| `jvm.threads.live` | `jvm_threads_live_threads` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество живых потоков | — | +| `jvm.threads.daemon` | `jvm_threads_daemon_threads` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество потоков-демонов | — | | `jvm.threads.peak` | `jvm_threads_peak_threads` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Пиковое количество потоков | — | -| `jvm.threads.states` | `jvm_threads_states_threads` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество потоков по состояниям | `state` | +| `jvm.threads.states` | `jvm_threads_states_threads` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество потоков по состоянию | `state` | | `process.cpu.usage` | `process_cpu_usage` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Использование CPU процессом | — | | `system.cpu.usage` | `system_cpu_usage` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Использование CPU системой | — | | `system.cpu.count` | `system_cpu_count` | [Gauge](https://docs.micrometer.io/micrometer/reference/concepts/gauges.html) | Количество доступных процессоров | — | diff --git a/mkdocs/docs/ru/documentation/netty.md b/mkdocs/docs/ru/documentation/netty.md index 68031ed..339494f 100644 --- a/mkdocs/docs/ru/documentation/netty.md +++ b/mkdocs/docs/ru/documentation/netty.md @@ -1,21 +1,63 @@ --- -description: "Explains Kora Netty customization and transport configuration used by HTTP clients, gRPC clients and servers, and Vert.x integrations. Use when working with NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." +description: "Explains Kora Netty customization and transport configuration used by HTTP Async clients, gRPC clients and gRPC servers. Use when working with NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora Netty customization and transport configuration used by HTTP clients, gRPC clients and servers, and Vert.x integrations; key triggers include NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." + use_when: "Use this file for Kora docs or implementation questions about Kora Netty customization and transport configuration used by HTTP Async clients, gRPC clients and gRPC servers; key triggers include NettyModule, EventLoopGroup, NettyTransport, Epoll, KQueue, NIO." --- -Функционал настраивающий работу Netty компонент которые используются другими модулями как [Vertx](database-vertx.md), [HTTP Async клиент](http-client.md#asynchttpclient), [gRPC клиент](grpc-client.md), [gRPC сервер](grpc-server.md). +Netty — это библиотека для сетевого взаимодействия, построенная вокруг неблокирующего ввода-вывода и модели `event loop`. +В Kora она используется как низкоуровневый механизм сетевого `транспорта` для модулей, которым нужно эффективно обрабатывать соединения и сетевые события. -Сам модуль самостоятельно не предоставляет какой-либо пользы, -а лишь служит для настройки [Netty транспорта и цикла событий Netty](https://netty.io/4.1/api/io/netty/channel/EventLoop.html) в рамках Kora. +Функционал настраивает работу общих компонентов Netty, которые используются другими модулями: [асинхронным HTTP-клиентом](http-client.md#asynchttpclient), [gRPC-клиентом](grpc-client.md), [gRPC-сервером](grpc-server.md). +Эти настройки полезны, когда приложению нужно управлять сетевым `транспортом`, количеством потоков обработки ввода-вывода или выбором `платформенного транспорта`. +Обычно значения по умолчанию подходят для большинства сервисов, но при высокой сетевой нагрузке или особых требованиях к окружению их можно задать явно. + +Сам модуль не предоставляет отдельный пользовательский `программный интерфейс`, +а служит для настройки [`транспорта Netty` и `цикла событий Netty`](https://netty.io/4.1/api/io/netty/channel/EventLoop.html) в рамках Kora. ## Подключение { #connection } -Модуль будет транзитивно предоставлен использующими его зависимостям. +Обычно модуль не требуется подключать вручную: его добавляют как транзитивную зависимость модули Kora, которым нужен Netty. + +## Что предоставляется { #what-it-provides } + +При подключении модуля `NettyCommonModule` добавляет в контейнер зависимостей следующие общие компоненты. +Модули-потребители ([асинхронный HTTP-клиент](http-client.md#asynchttpclient), [gRPC-клиент](grpc-client.md), [gRPC-сервер](grpc-server.md)) внедряют их вместо создания собственных потоков Netty: + +- **`NettyTransportConfig`** — конфигурация, привязанная к секции `netty` (предпочтительный [транспорт](#transport) и [количество рабочих потоков](#configuration)). +- **Рабочая `EventLoopGroup`** с тегом `@Tag(NettyCommonModule.WorkerLoopGroup.class)` — общий `цикл событий`, который обрабатывает соединения и сетевой ввод-вывод. Его размер задается параметром `threads`, и его используют как клиенты, так и серверы. +- **`EventLoopGroup` для приема соединений (boss)** с тегом `@Tag(NettyCommonModule.BossLoopGroup.class)` — отдельная группа, фиксированная на `1` поток, которую используют только серверные компоненты (например, [gRPC-сервер](grpc-server.md)) для приема входящих соединений; параметр `threads` на нее не влияет. +- **[`NettyChannelFactory`](#channel-factory)** — фабрика, создающая каналы Netty, соответствующие выбранному [транспорту](#transport). + +Обе группы `цикла событий` управляются [жизненным циклом](container.md#component-lifecycle) Kora: они корректно останавливаются после освобождения всех зависимых компонентов, поэтому ручное управление не требуется. + +Продвинутые модули, которые строят собственный `транспорт` Netty, могут внедрять эти компоненты напрямую: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MyNettyTransport { + + public MyNettyTransport(@Tag(NettyCommonModule.WorkerLoopGroup.class) EventLoopGroup workerGroup, + NettyChannelFactory channelFactory) { + // build a client or server bootstrap on the shared event loop + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MyNettyTransport( + @Tag(NettyCommonModule.WorkerLoopGroup::class) workerGroup: EventLoopGroup, + channelFactory: NettyChannelFactory, + ) + ``` ## Конфигурация { #configuration } -Пример конфигурации описанной в классе `NettyTransportConfig`: +Пример конфигурации, описанной в классе `NettyTransportConfig`: ===! ":material-code-json: `Hocon`" @@ -26,11 +68,8 @@ agent: } ``` - 1. Предпочитаемый [траснпорт](https://netty.io/wiki/native-transports.html) если доступен на пути как зависимость, по умолчанию выбирается в порядке доступности: - 1. `Epoll` (надо подключить [зависимость](https://mvnrepository.com/artifact/io.netty/netty-transport-native-epoll)) - 2. `KQueue` (надо подключить [зависимость](https://mvnrepository.com/artifact/io.netty/netty-transport-native-kqueue)) - 3. `Nio` - 2. Количество потоков [цикла событий Netty](https://netty.io/4.1/api/io/netty/channel/EventLoop.html), по умолчанию равен кол-во ядер процессора умноженных на 2 + 1. Предпочитаемый [транспорт](https://netty.io/wiki/native-transports.html): `NIO`, `EPOLL` или `KQUEUE` (по умолчанию не указано, необязательно). + 2. Количество потоков `worker event loop` (по умолчанию: количество доступных ядер процессора, умноженное на `2`). Для серверных компонентов дополнительно создается `boss event loop` с `1` потоком, значение `threads` на него не влияет. === ":simple-yaml: `YAML`" @@ -40,8 +79,74 @@ agent: threads: 2 #(2)! ``` - 1. Предпочитаемый [траснпорт](https://netty.io/wiki/native-transports.html) если доступен на пути как зависимость, по умолчанию выбирается в порядке доступности: - 1. `Epoll` (надо подключить [зависимость](https://mvnrepository.com/artifact/io.netty/netty-transport-native-epoll)) - 2. `KQueue` (надо подключить [зависимость](https://mvnrepository.com/artifact/io.netty/netty-transport-native-kqueue)) - 3. `Nio` - 2. Количество потоков [цикла событий Netty](https://netty.io/4.1/api/io/netty/channel/EventLoop.html), по умолчанию равен кол-во ядер процессора умноженных на 2 + 1. Предпочитаемый [транспорт](https://netty.io/wiki/native-transports.html): `NIO`, `EPOLL` или `KQUEUE` (по умолчанию не указано, необязательно). + 2. Количество потоков `worker event loop` (по умолчанию: количество доступных ядер процессора, умноженное на `2`). Для серверных компонентов дополнительно создается `boss event loop` с `1` потоком, значение `threads` на него не влияет. + +## Транспорт { #transport } + +Параметр `transport` задает предпочтительный `транспорт` Netty: + +- `NIO` - стандартный `транспорт` Java NIO, доступен всегда. +- `EPOLL` - `платформенный транспорт` Linux. +- `KQUEUE` - `платформенный транспорт` macOS / BSD. + +Если параметр `transport` не задан, Kora выбирает первый доступный транспорт в порядке: + +1. `EPOLL` +2. `KQUEUE` +3. `NIO` + +Если указанный `платформенный транспорт` недоступен во время выполнения, Kora использует первый доступный `транспорт` из этого же порядка. + +## Платформенный транспорт { #native-transport } + +Для использования `EPOLL` или `KQUEUE` соответствующая платформенная зависимость Netty должна быть доступна в `пути классов времени выполнения`: + +- [`io.netty:netty-transport-native-epoll`](https://mvnrepository.com/artifact/io.netty/netty-transport-native-epoll) для Linux. +- [`io.netty:netty-transport-native-kqueue`](https://mvnrepository.com/artifact/io.netty/netty-transport-native-kqueue) для macOS / BSD. + +При подключении платформенной зависимости требуется выбрать `классификатор` под целевую платформу, например `linux-x86_64`, `osx-x86_64` или `osx-aarch_64`. + +???+ tip "Совет" + + Обычно достаточно не задавать `transport` явно и оставить автоматический выбор. `Платформенный транспорт` стоит подключать осознанно: например, если он нужен для производительности или возможностей Netty, недоступных в `NIO`. + +## Фабрика каналов { #channel-factory } + +`NettyChannelFactory` — это общий внедряемый компонент, который создает экземпляры [`ChannelFactory`](https://netty.io/4.1/api/io/netty/channel/ChannelFactory.html) Netty, соответствующие выбранному [транспорту](#transport). +Это продвинутая точка внедрения для модулей, которые строят собственный клиентский или серверный `bootstrap` Netty и хотят получать каналы, согласованные с выбранным `транспортом`: + +- `getClientFactory()` / `getClientFactory(boolean domainSocket)` — фабрика клиентских каналов. +- `getServerFactory()` / `getServerFactory(boolean domainSocket)` — фабрика серверных каналов. + +Перегрузки без аргументов создают стандартные сокет-каналы `TCP`. +Передача `domainSocket = true` запрашивает канал [Unix domain socket](https://en.wikipedia.org/wiki/Unix_domain_socket): его поддерживают `платформенные транспорты` `EPOLL` и `KQUEUE`, тогда как реализация `NIO` в настоящее время возвращается к стандартным сокет-каналам. + +## Фабрика потоков { #thread-factory } + +Обе группы `цикла событий` — рабочая и boss — принимают необязательную [`ThreadFactory`](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadFactory.html). +Чтобы настроить именование или приоритет потоков Netty, предоставьте компонент `ThreadFactory` с тегом `@Tag(NettyCommonModule.class)`; при его наличии Kora использует его для обеих групп: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends AsyncHttpClientModule { + + @Tag(NettyCommonModule.class) + default ThreadFactory nettyThreadFactory() { + return new DefaultThreadFactory("netty-io"); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : AsyncHttpClientModule { + + @Tag(NettyCommonModule::class) + fun nettyThreadFactory(): ThreadFactory = DefaultThreadFactory("netty-io") + } + ``` diff --git a/mkdocs/docs/ru/documentation/openapi-codegen.md b/mkdocs/docs/ru/documentation/openapi-codegen.md index 85000e8..0c75438 100644 --- a/mkdocs/docs/ru/documentation/openapi-codegen.md +++ b/mkdocs/docs/ru/documentation/openapi-codegen.md @@ -1,19 +1,21 @@ --- -description: "Explains Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support. Use when working with openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth." +description: "Explains Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support. Use when working with openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth, prefixPath, requestInDelegateParams, HttpClientTokenProvider, PrincipalWithScopes, ApiSecurity." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support; key triggers include openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth." + use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI code generation for HTTP clients and servers, generator options, tags, validation, interceptors, authorization, and JsonNullable support; key triggers include openapi-generator, @HttpClient, @HttpController, @InterceptWith, @Tag, @Validate, JsonNullable, primaryAuth, prefixPath, requestInDelegateParams, HttpClientTokenProvider, PrincipalWithScopes, ApiSecurity." --- -Модуль для создания декларативных HTTP-обработчиков [HTTP сервера](http-server.md) -либо создания декларативных [HTTP клиентов](http-client.md) из OpenAPI контрактов с использованием [OpenAPI Generator плагином](https://openapi-generator.tech/docs/plugins#gradle). +Этот модуль генерирует код Kora из контракта `OpenAPI` с помощью [OpenAPI Generator](https://openapi-generator.tech/docs/plugins#gradle). +Из единого описания API можно создать декларативные обработчики [HTTP-сервера](http-server.md) или декларативные [HTTP-клиенты](http-client.md), +а также модели запросов и ответов, мапперы, обработку авторизации и дополнительные аннотации. +Такой подход полезен, когда `OpenAPI` является источником истины для транспортного контракта, а код приложения должен автоматически ему следовать. -Если нужен пошаговый разбор перед справочным описанием, смотрите [OpenAPI HTTP сервер](../guides/openapi-http-server.md), [OpenAPI HTTP сервер продвинутый](../guides/openapi-http-server-advanced.md) и [OpenAPI HTTP клиент](../guides/openapi-http-client.md). +Если нужен пошаговый разбор перед справочным описанием, смотрите [OpenAPI HTTP-сервер](../guides/openapi-http-server.md), [продвинутый OpenAPI HTTP-сервер](../guides/openapi-http-server-advanced.md) и [OpenAPI HTTP-клиент](../guides/openapi-http-client.md). ## Подключение { #dependency } ===! ":fontawesome-brands-java: `Java`" - Зависимость генератора `build.gradle`: + Зависимость генератора в `build.gradle`: ```groovy buildscript { dependencies { @@ -22,18 +24,18 @@ agent: } ``` - Зависимость плагина `build.gradle`: + Зависимость плагина в `build.gradle`: ```groovy plugins { id "org.openapi.generator" version "7.14.0" } - - Использование других версий плагина не гарантируется т.к. может быть не совместимо на уровне кода. ``` + Работоспособность других версий плагина не гарантируется, поскольку API `OpenAPI Generator` может быть несовместимо на уровне кода. + === ":simple-kotlin: `Kotlin`" - [Зависимость](general.md#dependencies) `build.gradle.kts`: + [Зависимость](general.md#dependencies) в `build.gradle.kts`: ```groovy buildscript { dependencies { @@ -42,49 +44,323 @@ agent: } ``` - Зависимость плагина `build.gradle.kts`: + Зависимость плагина в `build.gradle.kts`: ```groovy plugins { id("org.openapi.generator") version("7.14.0") } - - Использование других версий плагина не гарантируется т.к. может быть не совместимо на уровне кода. ``` -Требует подключения [HTTP сервера](http-server.md) либо [HTTP клиента](http-client.md). + Работоспособность других версий плагина не гарантируется, поскольку API `OpenAPI Generator` может быть несовместимо на уровне кода. + +Сгенерированному коду также требуется модуль [HTTP-сервера](http-server.md) или [HTTP-клиента](http-client.md) в зависимости от выбранного режима генерации. ## Конфигурация { #configuration } -Конфигурировать требуется параметры [плагина OpenAPI Generator](https://openapi-generator.tech/docs/plugins#gradle): +Настройте параметры [плагина OpenAPI Generator](https://openapi-generator.tech/docs/plugins#gradle): + +- Параметры `Gradle`-плагина описаны в [документации плагина](https://github.com/OpenAPITools/openapi-generator/blob/v7.14.0/modules/openapi-generator-gradle-plugin/README.adoc). +- Параметр плагина `configOptions` описан в [документации по конфигурации](https://openapi-generator.tech/docs/configuration/). +- Параметр плагина `openapiNormalizer` описан в [документации по настройке](https://openapi-generator.tech/docs/customization/#normalizer-opts). + +### Общие параметры { #common-opts } + +Помимо специфичных для Kora `configOptions`, `GenerateTask` принимает общие параметры `OpenAPI Generator`. +Они определяют, откуда читать контракт, куда помещать сгенерированные файлы, какие пакеты использовать и как предобрабатывать описание `OpenAPI`. +В проектах Kora эти параметры обычно задаются явно, поскольку сгенерированный код затем добавляется в обычную компиляцию проекта. + +| Параметр | Описание | +| -------- | -------- | +| `generatorName` | Имя генератора (`обязательный`, без значения по умолчанию). Для Kora всегда указывайте `kora`. | +| `inputSpec` | Путь к файлу `OpenAPI` (`обязательный`, без значения по умолчанию). Обычно это файл в `src/main/resources/openapi`, например `$projectDir/src/main/resources/openapi/openapi.yaml`. | +| `outputDir` | Каталог для сгенерированных файлов (по умолчанию не указан, необязательный). В проектах Kora это обычно каталог в `build`, например `$buildDir/generated/openapi`, который добавляется в основной набор исходного кода (source set). | +| `apiPackage` | Пакет для сгенерированных интерфейсов API, контроллеров, классов `delegate` и мапперов (по умолчанию: `org.openapitools.api`). Рекомендуется указывать его явно, например `ru.tinkoff.kora.example.openapi.api`. | +| `modelPackage` | Пакет для моделей, сгенерированных из схем `OpenAPI` (по умолчанию: `org.openapitools.model`). Рекомендуется указывать его явно, например `ru.tinkoff.kora.example.openapi.model`. | +| `invokerPackage` | Вспомогательный пакет генератора (по умолчанию: `org.openapitools.api`). Рекомендуется указывать его явно рядом с `apiPackage` и `modelPackage`, например `ru.tinkoff.kora.example.openapi.invoker`. | +| `configOptions` | Специфичные для генератора параметры (по умолчанию: `{}`). Для Kora здесь задаются `mode`, `clientConfigPrefix`, `enableServerValidation`, `interceptors` и другие параметры, описанные ниже. | +| `globalProperties` | Ограничивает, какие сущности генерируются (по умолчанию: `{}`). Полезно, когда нужно сгенерировать только `apis`, только `models` или отдельные модели и операции. Используйте осторожно: обычным клиентам и серверам Kora, как правило, нужны классы API, модели и мапперы вместе. | +| `openapiNormalizer` | Предобрабатывает контракт `OpenAPI` перед генерацией (по умолчанию: `{}`). Часто используется, чтобы отключить стандартные преобразования через `DISABLE_ALL`, сгенерировать только выбранные операции через `FILTER` или управлять правилами вроде `SIMPLIFY_ONEOF_ANYOF`. | +| `importMappings` | Сопоставляет имя схемы с существующим классом (по умолчанию: `{}`). Полезно, когда модель написана вручную или приходит из другого модуля, например `Money: "com.example.Money"`. | +| `typeMappings` | Сопоставляет тип `OpenAPI Generator` с типом языка (по умолчанию: `{}`). Используется для точечной замены типов, например замены `OffsetDateTime` на специфичный для проекта тип времени. | +| `schemaMappings` | Сопоставляет схему `OpenAPI` с внешним типом без генерации модели (по умолчанию: `{}`). Аналогично `importMappings`, но настраивается на уровне схемы и полезно для переиспользования общих DTO. | +| `skipValidateSpec` | Пропускает валидацию контракта `OpenAPI` перед генерацией (по умолчанию: `false`). В обычных сборках валидацию лучше оставлять включённой; используйте `true` только временно для внешних контрактов, которые нельзя быстро исправить. | +| `cleanupOutput` | Очищает `outputDir` перед генерацией (по умолчанию: `false`). Полезно, когда контракт часто меняется и файлы удалённых операций или моделей должны исчезать. Не указывайте в `outputDir` каталог с написанным вручную кодом. | + +Пример с общими параметрами: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + def openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient", GenerateTask) { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/openapi.yaml" + outputDir = "$buildDir/generated/openapi/client" + + def corePackage = "ru.tinkoff.kora.example.openapi" + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + + skipValidateSpec = false + cleanupOutput = true + openapiNormalizer = [ + DISABLE_ALL: "true", + FILTER: "tag:public|billing" + ] + configOptions = [ + mode: "java-client", + clientConfigPrefix: "httpClient.billing", + filterWithModels: "true" + ] + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + val openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient") { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/openapi.yaml" + outputDir = "$buildDir/generated/openapi/client" + + val corePackage = "ru.tinkoff.kora.example.openapi" + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + + skipValidateSpec = false + cleanupOutput = true + openapiNormalizer = mapOf( + "DISABLE_ALL" to "true", + "FILTER" to "tag:public|billing" + ) + configOptions = mapOf( + "mode" to "kotlin-client", + "clientConfigPrefix" to "httpClient.billing", + "filterWithModels" to "true" + ) + } + ``` + +Используйте `globalProperties` только для узких задач генерации, например при извлечении нескольких моделей в промежуточный модуль: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + globalProperties = [ + models: "User,Order", + apis: "false", + supportingFiles: "false" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + globalProperties = mapOf( + "models" to "User,Order", + "apis" to "false", + "supportingFiles" to "false" + ) + ``` + +### Параметры нормализации { #normalizer-opts } + +`openapiNormalizer` изменяет входной контракт `OpenAPI` перед генерацией. Это не параметр Kora, а общий механизм `OpenAPI Generator`. +Для Kora он особенно полезен, когда один большой контракт используется несколькими приложениями или когда контракт содержит неоднозначные для генерации кода конструкции. + +| Правило | Описание | +| -------- | -------- | +| `DISABLE_ALL` | Отключает стандартные правила нормализации (по умолчанию: `false`). Начиная с `OpenAPI Generator 7` некоторые правила включены по умолчанию, поэтому предсказуемая генерация часто начинается с `DISABLE_ALL: "true"`, а затем явно включаются только нужные правила. | +| `FILTER` | Оставляет для генерации только выбранные операции (по умолчанию не указано, необязательно). Поддерживает один фильтр за раз: `operationId:name1\|name2`, `method:get\|post` или `tag:public\|billing`. Операции, которые не подходят, помечаются как `x-internal: true` и не генерируются. | +| `KEEP_ONLY_FIRST_TAG_IN_OPERATION` | Оставляет у операции только первый тег (по умолчанию: `false`). Полезно, когда у операций несколько тегов и они разбиваются на несколько классов API не так, как вы ожидаете. | +| `SET_TAGS_FOR_ALL_OPERATIONS` | Заменяет теги всех операций одним переданным значением (по умолчанию не указано, необязательно). Полезно, когда нужно принудительно получить один сгенерированный класс API. | +| `SET_TAGS_TO_OPERATIONID` | Устанавливает тег операции равным `operationId`, либо `default`, если `operationId` пуст (по умолчанию: `false`). Полезно для контрактов без пригодных тегов, когда нужна предсказуемая группировка операций. | +| `SET_TAGS_TO_VENDOR_EXTENSION` | Читает теги операций из указанного расширения, например `x-tags` (по умолчанию не указано, необязательно). Полезно, когда внешний контракт нельзя изменить, но в нём уже есть собственная группировка операций. | +| `FIX_DUPLICATED_OPERATIONID` | Добавляет числовой суффикс к повторяющимся значениям `operationId` (по умолчанию: `false`). Лучше исправить контракт, но это правило помогает временно сгенерировать код по внешнему описанию. | +| `SET_BEARER_AUTH_FOR_NAME` | Преобразует указанную схему безопасности в `bearerAuth` (по умолчанию не указано, необязательно). Полезно для внешних контрактов, где bearer-токен описан нестандартно, но в приложении его нужно обрабатывать как обычную схему bearer. | +| `REF_AS_PARENT_IN_ALLOF` | Помечает `$ref` внутри `allOf` как родительскую схему через `x-parent: true` (по умолчанию: `false`). Может помочь контрактам, которые моделируют наследование через `allOf`. | +| `SIMPLIFY_ONEOF_ANYOF` | Упрощает некоторые конструкции `oneOf`/`anyOf`, например переносит вариант `null` в `nullable: true` и убирает одиночные обёртки (включено по умолчанию в `OpenAPI Generator 7`, если не задан `DISABLE_ALL`). Для Kora это может менять форму сгенерированных моделей, поэтому включайте его осознанно. | +| `SIMPLIFY_ANYOF_STRING_AND_ENUM_STRING` | Упрощает `anyOf`, составленный из `string` и строкового перечисления, до `string` (по умолчанию: `false`). Это может помочь с контрактами, где ограничение перечисления не важно для кода. | +| `SIMPLIFY_BOOLEAN_ENUM` | Преобразует булево перечисление в обычный `boolean` (включено по умолчанию в `OpenAPI Generator 7`, если не задан `DISABLE_ALL`). | +| `REFACTOR_ALLOF_WITH_PROPERTIES_ONLY` | Переносит свойства из схемы, содержащей одновременно `allOf` и `properties`, в отдельную схему внутри `allOf` (включено по умолчанию в `OpenAPI Generator 7`, если не задан `DISABLE_ALL`). Это может помочь наследованию, но строгие контракты стоит проверять после генерации. | +| `NORMALIZE_31SPEC` | Нормализует некоторые конструкции `OpenAPI 3.1` в форму, которую генератор понимает лучше (по умолчанию: `false`). Полезно для контрактов `3.1`, когда генерация не удаётся на новых формах схем. | +| `REMOVE_X_INTERNAL` | Удаляет `x-internal: true` из операций и моделей (по умолчанию: `false`). Используйте, только когда контракт уже содержит `x-internal`, но конкретная задача генерации должна принудительно вернуть такие операции. | +| `SET_CONTAINER_TO_NULLABLE` | Помечает типы-контейнеры `array`, `set` или `map` как `nullable` (по умолчанию не указано, необязательно). Используйте, только когда во внешнем контракте систематически отсутствует `nullable` у таких полей. | +| `SET_PRIMITIVE_TYPES_TO_NULLABLE` | Помечает примитивные типы `string`, `integer`, `number` или `boolean` как `nullable` (по умолчанию не указано, необязательно). Это существенно меняет сигнатуры моделей, поэтому применяйте его только к проблемным внешним контрактам. | + +Пример генерации только публичной части контракта: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + openapiNormalizer = [ + DISABLE_ALL: "true", + FILTER: "tag:public|billing" + ] + configOptions = [ + mode: "java-client", + filterWithModels: "true" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + openapiNormalizer = mapOf( + "DISABLE_ALL" to "true", + "FILTER" to "tag:public|billing" + ) + configOptions = mapOf( + "mode" to "kotlin-client", + "filterWithModels" to "true" + ) + ``` + +Сам по себе `FILTER` исключает только операции. Если после фильтрации нужно также удалить неиспользуемые модели, включите параметр Kora `filterWithModels`. +Для более сложного отбора обычно создают отдельные задачи генерации с разными значениями `FILTER`, например одну с `tag:billing`, а другую с `operationId:createUser|getUser`. + +Пример нормализации тегов для контракта без удобной группировки: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + openapiNormalizer = [ + DISABLE_ALL: "true", + SET_TAGS_TO_VENDOR_EXTENSION: "x-kora-tag", + FIX_DUPLICATED_OPERATIONID: "true" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + openapiNormalizer = mapOf( + "DISABLE_ALL" to "true", + "SET_TAGS_TO_VENDOR_EXTENSION" to "x-kora-tag", + "FIX_DUPLICATED_OPERATIONID" to "true" + ) + ``` + +### Параметры моделей { #model-opts } + +Kora также поддерживает несколько `configOptions`, управляющих мапперами `JSON` и общей генерацией моделей. +Они не зависят от того, генерируется клиент или сервер. + +| Параметр | Описание | +| -------- | -------- | +| `jsonAnnotation` | Аннотация-тег, используемая для внедрения мапперов `JSON` в сгенерированные мапперы запросов и ответов (по умолчанию: `ru.tinkoff.kora.json.common.annotation.Json`). | +| `objectType` | Тип для схем `type: object` без более точного описания. `Java` по умолчанию использует `java.lang.Object`, а `Kotlin` — `kotlin.Any`. Например, укажите `com.fasterxml.jackson.databind.JsonNode`, если приложение хочет обрабатывать произвольный `JSON` как дерево. | +| `disableHtmlEscaping` | Отключает экранирование HTML-символов в строках `JSON` (по умолчанию: `false`). Обычно значение по умолчанию оставляют. | +| `ignoreAnyOfInEnum` | Игнорирует `anyOf` при генерации перечислений (по умолчанию: `false`). Может помочь с контрактами, где перечисление описано через смешанные конструкции `anyOf`. | +| `discriminatorCaseSensitive` | Управляет чувствительностью к регистру при поиске значения дискриминатора для полиморфных (`oneOf`) моделей с дискриминатором (по умолчанию: `true`). Установите `false`, когда входящие значения дискриминатора могут отличаться регистром от определения в схеме. | +| `additionalModelTypeAnnotations` | Дополнительные аннотации на типах моделей (по умолчанию не указано, необязательно). Несколько аннотаций разделяются `;`, например `@Deprecated;@MyAnnotation`. | +| `additionalEnumTypeAnnotations` | Дополнительные аннотации на типах перечислений (по умолчанию не указано, необязательно). Несколько аннотаций разделяются `;`. | + +Пример: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-client", + jsonAnnotation: "ru.tinkoff.kora.json.common.annotation.Json", + objectType: "com.fasterxml.jackson.databind.JsonNode", + additionalModelTypeAnnotations: "@Deprecated" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "jsonAnnotation" to "ru.tinkoff.kora.json.common.annotation.Json", + "objectType" to "com.fasterxml.jackson.databind.JsonNode", + "additionalModelTypeAnnotations" to "@Deprecated" + ) + ``` + +### Несколько генераторов { #multiple-gens } + +В одном модуле можно зарегистрировать несколько задач `GenerateTask`, например чтобы сгенерировать два независимых контракта +или сгенерировать клиент для одного контракта и сервер для другого. Каждая задача пишет в один и тот же `outputDir` и добавляется в один и тот же набор исходного кода (source set), +поэтому единственное требование — чтобы сгенерированные пакеты не пересекались. Задайте каждой задаче собственные `apiPackage`/`modelPackage`/`invokerPackage`. + +===! ":fontawesome-brands-java: `Java`" -- Настройка параметров Gradle плагина в [документации](https://github.com/OpenAPITools/openapi-generator/blob/v7.14.0/modules/openapi-generator-gradle-plugin/README.adoc). -- Настройка `configOptions` параметра плагина в [документации](https://openapi-generator.tech/docs/generators/java/#config-options). -- Настройка `openapiNormalizer` параметра плагина в [документации](https://openapi-generator.tech/docs/customization/#openapi-normalizer). + ```groovy + def openApiGeneratePetV2 = tasks.register("openApiGeneratePetV2", GenerateTask) { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV2.yaml" + outputDir = "$buildDir/generated/openapi" + def corePackage = "ru.tinkoff.kora.example.openapi.petV2" //(1)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = [mode: "java-client", clientConfigPrefix: "httpClient.petV2"] + } + sourceSets.main { java.srcDirs += openApiGeneratePetV2.get().outputDir } + compileJava.dependsOn openApiGeneratePetV2 + + def openApiGeneratePetV3 = tasks.register("openApiGeneratePetV3", GenerateTask) { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV3.yaml" + outputDir = "$buildDir/generated/openapi" + def corePackage = "ru.tinkoff.kora.example.openapi.petV3" //(2)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = [mode: "java-reactive-client", clientConfigPrefix: "httpClient.petV3"] + } + sourceSets.main { java.srcDirs += openApiGeneratePetV3.get().outputDir } + compileJava.dependsOn openApiGeneratePetV3 + ``` + + 1. Изолированный пакет для первого контракта + 2. Другой пакет для второго контракта, чтобы имена классов не конфликтовали + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + val openApiGeneratePetV2 = tasks.register("openApiGeneratePetV2") { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV2.yaml" + outputDir = "$buildDir/generated/openapi" + val corePackage = "ru.tinkoff.kora.example.openapi.petV2" //(1)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = mapOf("mode" to "kotlin-client", "clientConfigPrefix" to "httpClient.petV2") + } + kotlin.sourceSets.main { kotlin.srcDir(openApiGeneratePetV2.get().outputDir) } + tasks.withType { dependsOn(openApiGeneratePetV2) } + + val openApiGeneratePetV3 = tasks.register("openApiGeneratePetV3") { + generatorName = "kora" + inputSpec = "$projectDir/src/main/resources/openapi/petstoreV3.yaml" + outputDir = "$buildDir/generated/openapi" + val corePackage = "ru.tinkoff.kora.example.openapi.petV3" //(2)! + apiPackage = "${corePackage}.api" + modelPackage = "${corePackage}.model" + invokerPackage = "${corePackage}.invoker" + configOptions = mapOf("mode" to "kotlin-suspend-client", "clientConfigPrefix" to "httpClient.petV3") + } + kotlin.sourceSets.main { kotlin.srcDir(openApiGeneratePetV3.get().outputDir) } + tasks.withType { dependsOn(openApiGeneratePetV3) } + ``` + + 1. Изолированный пакет для первого контракта + 2. Другой пакет для второго контракта, чтобы имена классов не конфликтовали ## Клиент { #client } -Минимальный пример настройки плагина для создания декларативного HTTP клиента: +Минимальная конфигурация плагина для создания декларативного HTTP-клиента: ===! ":fontawesome-brands-java: `Java`" - Доступные Kora параметры плагина (`configOptions`): - - - `clientConfigPrefix` - префикс конфигурации созданных HTTP-клиентов. Значение `строка`. - - `tags` - возможность проставлять дополнительные теги на созданные HTTP-клиенты - - `interceptors` - возможность указывать перехватчики для HTTP-клиентов - - `primaryAuth` - указать какой [механизм авторизации](http-client.md#custom-response) использовать как основной если указано несколько [securitySchemes]((https://swagger.io/docs/specification/authentication/)) в OpenAPI. Значение `строка`. - - `securityConfigPrefix` - префикс конфигурации механизм авторизации [Basic](http-client.md#basic)/[ApiKey](http-client.md#apikey) (путь конфигурации будет заданный префикс + имя [securitySchemes]((https://swagger.io/docs/specification/authentication/)) в OpenAPI, либо просто имя в OpenAPI если префикс не задан). Значение `строка`. - - `authAsMethodArgument` - возможность указывать авторизацию как аргумент метода HTTP клиента, а не через перехватчик. Значения: `true`, `false` - - `authAllowMultiple` - генерировать перехватчики для [мульти-авторизации](https://swagger.io/docs/specification/v3_0/authentication/#using-multiple-authentication-types) если таковая указана в спецификации. Значения: `true`, `false` - - `additionalContractAnnotations` - возможность указывать дополнительные аннотации над методами HTTP-клиента - - `enableJsonNullable` - обрабатывать `nullable=true` и `required=false` поля схем как [JsonNullable](json.md#jsonnullable-wrapper) обертку. Значения: `true`, `false` - - `forceIncludeOptional` - проставлять принудительно `@JsonInclude(Always)` для `nullable=true` и `required=false` полей вместо `enableJsonNullable`. Значения: `true`, `false`. - - `forceIncludeNonRequired` - проставлять принудительно [@JsonInclude(Always)](json.md#serialization-levels) для только `required=false` полей. Значения: `true`, `false`. - - `filterWithModels` - фильтровать и исключать из генерации также ненужные модели когда указана опция [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) в `openapiNormalizer`. Значения: `true`, `false` - - `mode` в каком режиме работать генератору, доступные значения: - * `java-client` - создание синхронного клиента - * `java-async-client` - создание [CompletionStage](https://www.baeldung.com/java-completablefuture) клиента - * `java-reactive-client` - создание [реактивного](https://projectreactor.io/docs/core/release/reference/) клиента, требуется подключить [Project Reactor](https://mvnrepository.com/artifact/io.projectreactor/reactor-core) самостоятельно. + Для клиентов `configOptions.mode` поддерживает `java-client`, `java-async-client` и `java-reactive-client`. + Остальные параметры клиента описаны ниже в разделах про авторизацию, перехватчики, теги, модели и неявные заголовки. ```groovy def openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient", GenerateTask) { @@ -108,35 +384,20 @@ agent: compileJava.dependsOn openApiGenerateHttpClient //(9)! ``` - 1. Путь до OpenAPI файла из которого будут созданы классы - 2. Директория куда буду создаваться файлы - 3. Пакет от классов делегатов, контроллеров, преобразователей и тп. - 4. Пакет от классов моделей, DTO и тп. - 5. Пакет от классов вызова - 6. Режим работы плагина (создание Java клиента / Kotlin / Java сервера и тп) - 7. Префикс путь к файлу конфигурации клиента - 8. Регистрируем созданные классы как исходный код проекта - 9. Делаем компиляцию кода, зависимой от генерации классов HTTP-клиента (сначала генерируем, потом компилируем) + 1. Путь к файлу `OpenAPI`, по которому создаются классы + 2. Каталог, где создаются сгенерированные файлы + 3. Пакет для делегатов, контроллеров и мапперов + 4. Пакет для моделей и DTO + 5. Вспомогательный пакет генератора + 6. Режим плагина + 7. Префикс пути конфигурации клиента + 8. Регистрирует сгенерированные классы как исходный код проекта + 9. Ставит компиляцию кода в зависимость от генерации классов HTTP-клиента: сначала генерация, затем компиляция === ":simple-kotlin: `Kotlin`" - Доступные Kora параметры плагина (`configOptions`): - - - `clientConfigPrefix` - префикс конфигурации созданных HTTP-клиентов. Значение `строка`. - - `tags` - возможность проставлять дополнительные теги на созданные HTTP-клиенты - - `interceptors` - возможность указывать перехватчики для HTTP-клиентов - - `primaryAuth` - указать какой [механизм авторизации](http-client.md#custom-response) использовать как основной если указано несколько [securitySchemes]((https://swagger.io/docs/specification/authentication/)) в OpenAPI. Значение `строка`. - - `securityConfigPrefix` - префикс конфигурации механизм авторизации [Basic](http-client.md#basic)/[ApiKey](http-client.md#apikey) (путь конфигурации будет заданный префикс + имя [securitySchemes]((https://swagger.io/docs/specification/authentication/)) в OpenAPI, либо просто имя в OpenAPI если префикс не задан). Значение `строка`. - - `authAsMethodArgument` - возможность указывать авторизацию как аргумент метода HTTP клиента, а не через перехватчик. Значения: `true`, `false` - - `authAllowMultiple` - генерировать перехватчики для [мульти-авторизации](https://swagger.io/docs/specification/v3_0/authentication/#using-multiple-authentication-types) если таковая указана в спецификации. Значения: `true`, `false` - - `additionalContractAnnotations` - возможность указывать дополнительные аннотации над методами HTTP-клиента - - `enableJsonNullable` - обрабатывать `nullable=true` и `required=false` поля схем как [JsonNullable](json.md#jsonnullable-wrapper) обертку. Значения: `true`, `false` - - `forceIncludeOptional` - проставлять принудительно `@JsonInclude(Always)` для `nullable=true` и `required=false` полей вместо `enableJsonNullable`. Значения: `true`, `false`. - - `forceIncludeNonRequired` - проставлять принудительно [@JsonInclude(Always)](json.md#serialization-levels) для только `required=false` полей. Значения: `true`, `false`. - - `filterWithModels` - фильтровать и исключать из генерации также ненужные модели когда указана опция [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) в `openapiNormalizer`. Значения: `true`, `false` - - `mode` в каком режиме работать генератору, доступные значения: - * `kotlin-client` - создание синхронного клиента - * `kotlin-suspend-client` - создание suspend клиента + Для клиентов `configOptions.mode` поддерживает `kotlin-client` и `kotlin-suspend-client`. + Остальные параметры клиента описаны ниже в разделах про авторизацию, перехватчики, теги, модели и неявные заголовки. ```groovy val openApiGenerateHttpClient = tasks.register("openApiGenerateHttpClient") { @@ -160,29 +421,271 @@ agent: tasks.withType { dependsOn(openApiGenerateHttpClient) } //(9)! ``` - 1. Путь до OpenAPI файла из которого будут созданы классы - 2. Директория куда буду создаваться файлы - 3. Пакет от классов делегатов, контроллеров, преобразователей и тп. - 4. Пакет от классов моделей, DTO и тп. - 5. Пакет от классов вызова - 6. Режим работы плагина (создание Java клиента / Kotlin / Java сервера и тп) - 7. Префикс путь к файлу конфигурации клиента - 8. Регистрируем созданные классы как исходный код проекта - 9. Делаем компиляцию кода, зависимой от генерации классов HTTP-клиента (сначала генерируем, потом компилируем) + 1. Путь к файлу `OpenAPI`, по которому создаются классы + 2. Каталог, где создаются сгенерированные файлы + 3. Пакет для делегатов, контроллеров и мапперов + 4. Пакет для моделей и DTO + 5. Вспомогательный пакет генератора + 6. Режим плагина + 7. Префикс пути конфигурации клиента + 8. Регистрирует сгенерированные классы как исходный код проекта + 9. Ставит компиляцию кода в зависимость от генерации классов HTTP-клиента: сначала генерация, затем компиляция -После создания HTTP-клиент будет доступен для внедрения как зависимость по созданному интерфейсу. +После генерации HTTP-клиент доступен для внедрения зависимостей через сгенерированный интерфейс. -### Перехватчики { #interceptors } +### Использование клиента { #client-usage } + +Для каждого тега API генератор создаёт интерфейс, аннотированный [`@HttpClient`](http-client.md), с именем по тегу (например `PetApi`). +Он внедряется в компоненты как любой другой клиент Kora, без дополнительной регистрации: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class RootService { + + private final PetApi petApi; //(1)! + + public RootService(PetApi petApi) { + this.petApi = petApi; + } + } + ``` -Есть возможность на созданные клиенты с `@HttpClient` аннотацией поставить [перехватчики](http-client.md#response-entity). + 1. Сгенерированный интерфейс `@HttpClient`, внедряется напрямую -Значение - Json объект, ключом которого выступает тег апи из контракта, а значением объект с полями `type` и `tag`, -можно указывать как оба поля одновременно, так и опционально одно из них на выбор где: +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class RootService( + private val petApi: PetApi, //(1)! + ) + ``` + + 1. Сгенерированный интерфейс `@HttpClient`, внедряется напрямую -- `type` - класс реализации конкретного перехватчика -- `tag` - теги перехватчика (можно указать как массив строк) +Сгенерированный клиент читает конфигурацию по пути, заданному `clientConfigPrefix`, за которым следует имя сгенерированного интерфейса. +Для `clientConfigPrefix = "httpClient.petV2"` и интерфейса `PetApi` блок конфигурации — `httpClient.petV2.PetApi`. +Полный набор параметров клиента (`url`, `requestTimeout`, блоки для отдельных операций, `telemetry`) описан в документации по [HTTP-клиенту](http-client.md#configuration): -Для этого необходимо установить параметр `configOptions.interceptors`: +===! ":material-code-json: `Hocon`" + + ```javascript + httpClient.petV2.PetApi { + url = "https://localhost:8443" //(1)! + requestTimeout = "10s" //(2)! + getValuesConfig { //(3)! + requestTimeout = "20s" + } + telemetry.logging.enabled = true + } + ``` + + 1. Базовый URL целевой службы + 2. Таймаут запроса по умолчанию для всех операций + 3. Блок переопределения для отдельной операции, названный по `operationId` (здесь `getValues`) + +=== ":simple-yaml: `YAML`" + + ```yaml + httpClient: + petV2: + PetApi: + url: "https://localhost:8443" #(1)! + requestTimeout: "10s" #(2)! + getValuesConfig: #(3)! + requestTimeout: "20s" + telemetry: + logging: + enabled: true + ``` + + 1. Базовый URL целевой службы + 2. Таймаут запроса по умолчанию для всех операций + 3. Блок переопределения для отдельной операции, названный по `operationId` (здесь `getValues`) + +Сигнатуры методов клиента зависят от выбранного `mode`: + +| Режим | Пример возвращаемого типа | +| -------- | -------- | +| `java-client` | `PetApiResponses.GetPetByIdApiResponse` (блокирующее значение) | +| `java-async-client` | `CompletionStage` | +| `java-reactive-client` | `Mono` (требует `reactor-core`) | +| `kotlin-client` | `PetApiResponses.GetPetByIdApiResponse` (блокирующее значение) | +| `kotlin-suspend-client` | `suspend fun ...: PetApiResponses.GetPetByIdApiResponse` | + +Каждый метод возвращает запечатанную (`sealed`) обёртку `*ApiResponses`, подтипы которой кодируют HTTP-статус, так же как это делают [делегаты сервера](#delegate-response-types). + +### Авторизация клиента { #client-authorization } + +Если контракт `OpenAPI` описывает `securitySchemes`, генератор создаёт модуль `ApiSecurity` с компонентами для авторизации клиента. +Для `apiKey` и `basic` генерируются компоненты, читающие конфигурацию. Для `bearer` и `oauth` ожидается соответствующий помеченный тегом компонент `HttpClientTokenProvider`. + +`securityConfigPrefix` задаёт общий префикс конфигурации авторизации. Если префикс не указан, путём конфигурации становится имя из `securitySchemes`. +Если у операции несколько схем авторизации, можно указать `primaryAuth`; иначе генератор выбирает одну из схем и пишет предупреждение в лог. +Если включён `authAllowMultiple`, генератор создаёт составной перехватчик, применяющий несколько схем авторизации последовательно. +Если включён `authAsMethodArgument`, данные авторизации добавляются в сигнатуру метода клиента вместо сгенерированного перехватчика. + +#### apiKey и basic { #client-authorization-config } + +Для схем `apiKey` и `basic` генератор создаёт читатели конфигурации `@DefaultComponent` и перехватчики, поэтому не требуется никаких компонентов — только значения конфигурации. +Путь конфигурации — это `securityConfigPrefix`, за которым следует имя схемы (или просто имя схемы, если `securityConfigPrefix` не задан). +Схема `apiKey` читает одну строку; схема `basic` читает объект `username`/`password`: + +===! ":material-code-json: `Hocon`" + + ```javascript + openapiAuth { + apiKeyAuth = "MyAuthApiKey" //(1)! + basicAuth { //(2)! + username = "user" + password = "password" + } + } + ``` + + 1. Схема `apiKey` `apiKeyAuth`: значение, отправляемое сгенерированным `ApiKeyHttpClientInterceptor` в заголовке/параметре запроса/куки, объявленных схемой + 2. Схема `basic` `basicAuth`: учётные данные, оборачиваемые сгенерированным `BasicAuthHttpClientInterceptor` + +=== ":simple-yaml: `YAML`" + + ```yaml + openapiAuth: + apiKeyAuth: "MyAuthApiKey" #(1)! + basicAuth: #(2)! + username: "user" + password: "password" + ``` + + 1. Схема `apiKey` `apiKeyAuth`: значение, отправляемое сгенерированным `ApiKeyHttpClientInterceptor` в заголовке/параметре запроса/куки, объявленных схемой + 2. Схема `basic` `basicAuth`: учётные данные, оборачиваемые сгенерированным `BasicAuthHttpClientInterceptor` + +#### bearer и oauth { #client-authorization-token } + +Для схем `bearer` и `oauth` генератор ожидает компонент [`HttpClientTokenProvider`](http-client.md#token-provider), помеченный сгенерированным классом-маркером `ApiSecurity` +(например `ApiSecurity.BearerAuth`). Генератор автоматически оборачивает его в `BearerAuthHttpClientInterceptor`, поэтому предоставить нужно только провайдер токенов: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface ClientAuthModule { + + @Tag(ApiSecurity.BearerAuth.class) //(1)! + default HttpClientTokenProvider bearerTokenProvider() { + return request -> CompletableFuture.completedFuture("my-token"); //(2)! + } + } + ``` + + 1. Тег должен совпадать со сгенерированным классом-маркером для схемы + 2. Реальные реализации обычно здесь получают или обновляют токен + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface ClientAuthModule { + + @Tag(ApiSecurity.BearerAuth::class) //(1)! + fun bearerTokenProvider(): HttpClientTokenProvider { + return HttpClientTokenProvider { CompletableFuture.completedFuture("my-token") } //(2)! + } + } + ``` + + 1. Тег должен совпадать со сгенерированным классом-маркером для схемы + 2. Реальные реализации обычно здесь получают или обновляют токен + +#### Несколько схем { #client-authorization-multiple } + +Когда операция объявляет несколько схем безопасности, `primaryAuth` выбирает, какую из них применять; иначе генератор выбирает одну и пишет предупреждение в лог. +Чтобы применить несколько схем к одному запросу, включите `authAllowMultiple` — генератор построит составной перехватчик, выполняющий каждую схему последовательно. +Чтобы передавать учётные данные явно на каждый вызов вместо перехватчика, включите `authAsMethodArgument` — значение авторизации станет аргументом метода клиента: + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-client", + securityConfigPrefix: "openapiAuth", + primaryAuth: "apiKeyAuth", //(1)! + authAllowMultiple: "false", //(2)! + authAsMethodArgument: "false" //(3)! + ] + ``` + + 1. Схема, применяемая, когда у операции их перечислено несколько + 2. Применять каждую объявленную схему через составной перехватчик + 3. Добавить значение авторизации как аргумент метода вместо перехватчика + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "securityConfigPrefix" to "openapiAuth", + "primaryAuth" to "apiKeyAuth", //(1)! + "authAllowMultiple" to "false", //(2)! + "authAsMethodArgument" to "false" //(3)! + ) + ``` + + 1. Схема, применяемая, когда у операции их перечислено несколько + 2. Применять каждую объявленную схему через составной перехватчик + 3. Добавить значение авторизации как аргумент метода вместо перехватчика + +### Дополнительные аннотации { #additional-contract-annotations } + +Параметр `additionalContractAnnotations` добавляет аннотации над сгенерированными методами клиента или контроллера сервера. +Значение — это объект `JSON`, где ключ — тег API из контракта или `*` для всех операций, а значение — массив объектов с полем `annotation`. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-client", + additionalContractAnnotations: """ + { + "*": [ + { "annotation": "ru.tinkoff.example.CommonAnnotation" } + ], + "pet": [ + { "annotation": "ru.tinkoff.example.PetAnnotation" } + ] + } + """ + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "additionalContractAnnotations" to """{ + "*": [ + { "annotation": "ru.tinkoff.example.CommonAnnotation" } + ], + "pet": [ + { "annotation": "ru.tinkoff.example.PetAnnotation" } + ] + } + """ + ) + ``` + +### Перехватчики { #interceptors } + +Сгенерированные клиенты, аннотированные `@HttpClient`, можно также аннотировать [перехватчиками](http-client.md#interceptors). +Значение — это объект `JSON`, где ключ — тег API из контракта, а значение — массив объектов с полями `type` и `tag`. +Оба поля можно указать вместе или указать только одно из них: + +- `type` — класс реализации конкретного перехватчика +- `tag` — теги перехватчика: строка или массив строк + +Задайте `configOptions.interceptors`: ===! ":fontawesome-brands-java: `Java`" @@ -241,10 +744,10 @@ agent: ### Теги { #tags } -Есть возможность на созданные клиенты с `@HttpClient` аннотацией поставить параметры `httpClientTag` и `telemetryTag`. -Значение - Json объект, ключом которого выступает тег апи из контракта, а значением объект с полями `httpClientTag` и `telemetryTag`. +Сгенерированным клиентам, аннотированным `@HttpClient`, можно передать параметры `httpClientTag` и `telemetryTag`. +Значение — это объект `JSON`, где ключ — тег API из контракта, а значение — объект с полями `httpClientTag` и `telemetryTag`. -Для этого необходимо установить параметр `configOptions.tags`: +Задайте `configOptions.tags`: ===! ":fontawesome-brands-java: `Java`" @@ -253,11 +756,11 @@ agent: mode: "java-client", tags: """ { - "*": { // применится для всех тегов, кроме явно указанных (в данном случае instrument) + "*": { "httpClientTag": "some.tag.Common", "telemetryTag": "some.tag.Common" }, - "instrument": { // применится для instrument + "instrument": { "httpClientTag": "some.tag.Instrument", "telemetryTag": "some.tag.Instrument" } @@ -272,11 +775,11 @@ agent: configOptions = mapOf( "mode" to "kotlin-client", "tags" to """{ - "*": { // применится для всех тегов, кроме явно указанных (в данном случае instrument) + "*": { "httpClientTag": "some.tag.Common", "telemetryTag": "some.tag.Common" }, - "instrument": { // применится для instrument + "instrument": { "httpClientTag": "some.tag.Instrument", "telemetryTag": "some.tag.Instrument" } @@ -285,29 +788,68 @@ agent: ) ``` +## Неявные заголовки { #implicit-headers } + +По умолчанию заголовки из операции `OpenAPI` становятся аргументами сгенерированного метода. +Если некоторые заголовки предоставляются инфраструктурой, а не кодом приложения, их можно сделать неявными. + +- `implicitHeaders = true` делает неявными все заголовки из операций `OpenAPI`. +- `implicitHeadersRegex` делает неявными только заголовки, имена которых соответствуют регулярному выражению. + +Неявный заголовок убирается из сигнатуры метода, но остаётся в аннотациях `OpenAPI` в сгенерированном коде. +Это сохраняет заголовок в документации контракта, не требуя от кода приложения передавать его вручную. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-client", + implicitHeadersRegex: "X-Request-.*" + ] + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-client", + "implicitHeadersRegex" to "X-Request-.*" + ) + ``` + +## Модели { #models } + +Генератор создаёт модели запросов и ответов из схем `OpenAPI`. +Необязательные поля используют `@Nullable` в `Java` и nullable-тип `T?` в `Kotlin`. +Для схем с наследованием и дискриминатором `Java` может генерировать `sealed interface`, а `Kotlin` — `sealed interface` / классы в зависимости от схемы. + +### Необязательные nullable-поля { #json-nullable } + +Если поле одновременно имеет `nullable: true` и отсутствует в списке `required`, по умолчанию оно генерируется как обычное необязательное поле. +Если нужно различать три состояния — поле отсутствует в `JSON`, поле присутствует со значением `null` и поле присутствует со значением — включите `enableJsonNullable`. +В этом случае поле генерируется как [JsonNullable](json.md#jsonnullable-wrapper). + +`forceIncludeOptional` и `forceIncludeNonRequired` управляют сериализацией необязательных полей: + +- `forceIncludeOptional` устанавливает `@JsonInclude(Always)` для полей с `nullable: true` и `required: false` вместо использования `JsonNullable`. +- `forceIncludeNonRequired` устанавливает `@JsonInclude(Always)` для всех полей с `required: false`. + +`forceIncludeOptional` нельзя включить вместе с `enableJsonNullable`, поскольку оба режима решают одну и ту же задачу разными способами. + +### Фильтрация моделей { #filter-with-models } + +`OpenAPI Generator` может фильтровать операции через `openapiNormalizer.FILTER`. +Если дополнительно включён `filterWithModels`, генератор Kora пытается исключить неиспользуемые модели, оставшиеся после фильтрации операций. +Это полезно для больших контрактов, где приложение генерирует только часть API. + ## Сервер { #server } -Минимальный пример настройки плагина для создания обработчиков HTTP-сервера: +Минимальная конфигурация плагина для создания обработчиков HTTP-сервера: ===! ":fontawesome-brands-java: `Java`" - Доступные Kora параметры плагина (`configOptions`): - - - `enableServerValidation` - создавать ли валидаторы по описанию OpenAPI сецификации для сервера и включать ли валидацию на HTTP-обработчиках. Значения: `true`, `false` - - `enableServerValidationInterceptor` - Добавлять ли перехватчик валидатора для отдельного маппинга исключений валидации в HTTP-ответы. Значения: `true, false` - - `requestInDelegateParams` - прокидывать ли `HttpServerRequest` принудительно как аргумент метода. Значения: `true`, `false` - - `interceptors` - возможность указывать перехватчики для HTTP-контроллеров - - `additionalContractAnnotations` - возможность указывать дополнительные аннотации над методами контроллера - - `enableJsonNullable` - обрабатывать `nullable=true` и `required=false` поля схем как [JsonNullable](json.md#jsonnullable-wrapper) обертку. Значения: `true`, `false` - - `forceIncludeOptional` - проставлять принудительно [@JsonInclude(Always)](json.md#serialization-levels) для `nullable=true` и `required=false` полей вместо `enableJsonNullable`. Значения: `true`, `false`. - - `forceIncludeNonRequired` - проставлять принудительно [@JsonInclude(Always)](json.md#serialization-levels) для только `required=false` полей. Значения: `true`, `false`. - - `filterWithModels` - фильтровать и исключать из генерации также ненужные модели когда указана опция [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) в `openapiNormalizer`. Значения: `true`, `false` - - `prefixPath` - префикс пути обработчиков HTTP-сервера. Значение: `строка` - - `delegateMethodBodyMode` - способ генерации тела метода в delegate классе. `none` - не генерировать тело метода, `throw-exception` - бросать исключение в теле метода. Для `throw-exception` дополнительно будет сгенерирован модуль со стандартной реализацией Delegate класса, если в графе приложения нет другой реализации - - `mode` в каком режиме работать генератору, доступные значения: - * `java-server` - создание синхронного сервера - * `java-async-server` - создание [CompletionStage](https://www.baeldung.com/java-completablefuture) сервера - * `java-reactive-server` - создание [реактивного](https://projectreactor.io/docs/core/release/reference/) сервера, требуется подключить [Project Reactor](https://mvnrepository.com/artifact/io.projectreactor/reactor-core) самостоятельно. + Для серверов `configOptions.mode` поддерживает `java-server`, `java-async-server` и `java-reactive-server`. + Остальные параметры сервера описаны ниже в разделах про валидацию, классы `delegate`, перехватчики, модели и неявные заголовки. ```groovy def openApiGenerateHttpServer = tasks.register("openApiGenerateHttpServer", GenerateTask) { @@ -330,33 +872,19 @@ agent: compileJava.dependsOn openApiGenerateHttpServer //(8)! ``` - 1. Путь до OpenAPI файла из которого будут созданы классы - 2. Директория куда буду создаваться файлы - 3. Пакет от классов делегатов, контроллеров, преобразователей и тп. - 4. Пакет от классов моделей, DTO и тп. - 5. Пакет от классов вызова - 6. Режим работы плагина (создание Java клиента / Kotlin / Java сервера и тп) - 7. Регистрируем созданные классы как исходный код проекта - 8. Делаем компиляцию кода, зависимой от генерации классов HTTP-сервера (сначала генерируем, потом компилируем) + 1. Путь к файлу `OpenAPI`, по которому создаются классы + 2. Каталог, где создаются сгенерированные файлы + 3. Пакет для делегатов, контроллеров и мапперов + 4. Пакет для моделей и DTO + 5. Вспомогательный пакет генератора + 6. Режим плагина + 7. Регистрирует сгенерированные классы как исходный код проекта + 8. Ставит компиляцию кода в зависимость от генерации классов HTTP-сервера: сначала генерация, затем компиляция === ":simple-kotlin: `Kotlin`" - Доступные Kora параметры плагина (`configOptions`): - - - `enableServerValidation` - создавать ли валидаторы по описанию OpenAPI сецификации для сервера и включать ли валидацию на HTTP-обработчиках. Значения: `true`, `false` - - `enableServerValidationInterceptor` - Добавлять ли перехватчик валидатора для отдельного маппинга исключений валидации в HTTP-ответы. Значения: `true, false` - - `requestInDelegateParams` - прокидывать ли `HttpServerRequest` принудительно как аргумент метода. Значения: `true`, `false` - - `interceptors` - возможность указывать перехватчики для HTTP-контроллеров - - `additionalContractAnnotations` - возможность указывать дополнительные аннотации над методами контроллера - - `enableJsonNullable` - обрабатывать `nullable=true` и `required=false` поля схем как [JsonNullable](json.md#jsonnullable-wrapper) обертку. Значения: `true`, `false` - - `forceIncludeOptional` - проставлять принудительно [@JsonInclude(Always)](json.md#serialization-levels) для `nullable=true` и `required=false` полей вместо `enableJsonNullable`. Значения: `true`, `false`. - - `forceIncludeNonRequired` - проставлять принудительно [@JsonInclude(Always)](json.md#serialization-levels) для только `required=false` полей. Значения: `true`, `false`. - - `filterWithModels` - фильтровать и исключать из генерации также ненужные модели когда указана опция [FILTER](https://openapi-generator.tech/docs/customization/#available-filters) в `openapiNormalizer`. Значения: `true`, `false` - - `prefixPath` - префикс пути обработчиков HTTP-сервера. Значение: `строка` - - `delegateMethodBodyMode` - способ генерации тела метода в delegate классе. `none` - не генерировать тело метода, `throw-exception` - бросать исключение в теле метода. Для `throw-exception` дополнительно будет сгенерирован модуль со стандартной реализацией Delegate класса, если в графе приложения нет другой реализации - - `mode` в каком режиме работать генератору, доступные значения: - * `kotlin-server` - создание синхронного сервера - * `kotlin-suspend-server` - создание suspend сервера + Для серверов `configOptions.mode` поддерживает `kotlin-server` и `kotlin-suspend-server`. + Остальные параметры сервера описаны ниже в разделах про валидацию, классы `delegate`, перехватчики, модели и неявные заголовки. ```groovy val openApiGenerateHttpServer = tasks.register("openApiGenerateHttpServer") { @@ -379,20 +907,20 @@ agent: tasks.withType { dependsOn(openApiGenerateHttpServer) } //(8)! ``` - 1. Путь до OpenAPI файла из которого будут созданы классы - 2. Директория куда буду создаваться файлы - 3. Пакет от классов делегатов, контроллеров, преобразователей и тп. - 4. Пакет от классов моделей, DTO и тп. - 5. Пакет от классов вызова - 6. Режим работы плагина (создание Java клиента / Kotlin / Java сервера и тп) - 7. Регистрируем созданные классы как исходный код проекта - 8. Делаем компиляцию кода, зависимой от генерации классов HTTP-сервера (сначала генерируем, потом компилируем) + 1. Путь к файлу `OpenAPI`, по которому создаются классы + 2. Каталог, где создаются сгенерированные файлы + 3. Пакет для делегатов, контроллеров и мапперов + 4. Пакет для моделей и DTO + 5. Вспомогательный пакет генератора + 6. Режим плагина + 7. Регистрирует сгенерированные классы как исходный код проекта + 8. Ставит компиляцию кода в зависимость от генерации классов HTTP-сервера: сначала генерация, затем компиляция -После создания обработчики будут автоматически зарегистрированы. +После генерации обработчики регистрируются автоматически. ### Валидация { #validation } -Для генерации моделей и контроллеров с аннотациями из модуля [валидации](validation.md) необходимо установить опцию `enableServerValidation`: +Чтобы генерировать модели и контроллеры с аннотациями из модуля [валидации](validation.md), задайте `enableServerValidation`: ===! ":fontawesome-brands-java: `Java`" @@ -403,7 +931,7 @@ agent: ] ``` - 1. Включение валидации на стороне контроллера HTTP сервера + 1. Включает валидацию на стороне контроллера HTTP-сервера === ":simple-kotlin: `Kotlin`" @@ -414,19 +942,170 @@ agent: ) ``` - 1. Включение валидации на стороне контроллера HTTP сервера + 1. Включает валидацию на стороне контроллера HTTP-сервера -### Перехватчики { #interceptors-2 } +Когда `enableServerValidation` включён, генератор добавляет аннотации валидации к моделям и параметрам методов сервера, +а также добавляет `@Validate` к методам контроллера с валидируемыми параметрами. +`enableServerValidationInterceptor` управляет добавлением `ValidationHttpServerInterceptor`, который преобразует ошибки валидации в HTTP-ответы. +Если `enableServerValidationInterceptor` не указан явно, он считается включённым, когда включена валидация сервера. +Если указано `enableServerValidationInterceptor = false`, аннотации валидации остаются, но стандартный перехватчик ответа не добавляется. + +### Реализация делегата { #delegate-method-body } + +Генератор сервера создаёт контроллер и контракт `delegate`, в котором пользователь реализует логику приложения. +По умолчанию `delegateMethodBodyMode = none`, поэтому методы контракта `delegate` не получают стандартного тела и должны быть реализованы приложением. + +Если задано `delegateMethodBodyMode = throwException`, методы получают тело, выбрасывающее исключение, и генератор также создаёт модуль +с реализацией контракта `delegate` по умолчанию. Этот режим полезен, когда приложение нужно собрать до того, как реализованы все операции, или когда собственные реализации подключаются постепенно. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-server", + delegateMethodBodyMode: "throwException" + ] + ``` -Есть возможность на созданные контроллеры с `@HttpController` аннотацией поставить [перехватчики](http-server.md#custom-response). +=== ":simple-kotlin: `Kotlin`" -Значение - Json объект, ключом которого выступает тег апи из контракта, а значением объект с полями `type` и `tag`, -можно указывать как оба поля одновременно, так и опционально одно из них на выбор где: + ```groovy + configOptions = mapOf( + "mode" to "kotlin-server", + "delegateMethodBodyMode" to "throwException" + ) + ``` -- `type` - класс реализации конкретного перехватчика -- `tag` - теги перехватчика (можно указать как массив строк) +#### Ответы делегата { #delegate-response-types } -Для этого необходимо установить параметр `configOptions.interceptors`: +Каждый сгенерированный метод `delegate` возвращает запечатанную (`sealed`) обёртку `*ApiResponses`, подтипы которой кодируют HTTP-статус, объявленный в контракте. +Для операции `getPetById` с ответами `200` и `404` генератор создаёт `PetApiResponses.GetPetByIdApiResponse` с подтипами +`GetPetById200ApiResponse` (несущим тело через `content()`) и `GetPetById404ApiResponse`. Реализация возвращает подтип, соответствующий результату: + +===! ":fontawesome-brands-java: `Java`" + + Возвращаемый тип зависит от `mode`: `java-server` возвращает значение напрямую (показано здесь), `java-async-server` возвращает `CompletionStage<...>`, `java-reactive-server` возвращает `Mono<...>`: + + ```java + @Component + public final class PetDelegate implements PetApiDelegate { + + private final Map petMap = new ConcurrentHashMap<>(); + + @Override + public PetApiResponses.GetPetByIdApiResponse getPetById(long petId) { + var pet = petMap.get(petId); + if (pet == null) { + return new PetApiResponses.GetPetByIdApiResponse.GetPetById404ApiResponse(); //(1)! + } + return new PetApiResponses.GetPetByIdApiResponse.GetPetById200ApiResponse(pet); //(2)! + } + + @Override + public PetApiResponses.AddPetApiResponse addPet(Pet body) { + petMap.put(body.id(), body); + return new PetApiResponses.AddPetApiResponse.AddPet200ApiResponse(body); + } + } + ``` + + 1. Подтип статуса `404`, без тела + 2. Подтип статуса `200`, несущий тело ответа + +=== ":simple-kotlin: `Kotlin`" + + Возвращаемый тип зависит от `mode`: `kotlin-server` возвращает значение напрямую (показано здесь), `kotlin-suspend-server` использует `suspend`-метод: + + ```kotlin + @Component + class PetDelegate : PetApiDelegate { + + private val petMap = ConcurrentHashMap() + + override fun getPetById(petId: Long): PetApiResponses.GetPetByIdApiResponse { + val pet = petMap[petId] + return if (pet == null) { + PetApiResponses.GetPetByIdApiResponse.GetPetById404ApiResponse() //(1)! + } else { + PetApiResponses.GetPetByIdApiResponse.GetPetById200ApiResponse(pet) //(2)! + } + } + + override fun addPet(pet: Pet): PetApiResponses.AddPetApiResponse { + petMap[pet.id] = pet + return PetApiResponses.AddPetApiResponse.AddPet200ApiResponse(pet) + } + } + ``` + + 1. Подтип статуса `404`, без тела + 2. Подтип статуса `200`, несущий тело ответа + +#### Исходный запрос { #request-in-delegate } + +По умолчанию метод `delegate` получает только параметры, объявленные в контракте. Если реализации нужен доступ к исходному запросу +(например, чтобы прочитать инфраструктурный заголовок или удалённый адрес), включите `requestInDelegateParams`. Тогда генератор добавляет +`HttpServerRequest` первым параметром каждого метода `delegate`. Это параметр только для сервера. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-server", + requestInDelegateParams: "true" //(1)! + ] + ``` + + 1. Добавляет `HttpServerRequest _serverRequest` первым аргументом каждого метода делегата + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-server", + "requestInDelegateParams" to "true" //(1)! + ) + ``` + + 1. Добавляет `HttpServerRequest _serverRequest` первым аргументом каждого метода делегата + +#### Префикс пути контроллера { #prefix-path } + +`prefixPath` добавляет базовый путь в начало каждого маршрута сгенерированного контроллера HTTP-сервера. Это полезно, когда все операции должны обслуживаться под общим +сегментом (например `/api/v1`), которого нет в путях `OpenAPI`. + +===! ":fontawesome-brands-java: `Java`" + + ```groovy + configOptions = [ + mode: "java-server", + prefixPath: "/api/v1" //(1)! + ] + ``` + + 1. Путь контракта `/pet/{id}` становится `/api/v1/pet/{id}` + +=== ":simple-kotlin: `Kotlin`" + + ```groovy + configOptions = mapOf( + "mode" to "kotlin-server", + "prefixPath" to "/api/v1" //(1)! + ) + ``` + + 1. Путь контракта `/pet/{id}` становится `/api/v1/pet/{id}` + +### Перехватчики { #interceptors-2 } + +Сгенерированные контроллеры, аннотированные `@HttpController`, можно также аннотировать [перехватчиками](http-server.md#interceptors). +Значение — это объект `JSON`, где ключ — тег API из контракта, а значение — объект с полями `type` и `tag`. +Оба поля можно указать вместе или указать только одно из них: + +- `type` — класс реализации конкретного перехватчика +- `tag` — теги перехватчика: строка или массив строк + +Задайте `configOptions.interceptors`: ===! ":fontawesome-brands-java: `Java`" @@ -485,8 +1164,11 @@ agent: ### Авторизация { #authorization } -Kora предоставляет интерфейс для извлечения авторизационной информации в рамках перехватчика, -созданного для сервера из OpenAPI, можно вытаскивать любые типы авторизации [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/) +Когда контракт `OpenAPI` описывает `securitySchemes`, генератор сервера создаёт модуль `ApiSecurity` с одним классом-маркером на каждую схему: +`ApiSecurity.BearerAuth`, `ApiSecurity.BasicAuth`, `ApiSecurity.ApiKeyAuth` и `ApiSecurity.OAuth` +(обрабатывающие [Basic/ApiKey/Bearer/OAuth](https://swagger.io/docs/specification/authentication/)). +Для каждой схемы приложение должно предоставить компонент `HttpServerPrincipalExtractor`, помеченный соответствующим классом-маркером. +Извлекатель получает запрос и разобранное значение учётных данных и возвращает аутентифицированный `Principal`: ===! ":fontawesome-brands-java: `Java`" @@ -496,11 +1178,28 @@ Kora предоставляет интерфейс для извлечения @Tag(ApiSecurity.BearerAuth.class) default HttpServerPrincipalExtractor bearerHttpServerPrincipalExtractor() { - return (request, value) -> CompletableFuture.completedFuture(new MyPrincipal(request.headers().getFirst("Authorization"))); + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); + } + + @Tag(ApiSecurity.BasicAuth.class) + default HttpServerPrincipalExtractor basicHttpServerPrincipalExtractor() { + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); + } + + @Tag(ApiSecurity.ApiKeyAuth.class) + default HttpServerPrincipalExtractor apiKeyHttpServerPrincipalExtractor() { + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); + } + + @Tag(ApiSecurity.OAuth.class) + default HttpServerPrincipalExtractor oauthHttpServerPrincipalExtractor() { //(1)! + return (request, value) -> CompletableFuture.completedFuture(new UserPrincipal("name")); } } ``` + 1. Схемы `OAuth` объявляют области доступа (scopes), поэтому извлекатель возвращает `PrincipalWithScopes` + === ":simple-kotlin: `Kotlin`" ```kotlin @@ -509,22 +1208,65 @@ Kora предоставляет интерфейс для извлечения @Tag(ApiSecurity.BearerAuth::class) fun bearerHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { - return HttpServerPrincipalExtractor { request, value -> - CompletableFuture.completedFuture( - MyPrincipal(request.headers().getFirst("Authorization")) - ) - } + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + + @Tag(ApiSecurity.BasicAuth::class) + fun basicHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + + @Tag(ApiSecurity.ApiKeyAuth::class) + fun apiKeyHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + + @Tag(ApiSecurity.OAuth::class) + fun oauthHttpServerPrincipalExtractor(): HttpServerPrincipalExtractor { //(1)! + return HttpServerPrincipalExtractor { _, _ -> CompletableFuture.completedFuture(UserPrincipal("name")) } + } + } + ``` + + 1. Схемы `OAuth` объявляют области доступа (scopes), поэтому извлекатель возвращает `PrincipalWithScopes` + +Для `OAuth` возвращаемый principal должен реализовывать `PrincipalWithScopes`, чтобы сгенерированный контроллер мог проверять области доступа, объявленные для каждой операции. +Извлекатель нужен только тем схемам, которые контракт действительно использует; класс-маркер существует для каждой объявленной схемы: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public record UserPrincipal(String name) implements PrincipalWithScopes { + + @Override + public Collection scopes() { + return List.of("read", "write"); //(1)! + } + } + ``` + + 1. Области доступа, предоставленные этому principal, сверяются с областями, требуемыми операцией + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + data class UserPrincipal(val name: String) : PrincipalWithScopes { + + override fun scopes(): Collection { + return listOf("read", "write") //(1)! } } ``` -## Совет { #recommendations } + 1. Области доступа, предоставленные этому principal, сверяются с областями, требуемыми операцией + +## Рекомендации { #recommendations } ???+ tip "Совет" - В случае если у вас что-то не создается посредствам плагина, либо поведение отличается от желаемого или других версий, - требуется тщательно проверить настройки [конфигурации плагина](#configuration) и изучить их, - так как они могут влиять на результаты того как создаются классы. + Если что-то не генерируется плагином или поведение отличается от ожидаемого либо от других версий, + внимательно проверьте [конфигурацию плагина](#configuration) и изучите настройки, + поскольку они могут влиять на то, как генерируются классы. - Начиная с `7.0.0` версии плагина, включенное по умолчанию `SIMPLIFY_ONEOF_ANYOF` правило у параметра `openapiNormalizer` - может вести к некоторым не очевидным результатам генератора. + Начиная с версии плагина `7.0.0`, правило `SIMPLIFY_ONEOF_ANYOF`, включённое по умолчанию в `openapiNormalizer`, + может приводить к некоторым неочевидным результатам генератора. diff --git a/mkdocs/docs/ru/documentation/openapi-management.md b/mkdocs/docs/ru/documentation/openapi-management.md index 7445320..6dd2b56 100644 --- a/mkdocs/docs/ru/documentation/openapi-management.md +++ b/mkdocs/docs/ru/documentation/openapi-management.md @@ -1,13 +1,16 @@ --- -description: "Explains Kora OpenAPI management module for serving generated OpenAPI specifications through the management HTTP server. Use when working with OpenApiManagementModule, OpenAPI, management endpoint, private HTTP server." +description: "Описывает модуль управления OpenAPI в Kora для публикации сгенерированных спецификаций OpenAPI, а также страниц Swagger UI и RapiDoc через публичный HTTP-сервер. Используйте при работе с OpenApiManagementModule, OpenApiManagementConfig, маршруты OpenAPI, Swagger UI, RapiDoc." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI management module for serving generated OpenAPI specifications through the management HTTP server; key triggers include OpenApiManagementModule, OpenAPI, management endpoint, private HTTP server." + use_when: "Use this file for Kora docs or implementation questions about Kora OpenAPI management module for serving generated OpenAPI specifications, Swagger UI, and RapiDoc pages through the public HTTP server; key triggers include OpenApiManagementModule, OpenApiManagementConfig, OpenAPI endpoint, Swagger UI, RapiDoc, /openapi, /swagger-ui, /rapidoc." --- -Модуль для предоставления OpenAPI файла из приложения, -а также [Swagger UI](https://swagger.io/tools/swagger-ui/) и [Rapidoc](https://rapidocweb.com/) для отображения OpenAPI. +Модуль `openapi-management` предоставляет из приложения готовые файлы `OpenAPI`, а также страницы [Swagger UI](https://swagger.io/tools/swagger-ui/) и [RapiDoc](https://rapidocweb.com/) для их просмотра. +`OpenAPI` — это машиночитаемый контракт HTTP API: по нему удобно проверять доступные операции, модели данных и параметры запросов. -Если нужен пошаговый разбор перед справочным описанием, смотрите [OpenAPI HTTP сервер](../guides/openapi-http-server.md). +Модуль не создает контракт из кода, а публикует уже существующие файлы из ресурсов приложения. +Это полезно для локальной разработки, тестовых окружений и служебного доступа к описанию API без отдельного сервера документации. + +Если нужен пошаговый разбор перед справочным описанием, смотрите [HTTP-сервер OpenAPI](../guides/openapi-http-server.md). ## Подключение { #dependency } @@ -37,11 +40,12 @@ agent: interface Application : OpenApiManagementModule ``` -Требует подключения [HTTP сервера](http-server.md). +Требует подключения модуля [HTTP-сервера](http-server.md), так как регистрирует собственные `GET`-обработчики для выдачи файлов и страниц просмотра. +Это обычные бины `HttpServerRequestHandler`, которые собирает **публичный** HTTP-сервер, поэтому пути `/openapi`, `/swagger-ui` и `/rapidoc` доступны на публичном HTTP-порту, а не на приватном (management) порту. ## Конфигурация { #configuration } -Пример конфигурации описанной в классе `OpenApiManagementConfig`: +Пример конфигурации, описанной в классе `OpenApiManagementConfig`: ===! ":material-code-json: `Hocon`" @@ -63,15 +67,16 @@ agent: } ``` - 1. Относительный путь до OpenAPI файлов в `resources` директории, можно указывать как один файл, так и несколько файлов - 2. Вкл/Выкл контроллера который отдает OpenAPI - 3. Путь по которому будет доступен OpenAPI - 1. Если указан один OpenAPI файл, является целиком путем по которому доступен файл - 2. Если указаны несколько OpenAPI файлов, является префиксом к пути перед именем файла `/openapi/{fileName}`, берется указанный путь и к нему добавляется имя файла без диреторий и его расширения, в случае файла `someDirectory/my-openapi-1.yaml` путь к файлу будет `/openapi/my-openapi-1` - 4. Вкл/Выкл контроллера который отдает SwaggerUI - 5. Путь по которому будет доступен SwaggerUI - 6. Вкл/Выкл контроллера который отдает Rapidoc - 7. Путь по которому будет доступен Rapidoc + 1. Путь к файлу `OpenAPI` или список путей относительно ресурсов приложения (обязательный, по умолчанию не указан). + 2. Включает выдачу файлов `OpenAPI` через HTTP-обработчик (по умолчанию: `false`). + 3. Путь, по которому доступны файлы `OpenAPI` (по умолчанию: `/openapi`). + Если указан один файл, он доступен ровно по этому пути. + Если указано несколько файлов, путь становится префиксом вида `/openapi/{file}`. + Значение `{file}` берется из имени файла без директорий и без расширения `.json`, `.yml` или `.yaml`: файл `someDirectory/my-openapi-1.yaml` будет доступен по пути `/openapi/my-openapi-1`. + 4. Включает страницу `Swagger UI` (по умолчанию: `false`). + 5. Путь, по которому доступна страница `Swagger UI` (по умолчанию: `/swagger-ui`). + 6. Включает страницу `RapiDoc` (по умолчанию: `false`). + 7. Путь, по которому доступна страница `RapiDoc` (по умолчанию: `/rapidoc`). === ":simple-yaml: `YAML`" @@ -89,22 +94,45 @@ agent: endpoint: "/rapidoc" #(7)! ``` - 1. Относительный путь до OpenAPI файлов в `resources` директории, можно указывать как один файл, так и несколько файлов - 2. Вкл/Выкл контроллера который отдает OpenAPI - 3. Путь по которому будет доступен OpenAPI - 1. Если указан один OpenAPI файл, является целиком путем по которому доступен файл - 2. Если указаны несколько OpenAPI файлов, является префиксом к пути перед именем файла `/openapi/{fileName}`, берется указанный путь и к нему добавляется имя файла без диреторий и его расширения, в случае файла `someDirectory/my-openapi-1.yaml` путь к файлу будет `/openapi/my-openapi-1` - 4. Вкл/Выкл контроллера который отдает SwaggerUI - 5. Путь по которому будет доступен SwaggerUI - 6. Вкл/Выкл контроллера который отдает Rapidoc - 7. Путь по которому будет доступен Rapidoc + 1. Путь к файлу `OpenAPI` или список путей относительно ресурсов приложения (обязательный, по умолчанию не указан). + 2. Включает выдачу файлов `OpenAPI` через HTTP-обработчик (по умолчанию: `false`). + 3. Путь, по которому доступны файлы `OpenAPI` (по умолчанию: `/openapi`). + Если указан один файл, он доступен ровно по этому пути. + Если указано несколько файлов, путь становится префиксом вида `/openapi/{file}`. + Значение `{file}` берется из имени файла без директорий и без расширения `.json`, `.yml` или `.yaml`: файл `someDirectory/my-openapi-1.yaml` будет доступен по пути `/openapi/my-openapi-1`. + 4. Включает страницу `Swagger UI` (по умолчанию: `false`). + 5. Путь, по которому доступна страница `Swagger UI` (по умолчанию: `/swagger-ui`). + 6. Включает страницу `RapiDoc` (по умолчанию: `false`). + 7. Путь, по которому доступна страница `RapiDoc` (по умолчанию: `/rapidoc`). + +Файлы читаются из ресурсов приложения при первом обращении и затем кэшируются в памяти (последующие запросы возвращают закэшированные байты). +Для файлов с расширением `.json` используется тип ответа `text/json; charset=utf-8`, для всех остальных файлов — `text/x-yaml; charset=utf-8`. + +При нескольких файлах `Swagger UI` показывает список доступных контрактов, а `RapiDoc` открывает первый файл из списка. + +Когда настроено несколько файлов, запрос к `/openapi/{file}` с неизвестным именем `{file}` возвращает `404` (`OpenAPI file not registered`), а запрос с пустым значением `{file}` возвращает `400` (`OpenAPI file not specified`). +Если настроенный ресурс не удается найти или прочитать в момент запроса, обработчик возвращает `404` или `500` соответственно, иначе он отвечает `200` и содержимым файла. + +## Маршруты { #endpoints } + +При включенной выдаче модуль регистрирует на публичном HTTP-сервере следующие `GET`-маршруты (пути показаны со значениями `endpoint` по умолчанию): + +| Маршрут | Обработчик | Включается через | +|-------|-----------------|------------| +| `GET /openapi` (один файл) или `GET /openapi/{file}` (несколько файлов) | `OpenApiHttpServerHandler` | `enabled = true` | +| `GET /swagger-ui` | `SwaggerUIHttpServerHandler` | `swaggerui.enabled = true` | +| `GET /swagger-ui/oauth2-redirect` | `SwaggerOauthHttpServerHandler` | регистрируется автоматически вместе со `Swagger UI` | +| `GET /rapidoc` | `RapidocHttpServerHandler` | `rapidoc.enabled = true` | + +Каждый маршрут использует значение `endpoint` из своей секции конфигурации, поэтому переопределение `endpoint` переносит соответствующий маршрут. +Путь `OAuth2`-перенаправления всегда равен `swaggerui.endpoint` с добавленным суффиксом `/oauth2-redirect`. -## Совет { #recommendations } +## Рекомендации { #recommendations } -???+ warning "Совет" +???+ warning "Рекомендация" - Мы советуем использовать подход когда первичен [контракт и по нему создается код](openapi-codegen.md), - в таком подходе отображается этот самый файл контракт. + Мы советуем использовать подход, при котором сначала создается [контракт, а затем по нему генерируется код](openapi-codegen.md). + В этом случае модуль публикует тот же файл контракта, который используется для генерации. - В случае же когда первичен код и по нему предполагается создавать файл контракт, можно использовать [Swagger Gradle Plugin](https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-gradle-plugin/README.md) - вкупе с [набором Swagger аннотаций](https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations) по которым будет создаваться файл контракт. + Если сначала пишется код, а контракт должен создаваться по нему, можно использовать [Swagger Gradle Plugin](https://github.com/swagger-api/swagger-core/blob/master/modules/swagger-gradle-plugin/README.md) + вместе с [аннотациями Swagger](https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations). diff --git a/mkdocs/docs/ru/documentation/probes.md b/mkdocs/docs/ru/documentation/probes.md index 8f3d26a..796ba1b 100644 --- a/mkdocs/docs/ru/documentation/probes.md +++ b/mkdocs/docs/ru/documentation/probes.md @@ -1,85 +1,281 @@ --- -description: "Explains Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting. Use when working with ReadinessProbe, LivenessProbe, ProbeFailure, ProbesModule, CircuitBreaker." +description: "Explains Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting. Use when working with ReadinessProbe, LivenessProbe, LivenessProbeFailure, ReadinessProbeFailure, CircuitBreaker." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting; key triggers include ReadinessProbe, LivenessProbe, ProbeFailure, ProbesModule, CircuitBreaker." + use_when: "Use this file for Kora docs or implementation questions about Kora readiness and liveness probes, probe configuration, dependency health checks, and Kubernetes-style availability reporting; key triggers include ReadinessProbe, LivenessProbe, LivenessProbeFailure, ReadinessProbeFailure, CircuitBreaker." --- -Функционал дающий приложению два метода для получения проб на служебном порту для предоставления информации о жизнеспособности и готовности сервиса. +Пробы позволяют проверять `жизнеспособность` (liveness) и `готовность` (readiness) приложения через служебный HTTP-порт. +Их обычно используют оркестраторы и балансировщики нагрузки, чтобы понять, можно ли отправлять приложению запросы и нужно ли перезапускать его экземпляр. +Наличие двух отдельных проб помогает отличать временную неспособность принимать трафик от состояния, когда сам процесс следует считать неисправным. -Предоставляется по средствам подключения [служебного HTTP сервера](http-server.md). +Пробами управляет [служебный HTTP-сервер](http-server.md). По умолчанию он работает на порту `8085`. +Интерфейсы `LivenessProbe` и `ReadinessProbe` находятся в базовом модуле `ru.tinkoff.kora:common` (транзитивная зависимость любого приложения Kora), +а конечные точки, которые их предоставляют, поставляются модулем [HTTP-сервера](http-server.md), поэтому для добавления пробы не требуется никакой дополнительной зависимости. + +Обе конечные точки проб всегда присутствуют на служебном сервере, даже если приложение не регистрирует ни одной собственной пробы соответствующего вида — в этом случае конечная точка просто сообщает об успехе. Если нужен пошаговый разбор перед справочным описанием, смотрите [Наблюдаемость](../guides/observability.md). -## Жизнеспособности { #liveness } +## Жизнеспособность { #liveness } -Эта проба отвечает за признак — является ли приложение живым в данный момент. Kora старается начать отдавать эту пробу как можно раньше, чтобы оркестраторы точно знали, что нет проблем при старте и не пытались сделать рестарт приложения. +Эта проба показывает, что приложение живо и его не нужно перезапускать. Kora старается начать отдавать эту пробу как можно раньше, чтобы оркестратор не перезапускал приложение во время штатного запуска. -Пример конфигурации пути HTTP сервера для получения проб, описанной в классе `HttpServerConfig` (указаны значения по умолчанию): +Пример конфигурации пути служебного HTTP-сервера, описанной в классе `HttpServerConfig` (указано значение по умолчанию): ===! ":material-code-json: `Hocon`" ```javascript httpServer { - privateApiHttpLivenessPath = "/system/liveness" + privateApiHttpLivenessPath = "/system/liveness" //(1)! } ``` + 1. Путь пробы `жизнеспособности` на служебном HTTP-сервере (по умолчанию: `/system/liveness`). + === ":simple-yaml: `YAML`" ```yaml httpServer: - privateApiHttpLivenessPath: "/system/liveness" + privateApiHttpLivenessPath: "/system/liveness" #(1)! ``` -Для создания собственной пробы жизнеспособности требуется чтобы компонент реализовывал интерфейс: + 1. Путь пробы `жизнеспособности` на служебном HTTP-сервере (по умолчанию: `/system/liveness`). + +Чтобы создать собственную пробу `жизнеспособности`, зарегистрируйте [компонент](container.md), реализующий интерфейс `LivenessProbe`: + ```java public interface LivenessProbe { @Nullable - LivenessProbeFailure probe(); + LivenessProbeFailure probe() throws Exception; } ``` -В случае ошибки проба должна возвращать `LivenessProbeFailure`, а в случае успеха `null`. +В случае успеха проба должна возвращать `null`, а при ошибке — `LivenessProbeFailure` с описанием проблемы. +`LivenessProbeFailure` — это запись, единственное поле `message` которой становится телом ответа `503`: + +```java +public record LivenessProbeFailure(String message) {} +``` + +Метод `probe()` объявлен с `throws Exception`, поэтому реализация может напрямую вызывать API, бросающие проверяемые исключения. +Выброшенное исключение трактуется как сбой — точные коды статуса и тела ответа смотрите в разделе [Ответ](#response). + +===! ":fontawesome-brands-java: `Java`" -## Готовности { #readiness } + ```java + import ru.tinkoff.kora.common.Component; + import ru.tinkoff.kora.common.liveness.LivenessProbe; + import ru.tinkoff.kora.common.liveness.LivenessProbeFailure; -Эта проба отвечает за признак — является ли приложение готовым к работе в данный момент. + @Component + public final class ApplicationHealthProbe implements LivenessProbe { + + @Override + public LivenessProbeFailure probe() { + return null; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.Component + import ru.tinkoff.kora.common.liveness.LivenessProbe + import ru.tinkoff.kora.common.liveness.LivenessProbeFailure + + @Component + class ApplicationHealthProbe : LivenessProbe { + override fun probe(): LivenessProbeFailure? = null + } + ``` -Пример конфигурации пути HTTP сервера для получения проб, описанной в классе `HttpServerConfig` (указаны значения по умолчанию): +## Готовность { #readiness } + +Эта проба показывает, что приложение готово принимать рабочую нагрузку. + +Пример конфигурации пути служебного HTTP-сервера, описанной в классе `HttpServerConfig` (указано значение по умолчанию): ===! ":material-code-json: `Hocon`" ```javascript httpServer { - privateApiHttpReadinessPath = "/system/readiness" + privateApiHttpReadinessPath = "/system/readiness" //(1)! } ``` + 1. Путь пробы `готовности` на служебном HTTP-сервере (по умолчанию: `/system/readiness`). + === ":simple-yaml: `YAML`" ```yaml httpServer: - privateApiHttpReadinessPath: "/system/readiness" + privateApiHttpReadinessPath: "/system/readiness" #(1)! ``` -Для создания собственной пробы жизнеспособности требуется чтобы компонент реализовывал интерфейс: + 1. Путь пробы `готовности` на служебном HTTP-сервере (по умолчанию: `/system/readiness`). + +Чтобы создать собственную пробу `готовности`, зарегистрируйте [компонент](container.md), реализующий интерфейс `ReadinessProbe`: + ```java public interface ReadinessProbe { @Nullable - ReadinessProbeFailure probe(); + ReadinessProbeFailure probe() throws Exception; } ``` -В случае ошибки проба должна возвращать `ReadinessProbeFailure`, а в случае успеха `null`. +В случае успеха проба должна возвращать `null`, а при ошибке — `ReadinessProbeFailure` с описанием проблемы. +`ReadinessProbeFailure` — это запись, единственное поле `message` которой становится телом ответа `503`: + +```java +public record ReadinessProbeFailure(String message) {} +``` + +Как и в случае с `LivenessProbe`, метод `probe()` объявлен с `throws Exception`, и выброшенное исключение трактуется как сбой. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.common.Component; + import ru.tinkoff.kora.common.readiness.ReadinessProbe; + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure; + + import java.time.Duration; + import java.time.Instant; + + @Component + public final class CustomReadinessProbe implements ReadinessProbe { + + private static final Duration WARMUP_PERIOD = Duration.ofMillis(500); + + private final Instant startedAt = Instant.now(); + + @Override + public ReadinessProbeFailure probe() { + var readyAt = startedAt.plus(WARMUP_PERIOD); + if (Instant.now().isBefore(readyAt)) { + return new ReadinessProbeFailure("Service is warming up"); + } + return null; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.Component + import ru.tinkoff.kora.common.readiness.ReadinessProbe + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure + import java.time.Duration + import java.time.Instant + + @Component + class CustomReadinessProbe : ReadinessProbe { + private val startedAt = Instant.now() + + override fun probe(): ReadinessProbeFailure? { + val readyAt = startedAt.plus(Duration.ofMillis(500)) + return if (Instant.now().isBefore(readyAt)) { + ReadinessProbeFailure("Service is warming up") + } else { + null + } + } + } + ``` + +## Несколько проб { #multiple-probes } + +Kora автоматически собирает **каждый** зарегистрированный [компонент](container.md), реализующий `LivenessProbe` (или `ReadinessProbe`) — +связывать их между собой вручную не нужно. Каждая конечная точка выполняет все пробы своего вида и агрегирует результат: + +- Конечная точка возвращает `200 OK` только тогда, когда успешны **все** пробы этого вида. +- Единственная неуспешная проба заставляет всю конечную точку вернуть `503`, а телом ответа становится сообщение этой неуспешной пробы. +- Если ни одна проба этого вида не зарегистрирована, конечная точка возвращает `200 OK` — служебный сервер всегда предоставляет оба пути. + +Это позволяет разбить независимые условия готовности или жизнеспособности на несколько небольших узконаправленных компонентов-проб. + +===! ":fontawesome-brands-java: `Java`" + + ```java + import ru.tinkoff.kora.common.Component; + import ru.tinkoff.kora.common.readiness.ReadinessProbe; + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure; + + @Component + public final class ComponentReadinessProbe implements ReadinessProbe { //(1)! + + private final SomeComponent component; + + public ComponentReadinessProbe(SomeComponent component) { + this.component = component; + } + + @Override + public ReadinessProbeFailure probe() { + if (component.isInitialized()) { //(2)! + return null; + } + return new ReadinessProbeFailure("SomeComponent is not initialized yet"); + } + } + ``` + + 1. Компонентов `ReadinessProbe` может быть сколько угодно; конечная точка завершается неуспехом, если неуспешен хотя бы один из них + 2. Проверяйте состояние **внутреннего** компонента, а не внешней зависимости + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + import ru.tinkoff.kora.common.Component + import ru.tinkoff.kora.common.readiness.ReadinessProbe + import ru.tinkoff.kora.common.readiness.ReadinessProbeFailure + + @Component + class ComponentReadinessProbe( + private val component: SomeComponent + ) : ReadinessProbe { //(1)! + + override fun probe(): ReadinessProbeFailure? { + return if (component.isInitialized) { //(2)! + null + } else { + ReadinessProbeFailure("SomeComponent is not initialized yet") + } + } + } + ``` + + 1. Компонентов `ReadinessProbe` может быть сколько угодно; конечная точка завершается неуспехом, если неуспешен хотя бы один из них + 2. Проверяйте состояние **внутреннего** компонента, а не внешней зависимости + +## Ответ { #response } + +Каждая конечная точка пробы обслуживается [служебным HTTP-сервером](http-server.md) и возвращает тело `text/plain` вместе с кодом статуса: + +- `200 OK` — тело `OK` — все зарегистрированные пробы вернули `null` либо ни одна проба этого вида не зарегистрирована. +- `503 Service Unavailable` — телом становится `message` возвращённого `LivenessProbeFailure` / `ReadinessProbeFailure` — как минимум одна проба сообщила о сбое. +- `503 Service Unavailable` — тело `Probe failed: ` — проба выбросила исключение; выброшенное исключение трактуется как сбой. +- `503 Service Unavailable` — тело `Probe is not ready yet` — компонент пробы ещё не инициализирован в контейнере зависимостей. +- `408 Request Timeout` — тело `Probe failed: timeout` — выполнение пробы не завершилось за `30` секунд. + +Конечная точка отвечает, как только становится известен агрегированный результат; поэтому проверки здоровья оркестратора и балансировщика нагрузки могут ориентироваться либо на код статуса, либо на текстовое тело ответа. + +## Рекомендации { #recommendations } -## Совет { #recommendations } +???+ warning "Рекомендация" -???+ warning "Совет" + **Не рекомендуется делать пробы, которые напрямую проверяют внешние зависимости: базы данных, очереди или другие сервисы.** - **Мы крайне не советуем делать пробы, проверяющие внешние зависимости, такие как базы данных или другие сервисы.** + Временная недоступность внешней зависимости не должна автоматически приводить к перезапуску приложения. Для таких случаев используйте шаблон [CircuitBreaker](resilient.md#circuitbreaker). - В случае недоступности внешних зависимостей рекомендуется использовать шаблон [Прерыватель](resilient.md#circuitbreaker). +Проба должна отражать состояние **самого** приложения, а не систем, с которыми оно взаимодействует. +Хорошие примеры — это `ReadinessProbe`, которая возвращает ошибку, пока сервис прогревается, +или та, что проверяет, завершил ли внутренний компонент свою инициализацию. -Хорошим примером для `ReadinessProbe` может служить проба, которая возвращает ошибку во время прогрева сервиса. +Каждая проба выполняется на выделенном исполнителе (исполнитель на виртуальных потоках, если он доступен, иначе `ForkJoinPool.commonPool()`), +поэтому тело пробы может блокироваться, не задерживая служебный HTTP-сервер. При этом вся конечная точка по-прежнему ограничена тайм-аутом в `30` секунд, +по истечении которого она отвечает `408`, поэтому держите логику пробы быстрой и избегайте длительной или неограниченной работы. diff --git a/mkdocs/docs/ru/documentation/resilient.md b/mkdocs/docs/ru/documentation/resilient.md index a8f425d..62e5503 100644 --- a/mkdocs/docs/ru/documentation/resilient.md +++ b/mkdocs/docs/ru/documentation/resilient.md @@ -1,13 +1,16 @@ --- -description: "Explains Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, telemetry, configuration, and supported signatures. Use when working with @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." +description: "Explains Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, imperative managers, exceptions, telemetry, configuration, and supported signatures. Use when working with @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, telemetry, configuration, and supported signatures; key triggers include @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." + use_when: "Use this file for Kora docs or implementation questions about Kora resilience aspects for circuit breakers, retries, timeouts, fallback methods, imperative managers, exceptions, telemetry, configuration, and supported signatures; key triggers include @CircuitBreaker, @Retry, @Timeout, @Fallback, CircuitBreakerManager, RetryManager, TimeoutManager, FallbackManager, RetryState, CallNotPermittedException, RetryExhaustedException, TimeoutExhaustedException, CircuitBreakerConfig, RetryConfig, TimeoutConfig, ResilientModule." --- -Модуль для создания отказоустойчивого приложения с использованием таких подходов как [прерыватель](#circuitbreaker), -[резервный метод](#fallback), [повторитель](#retry) и [ограничитель](#timeout) с помощью аннотаций аспектов в декларативном стиле. +Модуль для построения отказоустойчивого приложения с помощью таких механизмов, как [CircuitBreaker](#circuitbreaker), +[Fallback](#fallback), [Retry](#retry) и [Timeout](#timeout). +Эти механизмы можно применять декларативно через аннотации-аспекты либо напрямую через компоненты-менеджеры, когда защита требуется в императивном коде. -Если нужен пошаговый разбор перед справочным описанием, смотрите [Отказоустойчивость](../guides/resilient.md). +`ResilientModule` объединяет `CircuitBreakerModule`, `RetryModule`, `TimeoutModule` и `FallbackModule`. + +Пошаговый разбор перед справочным описанием смотрите в разделе [Отказоустойчивость](../guides/resilient.md). ## Подключение { #dependency } @@ -37,28 +40,30 @@ agent: interface Application : ResilientModule ``` -## Прерыватель { #circuitbreaker } +## CircuitBreaker { #circuitbreaker } + +`CircuitBreaker` — это прокси, который управляет потоком запросов к конкретному методу +и может временно запретить выполнение этого метода, если тот выбрасывает много исключений, попадающих под настроенный фильтр (`CircuitBreakerPredicate`). -Прерыватель (`CircuitBreaker`) – это прокси, который контролирует поток к запросам конкретного метода -и может прекращать временно выполнение этого метода если метод бросает много исключений соответствующих заданным требованиям фильтра (`CircuitBreakerPredicate`). +Цель применения CircuitBreaker — дать системе время исправить ошибку, вызвавшую сбой, прежде чем позволить приложению снова попытаться выполнить операцию. +Паттерн `CircuitBreaker` обеспечивает стабильность на время восстановления системы после сбоя и снижает влияние на производительность. +`CircuitBreaker` может находиться в одном из нескольких состояний: `CLOSED`, `OPEN`, `HALF_OPEN`. -Цель применения прерывателя — дать системе время на исправление ошибки, которая вызвала сбой, прежде чем разрешить приложению попытаться выполнить операцию еще раз. -Шаблон прерыватель обеспечивает стабильность, пока система восстанавливается после сбоя и снижает влияние на производительность. -Прерыватель может находиться в нескольких состояниях в зависимости от поведения (`OPEN, CLOSED, HALF_OPEN`) +- `CLOSED`: запрос приложения передается защищаемой операции. Прокси подсчитывает недавние сбои в пределах настроенного числа операций (`slidingWindowSize`), проходящих через него, и увеличивает этот счетчик, когда операция завершается неуспешно. + Если количество запросов превышает минимально необходимое для расчета (`minimumRequiredCalls`) и число недавних сбоев превышает настроенный порог (`failureRateThreshold`), прокси переходит в `OPEN`. +- `OPEN`: находясь в этом состоянии, запрос приложения немедленно завершается с ошибкой, и приложению возвращается исключение. + В этот момент прокси запускает таймер ожидания (`waitDurationInOpenState`), и по его истечении прокси переходит в `HALF_OPEN`. +- `HALF_OPEN`: ограниченному числу запросов (`permittedCallsInHalfOpenState`) от приложения разрешается пройти и вызвать операцию. Если эти запросы успешны, считается, что ошибка, ранее вызвавшая + сбой, устранена, и `CircuitBreaker` переходит в состояние `CLOSED` (счетчик сбоев сбрасывается). Если какой-либо запрос завершается сбоем, `CircuitBreaker` считает, что + неисправность все еще присутствует, поэтому возвращается в состояние `OPEN` и перезапускает таймер ожидания (`waitDurationInOpenState`), чтобы дать системе дополнительное время на восстановление после сбоя. -- `CLOSED`: Запрос приложения перенаправляется на операцию. Прокси ведет подсчет числа недавних сбоев в рамках установленного кол-ва операций (`slidingWindowSize`) поступающих через прокси, и если вызов операции не завершился успешно, прокси увеличивает это число. - Если число запросов превысило установленный минимальный потолок необходимый для подсчетов (`minimumRequiredCalls`) и число недавних сбоев превышает заданный порог (`failureRateThreshold`) в течение заданного периода времени, прокси переводится в состояние `OPEN`. -- `OPEN`: Во время нахождения в таком статусе запрос от приложения немедленно завершает с ошибкой и исключение возвращается в приложение. - На этом этапе прокси запускает таймер времени ожидания (`waitDurationInOpenState`), и по истечении времени этого таймера прокси переводится в состояние `HALF-OPEN`. -- `HALF-OPEN`: Ограниченному числу запросов (`permittedCallsInHalfOpenState`) от приложения разрешено проходить через операцию и вызывать ее. Если эти запросы выполняются успешно, предполагается, что ошибка, которая ранее вызывала - сбой, устранена, а автоматический выключатель переходит в состояние `CLOSED` (счетчик сбоев сбрасывается). Если какой-либо запрос завершается со сбоем, автоматическое выключение предполагает, что - неисправность все еще присутствует, поэтому он возвращается в состояние `OPEN` и перезапускает таймер времени ожидания (`waitDurationInOpenState`), чтобы дать системе дополнительное время на восстановление после сбоя. +Состояние `HALF_OPEN` помогает предотвратить лавинообразный рост числа запросов к сервису: после начала восстановления сервис некоторое время может справляться лишь с ограниченным числом запросов. -Состояние `HALF-OPEN` помогает предотвратить быстрый рост запросов к сервису. Т.к. после начала работы сервиса, некоторое время он может быть способен обрабатывать ограниченное число запросов до полного восстановления. +Изначально находится в состоянии `CLOSED`. -Изначально имеет состояние `CLOSED`. +### Декларативное использование { #declarative-usage } -### Декларативный подход { #declarative-usage } +Если `CircuitBreaker` находится в состоянии `OPEN`, вызов завершается с `CallNotPermittedException`. ===! ":fontawesome-brands-java: `Java`" @@ -86,11 +91,12 @@ agent: ### Конфигурация { #configuration } -Существует конфигурация по умолчанию, которая применяется ко всем прерывателям при создании -и затем применяются именованные настройки конкретного прерывателя для переопределения настроек по умолчанию. -Можно изменить настройки по умолчанию для всех прерывателей одновременно изменив конфигурацию по умолчанию (`default`). +Существует конфигурация по умолчанию, которая применяется к CircuitBreaker при его создании, +после чего именованные настройки конкретного CircuitBreaker применяются поверх настроек по умолчанию. + +Вы можете изменить настройки по умолчанию сразу для всех CircuitBreaker, изменив конфигурацию `default`. -Пример полной конфигурации, описанной в классе `CircuitBreakerConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации, описанной в классе `CircuitBreakerConfig` (указаны примерные значения или значения по умолчанию): ===! ":material-code-json: `Hocon`" @@ -104,17 +110,20 @@ agent: waitDurationInOpenState = "25s" //(4)! permittedCallsInHalfOpenState = 15 //(5)! enabled = true //(6)! + failurePredicateName = "MyPredicate" //(7)! } } } ``` - 1. Предельное кол-во запросов в рамках которых рассчитывается `failureRateThreshold` для определения состояния (**обязательный**) - 2. Минимальное кол-во запросов необходимое для начала расчета состояния (**обязательный**) - 3. Процент неуспешных запросов который необходим для перехода в состояния `OPEN` (имеет значения от *1 до 100*) (**обязательный**) - 4. Время ожидания в статусе `OPEN`, после которого осуществляется переход в статус `HALF-OPEN` (**обязательный**) - 5. Необходимое кол-во запросов в статусе `HALF-OPEN` которые должны завершится успехом для перехода в `CLOSED` (**обязательный**) - 6. Включить или отключить прерыватель (по умолчанию `true`) + 1. Максимальное число запросов, используемых для расчета `failureRateThreshold` и определения состояния (`required`, значение по умолчанию не задано). + 2. Минимальное число запросов, необходимое для начала расчета состояния (`required`, значение по умолчанию не задано). + 3. Процент неуспешных запросов, необходимый для перехода в `OPEN`; значение должно быть от `1` до `100` (`required`, значение по умолчанию не задано). + 4. Время ожидания в `OPEN`, по истечении которого выполняется переход в `HALF_OPEN` (`required`, значение по умолчанию не задано). + 5. Число запросов в `HALF_OPEN`, которые должны завершиться успешно для перехода в `CLOSED` (`required`, значение по умолчанию не задано). + 6. Включение или отключение `CircuitBreaker` (по умолчанию: `true`). + 7. Имя фильтра исключений из `CircuitBreakerPredicate#name()` (по умолчанию учитываются все ошибки). + === ":simple-yaml: `YAML`" @@ -128,16 +137,18 @@ agent: waitDurationInOpenState: "25s" #(4)! permittedCallsInHalfOpenState: 15 #(5)! enabled: true #(6)! + failurePredicateName: "MyPredicate" #(7)! ``` - 1. Предельное кол-во запросов в рамках которых рассчитывается `failureRateThreshold` для определения состояния (**обязательный**) - 2. Минимальное кол-во запросов необходимое для начала расчета состояния (**обязательный**) - 3. Процент неуспешных запросов который необходим для перехода в состояния `OPEN` (имеет значения от *1 до 100*) (**обязательный**) - 4. Время ожидания в статусе `OPEN`, после которого осуществляется переход в статус `HALF-OPEN` (**обязательный**) - 5. Необходимое кол-во запросов в статусе `HALF-OPEN` которые должны завершится успехом для перехода в `CLOSED` (**обязательный**) - 6. Включить или отключить прерыватель (по умолчанию `true`) + 1. Максимальное число запросов, используемых для расчета `failureRateThreshold` и определения состояния (`required`, значение по умолчанию не задано). + 2. Минимальное число запросов, необходимое для начала расчета состояния (`required`, значение по умолчанию не задано). + 3. Процент неуспешных запросов, необходимый для перехода в `OPEN`; значение должно быть от `1` до `100` (`required`, значение по умолчанию не задано). + 4. Время ожидания в `OPEN`, по истечении которого выполняется переход в `HALF_OPEN` (`required`, значение по умолчанию не задано). + 5. Число запросов в `HALF_OPEN`, которые должны завершиться успешно для перехода в `CLOSED` (`required`, значение по умолчанию не задано). + 6. Включение или отключение `CircuitBreaker` (по умолчанию: `true`). + 7. Имя фильтра исключений из `CircuitBreakerPredicate#name()` (по умолчанию учитываются все ошибки). -Пример переопределения именованных настроек для определенного прерывателя: +Пример переопределения именованных настроек конкретного CircuitBreaker: ===! ":material-code-json: `Hocon`" @@ -160,14 +171,25 @@ agent: waitDurationInOpenState: "50s" ``` -Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#resilience). +!!! warning "Ограничения" + + Следующие условия проверяются при старте приложения — нарушение любого из них приводит к ошибке сборки графа: + `failureRateThreshold` должен быть в диапазоне `1..100`; `slidingWindowSize` ≥ `1`; `minimumRequiredCalls` ≥ `1` **и** ≤ `slidingWindowSize`; `permittedCallsInHalfOpenState` ≥ `1`. + Для каждого `@CircuitBreaker` **должна** присутствовать либо именованная, либо `default` конфигурация, иначе запуск завершится ошибкой. + +!!! note "Примечание" + + Установка `enabled = false` превращает аспект в прозрачный проброс — метод вызывается напрямую, без размыкания цепи. + `failurePredicateName` по умолчанию равен `KoraCircuitBreakerPredicate` (учитывает каждую ошибку); собственный `CircuitBreakerPredicate` может использоваться несколькими прерывателями через ссылку на его `name()`. + +Метрики модуля описаны в разделе [Справочник метрик](metrics.md#resilience). ### Фильтрация исключений { #exception-filtering } -Для регистрации какие ошибки следует записывать как ошибки со стороны прерывателя, можно переопределить фильтр по умолчанию, -требуется реализовать `CircuitBreakerPredicate` и зарегистрировать свой компонент в контексте и указать в конфигурации прерывателя его имя возвращаемое в методе `name()`. +Чтобы задать, какие ошибки должны учитываться как ошибки CircuitBreaker, вы можете переопределить фильтр по умолчанию: +необходимо реализовать `CircuitBreakerPredicate`, зарегистрировать свой компонент в контексте и указать в конфигурации CircuitBreaker его имя, возвращаемое методом `name()`. -По умолчанию прерыватель учитывает все ошибки. +По умолчанию `CircuitBreaker` учитывает все ошибки. ===! ":fontawesome-brands-java: `Java`" @@ -213,7 +235,7 @@ agent: } ``` - 1. Имя предиката из метода `name()` + 1. Имя фильтра исключений из `CircuitBreakerPredicate#name()` (по умолчанию учитываются все ошибки). === ":simple-yaml: `YAML`" @@ -224,12 +246,12 @@ agent: failurePredicateName: "MyPredicate" #(1)! ``` - 1. Имя предиката из метода `name()` + 1. Имя фильтра исключений из `CircuitBreakerPredicate#name()` (по умолчанию учитываются все ошибки). -### Императивный подход { #imperative-usage } +### Императивное использование { #imperative-usage } -Можно использовать прерыватель в императивном коде, для этого понадобиться внедрить как зависимость `CircuitBreakerManager` -и взять из него прерыватель по имени конфигурации которая указывалась бы в аннотации: +Прерыватель можно использовать в императивном коде: внедрите `CircuitBreakerManager` +и получите из него `CircuitBreaker` по имени конфигурации, которое было бы указано в аннотации: ===! ":fontawesome-brands-java: `Java`" @@ -271,13 +293,74 @@ agent: } ``` -## Повторитель { #retry } +Чтобы вернуть резервное значение вместо выбрасывания `CallNotPermittedException`, когда прерыватель находится в `OPEN`, используйте перегрузку `accept`, принимающую второй `Supplier`: -Повторитель (`Retry`) - предоставляет возможность настраивать политику повторного вызова проаннотированных методов. -Позволяет указать когда требуется повторить попытку выполнения метода, настроить параметры повторения, -в случае если методом было брошено исключение соответствующая заданным требованиям фильтра (`RetryPredicate`). +===! ":fontawesome-brands-java: `Java`" + + ```java + public String doWork() { + var circuitBreaker = manager.get("custom"); + return circuitBreaker.accept(this::doSomeWork, () -> "fallback"); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun doWork(): String { + val circuitBreaker = manager["custom"] + return circuitBreaker.accept({ doSomeWork() }, { "fallback" }) + } + ``` -### Декларативный подход { #declarative-usage-2 } +Когда защищаемый вызов нельзя обернуть в единственный `Supplier`, получите и освободите разрешение вручную. +Вызовите `acquire()` (выбрасывает `CallNotPermittedException`, когда прерыватель в `OPEN` либо в `HALF_OPEN` без оставшихся пробных вызовов), чтобы получить разрешение, затем **всегда** сообщайте о результате через `releaseOnSuccess()` или `releaseOnError(Throwable)` — иначе прерыватель теряет разрешение, и его учет становится некорректным: + +===! ":fontawesome-brands-java: `Java`" + + ```java + public String doWork() { + var circuitBreaker = manager.get("custom"); + circuitBreaker.acquire(); // throws CallNotPermittedException when the call is not permitted + try { + var result = doSomeWork(); + circuitBreaker.releaseOnSuccess(); + return result; + } catch (Throwable e) { + circuitBreaker.releaseOnError(e); + throw e; + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun doWork(): String { + val circuitBreaker = manager["custom"] + circuitBreaker.acquire() // throws CallNotPermittedException when the call is not permitted + try { + val result = doSomeWork() + circuitBreaker.releaseOnSuccess() + return result + } catch (e: Throwable) { + circuitBreaker.releaseOnError(e) + throw e + } + } + ``` + +`tryAcquire()` — это альтернатива без выбрасывания исключения: она возвращает `false`, когда вызов не разрешен, поэтому вы можете ветвить логику без перехвата `CallNotPermittedException`. +Когда `acquire()` все же выбрасывает исключение, текущее [состояние](#circuitbreaker) прерывателя (`OPEN` или `HALF_OPEN`) доступно через `CallNotPermittedException#state()`. + +## Retry { #retry } + +`Retry` предоставляет возможность настроить повторный вызов аннотированных методов. +Он позволяет задать, когда метод должен повторяться, и настроить параметры повторов, когда метод выбрасывает исключение, попадающее под настроенный фильтр (`RetryPredicate`). + +### Декларативное использование { #declarative-usage-2 } + +Если все попытки исчерпаны, вызов завершается с `RetryExhaustedException`. ===! ":fontawesome-brands-java: `Java`" @@ -305,11 +388,12 @@ agent: ### Конфигурация { #configuration-2 } -Существует конфигурация по умолчанию, которая применяется ко всем повторителям при создании -и затем применяются именованные настройки конкретного повторителя для переопределения настроек по умолчанию. -Можно изменить настройки по умолчанию для всех повторителей одновременно изменив конфигурацию по умолчанию (`default`). +Существует конфигурация `default`, которая применяется к `Retry` при создании, +после чего именованные настройки конкретного `Retry` применяются поверх настроек по умолчанию. + +Настройки по умолчанию можно изменить сразу для всех `Retry`, изменив конфигурацию `default`. -Пример полной конфигурации, описанной в классе `RetryConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации, описанной в классе `RetryConfig` (указаны значения по умолчанию или примерные значения): ===! ":material-code-json: `Hocon`" @@ -321,15 +405,17 @@ agent: attempts = 2 //(2)! delayStep = "100ms" //(3)! enabled = true //(4)! + failurePredicateName = "MyPredicate" //(5)! } } } ``` - 1. Начальное время задержки для операции при повторения (**обязательный**) - 2. Кол-во попыток для операции (**обязательный**) - 3. Шаг задержки который аккумулируется в следствии последующих попыток - 4. Включить или отключить повторитель (по умолчанию `true`) + 1. Начальная задержка перед повторным вызовом (`required`, значение по умолчанию не задано). + 2. Число попыток повтора (`required`, значение по умолчанию не задано). + 3. Приращение задержки для последующих попыток (по умолчанию: `0`). + 4. Включение или отключение `Retry` (по умолчанию: `true`). + 5. Имя фильтра исключений из `RetryPredicate#name()` (по умолчанию учитываются все ошибки). === ":simple-yaml: `YAML`" @@ -341,19 +427,32 @@ agent: attempts: 2 #(2)! delayStep: "100ms" #(3)! enabled: true #(4)! + failurePredicateName: "MyPredicate" #(5)! ``` - 1. Начальное время задержки для операции при повторения (**обязательный**) - 2. Кол-во попыток для операции (**обязательный**) - 3. Шаг задержки который аккумулируется в следствии последующих попыток - 4. Включить или отключить повторитель (по умолчанию `true`) + 1. Начальная задержка перед повторным вызовом (`required`, значение по умолчанию не задано). + 2. Число попыток повтора (`required`, значение по умолчанию не задано). + 3. Приращение задержки для последующих попыток (по умолчанию: `0`). + 4. Включение или отключение `Retry` (по умолчанию: `true`). + 5. Имя фильтра исключений из `RetryPredicate#name()` (по умолчанию учитываются все ошибки). + +!!! warning "Ограничения и прогрессия задержки" + + `delay` и `attempts` обязательны (берутся из именованной или `default` конфигурации), а `attempts` должно быть `≥ 0`; отсутствие `delay`/`attempts` или отрицательное `attempts` приводит к ошибке старта приложения. + `attempts` считает повторы **после** первоначального вызова, поэтому `attempts = 2` допускает в сумме до `3` выполнений. + Каждый повтор ждет на `delayStep` (по умолчанию `0`) дольше предыдущего, так что задержки составляют `delay`, `delay + delayStep`, `delay + 2·delayStep`, … . + +!!! note "Примечание" + + Установка `enabled = false` превращает `@Retry` в прозрачный проброс (метод выполняется один раз). + `failurePredicateName` по умолчанию равен `KoraRetryPredicate` (повторяет при каждой ошибке); собственный `RetryPredicate` может использоваться несколькими повторителями через ссылку на его `name()`. ### Фильтрация исключений { #exception-filtering-2 } -Для регистрации какие ошибки следует записывать как ошибки со стороны повторителя, можно переопределить фильтр по умолчанию, -требуется реализовать `RetryPredicate` и зарегистрировать свой компонент в контексте и указать в конфигурации повторителя его имя возвращаемое в методе `name()`. +Чтобы задать, какие ошибки должны учитываться как ошибки на стороне Retry, вы можете переопределить фильтр по умолчанию: +необходимо реализовать `RetryPredicate`, зарегистрировать его компонент в контексте и указать в конфигурации Retry его имя, возвращаемое методом `name()`. -По умолчанию повторитель учитывает все ошибки. +По умолчанию `Retry` учитывает все ошибки. ===! ":fontawesome-brands-java: `Java`" @@ -399,7 +498,7 @@ agent: } ``` - 1. Имя предиката из метода `name()` + 1. Имя фильтра исключений из `RetryPredicate#name()` (по умолчанию учитываются все ошибки). === ":simple-yaml: `YAML`" @@ -410,12 +509,12 @@ agent: failurePredicateName: "MyPredicate" #(1)! ``` - 1. Имя предиката из метода `name()` + 1. Имя фильтра исключений из `RetryPredicate#name()` (по умолчанию учитываются все ошибки). -### Императивный подход { #imperative-usage-2 } +### Императивное использование { #imperative-usage-2 } -Можно использовать повторитель в императивном коде, для этого понадобиться внедрить как зависимость `RetryManager` -и взять из него повторитель по имени конфигурации которая указывалась бы в аннотации: +Повторитель можно использовать в императивном коде: внедрите `RetryManager` +и получите из него `Retry` по имени конфигурации, которое было бы указано в аннотации: ===! ":fontawesome-brands-java: `Java`" @@ -457,11 +556,85 @@ agent: } ``` -## Ограничитель { #timeout } +Чтобы вернуть резервное значение вместо выбрасывания `RetryExhaustedException`, когда все попытки исчерпаны, передайте второй `Supplier`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var retry = manager.get("custom"); + return retry.retry(this::doSomeWork, () -> "fallback"); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val retry = manager["custom"] + return retry.retry({ doSomeWork() }, { "fallback" }) + ``` + +Для асинхронного императивного кода есть перегрузка, которая повторяет `Supplier>` и возвращает `CompletionStage`, планируя каждую попытку после настроенной задержки без блокировки вызывающего потока. + +#### Ручное управление состоянием повтора { #manual-retry-state } + +Для полного контроля над циклом повторов используйте `retry.asState()`, который возвращает `RetryState`. +Он реализует `AutoCloseable`, поэтому оберните его в try-with-resources (Java) или `use` (Kotlin), чтобы записать метрики по завершении. +При каждом перехваченном исключении вызывайте `onException(Throwable)`, который возвращает `RetryStatus`: + +- `ACCEPTED` — разрешена еще одна попытка; вызовите `doDelay()` (блокирует на время текущей задержки) и повторите. +- `REJECTED` — исключение отклонено `RetryPredicate` и не должно повторяться; пробросьте его. +- `EXHAUSTED` — все попытки исчерпаны; выбросьте `RetryExhaustedException` (или вернитесь к значению по умолчанию). + +`getAttempts()` / `getAttemptsMax()` сообщают о прогрессе, а `getDelayNanos()` возвращает следующую задержку. + +===! ":fontawesome-brands-java: `Java`" + + ```java + public String doWork() { + var retry = manager.get("custom"); + try (var state = retry.asState()) { + while (true) { + try { + return doSomeWork(); + } catch (Exception e) { + switch (state.onException(e)) { + case ACCEPTED -> state.doDelay(); // wait, then loop and retry + case REJECTED -> throw e; // not retryable + case EXHAUSTED -> throw new RetryExhaustedException("custom", state.getAttemptsMax(), e); + } + } + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + fun doWork(): String { + val retry = manager["custom"] + retry.asState().use { state -> + while (true) { + try { + return doSomeWork() + } catch (e: Exception) { + when (state.onException(e)) { + Retry.RetryState.RetryStatus.ACCEPTED -> state.doDelay() // wait, then loop and retry + Retry.RetryState.RetryStatus.REJECTED -> throw e // not retryable + Retry.RetryState.RetryStatus.EXHAUSTED -> throw RetryExhaustedException("custom", state.attemptsMax, e) + } + } + } + } + } + ``` + +## Timeout { #timeout } + +`Timeout` задает максимальное время выполнения аннотированного метода. -Ограничитель времени (`Timeout`) - предоставляет возможность задавать максимальное время работы проаннотированного метода. +### Декларативное использование { #declarative-usage-3 } -### Декларативный подход { #declarative-usage-3 } +Если метод не завершается в пределах `duration`, вызов завершается с `TimeoutExhaustedException`. ===! ":fontawesome-brands-java: `Java`" @@ -499,11 +672,12 @@ agent: ### Конфигурация { #configuration-3 } -Существует конфигурация по умолчанию, которая применяется к ограничителю при создании -и затем применяются именованные настройки конкретного ограничителя для переопределения настроек по умолчанию. -Можно изменить настройки по умолчанию для всех ограничителей одновременно изменив конфигурацию по умолчанию (`default`). +Существует конфигурация `default`, которая применяется к Timeout при его создании, +после чего именованные настройки конкретного Timeout применяются поверх настроек по умолчанию. -Пример полной конфигурации, описанной в классе `TimeoutConfig` (указаны примеры значений или значения по умолчанию): +Настройки по умолчанию можно изменить сразу для всех Timeout, изменив конфигурацию `default`. + +Пример полной конфигурации, описанной в классе `TimeoutConfig` (указаны значения по умолчанию или примерные значения): ===! ":material-code-json: `Hocon`" @@ -518,8 +692,8 @@ agent: } ``` - 1. Предельное время работы операции после которого будет брошен `TimeoutExhaustedException` (**обязательный**) - 2. Включить или отключить ограничитель (по умолчанию `true`) + 1. Ограничение времени операции, по превышении которого будет выброшено `TimeoutExhaustedException` (`required`, значение по умолчанию не задано). + 2. Включение или отключение `Timeout` (по умолчанию: `true`). === ":simple-yaml: `YAML`" @@ -527,17 +701,22 @@ agent: resilient: timeout: default: - delay: "1s" #(1)! + duration: "1s" #(1)! enabled: true #(2)! ``` - 1. Предельное время работы операции после которого будет брошен `TimeoutExhaustedException` (**обязательный**) - 2. Включить или отключить ограничитель (по умолчанию `true`) + 1. Ограничение времени операции, по превышении которого будет выброшено `TimeoutExhaustedException` (`required`, значение по умолчанию не задано). + 2. Включение или отключение `Timeout` (по умолчанию: `true`). + +!!! note "Примечание" -### Императивный подход { #imperative-usage-3 } + `duration` обязателен (берется из именованной или `default` конфигурации), и без него запуск завершается ошибкой. + Установка `enabled = false` превращает `@Timeout` в прозрачный проброс — метод выполняется без ограничения по времени. -Можно использовать ограничитель времени в императивном коде, для этого понадобиться внедрить как зависимость `TimeoutManager` -и взять из него ограничитель по имени конфигурации которая указывалась бы в аннотации: +### Императивное использование { #imperative-usage-3 } + +Ограничитель времени можно использовать в императивном коде: внедрите `TimeoutManager` +и получите из него `Timeout` по имени конфигурации, которое было бы указано в аннотации: ===! ":fontawesome-brands-java: `Java`" @@ -579,14 +758,31 @@ agent: } ``` -## Резервный метод { #fallback } +`Timeout` также предоставляет `execute(Runnable)` для операций, ничего не возвращающих, а `timeout()` возвращает настроенную `Duration`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + var timeout = manager.get("custom"); + Duration limit = timeout.timeout(); // configured duration + timeout.execute(() -> { /* do some work */ }); // Runnable variant, throws TimeoutExhaustedException on timeout + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val timeout = manager["custom"] + val limit: Duration = timeout.timeout() // configured duration + timeout.execute(Runnable { /* do some work */ }) // Runnable variant, throws TimeoutExhaustedException on timeout + ``` -Резервный метод (`Fallback`) - предоставляет возможность указания метода который будет вызван в случае -если исключение брошенное проаннотированным методом будет удовлетворено фильтрам (`FallbackPredicate`). +## Fallback { #fallback } -Метод **должен совпадать** по сигнатуре возвращаемого результата. +`Fallback` позволяет указать метод, который будет вызван, когда исключение, выброшенное аннотированным методом, попадает под настроенные фильтры (`FallbackPredicate`). -### Декларативный подход { #declarative-usage-4 } +Резервный метод **должен совпадать** по типу возвращаемого значения с аннотированным методом. + +### Декларативное использование { #declarative-usage-4 } Пример резервного метода без аргументов: @@ -616,11 +812,11 @@ agent: @Fallback(value = "custom", method = "getFallback()") fun value(): String = "value" - fun fallback(): String = "fallback" + fun getFallback(): String = "fallback" } ``` -Пример резервного метода с аргументами: +Пример для *Fallback* с аргументами: ===! ":fontawesome-brands-java: `Java`" @@ -628,7 +824,7 @@ agent: @Component public class SomeService { - @Fallback(value = "custom", method = "getFallback(arg3, arg1)") //(1)! + @Fallback(value = "custom", method = "getFallback(arg3, arg1)") // Passes the arguments of the annotated method in the specified order to the Fallback method public String getValue(String arg1, Integer arg2, Long arg3) { return "value"; } @@ -638,8 +834,6 @@ agent: } } ``` - - 1. Передает аргументы проаннотированного метода в указанном порядке в резервный метод === ":simple-kotlin: `Kotlin`" @@ -647,22 +841,22 @@ agent: @Component open class SomeService { - @Fallback(value = "custom", method = "getFallback(arg3, arg1)") //(1)! + // Passes the arguments of the annotated method in the specified order to the Fallback method + @Fallback(value = "custom", method = "getFallback(arg3, arg1)") fun getValue(arg1: String, arg2: Int, arg3: Long): String = "value" fun getFallback(argLong: Long, argString: String): String = "fallback" } ``` - 1. Передает аргументы проаннотированного метода в указанном порядке в резервный метод - ### Конфигурация { #configuration-4 } -Существует конфигурация по умолчанию, которая применяется ко всем резервным метода при создании -и затем применяются именованные настройки конкретного резервного метода для переопределения настроек по умолчанию. -Можно изменить настройки по умолчанию для всех резервных методов одновременно изменив конфигурацию по умолчанию (`default`). +Существует конфигурация `default`, которая применяется к Fallback при создании, +после чего именованные настройки конкретного Fallback применяются поверх настроек по умолчанию. + +Настройки по умолчанию можно изменить сразу для всех Fallback, изменив конфигурацию `default`. -Пример полной конфигурации, описанной в классе `FallbackConfig` (указаны примеры значений или значения по умолчанию): +Пример полной конфигурации, описанной в классе `FallbackConfig` (указаны значения по умолчанию или примерные значения): ===! ":material-code-json: `Hocon`" @@ -677,8 +871,8 @@ agent: } ``` - 1. Имя предиката из метода `name()` - 2. Включить или отключить резервный метод (по умолчанию `true`) + 1. Имя фильтра исключений из `FallbackPredicate#name()` (по умолчанию учитываются все ошибки). + 2. Включение или отключение `Fallback` (по умолчанию: `true`). === ":simple-yaml: `YAML`" @@ -690,14 +884,21 @@ agent: enabled: true #(2)! ``` - 1. Имя предиката из метода `name()` - 2. Включить или отключить резервный метод (по умолчанию `true`) + 1. Имя фильтра исключений из `FallbackPredicate#name()` (по умолчанию учитываются все ошибки). + 2. Включение или отключение `Fallback` (по умолчанию: `true`). + +!!! note "Примечание" + + В отличие от других аспектов, у `@Fallback` нет обязательных свойств — без конфигурации он использует значения по умолчанию. + Установка `enabled = false` отключает резервный вариант, так что исходное исключение пробрасывается дальше. + `failurePredicateName` по умолчанию равен `KoraFallbackPredicate` (запускает резервный вариант при каждой ошибке); собственный `FallbackPredicate` может использоваться несколькими резервными вариантами через ссылку на его `name()`. ### Фильтрация исключений { #exception-filtering-3 } -Для регистрации какие ошибки следует записывать как ошибки со стороны резервного метода, можно переопределить фильтр по умолчанию, -требуется реализовать `FallbackPredicate` и зарегистрировать свой компонент в контексте -и указать в конфигурации резервного метода его имя возвращаемое в методе `name()`. +Чтобы задать, какие ошибки должны учитываться как ошибки Fallback, вы можете переопределить фильтр по умолчанию: +необходимо реализовать `FallbackPredicate`, зарегистрировать свой компонент в контексте и указать в конфигурации Fallback его имя, возвращаемое методом `name()`. + +По умолчанию `Fallback` учитывает все ошибки. ===! ":fontawesome-brands-java: `Java`" @@ -729,10 +930,10 @@ agent: } ``` -### Императивный подход { #imperative-usage-4 } +### Императивное использование { #imperative-usage-4 } -Можно использовать резервный метод в императивном коде, для этого понадобиться внедрить как зависимость `FallbackManager` -и взять из него резервный метод по имени конфигурации которая указывалась бы в аннотации: +Резервный метод можно использовать в императивном коде: внедрите `FallbackManager` +и получите из него `Fallback` по имени конфигурации, которое было бы указано в аннотации: ===! ":fontawesome-brands-java: `Java`" @@ -774,12 +975,44 @@ agent: } ``` -## Комбинация { #combination } +Для операций, ничего не возвращающих, используйте перегрузку с `Runnable`; а `canFallback(Throwable)` сообщает, запустит ли заданное исключение резервный вариант согласно настроенному `FallbackPredicate`: + +===! ":fontawesome-brands-java: `Java`" -Можно совмещать одновременно над одним методом все вышеперечисленные аннотации. + ```java + var fallback = manager.get("custom"); -Порядок применения аннотаций зависит от порядка объявления аннотаций. -Вы можете поменять порядок по своему желанию и комбинировать его с другими аннотациями, которые точно также применяются в порядке объявления. + // canFallback tells whether the exception would trigger the fallback + if (fallback.canFallback(exception)) { + // exception matches the configured FallbackPredicate + } + + // Runnable variant for operations that return nothing + fallback.fallback( + () -> { /* primary action */ }, + () -> { /* fallback action */ }); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val fallback = manager["custom"] + + // canFallback tells whether the exception would trigger the fallback + if (fallback.canFallback(exception)) { + // exception matches the configured FallbackPredicate + } + + // Runnable variant for operations that return nothing + fallback.fallback(Runnable { /* primary action */ }, Runnable { /* fallback action */ }) + ``` + +## Комбинирование { #combination } + +Все перечисленные выше аннотации можно комбинировать одновременно над одним методом. + +Порядок применения аннотаций зависит от порядка их объявления. +Вы можете менять порядок по своему усмотрению и комбинировать с другими аннотациями, которые также применяются в порядке объявления. ===! ":fontawesome-brands-java: `Java`" @@ -819,14 +1052,14 @@ agent: В примере выше: -1. Применяется `@Timeout` который, говорит что метод не должен выполняться дольше времени указанного в конфигурации -2. Применяется `@Retry` который будет пытаться повторить выполнение метода указанное в конфигурации кол-во раз в случае, если метод бросил исключение по цепочке (включая `@Timeout`) -3. Применяется `@CircuitBreaker` который будет работать согласно конфигурации и [состоянию](#circuitbreaker) в зависимости успешного результата метода или если метод бросил исключение по цепочке (включая `@Timeout` & `@Retry`) -4. Применяется `@Fallback` который будет вызвать `getFallback` метод с аргументом `arg1` в случае если метод бросил исключение по цепочке (включая `@Timeout` & `@Retry` & `@CircuitBreaker`) +1. Применяется `@Timeout` и проверяет, что метод не выполняется дольше времени, указанного в конфигурации. +2. Применяется `@Retry` и пытается повторить выполнение метода настроенное число раз, если метод выбрасывает исключение в цепочке, включая исключение из `@Timeout`. +3. Применяется `@CircuitBreaker` и работает согласно своей конфигурации и [состоянию](#circuitbreaker), в зависимости от успешного результата метода или исключения в цепочке, включая исключения из `@Timeout` и `@Retry`. +4. Применяется `@Fallback` и вызывает метод `getFallback` с аргументом `arg1`, если метод выбрасывает исключение в цепочке, включая исключения из `@Timeout`, `@Retry` и `@CircuitBreaker`. -Порядок вызова аспектов соответствует порядку аннотаций над методом, сверху внизу. +Порядок вызова аспектов следует порядку аннотаций на методе: сверху вниз. -Пример конфигурации всех аспектов: +Пример конфигурации для всех аспектов: ===! ":material-code-json: `Hocon`" @@ -875,28 +1108,84 @@ agent: attempts: 2 ``` +## Исключения { #exceptions } + +Все исключения отказоустойчивости наследуются от `ru.tinkoff.kora.resilient.ResilientException` (это `RuntimeException`), который предоставляет `name()` — имя конфигурации аспекта, вызвавшего его. + +| Исключение | Выбрасывается | Дополнительный API | +|------------------------------|---------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------| +| `ResilientException` | базовый тип для всех перечисленных ниже | `name()` | +| `CallNotPermittedException` | `@CircuitBreaker` / `CircuitBreaker#acquire()`, когда прерыватель в `OPEN` либо в `HALF_OPEN` без оставшихся пробных вызовов | `state()` возвращает `CircuitBreaker.State` (`OPEN` / `HALF_OPEN`) | +| `RetryExhaustedException` | `@Retry` / `Retry#retry(...)`, когда каждая попытка завершилась неудачей | `name()`; в сообщении указано число попыток, последний сбой доступен через `getCause()` | +| `TimeoutExhaustedException` | `@Timeout` / `Timeout#execute(...)`, когда метод превышает `duration` | `name()` | + +**Описание** — аспекты отказоустойчивости сигнализируют о сбое, выбрасывая одно из этих непроверяемых исключений из защищаемого метода. + +**Причины** + +- `CallNotPermittedException` — прерыватель размыкает вызовы, потому что доля сбоев достигла `failureRateThreshold`; вызов был отклонен без обращения к методу. +- `RetryExhaustedException` — метод продолжал выбрасывать повторяемое исключение, пока не было достигнуто `attempts`; исходный сбой доступен через `getCause()`. +- `TimeoutExhaustedException` — метод не завершился в пределах `duration`. + +**Рекомендации** + +- Перехватывайте `ResilientException`, чтобы единообразно обрабатывать любой сбой отказоустойчивости, либо перехватывайте конкретный тип, когда обработка различается. +- Когда аспекты [комбинируются](#combination), исключение нижележащего аспекта распространяется вверх по цепочке: например, `TimeoutExhaustedException` из `@Timeout` наблюдается `@Retry`, затем `@CircuitBreaker` и, наконец, `@Fallback`. Предпочитайте метод `@Fallback` или императивный резервный вариант превращению этих исключений в ошибки, видимые пользователю. + +Пример обработки: + +===! ":fontawesome-brands-java: `Java`" + + ```java + try { + return service.getValue(); + } catch (CallNotPermittedException e) { + log.warn("CircuitBreaker '{}' is {}", e.name(), e.state()); + return cachedValue(); + } catch (TimeoutExhaustedException | RetryExhaustedException e) { + log.warn("Resilient '{}' failed", e.name(), e); + return cachedValue(); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + try { + return service.value() + } catch (e: CallNotPermittedException) { + log.warn("CircuitBreaker '{}' is {}", e.name(), e.state()) + return cachedValue() + } catch (e: ResilientException) { // TimeoutExhaustedException, RetryExhaustedException, ... + log.warn("Resilient '{}' failed", e.name(), e) + return cachedValue() + } + ``` + ## Сигнатуры { #signatures } -Доступные сигнатуры для методов которые поддерживают аннотации из коробки: +Доступные сигнатуры методов, поддерживаемые этими аннотациями «из коробки»: +Все четыре аннотации поддерживают обычные синхронные методы, асинхронные типы и реактивные типы, но фактический набор зависит от языка и обработчика. ===! ":fontawesome-brands-java: `Java`" - Класс не должен быть `final`, чтобы аспекты работали. + Класс должен быть не `final`, чтобы аспекты работали. - Под `T` подразумевается тип возвращаемого значения, либо `Void`. + `T` обозначает тип возвращаемого значения. + - `void myMethod()` - `T myMethod()` - `Optional myMethod()` - - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `CompletionStage myMethod()` / `CompletableFuture myMethod()` ([CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html)) + - `Mono myMethod()` ([Project Reactor](https://projectreactor.io/docs/core/release/reference/), требуется [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` ([Project Reactor](https://projectreactor.io/docs/core/release/reference/), требуется [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" Класс должен быть `open`, чтобы аспекты работали. - Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `Unit`. + Под `T` понимается тип возвращаемого значения, либо `T?`, либо `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `suspend myMethod(): T` ([Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine), требуется [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `myMethod(): Flow` ([Kotlin Coroutines](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine), требуется [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) diff --git a/mkdocs/docs/ru/documentation/s3-client.md b/mkdocs/docs/ru/documentation/s3-client.md index 044c4a0..c678398 100644 --- a/mkdocs/docs/ru/documentation/s3-client.md +++ b/mkdocs/docs/ru/documentation/s3-client.md @@ -6,24 +6,26 @@ agent: ??? warning "Экспериментальный модуль" - **Эксперементальный** модуль является полностью рабочим и протестированным, но требует дополнительной апробации и аналитики по использованию, - по этой причине API может потенциально притерпеть незначительные изменения перед полной готовностью. + **Экспериментальный** модуль полностью работает и протестирован, но требует дополнительной апробации и аналитики использования, + поэтому API потенциально может претерпеть незначительные изменения до того, как станет полностью стабильным. -Модуль предоставляет тонкий слой абстракции для создания S3-клиентов -с помощью аннотаций в декларативном стиле, либо использование клиентов в императивном стиле для работы с [хранилищем S3](https://aws.amazon.com/ru/s3/faqs/). +Модуль предоставляет слой абстракции для работы с [S3-совместимым объектным хранилищем](https://aws.amazon.com/s3/faqs/): +можно создавать декларативные `S3`-клиенты с помощью аннотаций либо внедрять готовые к использованию императивные клиенты. +Декларативный клиент удобен для типовых операций с объектами и ключами, тогда как императивный клиент полезен, когда операциями +нужно управлять напрямую в коде. Если нужен пошаговый разбор перед справочным описанием, смотрите [S3](../guides/s3.md). ## AWS { #aws } -Реализация S3-клиента основанная на библиотеке [AWS](https://github.com/aws/aws-sdk-java-v2). +Реализация `S3`-клиента основана на [библиотеке AWS](https://github.com/aws/aws-sdk-java-v2). -Для работы и внедрения доступны компоненты: +Компоненты, доступные для внедрения: -- Императивные [Kora S3 клиенты](#client-imperative) -- `S3Client` синхронный AWS S3 клиент -- `S3AsyncClient` асинхронный AWS S3 клиент -- `S3AsyncClient` с тегом `@Tag(MultipartUpload.class)` асинхронный AWS S3 клиент для пакетной загрузки +- Императивные [Kora S3-клиенты](#client-imperative) +- `S3Client` синхронный AWS S3-клиент +- `S3AsyncClient` асинхронный AWS S3-клиент +- `S3AsyncClient` с тегом `@Tag(MultipartUpload.class)` асинхронный AWS S3-клиент для пакетной загрузки ### Подключение { #dependency } @@ -53,145 +55,186 @@ agent: interface Application : AwsS3ClientModule ``` -Требуется подключить любой модуль [HTTP-клиента](http-client.md). +Требуется добавить любой модуль [HTTP-клиента](http-client.md). ### Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `AwsS3ClientConfig` и `S3Config` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации S3 клиента: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript s3client { - aws { - addressStyle = "PATH" //(1)! - requestTimeout = "45s" //(2)! - checksumValidationEnabled = false //(3)! - chunkedEncodingEnabled = true //(4)! - upload { - bufferSize = "32MiB" //(5)! - partSize = "8MiB" //(6)! - } - } - - url = "http://localhost:9000" //(7)! - accessKey = "someKey" //(8)! - secretKey = "someSecret" //(9)! - region = "aws-global" //(10)! - telemetry { - logging { - enabled = false //(11)! - } - metrics { - enabled = true //(12)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(13)! - tags = { // (14)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(15)! - attributes = { // (16)! - "key1" = "value1" - "key2" = "value2" - } - } - } + url = "http://localhost:9000" //(1)! + accessKey = "someKey" //(2)! + secretKey = "someSecret" //(3)! + region = "aws-global" //(4)! } ``` - 1. Какой тип доступа к [файлам использовать](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), может иметь значения `PATH` или `VIRTUAL_HOSTED` - 2. Максимальное время выполнения операции - 3. Проверять ли контрольную суммы [MD5 файлов перед загрузкой и при получении](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) из AWS - 4. Кодировать ли в виде кусков при [подписании данных файла](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#chunkedEncodingEnabled(java.lang.Boolean)) при загрузке в AWS - 5. Максимальный размер буфера при загрузке файлов (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 6. Максимальный размер кусочка файла при единовременной загрузке файла (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 7. URL хранилища S3 - 8. Ключ доступа к S3 - 9. Секрет доступа к S3 - 10. Регион хранилища S3 - 11. Включает логгирование модуля (по умолчанию `false`) - 12. Включает метрики модуля (по умолчанию `true`) - 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 14. Настройка тегов для метрик (опционально) - 15. Включает трассировку модуля (по умолчанию `true`) - 16. Настройка атрибутов для трассировки (опционально) + 1. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 2. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 3. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 4. Регион хранилища `S3` (по умолчанию: `aws-global`) === ":simple-yaml: `YAML`" ```yaml s3client: - aws: - addressStyle: "PATH" #(1)! - requestTimeout: "45s" #(2)! - checksumValidationEnabled: false #(3)! - chunkedEncodingEnabled: true #(4)! - upload: - bufferSize: "32MiB" #(5)! - partSize: "8MiB" #(6)! - - url: "http://localhost:9000" #(7)! - accessKey: "someKey" #(8)! - secretKey: "someSecret" #(9)! - region: "aws-global" #(10)! - telemetry: - logging: - enabled: false #(11)! - metrics: - enabled: true #(12)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! - tags: #(14)! - key1: value1 - key2: value2 - tracing: - enabled: true #(15)! - attributes: #(16)! - key1: value1 - key2: value2 - ``` - - 1. Какой тип доступа к [файлам использовать](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), может иметь значения `PATH` или `VIRTUAL_HOSTED` - 2. Максимальное время выполнения операции - 3. Проверять ли контрольную суммы [MD5 файлов перед загрузкой и при получении](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) из AWS - 4. Кодировать ли в виде кусков при [подписании данных файла](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#chunkedEncodingEnabled(java.lang.Boolean)) при загрузке в AWS - 5. Максимальный размер буфера при загрузке файлов (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 6. Максимальный размер кусочка файла при единовременной загрузке файла (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 7. URL хранилища S3 - 8. Ключ доступа к S3 - 9. Секрет доступа к S3 - 10. Регион хранилища S3 - 11. Включает логгирование модуля (по умолчанию `false`) - 12. Включает метрики модуля (по умолчанию `true`) - 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 14. Настройка тегов для метрик (опционально) - 15. Включает трассировку модуля (по умолчанию `true`) - 16. Настройка атрибутов для трассировки (опционально) - -Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#s3-client). + url: "http://localhost:9000" #(1)! + accessKey: "someKey" #(2)! + secretKey: "someSecret" #(3)! + region: "aws-global" #(4)! + ``` + + 1. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 2. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 3. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 4. Регион хранилища `S3` (по умолчанию: `aws-global`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классах `AwsS3ClientConfig` и `S3Config` (указаны примеры значений или значения по умолчанию): + + ===! ":material-code-json: `HOCON`" + + ```javascript + s3client { + aws { + addressStyle = "PATH" //(1)! + requestTimeout = "45s" //(2)! + checksumValidationEnabled = false //(3)! + chunkedEncodingEnabled = true //(4)! + upload { + bufferSize = "32MiB" //(5)! + partSize = "8MiB" //(6)! + } + } + + url = "http://localhost:9000" //(7)! + accessKey = "someKey" //(8)! + secretKey = "someSecret" //(9)! + region = "aws-global" //(10)! + telemetry { + logging { + enabled = false //(11)! + } + metrics { + enabled = true //(12)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(13)! + tags = { // (14)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(15)! + attributes = { // (16)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Стиль доступа к объектам, может иметь значения `PATH` или `VIRTUAL_HOSTED` (по умолчанию: `PATH`) + 2. Максимальное время выполнения операции (по умолчанию: `45s`) + 3. Проверять ли [контрольную сумму MD5 перед загрузкой и при получении](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) из `AWS` (по умолчанию: `false`) + 4. Использовать ли частичное (chunked) кодирование при подписании данных файла во время загрузки в `AWS` (по умолчанию: `true`) + 5. Максимальный размер буфера для загрузки файлов (по умолчанию: `32MiB`) + 6. Максимальный размер части файла при загрузке одного файла (по умолчанию: `8MiB`) + 7. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 8. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 9. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 10. Регион хранилища `S3` (по умолчанию: `aws-global`) + 11. Включает логирование модуля (по умолчанию: `false`) + 12. Включает метрики модуля (по умолчанию: `true`) + 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Настройка тегов метрик (по умолчанию: `{}`) + 15. Включает трассировку модуля (по умолчанию: `true`) + 16. Настройка атрибутов трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + s3client: + aws: + addressStyle: "PATH" #(1)! + requestTimeout: "45s" #(2)! + checksumValidationEnabled: false #(3)! + chunkedEncodingEnabled: true #(4)! + upload: + bufferSize: "32MiB" #(5)! + partSize: "8MiB" #(6)! + + url: "http://localhost:9000" #(7)! + accessKey: "someKey" #(8)! + secretKey: "someSecret" #(9)! + region: "aws-global" #(10)! + telemetry: + logging: + enabled: false #(11)! + metrics: + enabled: true #(12)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(13)! + tags: #(14)! + key1: value1 + key2: value2 + tracing: + enabled: true #(15)! + attributes: #(16)! + key1: value1 + key2: value2 + ``` + + 1. Стиль доступа к объектам, может иметь значения `PATH` или `VIRTUAL_HOSTED` (по умолчанию: `PATH`) + 2. Максимальное время выполнения операции (по умолчанию: `45s`) + 3. Проверять ли [контрольную сумму MD5 перед загрузкой и при получении](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#checksumValidationEnabled(java.lang.Boolean)) из `AWS` (по умолчанию: `false`) + 4. Использовать ли частичное (chunked) кодирование при подписании данных файла во время загрузки в `AWS` (по умолчанию: `true`) + 5. Максимальный размер буфера для загрузки файлов (по умолчанию: `32MiB`) + 6. Максимальный размер части файла при загрузке одного файла (по умолчанию: `8MiB`) + 7. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 8. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 9. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 10. Регион хранилища `S3` (по умолчанию: `aws-global`) + 11. Включает логирование модуля (по умолчанию: `false`) + 12. Включает метрики модуля (по умолчанию: `true`) + 13. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 14. Настройка тегов метрик (по умолчанию: `{}`) + 15. Включает трассировку модуля (по умолчанию: `true`) + 16. Настройка атрибутов трассировки (по умолчанию: `{}`) + +Метрики модуля описаны в разделе [Справочник по метрикам](metrics.md#s3-client). ### Формат ответа { #response-format } -В случае использование AWS модуля, есть возможность получать специальные форматы ответа специфичные только AWS библиотеке: +При использовании модуля `AWS` можно возвращать специальные форматы ответа, специфичные для библиотеки `AWS`: -| Операция | Формат ответа | -|-----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Операция | Формат ответа | +|-----------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Получение файла](#get-file) | [GetObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/GetObjectResponse.html) / [ResponseInputStream](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/ResponseInputStream.html) | | [Получение метаданных файла](#metadata) | [HeadObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/HeadObjectResponse.html) | -| [Перечисление файлов](#list-files) | [ListObjectsV2Response](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/ListObjectsV2Response.html) | -| [Добавление файла](#add-file) | [PutObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/PutObjectResponse.html) | -| [Удаление файла](#delete-file) | [DeleteObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/DeleteObjectResponse.html) / [DeleteObjectsResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/DeleteObjectsResponse.html) | +| [Получение списка файлов](#list-files) | [ListObjectsV2Response](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/ListObjectsV2Response.html) | +| [Добавление файла](#add-file) | [PutObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/PutObjectResponse.html) | +| [Удаление файла](#delete-file) | [DeleteObjectResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/DeleteObjectResponse.html) / [DeleteObjectsResponse](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/DeleteObjectsResponse.html) | + +Для операций `@S3.Get`, получающих объект или метаданные, отсутствие объекта можно описать в типе ответа. +В `Java` поддерживаются `Optional`, `Optional`, `Optional`, +`Optional>` и `Optional`. +В `Kotlin` для этого используются nullable-типы ответа: `S3Object?`, `S3ObjectMeta?`, `GetObjectResponse?`, +`ResponseInputStream?` и `HeadObjectResponse?`. ## Minio { #minio } -Реализация S3-клиента основанная на библиотеке [Minio](https://github.com/minio/minio-java). -Учитывайте что реализация использует [OkHttp](https://github.com/square/okhttp) написанный на Kotlin и использует соответствующие зависимости. +Реализация `S3`-клиента основана на библиотеке [Minio](https://github.com/minio/minio-java). +Учитывайте, что реализация использует [OkHttp](https://github.com/square/okhttp), написанную на `Kotlin`, и её зависимости. -Для работы и внедрения доступны компоненты: +Компоненты, доступные для внедрения: -- Императивные [Kora S3 клиенты](#client-imperative) -- `MinioClient` синхронный Minio S3 клиент -- `MinioAsyncClient` асинхронный Minio S3 клиент +- Императивные [Kora S3-клиенты](#client-imperative) +- `MinioClient` синхронный Minio S3-клиент +- `MinioAsyncClient` асинхронный Minio S3-клиент ### Подключение { #dependency-2 } @@ -221,100 +264,153 @@ agent: interface Application : MinioS3ClientModule ``` -Можно подключить [OkHttp модуль](http-client.md#okhttp) либо будет создан стандартный HTTP-клиент. +Можно добавить зависимость [модуля OkHttp](http-client.md#okhttp), иначе будет автоматически создан стандартный HTTP-клиент. ### Конфигурация { #configuration-2 } -Пример полной конфигурации, описанной в классе `MinioS3ClientConfig` и `S3Config` (указаны примеры значений или значения по умолчанию): +Основные параметры конфигурации Minio S3 клиента: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript s3client { - minio { - addressStyle = "PATH" //(1)! - requestTimeout = "45s" //(2)! - upload { - partSize = "8MiB" //(3)! - } - } - - url = "http://localhost:9000" //(4)! - accessKey = "someKey" //(5)! - secretKey = "someSecret" //(6)! - region = "aws-global" //(7)! - telemetry { - logging { - enabled = false //(8)! - } - metrics { - enabled = true //(9)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(10)! - } - tracing { - enabled = true //(11)! - } - } + url = "http://localhost:9000" //(1)! + accessKey = "someKey" //(2)! + secretKey = "someSecret" //(3)! + region = "aws-global" //(4)! } ``` - 1. Какой тип доступа к [файлам использовать](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), может иметь значения `PATH` или `VIRTUAL_HOSTED` - 2. Максимальное время выполнения операции - 3. Максимальный размер кусочка файла при единовременной загрузке файла (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 4. URL хранилища S3 - 5. Ключ доступа к S3 - 6. Секрет доступа к S3 - 7. Регион хранилища S3 - 8. Включает логгирование модуля (по умолчанию `false`) - 9. Включает метрики модуля (по умолчанию `true`) - 10. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 11. Включает трассировку модуля (по умолчанию `true`) + 1. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 2. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 3. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 4. Регион хранилища `S3` (по умолчанию: `aws-global`) === ":simple-yaml: `YAML`" ```yaml s3client: - minio: - addressStyle: "PATH" #(1)! - requestTimeout: "45s" #(1)! - upload: - partSize: "8MiB" #(2)! - - url: "http://localhost:9000" #(3)! - accessKey: "someKey" #(4)! - secretKey: "someSecret" #(5)! - region: "aws-global" #(6)! - telemetry: - logging: - enabled: false #(7)! - metrics: - enabled: true #(8)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(9)! - telemetry: - enabled: true #(10)! - ``` - - 1. Какой тип доступа к [файлам использовать](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Configuration.Builder.html#pathStyleAccessEnabled(java.lang.Boolean)), может иметь значения `PATH` или `VIRTUAL_HOSTED` - 2. Максимальное время выполнения операции - 3. Максимальный размер кусочка файла при единовременной загрузке файла (указывается как число в байтах / либо как `4MiB` / `4MB` / `1000Kb` и тп) - 4. URL хранилища S3 - 5. Ключ доступа к S3 - 6. Секрет доступа к S3 - 7. Регион хранилища S3 - 8. Включает логгирование модуля (по умолчанию `false`) - 9. Включает метрики модуля (по умолчанию `true`) - 10. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 11. Включает трассировку модуля (по умолчанию `true`) - -## Клиент декларативный { #client-declarative } - -Предлагается использовать специальные аннотации для создания декларативного клиента: - -* `@S3.Client` — указывает что интерфейс является декларативным S3-клиентом -* `@S3.Get` — указывает что метод выполняет операцию [получения файла/метаданных](#dependency) -* `@S3.List` — указывает что метод выполняет операцию [получения списка файлов/метаданных](#dependency) -* `@S3.Put` — указывает что метод выполняет операцию [добавления файла](#dependency) -* `@S3.Delete` — указывает что метод выполняет операцию [удаления файла](#dependency) + url: "http://localhost:9000" #(1)! + accessKey: "someKey" #(2)! + secretKey: "someSecret" #(3)! + region: "aws-global" #(4)! + ``` + + 1. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 2. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 3. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 4. Регион хранилища `S3` (по умолчанию: `aws-global`) + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной в классах `MinioS3ClientConfig` и `S3Config` (указаны примеры значений или значения по умолчанию): + + ===! ":material-code-json: `HOCON`" + + ```javascript + s3client { + minio { + addressStyle = "PATH" //(1)! + requestTimeout = "45s" //(2)! + upload { + partSize = "8MiB" //(3)! + } + } + + url = "http://localhost:9000" //(4)! + accessKey = "someKey" //(5)! + secretKey = "someSecret" //(6)! + region = "aws-global" //(7)! + telemetry { + logging { + enabled = false //(8)! + } + metrics { + enabled = true //(9)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(10)! + tags = { // (11)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(12)! + attributes = { // (13)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + ``` + + 1. Стиль доступа к объектам, может иметь значения `PATH` или `VIRTUAL_HOSTED` (по умолчанию: `PATH`) + 2. Максимальное время выполнения операции (по умолчанию: `45s`) + 3. Максимальный размер части файла при загрузке одного файла (по умолчанию: `8MiB`) + 4. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 5. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 6. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 7. Регион хранилища `S3` (по умолчанию: `aws-global`) + 8. Включает логирование модуля (по умолчанию: `false`) + 9. Включает метрики модуля (по умолчанию: `true`) + 10. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 11. Настройка тегов метрик (по умолчанию: `{}`) + 12. Включает трассировку модуля (по умолчанию: `true`) + 13. Настройка атрибутов трассировки (по умолчанию: `{}`) + + === ":simple-yaml: `YAML`" + + ```yaml + s3client: + minio: + addressStyle: "PATH" #(1)! + requestTimeout: "45s" #(2)! + upload: + partSize: "8MiB" #(3)! + + url: "http://localhost:9000" #(4)! + accessKey: "someKey" #(5)! + secretKey: "someSecret" #(6)! + region: "aws-global" #(7)! + telemetry: + logging: + enabled: false #(8)! + metrics: + enabled: true #(9)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(10)! + tags: #(11)! + key1: value1 + key2: value2 + tracing: + enabled: true #(12)! + attributes: #(13)! + key1: value1 + key2: value2 + ``` + + 1. Стиль доступа к объектам, может иметь значения `PATH` или `VIRTUAL_HOSTED` (по умолчанию: `PATH`) + 2. Максимальное время выполнения операции (по умолчанию: `45s`) + 3. Максимальный размер части файла при загрузке одного файла (по умолчанию: `8MiB`) + 4. `URL` хранилища `S3` (`обязательный`, по умолчанию не указано) + 5. Ключ доступа к `S3` (`обязательный`, по умолчанию не указано) + 6. Секрет доступа к `S3` (`обязательный`, по умолчанию не указано) + 7. Регион хранилища `S3` (по умолчанию: `aws-global`) + 8. Включает логирование модуля (по умолчанию: `false`) + 9. Включает метрики модуля (по умолчанию: `true`) + 10. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 11. Настройка тегов метрик (по умолчанию: `{}`) + 12. Включает трассировку модуля (по умолчанию: `true`) + 13. Настройка атрибутов трассировки (по умолчанию: `{}`) + +## Декларативный клиент { #client-declarative } + +Для создания декларативного клиента предлагается использовать специальные аннотации: + +* `@S3.Client` - указывает, что интерфейс является декларативным S3-клиентом +* `@S3.Get` - указывает, что метод выполняет [операцию получения файла/метаданных](#get-file) +* `@S3.List` - указывает, что метод выполняет [операцию получения списка файлов/метаданных](#list-files) +* `@S3.Put` - указывает, что метод выполняет [операцию добавления файла](#add-file) +* `@S3.Delete` - указывает, что метод выполняет [операцию удаления файла](#delete-file) ===! ":fontawesome-brands-java: `Java`" @@ -322,8 +418,8 @@ agent: @S3.Client public interface SomeClient { - @S3.Get - S3Object operation(String key); + @S3.Get + S3Object operation(String key); } ``` @@ -333,7 +429,7 @@ agent: @S3.Client interface SomeClient { - @S3.Get + @S3.Get fun operation(key: String): S3Object } ``` @@ -348,12 +444,12 @@ agent: @S3.Client("s3client.someClient") //(1)! public interface SomeClient { - @S3.Get - S3Object operation(String key); + @S3.Get + S3Object operation(String key); } ``` - 1. Путь до конфигурации конкретно этого клиента + 1. Путь к конфигурации данного конкретного клиента === ":simple-kotlin: `Kotlin`" @@ -361,16 +457,21 @@ agent: @S3.Client("s3client.someClient") //(1)! interface SomeClient { - @S3.Get + @S3.Get fun operation(key: String): S3Object } ``` - 1. Путь до конфигурации конкретно этого клиента + 1. Путь к конфигурации данного конкретного клиента + +`@S3.Client` без аргументов эквивалентна `@S3.Client("")`: значение `value` аннотации пустое, +и `S3ClientConfig` будет считана из пустого пути через `Config.get("")`. +На практике обычно лучше указывать явный путь, например `@S3.Client("s3client.someClient")`, +чтобы конфигурация `bucket` была отделена от других клиентов. -Пример конфигурации в случае пути `s3client.someClient` описанной в классе `S3ClientConfig`: +Конфигурация для случая пути `s3client.someClient`, описанная в классе `S3ClientConfig`: -===! ":material-code-json: `Hocon`" +===! ":material-code-json: `HOCON`" ```javascript s3client { @@ -380,7 +481,7 @@ agent: } ``` - 1. Корзина ([bucket](https://aws.amazon.com/ru/s3/faqs/)) где будут хранится файлы + 1. Бакет ([bucket](https://aws.amazon.com/s3/faqs/)), в котором будут храниться файлы (`обязательный`, по умолчанию не указано) === ":simple-yaml: `YAML`" @@ -390,12 +491,12 @@ agent: bucket: "someBucket" #(1)! ``` - 1. Корзина ([bucket](https://aws.amazon.com/ru/s3/faqs/)) где будут хранится файлы + 1. Бакет ([bucket](https://aws.amazon.com/s3/faqs/)), в котором будут храниться файлы (`обязательный`, по умолчанию не указано) -### Получение файла { #get-file } +#### Получение файла { #get-file } -Секция описывает операцию получения файла/метаданных с помощью декларативного S3-клиента. -Предлагается использовать аннотацию `@S3.Get` для указания операции. +В разделе описана операция получения файла/метаданных с помощью декларативного S3-клиента. +Для указания операции предлагается использовать аннотацию `@S3.Get`. ===! ":fontawesome-brands-java: `Java`" @@ -411,9 +512,9 @@ agent: } ``` - 1. Операция получения файла - 2. Получение в ответ файла вместе данными - 3. Ключ файла можно указать в аннотации + 1. операция получения файла + 2. файл вместе с данными в ответе + 3. ключ файла можно указать в аннотации === ":simple-kotlin: `Kotlin`" @@ -429,15 +530,15 @@ agent: } ``` - 1. Операция получения файла - 2. Получение в ответ файла вместе данными - 3. Ключ файла можно указать в аннотации + 1. операция получения файла + 2. файл вместе с данными в ответе + 3. ключ файла можно указать в аннотации #### Метаданные { #metadata } -Операция получения файла по ключу может возвращать как полный файл вместе с данными `S3Object`, -так и облегченный вариант в виде метаданных файла без данных `S3ObjectMeta`, -такой метод значительно выполняется быстрее тк не возвращает данные файла. +Операция получения файла по ключу может возвращать либо полный файл `S3Object` вместе с данными, +либо облегчённую версию в виде метаданных файла `S3ObjectMeta` без данных; +этот способ значительно быстрее, поскольку не возвращает данные файла. ===! ":fontawesome-brands-java: `Java`" @@ -450,7 +551,7 @@ agent: } ``` - 1. Получение в ответ метаданные файла + 1. Получение метаданных файла в ответе === ":simple-kotlin: `Kotlin`" @@ -463,11 +564,11 @@ agent: } ``` - 1. Получение в ответ метаданные файла + 1. Получение метаданных файла в ответе #### Шаблон ключа { #key-template } -Ключ можно указывать также как шаблон и подставлять туда аргументы метода как части шаблона, +Ключ также можно задать в виде шаблона и подставлять в него аргументы метода как часть шаблона; все аргументы метода должны быть частью составного ключа. ===! ":fontawesome-brands-java: `Java`" @@ -481,7 +582,7 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон ключа, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` + 1. Шаблон, используемый для построения ключа: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` 2. Все аргументы метода должны быть частью шаблона ключа === ":simple-kotlin: `Kotlin`" @@ -495,13 +596,13 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон ключа, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` + 1. Шаблон, используемый для построения ключа: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` 2. Все аргументы метода должны быть частью шаблона ключа -#### Множество ключей { #multiple-keys } +#### Несколько ключей { #multiple-keys } -Можно также получать сразу множество файлов по ключам как полный файл вместе с данными `S3Object`, -так и облегченный вариант в виде множества метаданных файлов без данных `S3ObjectMeta`. +Также можно получать несколько файлов по ключам — либо как полные объекты с данными (`S3Object`), +либо как облегчённые метаданные без данных объекта (`S3ObjectMeta`). ===! ":fontawesome-brands-java: `Java`" @@ -514,8 +615,8 @@ agent: } ``` - 1. Операция получения файла для множества ключей **не должна** содержать шаблон ключа - 2. Операция должна принимать список ключей и отдавать список `S3Object` либо `S3ObjectMeta` + 1. Операция получения по нескольким ключам **не должна** содержать шаблон ключа + 2. Операция должна принимать список ключей и возвращать список `S3Object` или `S3ObjectMeta` === ":simple-kotlin: `Kotlin`" @@ -528,16 +629,53 @@ agent: } ``` - 1. Операция получения файла для множества ключей **не должна** содержать шаблон ключа - 2. Операция должна принимать список ключей и отдавать список `S3Object` либо `S3ObjectMeta` + 1. Операция получения по нескольким ключам **не должна** содержать шаблон ключа + 2. Операция должна принимать список ключей и возвращать список `S3Object` или `S3ObjectMeta` -### Перечисление файлов { #list-files } +#### Необязательный ответ { #optional-get } -Секция описывает операцию получения списка файлов/метаданных с помощью декларативного S3-клиента. -Предлагается использовать аннотацию `@S3.List` для указания операции. +Если отсутствие файла не должно приводить к `S3NotFoundException`, результат `@S3.Get` можно сделать необязательным. +Для стандартных типов `Kora` в `Java` используются `Optional` и `Optional`; +модуль `AWS` также поддерживает `Optional`, +`Optional>` и `Optional`. +В `Kotlin` для тех же случаев используются nullable-типы ответа. -Можно указывать [префикс ключа](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html), чтобы делать выборку по нужным ключам подходящим под префикс, -также можно указывать ограничение по выборке файлов, максимальное количество файлов для операции `1000`: +===! ":fontawesome-brands-java: `Java`" + + ```java + @S3.Client("s3client.someClient") + public interface SomeClient { + + @S3.Get + Optional object(String key); + + @S3.Get + Optional meta(String key); + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @S3.Client("s3client.someClient") + interface SomeClient { + + @S3.Get + fun object(key: String): S3Object? + + @S3.Get + fun meta(key: String): S3ObjectMeta? + } + ``` + +### Получение списка файлов { #list-files } + +В разделе описана операция получения списка файлов/метаданных с помощью декларативного S3-клиента. +Для указания операции предлагается использовать аннотацию `@S3.List`. + +Можно указать [префикс ключа](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html), чтобы выбрать ключи, соответствующие этому префиксу, +а также задать ограничение на выборку файлов с помощью параметра `limit` аннотации `@S3.List`. +Значение `limit` должно находиться в диапазоне `1..1000`, значение по умолчанию — `1000`. ===! ":fontawesome-brands-java: `Java`" @@ -556,9 +694,9 @@ agent: } ``` - 1. Префикс можно передать как аргумент метода, если он не указан в аннотации - 2. Префикс можно указать в аннотации - 3. Можно указывать ограничение по выборке файлов для операции перечисления, максимальное количество файлов для операции `1000`: + 1. префикс можно передать как аргумент метода, если он не указан в аннотации + 2. префикс можно указать в аннотации + 3. Можно указать ограничение выборки файлов для операции получения списка через `limit`; допустимый диапазон — `1..1000`, значение по умолчанию — `1000` === ":simple-kotlin: `Kotlin`" @@ -577,15 +715,15 @@ agent: } ``` - 1. Префикс можно передать как аргумент метода, если он не указан в аннотации - 2. Префикс можно указать в аннотации - 3. Можно указывать ограничение по выборке файлов для операции перечисления, максимальное количество файлов для операции `1000`: + 1. префикс можно передать как аргумент метода, если он не указан в аннотации + 2. префикс можно указать в аннотации + 3. Можно указать ограничение выборки файлов для операции получения списка через `limit`; допустимый диапазон — `1..1000`, значение по умолчанию — `1000` #### Метаданные { #metadata-2 } -Операция получения файла по ключу может возвращать как полный файл вместе с данными `S3ObjectList`, -так и облегченный вариант в виде метаданных файла без данных `S3ObjectMetaList`, -такой метод значительно выполняется быстрее тк не возвращает данные файла. +Операция получения списка может возвращать либо полный список файлов `S3ObjectList` вместе с данными, +либо облегчённую версию в виде метаданных файлов `S3ObjectMetaList` без данных; +этот способ значительно быстрее, поскольку не возвращает данные файлов. ===! ":fontawesome-brands-java: `Java`" @@ -598,7 +736,7 @@ agent: } ``` - 1. Получение в ответ метаданные файлов + 1. Получение метаданных файлов в ответе === ":simple-kotlin: `Kotlin`" @@ -611,11 +749,11 @@ agent: } ``` - 1. Получение в ответ метаданные файлов + 1. Получение метаданных файлов в ответе #### Шаблон префикса { #prefix-template } -Префикс можно указывать также как шаблон и подставлять туда аргументы метода как части шаблона, +Префикс также можно задать в виде шаблона и подставлять в него аргументы метода как часть шаблона; все аргументы метода должны быть частью составного ключа. ===! ":fontawesome-brands-java: `Java`" @@ -629,7 +767,7 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон префикса, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` + 1. Шаблон, используемый для построения префикса: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` === ":simple-kotlin: `Kotlin`" @@ -642,11 +780,11 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон префикса, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` + 1. Шаблон, используемый для построения префикса: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` #### Разделитель { #separator } -Можно указывать разделитель для [префикса ключа](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html), исключать нужные файлы из выборки: +Можно указать разделитель для [префикса ключа](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html), чтобы отфильтровать результат получения списка: ===! ":fontawesome-brands-java: `Java`" @@ -659,7 +797,7 @@ agent: } ``` - 1. Указывается разделитель по которому будет фильтроваться перечисления файлов + 1. Разделитель, используемый для фильтрации списка файлов === ":simple-kotlin: `Kotlin`" @@ -672,14 +810,14 @@ agent: } ``` - 1. Указывается разделитель по которому будет фильтроваться перечисления файлов + 1. Разделитель, используемый для фильтрации списка файлов ### Добавление файла { #add-file } -Секция описывает операцию добавления файла с помощью декларативного S3-клиента. -Предлагается использовать аннотацию `@S3.Put` для операции. +В разделе описана операция добавления файла с помощью декларативного S3-клиента. +Для операции предлагается использовать аннотацию `@S3.Put`. -Требуется указать ключ и тело файла для добавления: +Требуется указать ключ и тело добавляемого файла: ===! ":fontawesome-brands-java: `Java`" @@ -696,9 +834,9 @@ agent: } ``` - 1. Ключ файла по которому он будет добавлен в хранилище - 2. Само тело файла которое будет добавлено в хранилище - 3. Ключ также можно указать в аннотации если он статичен + 1. Ключ файла, по которому он будет добавлен в хранилище + 2. само тело файла, которое будет добавлено в хранилище + 3. ключ также можно указать в аннотации, если он статический === ":simple-kotlin: `Kotlin`" @@ -714,21 +852,162 @@ agent: } ``` - 1. Ключ файла по которому он будет добавлен в хранилище - 2. Само тело файла которое будет добавлено в хранилище - 3. Ключ также можно указать в аннотации если он статичен + 1. Ключ файла, по которому он будет добавлен в хранилище + 2. само тело файла, которое будет добавлено в хранилище + 3. ключ также можно указать в аннотации, если он статический #### Тело файла { #file-body } -Тело файла (`S3Body`) может быть создано из `byte[]` / `ByteBuffer` / `InputStream` / `Flow.Publisher` через соответсвующее статические методы конструкторы. +Тело файла (`S3Body`) можно создать из `byte[]`, `ByteBuffer`, `InputStream` или `Flow.Publisher` +с помощью соответствующих статических фабричных методов. Каждый фабричный метод имеет перегрузки, дополнительно принимающие +значения `type` (`Content-Type`) и `encoding` (`Content-Encoding`): + +| Фабричный метод | Источник | Размер | Описание | +|-----------------------------------------------|------------------------------|------------|--------------------------------------------------------------------------------------------------------| +| `S3Body.ofBytes(byte[])` | `byte[]` | Известен | Тело из массива байтов в памяти | +| `S3Body.ofBuffer(ByteBuffer)` | `ByteBuffer` | Известен | Тело из буфера в памяти (в качестве размера используется `remaining()`) | +| `S3Body.ofInputStream(InputStream, long)` | `InputStream` | Известен | Потоковое тело, точная длина которого передаётся явно через аргумент `size` | +| `S3Body.ofInputStreamReadAll(InputStream)` | `InputStream` | Известен | Считывает весь поток в память **немедленно**, затем ведёт себя как массив байтов | +| `S3Body.ofInputStreamUnbound(InputStream)` | `InputStream` | Неизвестен | Потоковое тело неизвестной длины (`size()` возвращает `-1`) | +| `S3Body.ofPublisher(Flow.Publisher)` | `Flow.Publisher` | Неизвестен | Реактивное потоковое тело неизвестной длины (`size()` возвращает `-1`) | +| `S3Body.ofPublisher(Flow.Publisher, long)` | `Flow.Publisher` | Известен | Реактивное потоковое тело, длина которого передаётся явно через аргумент `size` | + +Само тело предоставляет следующие методы доступа: + +| Метод | Описание | +|---------------------------------------------|-----------------------------------------------------------------------------------| +| `byte[] asBytes()` | Считывает всё тело в массив байтов (исчерпывает нижележащий поток) | +| `InputStream asInputStream()` | Возвращает тело как блокирующий `InputStream` | +| `Flow.Publisher asPublisher()` | Возвращает тело как реактивный `Flow.Publisher` | +| `long size()` | Длина содержимого в байтах или `-1`, если неизвестна (неограниченный поток / publisher) | +| `String type()` | `Content-Type` тела | +| `String encoding()` | `Content-Encoding` тела | + +Если файл очень большой или его длина неизвестна и требуется потоковая передача, рекомендуется создавать тело с помощью +`S3Body.ofPublisher(...)` или `S3Body.ofInputStreamUnbound(...)`. + +Если тип файла не указан, будет использован `application/octet-stream`. +Для `@S3.Put` тело также можно передать напрямую как `byte[]` или `ByteBuffer`; в этом случае клиент сам создаёт `S3Body`. +Аннотация `@S3.Put` позволяет указать `type` и `encoding`, которые будут записаны как `Content-Type` и `Content-Encoding`. + +`HTTP`-сервер может передавать тело запроса в `S3` потоком, не читая весь файл в память заранее. +Для этого примите тело запроса как `Flow.Publisher` и передайте его в `S3Body.ofPublisher(...)`. +Если размер тела известен, например из заголовка `Content-Length`, лучше передать этот размер в `S3Body`; +если размер неизвестен, используйте перегрузку без размера, и размер будет считаться неизвестным. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController + public final class UploadController { + + private final S3KoraClient s3; + + public UploadController(S3KoraClient s3) { + this.s3 = s3; + } + + @HttpRoute(method = HttpMethod.PUT, path = "/files/{key}") + public HttpServerResponse upload(@Path String key, + @Header("Content-Type") @Nullable String contentType, + @Header("Content-Length") @Nullable Long contentLength, + Flow.Publisher body) { + var type = contentType == null ? "application/octet-stream" : contentType; + var s3Body = contentLength == null + ? S3Body.ofPublisher(body, type) + : S3Body.ofPublisher(body, contentLength, type); + + this.s3.put("documents", key, s3Body); + return HttpServerResponse.of(201); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController + class UploadController( + private val s3: S3KoraClient + ) { + + @HttpRoute(method = HttpMethod.PUT, path = "/files/{key}") + fun upload( + @Path key: String, + @Header("Content-Type") contentType: String?, + @Header("Content-Length") contentLength: Long?, + body: Flow.Publisher + ): HttpServerResponse { + val type = contentType ?: "application/octet-stream" + val s3Body = if (contentLength == null) { + S3Body.ofPublisher(body, type) + } else { + S3Body.ofPublisher(body, contentLength, type) + } + + s3.put("documents", key, s3Body) + return HttpServerResponse.of(201) + } + } + ``` + +В этом варианте `Kora` получает `Flow.Publisher` из тела `HTTP`-запроса через стандартный +`HttpServerRequestMapper`, а `S3`-клиент читает тот же поток во время загрузки. Обработчику не нужно вызывать +`asBytes()`, `asInputStream().readAllBytes()` или `S3Body.ofInputStreamReadAll(...)`, если цель — не держать весь файл в памяти. + +#### Тип и кодировка содержимого { #content-type } + +Вместо того чтобы самостоятельно конструировать `S3Body`, можно передать тело напрямую как `byte[]` или `ByteBuffer` и позволить +клиенту обернуть его в `S3Body`. В этом случае для построения тела используются атрибуты `type` (`Content-Type`) +и `encoding` (`Content-Encoding`) аннотации `@S3.Put`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @S3.Client("s3client.someClient") + public interface SomeClient { + + @S3.Put(value = "some-key", type = "image/jpeg", encoding = "gzip") //(1)! + void operation1(byte[] body); //(2)! + + @S3.Put("some-key") + void operation2(ByteBuffer body); //(3)! + } + ``` + + 1. `type` сопоставляется с `Content-Type`, а `encoding` — с `Content-Encoding` + 2. Когда тело имеет тип `byte[]` или `ByteBuffer`, клиент сам строит `S3Body`, используя `type`/`encoding` из аннотации + 3. Если не заданы ни `type`, ни `encoding`, в качестве `Content-Type` используется `application/octet-stream` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @S3.Client("s3client.someClient") + interface SomeClient { -Если файл очень большой либо не известна его длина и требуется потоковая загрузка, то рекомендуется создавать тело с помощь `S3Body.ofPublisher()` либо `S3Body.ofInputStreamUnbound()`. + @S3.Put(value = "some-key", type = "image/jpeg", encoding = "gzip") //(1)! + fun operation1(body: ByteArray) //(2)! -В случае если не будет указан тип файла, он будет проставлен как `application/octet-stream`. + @S3.Put("some-key") + fun operation2(body: ByteBuffer) //(3)! + } + ``` + + 1. `type` сопоставляется с `Content-Type`, а `encoding` — с `Content-Encoding` + 2. Когда тело имеет тип `ByteArray` или `ByteBuffer`, клиент сам строит `S3Body`, используя `type`/`encoding` из аннотации + 3. Если не заданы ни `type`, ни `encoding`, в качестве `Content-Type` используется `application/octet-stream` + +!!! warning "Тип тела" + + Тело операции `@S3.Put` должно быть `S3Body`, `byte[]` или `ByteBuffer`, иначе возникает ошибка компиляции. + Атрибуты `type` и `encoding` применяются только к «сырым» телам `byte[]`/`ByteBuffer`; когда передаётся готовый `S3Body`, + используются его собственные значения `type()`/`encoding()`, а атрибуты аннотации игнорируются. #### Шаблон ключа { #key-template-2 } -Ключ можно указывать также как шаблон и подставлять туда аргументы метода как части шаблона, +Ключ также можно задать в виде шаблона и подставлять в него аргументы метода как часть шаблона; все аргументы метода должны быть частью составного ключа. ===! ":fontawesome-brands-java: `Java`" @@ -742,8 +1021,8 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон ключа, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` - 2. Все аргументы метода должны быть частью шаблона ключа либо `S3Body` + 1. Шаблон, используемый для построения ключа: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` + 2. Все аргументы метода должны быть частью шаблона ключа либо иметь тип `S3Body` === ":simple-kotlin: `Kotlin`" @@ -756,13 +1035,13 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон ключа, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` - 2. Все аргументы метода должны быть частью шаблона ключа либо `S3Body` + 1. Шаблон, используемый для построения ключа: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` + 2. Все аргументы метода должны быть частью шаблона ключа либо иметь тип `S3Body` ### Удаление файла { #delete-file } -Секция описывает операцию удаление файла с помощью декларативного S3-клиента. -Предлагается использовать аннотацию `@S3.Delete` для операции. +В разделе описана операция удаления файла с помощью декларативного S3-клиента. +Для операции предлагается использовать аннотацию `@S3.Delete`. ===! ":fontawesome-brands-java: `Java`" @@ -778,8 +1057,8 @@ agent: } ``` - 1. Операция удаления файла - 2. Получение в ответ файла вместе данными + 1. операция удаления файла + 2. Ключ удаляемого файла 3. Ключ файла можно указать в аннотации === ":simple-kotlin: `Kotlin`" @@ -796,13 +1075,13 @@ agent: } ``` - 1. Операция удаления файла - 2. Получение в ответ файла вместе данными + 1. операция удаления файла + 2. Ключ удаляемого файла 3. Ключ файла можно указать в аннотации #### Шаблон ключа { #key-template-3 } -Ключ можно указывать также как шаблон и подставлять туда аргументы метода как части шаблона, +Ключ также можно задать в виде шаблона и подставлять в него аргументы метода как часть шаблона; все аргументы метода должны быть частью составного ключа. ===! ":fontawesome-brands-java: `Java`" @@ -816,7 +1095,7 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон ключа, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` + 1. Шаблон, используемый для построения ключа: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` 2. Все аргументы метода должны быть частью шаблона ключа === ":simple-kotlin: `Kotlin`" @@ -830,13 +1109,12 @@ agent: } ``` - 1. Шаблон по которому собирать шаблон ключа, каждый аргумент шаблона будет подставлен через `toString()`, аргументы в шаблоне указывается как имена аргументов метода в `{ковычках}` + 1. Шаблон, используемый для построения ключа: каждый аргумент шаблона подставляется через `toString()`, а аргументы шаблона указываются как имена аргументов метода в `{фигурных скобках}` 2. Все аргументы метода должны быть частью шаблона ключа -#### Множество ключей { #multiple-keys-2 } +#### Несколько ключей { #multiple-keys-2 } -Можно также получать сразу множество файлов по ключам как полный файл вместе с данными `S3Object`, -так и облегченный вариант в виде множества метаданных файлов без данных `S3ObjectMeta`. +Также можно удалять несколько файлов по ключам. ===! ":fontawesome-brands-java: `Java`" @@ -849,8 +1127,8 @@ agent: } ``` - 1. Операция получения файла для множества ключей **не должна** содержать шаблон ключа - 2. Операция должна принимать список ключей и отдавать `void` + 1. Операция удаления по нескольким ключам **не должна** содержать шаблон ключа + 2. Операция должна принимать список ключей и возвращать `void` === ":simple-kotlin: `Kotlin`" @@ -863,39 +1141,497 @@ agent: } ``` - 1. Операция получения файла для множества ключей **не должна** содержать шаблон ключа - 2. Операция должна принимать список ключей и отдавать `void` + 1. Операция удаления по нескольким ключам **не должна** содержать шаблон ключа + 2. Операция должна принимать список ключей и возвращать `void` ### Сигнатуры { #signatures } -Доступные сигнатуры для методов декларативного HTTP клиента из коробки: +Доступные из коробки сигнатуры методов декларативного `S3`-клиента: ===! ":fontawesome-brands-java: `Java`" - Под `T` подразумевается тип возвращаемого значения, либо `Void`. + Под `T` подразумевается тип возвращаемого значения. - `T myMethod()` - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) + - `CompletableFuture myMethod()` [CompletableFuture](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html) - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" - Под `T` подразумевается тип возвращаемого значения, либо `Unit`. + Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `Unit`. - `myMethod(): T` - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) -## Клиент императивный { #client-imperative } +## Модели { #models } + +И декларативные, и императивные клиенты возвращают один и тот же набор типов-моделей (если не используется +[нативный формат ответа](#response-format) модуля `AWS`). Все модели — интерфейсы только для чтения. + +### S3Object { #model-s3-object } + +Полный объект вместе с его данными, возвращаемый операциями [получения](#get-file) и доступный внутри [S3ObjectList](#model-s3-object-list): + +| Метод | Описание | +|---------------------|-----------------------------------------------------------------| +| `String key()` | Ключ объекта | +| `Instant modified()`| Время последнего изменения | +| `long size()` | Размер объекта в байтах | +| `S3Body body()` | [Тело](#file-body) объекта с данными | + +### S3ObjectMeta { #model-s3-object-meta } + +Облегчённые метаданные без данных объекта, возвращаемые операциями [получения](#metadata) метаданных и доступные внутри +[S3ObjectMetaList](#model-s3-object-meta-list). Получение метаданных быстрее, поскольку тело объекта не передаётся: -Можно внедрить императивный Kora клиента для работы с S3, предоставляется как клиент для синхронной так и асинхронной работы: +| Метод | Описание | +|----------------------|---------------------------------| +| `String key()` | Ключ объекта | +| `Instant modified()` | Время последнего изменения | +| `long size()` | Размер объекта в байтах | + +### S3ObjectList { #model-s3-object-list } + +Список полных объектов, возвращаемый операциями [получения списка](#list-files). Расширяет `S3ObjectMetaList`, поэтому также предоставляет префикс и метаданные: + +| Метод | Описание | +|-------------------------------|---------------------------------------------------| +| `String prefix()` | Префикс, использованный для получения списка | +| `List objects()` | Объекты, соответствующие префиксу (с данными) | +| `List metas()` | Метаданные объектов, соответствующих префиксу | + +### S3ObjectMetaList { #model-s3-object-meta-list } + +Список метаданных, возвращаемый операциями [получения списка](#metadata-2) метаданных: + +| Метод | Описание | +|-------------------------------|-------------------------------------------------| +| `String prefix()` | Префикс, использованный для получения списка | +| `List metas()` | Метаданные объектов, соответствующих префиксу | + +### S3ObjectUpload { #model-s3-object-upload } + +Результат операции [добавления файла](#add-file): + +| Метод | Описание | +|-----------------------|-----------------------------------------------------------------------------| +| `String versionId()` | Идентификатор версии загруженного объекта (если для бакета включено версионирование) | + +## Императивный клиент { #client-imperative } + +Для работы с `S3` можно внедрить императивный клиент `Kora`; предоставляются как синхронный, так и асинхронный клиенты: - `S3KoraClient` - клиент для синхронной работы - `S3KoraAsyncClient` - клиент для асинхронной работы -## Ошибки { #exceptions } +Оба клиента работают с явными параметрами `bucket` и `key` и поддерживают получение объектов или метаданных, получение списка объектов по префиксу, +загрузку `S3Body` и удаление одного или нескольких объектов. В отличие от декларативного клиента, они не привязаны к единственному `bucket` из +конфигурации — `bucket` передаётся в каждый метод явно. -В случае ошибки работы клиента будут брошены специальные ошибки: +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { -- `S3NotFoundException` - в случае если не найдет файл по указанному ключу -- `S3DeleteException` - в случае ошибки удаления файла -- `S3Exception` - в любом другом случае + private final S3KoraClient s3; + + public SomeService(S3KoraClient s3) { + this.s3 = s3; + } + + public byte[] download(String bucket, String key) { + S3Object object = s3.get(bucket, key); //(1)! + return object.body().asBytes(); + } + } + ``` + + 1. Выбрасывает `S3NotFoundException`, если объект отсутствует + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val s3: S3KoraClient + ) { + + fun download(bucket: String, key: String): ByteArray { + val obj = s3.get(bucket, key) //(1)! + return obj.body().asBytes() + } + } + ``` + + 1. Выбрасывает `S3NotFoundException`, если объект отсутствует + +### Синхронный клиент { #client-imperative-sync } + +Интерфейс `S3KoraClient` предоставляет следующие операции: + +| Метод | Описание | +|---------------------------------------------------------------------------------------|-------------------------------------------------------------------| +| `S3Object get(bucket, key)` | Получить один объект с данными | +| `S3ObjectMeta getMeta(bucket, key)` | Получить метаданные одного объекта | +| `List get(bucket, Collection keys)` | Получить несколько объектов с данными | +| `List getMeta(bucket, Collection keys)` | Получить метаданные нескольких объектов | +| `S3ObjectList list(bucket[, prefix[, delimiter, limit]])` | Получить список объектов по префиксу (с данными) | +| `S3ObjectMetaList listMeta(bucket[, prefix[, delimiter, limit]])` | Получить список метаданных объектов по префиксу | +| `List list(bucket, Collection prefixes[, delimiter, limit])` | Получить список объектов сразу для нескольких префиксов | +| `List listMeta(bucket, Collection prefixes[, delimiter, limit])` | Получить список метаданных объектов сразу для нескольких префиксов | +| `S3ObjectUpload put(bucket, key, S3Body body)` | Добавить объект и вернуть результат загрузки | +| `void delete(bucket, key)` | Удалить один объект | +| `void delete(bucket, Collection keys)` | Удалить несколько объектов (при неудаче выбрасывает `S3DeleteException`) | + +Перегрузки `list`/`listMeta` без `delimiter`/`limit` по умолчанию используют `null` для `delimiter` и `1000` для `limit`. +Аргумент `limit` должен находиться в диапазоне `1..1000`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + // получить один объект и его метаданные + S3Object object = s3.get("documents", "report.pdf"); + S3ObjectMeta meta = s3.getMeta("documents", "report.pdf"); + + // получить сразу несколько объектов + List objects = s3.get("documents", List.of("a.pdf", "b.pdf")); + + // получить список по префиксу с разделителем и ограничением + S3ObjectList list = s3.list("documents", "2024/", "/", 100); + for (S3Object o : list.objects()) { + // ... + } + + // получить список сразу для нескольких префиксов + List perPrefix = s3.listMeta("documents", List.of("2023/", "2024/")); + + // добавить объект + S3ObjectUpload upload = s3.put("documents", "report.pdf", S3Body.ofBytes(bytes)); + String versionId = upload.versionId(); + + // удалить один объект и пакет объектов + s3.delete("documents", "report.pdf"); + s3.delete("documents", List.of("a.pdf", "b.pdf")); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + // получить один объект и его метаданные + val obj = s3.get("documents", "report.pdf") + val meta = s3.getMeta("documents", "report.pdf") + + // получить сразу несколько объектов + val objects = s3.get("documents", listOf("a.pdf", "b.pdf")) + + // получить список по префиксу с разделителем и ограничением + val list = s3.list("documents", "2024/", "/", 100) + for (o in list.objects()) { + // ... + } + + // получить список сразу для нескольких префиксов + val perPrefix = s3.listMeta("documents", listOf("2023/", "2024/")) + + // добавить объект + val upload = s3.put("documents", "report.pdf", S3Body.ofBytes(bytes)) + val versionId = upload.versionId() + + // удалить один объект и пакет объектов + s3.delete("documents", "report.pdf") + s3.delete("documents", listOf("a.pdf", "b.pdf")) + ``` + +### Асинхронный клиент { #client-imperative-async } + +Интерфейс `S3KoraAsyncClient` повторяет `S3KoraClient` метод в метод, но каждая операция возвращает +[CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) +(`CompletionStage` для операций удаления): + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final S3KoraAsyncClient s3; + + public SomeService(S3KoraAsyncClient s3) { + this.s3 = s3; + } + + public CompletionStage download(String bucket, String key) { + return s3.get(bucket, key) + .thenApply(object -> object.body().asBytes()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val s3: S3KoraAsyncClient + ) { + + fun download(bucket: String, key: String): CompletionStage { + return s3.get(bucket, key) + .thenApply { it.body().asBytes() } + } + } + ``` + +## Нативные клиенты { #native-clients } + +Помимо декларативных и императивных клиентов `Kora`, для внедрения также доступны нижележащие нативные клиенты `SDK`. +Они полезны для расширенных операций, не покрываемых декларативным/императивным API (например, управление бакетами, копирование +объектов, предподписанные (presigned) URL и так далее). + +[Модуль AWS](#aws) предоставляет: + +- `S3Client` — синхронный клиент `AWS` +- `S3AsyncClient` — асинхронный клиент `AWS` +- `S3AsyncClient` с `@Tag(MultipartUpload.class)` — асинхронный клиент `AWS`, предварительно настроенный для [многочастной загрузки](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/internal/multipart/MultipartS3AsyncClient.html) в соответствии с `upload.partSize` и `upload.bufferSize` + +[Модуль Minio](#minio) предоставляет: + +- `MinioClient` — синхронный клиент `Minio` +- `MinioAsyncClient` — асинхронный клиент `Minio` + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class BucketService { + + private final S3Client s3Client; //(1)! + private final S3AsyncClient multipartClient; + + public BucketService(S3Client s3Client, + @Tag(MultipartUpload.class) S3AsyncClient multipartClient) { //(2)! + this.s3Client = s3Client; + this.multipartClient = multipartClient; + } + + public void ensureBucket(String bucket) { + s3Client.createBucket(b -> b.bucket(bucket)); + } + } + ``` + + 1. Нативный `S3Client` из `AWS`, внедряемый напрямую + 2. Асинхронный клиент с тегом `@Tag(MultipartUpload.class)` для многочастной загрузки + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class BucketService( + private val s3Client: S3Client, //(1)! + @Tag(MultipartUpload::class) private val multipartClient: S3AsyncClient //(2)! + ) { + + fun ensureBucket(bucket: String) { + s3Client.createBucket { it.bucket(bucket) } + } + } + ``` + + 1. Нативный `S3Client` из `AWS`, внедряемый напрямую + 2. Асинхронный клиент с тегом `@Tag(MultipartUpload::class)` для многочастной загрузки + +## Исключения { #exceptions } + +Если операция клиента завершается неудачей, выбрасывается одно из исключений `S3`. Все они наследуются от базового `S3Exception`, +который, в свою очередь, расширяет `RuntimeException`, поэтому их обработка необязательна и не проверяется компилятором. + +**Иерархия исключений:** + +``` +RuntimeException +└── S3Exception + ├── S3NotFoundException + └── S3DeleteException +``` + +Базовое исключение `S3Exception` предоставляет код ошибки и сообщение, сообщённые хранилищем: + +| Метод | Описание | +|-----------------------------|------------------------------------------------------| +| `String getErrorCode()` | Код ошибки хранилища (например, `NoSuchKey`) | +| `String getErrorMessage()` | Сообщение об ошибке хранилища | + +**Пример обработки:** + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final S3KoraClient s3; + + public SomeService(S3KoraClient s3) { + this.s3 = s3; + } + + public void call(String bucket) { + try { + s3.delete(bucket, List.of("a.pdf", "b.pdf")); + } catch (S3NotFoundException e) { + // Объект или бакет отсутствует: getErrorCode() возвращает NoSuchKey или NoSuchBucket + } catch (S3DeleteException e) { + // Один или несколько объектов не были удалены + for (S3DeleteException.Error error : e.getErrors()) { + // error.key(), error.bucket(), error.code(), error.message() + } + } catch (S3Exception e) { + // Любая другая ошибка хранилища: getErrorCode(), getErrorMessage() + } + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService( + private val s3: S3KoraClient + ) { + + fun call(bucket: String) { + try { + s3.delete(bucket, listOf("a.pdf", "b.pdf")) + } catch (e: S3NotFoundException) { + // Объект или бакет отсутствует: errorCode равен NoSuchKey или NoSuchBucket + } catch (e: S3DeleteException) { + // Один или несколько объектов не были удалены + for (error in e.errors) { + // error.key(), error.bucket(), error.code(), error.message() + } + } catch (e: S3Exception) { + // Любая другая ошибка хранилища: errorCode, errorMessage + } + } + } + ``` + +### S3NotFoundException { #not-found-exception } + +Выбрасывается, когда запрошенный объект или бакет не существует. + +**Причины:** + +- Ключ объекта не существует (`getErrorCode()` возвращает `NoSuchKey`) +- Бакет не существует (`getErrorCode()` возвращает `NoSuchBucket`) + +**Рекомендации:** + +- Сделайте результат `@S3.Get` [необязательным](#optional-get) (`Optional`/nullable), если отсутствие объекта — нормальный исход +- Проверьте `bucket` из конфигурации и запрошенный `key` + +### S3DeleteException { #delete-exception } + +Выбрасывается пакетными операциями `delete(bucket, keys)`, когда один или несколько объектов не удалось удалить. +Предоставляет список отдельных сбоев: + +| Метод | Описание | +|---------------------------|----------------------------------------------------------------| +| `List getErrors()` | Сбои по каждому объекту, каждый с `key()`, `bucket()`, `code()`, `message()` | + +**Рекомендации:** + +- Изучите `getErrors()`, чтобы определить, какие объекты не удалось обработать и почему +- Повторите неудавшиеся ключи отдельно, если сбой временный + +### S3Exception { #base-exception } + +Базовое исключение, выбрасываемое при любой другой ошибке хранилища или клиента, не связанной с отсутствием объекта или сбоем пакетного удаления. + +**Рекомендации:** + +- Логируйте `getErrorCode()` и `getErrorMessage()` для диагностики +- Включите [логирование](#configuration) клиента на уровне `DEBUG`, чтобы изучить нижележащий запрос/ответ + +## Тестирование { #testing } + +Декларативные и императивные `S3`-клиенты можно тестировать с помощью [@KoraAppTest](junit5.md) вместе с реальным +`S3`-совместимым хранилищем, запущенным в контейнере [Testcontainers](https://java.testcontainers.org/) (например, `Minio`). +Параметры подключения к хранилищу передаются в конфигурацию приложения через системные свойства: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @TestcontainersMinio( + mode = ContainerMode.PER_RUN, + bucket = @Bucket(value = SomeClientTests.BUCKET, create = Bucket.Mode.PER_METHOD, drop = Bucket.Mode.PER_METHOD)) + @KoraAppTest(Application.class) + class SomeClientTests implements KoraAppTestConfigModifier { + + static final String BUCKET = "simple"; + + @ConnectionMinio + private MinioConnection minioConnection; + + @TestComponent + private SomeClient client; + + @Override + public KoraConfigModification config() { + return KoraConfigModification + .ofSystemProperty("S3_URL", minioConnection.params().uri().toString()) + .withSystemProperty("S3_ACCESS_KEY", minioConnection.params().accessKey()) + .withSystemProperty("S3_SECRET_KEY", minioConnection.params().secretKey()) + .withSystemProperty("S3_BUCKET", BUCKET); + } + + @Test + void putAndGet() { + var value = "value".getBytes(StandardCharsets.UTF_8); + client.putObject("k1", S3Body.ofBytes(value)); + + var found = client.getObject("k1"); + assertArrayEquals(value, found.body().asBytes()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @TestcontainersMinio( + mode = ContainerMode.PER_RUN, + bucket = Bucket(value = [BUCKET], create = Bucket.Mode.PER_METHOD, drop = Bucket.Mode.PER_METHOD)) + @KoraAppTest(Application::class) + class SomeClientTests : KoraAppTestConfigModifier { + + @ConnectionMinio + lateinit var minioConnection: MinioConnection + + @TestComponent + lateinit var client: SomeClient + + override fun config(): KoraConfigModification = KoraConfigModification + .ofSystemProperty("S3_URL", minioConnection.params().uri().toString()) + .withSystemProperty("S3_ACCESS_KEY", minioConnection.params().accessKey()) + .withSystemProperty("S3_SECRET_KEY", minioConnection.params().secretKey()) + .withSystemProperty("S3_BUCKET", BUCKET) + + @Test + fun putAndGet() { + val value = "value".toByteArray() + client.putObject("k1", S3Body.ofBytes(value)) + + val found = client.getObject("k1") + assertArrayEquals(value, found.body().asBytes()) + } + + companion object { + const val BUCKET = "simple" + } + } + ``` diff --git a/mkdocs/docs/ru/documentation/scheduling.md b/mkdocs/docs/ru/documentation/scheduling.md index 47123d6..c5ddfad 100644 --- a/mkdocs/docs/ru/documentation/scheduling.md +++ b/mkdocs/docs/ru/documentation/scheduling.md @@ -4,17 +4,35 @@ agent: use_when: "Use this file for Kora docs or implementation questions about Kora scheduling for native and Quartz schedulers, fixed rate, fixed delay, one-shot and cron jobs, triggers, shutdown, and concurrency controls; key triggers include @ScheduleAtFixedRate, @ScheduleWithFixedDelay, @ScheduleOnce, @ScheduleWithCron, @ScheduleWithTrigger, @DisallowConcurrentExecution, SchedulingModule, QuartzModule." --- -Модуль для создания планировщиков в декларативном стиле с помощью аннотаций. +Модуль планирования Kora позволяет запускать методы приложения по расписанию в декларативном стиле через аннотации. +Во время компиляции Kora генерирует компоненты задач и связывает их с выбранным механизмом планирования. -## Встроенный { #native } +Доступны два варианта: собственный планировщик на основе `ScheduledExecutorService` из `JDK` и планировщик на основе `Quartz`. +Собственный вариант подходит для простых периодических задач внутри одного приложения, а `Quartz` полезен для `cron`-выражений, пользовательских экземпляров `Trigger` и дополнительных правил выполнения задач. -Создание планировщика с использованием стандартного [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) который -поставляется с JVM. +## Собственный планировщик { #native } -Для создания планировщика через аспекты используются специальные аннотации которые по сути дублируются сигнатуры методов `ScheduledExecutorService`. -Параметры аннотаций соответствуют параметрам методов `scheduleAtFixedRate`, `scheduleWithFixedDelay`, `schedule` соответственно. +Собственный планировщик использует стандартный [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html), который поставляется вместе с `JDK`. -Так же все аннотации имеют аргумент `config` при наличии которого значения параметра возьмутся из конфигурации по указанному пути. +Для создания задач через аспекты используются специальные аннотации, соответствующие методам `ScheduledExecutorService`. +Параметры аннотаций совпадают с параметрами методов `scheduleAtFixedRate`, `scheduleWithFixedDelay` и `schedule`. + +У всех аннотаций есть параметр `config`. +Если он указан, значения параметров берутся из конфигурации по этому пути и имеют приоритет над значениями из аннотации. +Конфигурация конкретной задачи также может содержать секцию `telemetry`, значения которой переопределяют общую телеметрию планировщика для этой задачи. + +Методы, выполняемые по расписанию, должны удовлетворять следующим требованиям: + +- Класс, в котором объявлен метод, должен быть компонентом в [графе зависимостей](container.md), например помеченным аннотацией `@Component`. +- Метод собственного планировщика не должен иметь аргументов (планировщик `Quartz` дополнительно допускает необязательный аргумент [JobExecutionContext](#job-context)). +- Возвращаемое значение метода игнорируется. +- В `Kotlin` метод не должен быть `suspend`-функцией. + +!!! warning "Интервал обязателен" + + `@ScheduleAtFixedRate` требует `period`, а `@ScheduleWithFixedDelay` требует `delay`. + Если не задан ни атрибут аннотации (его значение по умолчанию равно `0`), ни путь `config`, предоставляющий значение, + компиляция завершается ошибкой `Either period() or config() annotation parameter must be provided`. ### Подключение { #dependency } @@ -34,7 +52,7 @@ agent: === ":simple-kotlin: `Kotlin`" [Зависимость](general.md#dependencies) `build.gradle.kts`: - ```groovy + ```kotlin implementation("ru.tinkoff.kora:scheduling-jdk") ``` @@ -46,7 +64,7 @@ agent: ### Конфигурация { #configuration } -Пример полной конфигурации, описанной в классе `ScheduledExecutorServiceConfig` (указаны значения по умолчанию): +Полный пример конфигурации, описываемой классом `ScheduledExecutorServiceConfig`, со значениями по умолчанию: ===! ":material-code-json: `Hocon`" @@ -77,14 +95,14 @@ agent: } ``` - 1. Максимальное кол-во потоков у [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) - 2. Время ожидания выполнения задач перед выключением планировщика в случае [штатного завершения](container.md#component-lifecycle) - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Включает метрики модуля (по умолчанию `true`) - 5. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 6. Настройка тегов для метрик (опционально) - 7. Включает трассировку модуля (по умолчанию `true`) - 8. Настройка атрибутов для трассировки (опционально) + 1. Максимальное количество потоков в [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) (по умолчанию: `2`) + 2. Время ожидания завершения задач перед остановкой планировщика при [плавной остановке](container.md#component-lifecycle) (по умолчанию: `30s`) + 3. Включает логирование модуля (по умолчанию: `false`) + 4. Включает метрики модуля (по умолчанию: `true`) + 5. Настраивает [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Настраивает теги метрик (по умолчанию: `{}`) + 7. Включает трассировку модуля (по умолчанию: `true`) + 8. Настраивает атрибуты трассировки (по умолчанию: `{}`) === ":simple-yaml: `YAML`" @@ -108,24 +126,66 @@ agent: key2: value2 ``` - 1. Максимальное кол-во потоков у [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) - 2. Время ожидания выполнения задач перед выключением планировщика в случае [штатного завершения](container.md#component-lifecycle) - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Включает метрики модуля (по умолчанию `true`) - 5. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 6. Настройка тегов для метрик (опционально) - 7. Включает трассировку модуля (по умолчанию `true`) - 8. Настройка атрибутов для трассировки (опционально) + 1. Максимальное количество потоков в [ScheduledExecutorService](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ScheduledExecutorService.html) (по умолчанию: `2`) + 2. Время ожидания завершения задач перед остановкой планировщика при [плавной остановке](container.md#component-lifecycle) (по умолчанию: `30s`) + 3. Включает логирование модуля (по умолчанию: `false`) + 4. Включает метрики модуля (по умолчанию: `true`) + 5. Настраивает [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Настраивает теги метрик (по умолчанию: `{}`) + 7. Включает трассировку модуля (по умолчанию: `true`) + 8. Настраивает атрибуты трассировки (по умолчанию: `{}`) + +Метрики модуля описаны в разделе [Справочник метрик](metrics.md#scheduling). + +Конфигурация конкретной задачи также может содержать собственную секцию `telemetry`, которая переопределяет общую для планировщика `scheduling.telemetry` только для этой задачи. +Незаданные значения берутся из общей конфигурации, поэтому достаточно указать только то, что должно отличаться: + +===! ":material-code-json: `Hocon`" -Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#scheduling). + ```javascript + scheduling { + jobs { + fix-rate { + period = "50ms" + telemetry { + logging.enabled = true //(1)! + metrics.enabled = false //(2)! + } + } + } + } + ``` -### Фиксированный интервал { #fixed-rate } + 1. Переопределяет `scheduling.telemetry.logging.enabled` только для этой задачи + 2. Переопределяет `scheduling.telemetry.metrics.enabled` только для этой задачи -Планирование с запуском задач с фиксированными промежутками времени, независимо от того, завершилась ли предыдущая или нет -Такой подход может привести к одновременному выполнению нескольких задач. +=== ":simple-yaml: `YAML`" -Например, если период установлен в 10 секунд, а каждое выполнение задачи занимает 5 секунд, -то след задача запуститься через 5 секунд после выполнения предыдущей. + ```yaml + scheduling: + jobs: + fix-rate: + period: "50ms" + telemetry: + logging: + enabled: true #(1)! + metrics: + enabled: false #(2)! + ``` + + 1. Переопределяет `scheduling.telemetry.logging.enabled` только для этой задачи + 2. Переопределяет `scheduling.telemetry.metrics.enabled` только для этой задачи + +Наблюдаемость задач по расписанию также можно настроить в коде, зарегистрировав компонент, реализующий +`SchedulingLoggerFactory`, `SchedulingMetricsFactory`, `SchedulingTracerFactory` или целиком `SchedulingTelemetryFactory`. + +### Фиксированная частота { #fixed-rate } + +Планирование с запуском задач через фиксированный интервал времени независимо от того, завершилось ли предыдущее выполнение. +Это может приводить к одновременному выполнению нескольких задач. + +Например, если период равен 10 секундам, а каждое выполнение задачи занимает 5 секунд, +то следующая задача запускается через 5 секунд после завершения предыдущей. ===! ":fontawesome-brands-java: `Java`" @@ -155,7 +215,9 @@ agent: #### Конфигурация { #configuration-2 } -Возможно передача параметров через конфигурацию, она имеет приоритет перед указанными в аннотации параметрами: +Параметры можно передавать через конфигурацию; конфигурация имеет приоритет над значениями из аннотации. +Путь `config` произвольный, но по соглашению вкладывается в секцию `scheduling`, чтобы параметры задачи +и её `telemetry` находились вместе (как в [проекте-примере](https://github.com/kora-projects/kora-examples), `scheduling.jobs.fix-rate`): ===! ":fontawesome-brands-java: `Java`" @@ -163,7 +225,7 @@ agent: @Component public class SomeService { - @ScheduleAtFixedRate(config = "job") + @ScheduleAtFixedRate(config = "scheduling.jobs.fix-rate") void schedule() { // do something } @@ -176,45 +238,51 @@ agent: @Component class SomeService { - @ScheduleAtFixedRate(config = "job") + @ScheduleAtFixedRate(config = "scheduling.jobs.fix-rate") fun schedule() { // do something } } ``` -Пример конфигурации через файл: +Пример файла конфигурации: ===! ":material-code-json: `Hocon`" ```javascript - job { - initialDelay = "50ms" //(1)! - period = "50ms" //(2)! + scheduling { + jobs { + fix-rate { + initialDelay = "50ms" //(1)! + period = "50ms" //(2)! + } + } } ``` - 1. Начальный интервал задержки перед первой задачей - 2. Переодический интервал между задачами + 1. Начальная задержка перед первой задачей (по умолчанию: `0ms`) + 2. Периодический интервал между задачами (`обязательный`, без значения по умолчанию) === ":simple-yaml: `YAML`" ```yaml - job: - initialDelay: "50ms" #(1)! - period: "50ms" #(2)! + scheduling: + jobs: + fix-rate: + initialDelay: "50ms" #(1)! + period: "50ms" #(2)! ``` - 1. Начальный интервал задержки перед первой задачей - 2. Переодический интервал между задачами + 1. Начальная задержка перед первой задачей (по умолчанию: `0ms`) + 2. Периодический интервал между задачами (`обязательный`, без значения по умолчанию) ### Фиксированная задержка { #fixed-delay } -Планировщик ожидает фиксированный промежуток времени от окончания предыдущего исполнения задачи. -Выполнения нескольких задач не будет происходить одновременно. +Планировщик выдерживает фиксированный интервал времени от момента окончания предыдущего выполнения задачи. +Несколько выполнений одной и той же задачи не будут происходить одновременно. -Не имеет значения, сколько времени занимает текущее исполнение, -следующая задача запустится после завершения предыдущей задачи и заданного промежутка ожидания. +Не имеет значения, сколько длится текущее выполнение: +следующая задача запускается после того, как предыдущая задача завершилась и прошла настроенная задержка. ===! ":fontawesome-brands-java: `Java`" @@ -244,7 +312,7 @@ agent: #### Конфигурация { #configuration-3 } -Возможно передача параметров через конфигурацию, она имеет приоритет перед указанными в аннотации параметрами: +Параметры можно передавать через конфигурацию; она имеет приоритет над значениями из аннотации: ===! ":fontawesome-brands-java: `Java`" @@ -252,7 +320,7 @@ agent: @Component public class SomeService { - @ScheduleWithFixedDelay(config = "job") + @ScheduleWithFixedDelay(config = "scheduling.jobs.fix-delay") void schedule() { // do something } @@ -265,41 +333,47 @@ agent: @Component class SomeService { - @ScheduleWithFixedDelay(config = "job") + @ScheduleWithFixedDelay(config = "scheduling.jobs.fix-delay") fun schedule() { // do something } } ``` -Пример конфигурации через файл: +Пример файла конфигурации: ===! ":material-code-json: `Hocon`" ```javascript - job { - initialDelay = "50ms" //(1)! - delay = "50ms" //(2)! + scheduling { + jobs { + fix-delay { + initialDelay = "50ms" //(1)! + delay = "50ms" //(2)! + } + } } ``` - 1. Начальный интервал задержки перед первой задачей - 2. Переодический интервал задержки между задачами + 1. Начальная задержка перед первой задачей (по умолчанию: `0ms`) + 2. Периодическая задержка между задачами (`обязательный`, без значения по умолчанию) === ":simple-yaml: `YAML`" ```yaml - job: - initialDelay: "50ms" #(1)! - delay: "50ms" #(2)! + scheduling: + jobs: + fix-delay: + initialDelay: "50ms" #(1)! + delay: "50ms" #(2)! ``` - 1. Начальный интервал задержки перед первой задачей - 2. Переодический интервал задержки между задачами + 1. Начальная задержка перед первой задачей (по умолчанию: `0ms`) + 2. Периодическая задержка между задачами (`обязательный`, без значения по умолчанию) -### Одноразовый { #once } +### Однократно { #once } -Запускает одиножды задачу через определенный фиксированный интервал времени. +Запускает задачу один раз через настроенный интервал времени. ===! ":fontawesome-brands-java: `Java`" @@ -329,7 +403,7 @@ agent: #### Конфигурация { #configuration-4 } -Возможно передача параметров через конфигурацию, она имеет приоритет перед указанными в аннотации параметрами: +Параметры можно передавать через конфигурацию; она имеет приоритет над значениями из аннотации: ===! ":fontawesome-brands-java: `Java`" @@ -337,7 +411,7 @@ agent: @Component public class SomeService { - @ScheduleOnce(config = "job") + @ScheduleOnce(config = "scheduling.jobs.once") void schedule() { // do something } @@ -350,42 +424,87 @@ agent: @Component class SomeService { - @ScheduleOnce(config = "job") + @ScheduleOnce(config = "scheduling.jobs.once") fun schedule() { // do something } } ``` -Пример конфигурации через файл: +Пример файла конфигурации: ===! ":material-code-json: `Hocon`" ```javascript - job { - delay = "50ms" //(1)! + scheduling { + jobs { + once { + delay = "50ms" //(1)! + } + } } ``` - 1. Начальный интервал задержки перед задачей + 1. Задержка перед задачей (`обязательный`, без значения по умолчанию) === ":simple-yaml: `YAML`" ```yaml - job: - delay: "50ms" #(1)! + scheduling: + jobs: + once: + delay: "50ms" #(1)! + ``` + + 1. Задержка перед задачей (`обязательный`, без значения по умолчанию) + +### Плавная остановка { #graceful-shutdown } + +Во время [плавной остановки](container.md#component-lifecycle) собственный планировщик ожидает завершения задач в течение `scheduling.shutdownWait`. +Если задачу нужно остановить раньше, проверяйте [Thread.currentThread().isInterrupted()](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#isInterrupted--) и останавливайте работу вручную. + +### Программное планирование { #programmatic } + +Для планирования задач в императивном стиле можно внедрить компонент `JdkSchedulingExecutor`. +Он оборачивает тот же `ScheduledExecutorService`, что и аннотации, и предоставляет методы `scheduleAtFixedRate`, `scheduleWithFixedDelay` и `schedule`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { + + private final JdkSchedulingExecutor executor; + + public SomeService(JdkSchedulingExecutor executor) { + this.executor = executor; + } + + public void start() { + executor.scheduleAtFixedRate(() -> { + // do something + }, 50, 50, TimeUnit.MILLISECONDS); + } + } ``` - 1. Начальный интервал задержки перед задачей +=== ":simple-kotlin: `Kotlin`" -### Штатное завершение { #graceful-shutdown } + ```kotlin + @Component + class SomeService(private val executor: JdkSchedulingExecutor) { -Если вы хотите предварительно завершить обработку при [штатном завершении](container.md#component-lifecycle) сервиса не дожидаясь ее окончания, -требуется проверять [Thread.currentThread().isInterrupted()](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#isInterrupted--) статус и прекращать работу самостоятельно. + fun start() { + executor.scheduleAtFixedRate({ + // do something + }, 50, 50, TimeUnit.MILLISECONDS) + } + } + ``` ## Quartz { #quartz } -Реализация на основе библиотеки [Quartz](https://www.baeldung.com/quartz) как планировщика для создания аспектов. +Реализация на основе библиотеки [Quartz](https://www.quartz-scheduler.org/) используется для задач с расписанием по `cron`, пользовательских экземпляров `Trigger` и правил выполнения `Quartz`. ### Подключение { #dependency-2 } @@ -405,7 +524,7 @@ agent: === ":simple-kotlin: `Kotlin`" [Зависимость](general.md#dependencies) `build.gradle.kts`: - ```groovy + ```kotlin implementation("ru.tinkoff.kora:scheduling-quartz") ``` @@ -417,7 +536,9 @@ agent: ### Конфигурация { #configuration-5 } -Конфигурация указывается как значения [Properties](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/configuration/) в формате ключ и значение: +Конфигурация `Quartz` задаётся значениями [Properties](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/configuration/) в формате ключ-значение. +Настройки Kora для плавной остановки и телеметрии задаются в секции `scheduling`. +Конфигурация конкретной `cron`-задачи также может содержать секцию `telemetry`, значения которой переопределяют общую телеметрию планировщика для этой задачи. ===! ":material-code-json: `Hocon`" @@ -434,20 +555,30 @@ agent: metrics { enabled = true //(4)! slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(5)! + tags = { // (6)! + "key1" = "value1" + "key2" = "value2" + } } tracing { - enabled = true //(6)! + enabled = true //(7)! + attributes = { // (8)! + "key1" = "value1" + "key2" = "value2" + } } } } ``` - 1. Параметры настройки Quartz планировщика - 2. Ожидать ли выполнения задач перед выключением планировщика в случае [штатного завершения](container.md#component-lifecycle) - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Включает метрики модуля (по умолчанию `true`) - 5. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 6. Включает трассировку модуля (по умолчанию `true`) + 1. Параметры конфигурации планировщика `Quartz` (по умолчанию используются свойства из `quartz.properties` ниже) + 2. Ожидать ли завершения задач перед остановкой планировщика при [плавной остановке](container.md#component-lifecycle) (по умолчанию: `true`) + 3. Включает логирование модуля (по умолчанию: `false`) + 4. Включает метрики модуля (по умолчанию: `true`) + 5. Настраивает [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Настраивает теги метрик (по умолчанию: `{}`) + 7. Включает трассировку модуля (по умолчанию: `true`) + 8. Настраивает атрибуты трассировки (по умолчанию: `{}`) === ":simple-yaml: `YAML`" @@ -461,19 +592,27 @@ agent: enabled: false #(3)! metrics: enabled: true #(4)! - slo: [ 3, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! - telemetry: - enabled: true #(6)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! + tags: #(6)! + key1: value1 + key2: value2 + tracing: + enabled: true #(7)! + attributes: #(8)! + key1: value1 + key2: value2 ``` - 1. Параметры настройки Quartz планировщика - 2. Ожидать ли выполнения задач перед выключением планировщика в случае [штатного завершения](container.md#component-lifecycle) - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Включает метрики модуля (по умолчанию `true`) - 5. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 6. Включает трассировку модуля (по умолчанию `true`) + 1. Параметры конфигурации планировщика `Quartz` (по умолчанию используются свойства из `quartz.properties` ниже) + 2. Ожидать ли завершения задач перед остановкой планировщика при [плавной остановке](container.md#component-lifecycle) (по умолчанию: `true`) + 3. Включает логирование модуля (по умолчанию: `false`) + 4. Включает метрики модуля (по умолчанию: `true`) + 5. Настраивает [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрик (по умолчанию: `ru.tinkoff.kora.telemetry.common.TelemetryConfig.MetricsConfig#DEFAULT_SLO`) + 6. Настраивает теги метрик (по умолчанию: `{}`) + 7. Включает трассировку модуля (по умолчанию: `true`) + 8. Настраивает атрибуты трассировки (по умолчанию: `{}`) -По умолчанию используются настройки из: +Настройки по умолчанию используются из: ??? abstract "quartz.properties" @@ -493,11 +632,43 @@ agent: org.quartz.jobStore.class: org.quartz.simpl.RAMJobStore ``` -### Крон { #cron } +### Cron { #cron } + +Для запуска задач по расписанию используются [`cron`-выражения](http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html). + +Выражение `Quartz` состоит из шести обязательных полей и необязательного седьмого поля года, разделённых пробелами: + +| Поле | Допустимые значения | Обязательное | +|--------------|----------------------|--------------| +| Секунды | `0-59` | да | +| Минуты | `0-59` | да | +| Часы | `0-23` | да | +| День месяца | `1-31` | да | +| Месяц | `1-12` или `JAN-DEC` | да | +| День недели | `1-7` или `SUN-SAT` | да | +| Год | пусто, `1970-2099` | нет | -Использование [Cron](http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html) выражений для запуска запланированных задач. +Помимо обычных чисел, диапазонов (`8-10`), списков (`6,19`) и шагов (`0/30`), поддерживаются следующие специальные символы: -Запускает одиножды задачу через определенный фиксированный интервал времени. +| Символ | Значение | +|--------|----------------------------------------------------------------------------------------------------| +| `*` | Все значения поля (например, `*` в поле минут означает «каждую минуту») | +| `?` | Без конкретного значения, используется в поле дня месяца или дня недели, когда указано другое из них | +| `L` | Последний (последний день месяца или последний указанный день недели в месяце) | +| `W` | Ближайший будний день к указанному дню месяца | +| `#` | N-й указанный день недели в месяце, например `5#2` — это вторая пятница | + +Примеры выражений: + +| Выражение | Значение | +|---------------------|---------------------------------------------| +| `0 0 * * * ?` | В начале каждого часа каждого дня | +| `*/10 * * * * ?` | Каждые десять секунд | +| `0 0 8-10 * * ?` | В 8, 9 и 10 часов каждого дня | +| `0 0/30 8-10 * * ?` | В 8:00, 8:30, 9:00, 9:30, 10:00 и 10:30 | +| `0 0 0 L * ?` | В последний день месяца в полночь | +| `0 0 0 1W * ?` | В первый будний день месяца в полночь | +| `0 0 0 ? * 5#2` | Во вторую пятницу месяца в полночь | ===! ":fontawesome-brands-java: `Java`" @@ -505,14 +676,14 @@ agent: @Component public class SomeService { - @ScheduleWithCron("0 0 * * * * ?") //(1)! + @ScheduleWithCron("* * * ? * * *") //(1)! void schedule() { // do something } } ``` - 1. Cron выражение говорящее запускать задачу каждый час и каждый день + 1. `cron`-выражение, которое запускает задачу каждую секунду === ":simple-kotlin: `Kotlin`" @@ -520,18 +691,53 @@ agent: @Component class SomeService { - @ScheduleWithCron("0 0 * * * * ?") //(1)! + @ScheduleWithCron("* * * ? * * *") //(1)! fun schedule() { // do something } } ``` - 1. Cron выражение говорящее запускать задачу каждый час и каждый день + 1. `cron`-выражение, которое запускает задачу каждую секунду + +Атрибут `identity` задаёт [идентичность Quartz Trigger](https://www.quartz-scheduler.org/api/2.3.0/org/quartz/TriggerBuilder.html), +используемую для именования задачи, что полезно для идентификации и замены задач, особенно с кластерными или персистентными реализациями `JobStore`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { + + @ScheduleWithCron(value = "0 0 * * * ?", identity = "my-hourly-job") //(1)! + void schedule() { + // do something + } + } + ``` + + 1. `cron`-выражение, которое запускает задачу в начале каждого часа, зарегистрированное под идентичностью триггера `my-hourly-job` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService { + + @ScheduleWithCron(value = "0 0 * * * ?", identity = "my-hourly-job") //(1)! + fun schedule() { + // do something + } + } + ``` + + 1. `cron`-выражение, которое запускает задачу в начале каждого часа, зарегистрированное под идентичностью триггера `my-hourly-job` #### Конфигурация { #configuration-6 } -Возможно передача параметров через конфигурацию, она имеет приоритет перед указанными в аннотации параметрами: +Параметры можно передавать через конфигурацию; конфигурация имеет приоритет над значениями из аннотации. +Как и в случае собственного планировщика, путь `config` произвольный и по соглашению вкладывается в секцию `scheduling` +(как в [проекте-примере](https://github.com/kora-projects/kora-examples), `scheduling.jobs.quartz`): ===! ":fontawesome-brands-java: `Java`" @@ -539,7 +745,7 @@ agent: @Component public class SomeService { - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") void schedule() { // do something } @@ -552,7 +758,7 @@ agent: @Component class SomeService { - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") fun schedule() { // do something } @@ -564,25 +770,31 @@ agent: ===! ":material-code-json: `Hocon`" ```javascript - job { - cron = "0 0 * * * * ?" //(1)! + scheduling { + jobs { + quartz { + cron = "* * * ? * * *" //(1)! + } + } } ``` - 1. Cron выражение говорящее запускать задачу каждый час и каждый день + 1. `cron`-выражение, которое запускает задачу каждую секунду (`обязательный`, без значения по умолчанию) === ":simple-yaml: `YAML`" ```yaml - job: - cron: "0 0 * * * * ?" #(1)! + scheduling: + jobs: + quartz: + cron: "* * * ? * * *" #(1)! ``` - 1. Cron выражение говорящее запускать задачу каждый час и каждый день + 1. `cron`-выражение, которое запускает задачу каждую секунду (`обязательный`, без значения по умолчанию) -### Триггер { #trigger } +### Trigger { #trigger } -Предполагает создание собственного `триггера` на основе библиотеки Quartz и регистрация его в контейнере приложения с определенным тегом и последующее его использование через аннотацию. +Для пользовательского расписания можно создать `Trigger` из библиотеки `Quartz`, зарегистрировать его в графе зависимостей с тегом, а затем использовать этот тег в аннотации `@ScheduleWithTrigger`. ===! ":fontawesome-brands-java: `Java`" @@ -612,8 +824,8 @@ agent: } ``` - 1. Тег триггера - 2. Тег триггера + 1. Тег, используемый для регистрации `Trigger` в графе зависимостей. + 2. Тот же тег, используемый задачей для получения `Trigger`. === ":simple-kotlin: `Kotlin`" @@ -638,19 +850,20 @@ agent: @Component class SomeService { - @ScheduleWithTrigger(@Tag(SomeService.class)) //(2)! + @ScheduleWithTrigger(@Tag(SomeService::class)) //(2)! fun schedule() { // do something } } ``` - 1. Тег триггера - 2. Тег триггера + 1. Тег, используемый для регистрации `Trigger` в графе зависимостей. + 2. Тот же тег, используемый задачей для получения `Trigger`. -### Неконкурентный запуск { #non-concurrent-execution } +### Неконкурентное выполнение { #non-concurrent-execution } -Аннотация, которая говорит что метод с аннотацией не должен выполняться параллельно. +Аннотация `@DisallowConcurrentExecution` предотвращает одновременное выполнение одного и того же метода планировщиком `Quartz`. +Это аналог `org.quartz.DisallowConcurrentExecution` в `Kora`, который можно разместить на любом методе, помеченном `@Schedule*`. ===! ":fontawesome-brands-java: `Java`" @@ -659,7 +872,7 @@ agent: public class SomeService { @DisallowConcurrentExecution - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") void schedule() { // do something } @@ -673,20 +886,56 @@ agent: class SomeService { @DisallowConcurrentExecution - @ScheduleWithCron(config = "job") + @ScheduleWithCron(config = "scheduling.jobs.quartz") fun schedule() { // do something } } ``` -### Неизменное выполнение { #persistent-execution } +### Контекст задачи { #job-context } + +Метод, выполняемый по расписанию `Quartz`, может опционально объявить единственный аргумент `org.quartz.JobExecutionContext`. +Если он присутствует, `Kora` передаёт методу текущий контекст выполнения; если отсутствует, метод вызывается без аргументов. +Контекст даёт доступ к `org.quartz.JobDataMap` задачи, что является способом чтения и записи состояния, связанного с задачей: -Аннотация, которая говорит обновить принудительно `org.quartz.JobDataMap` обновить во время выполнения и требует, -чтобы планировщик повторно сохранил `org.quartz.JobDataMap` по завершении выполнения. +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { -Рекомендуется использовать совместно с аннотацией `@DisallowConcurrentExecution`, -чтобы избежать конфликтов при хранении данных при одновременном выполнении задач. + @ScheduleWithCron(config = "scheduling.jobs.quartz") + void schedule(JobExecutionContext context) { + JobDataMap data = context.getJobDetail().getJobDataMap(); + int counter = data.containsKey("counter") ? data.getInt("counter") : 0; + data.put("counter", counter + 1); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService { + + @ScheduleWithCron(config = "scheduling.jobs.quartz") + fun schedule(context: JobExecutionContext) { + val data = context.jobDetail.jobDataMap + val counter = if (data.containsKey("counter")) data.getInt("counter") else 0 + data.put("counter", counter + 1) + } + } + ``` + +### Сохранение данных задачи { #persistent-execution } + +Аннотация `@PersistJobDataAfterExecution` указывает `Quartz` сохранять обновлённый `org.quartz.JobDataMap` после выполнения задачи, +чтобы изменения, внесённые через [JobExecutionContext](#job-context), были видны при следующем выполнении. + +Её рекомендуется использовать вместе с `@DisallowConcurrentExecution`, +чтобы избежать конфликтов при сохранении данных во время одновременного выполнения задачи. ===! ":fontawesome-brands-java: `Java`" @@ -694,24 +943,67 @@ agent: @Component public class SomeService { + @DisallowConcurrentExecution @PersistJobDataAfterExecution - @ScheduleWithCron(config = "job") - void schedule() { - // do something + @ScheduleWithCron(config = "scheduling.jobs.quartz") + void schedule(JobExecutionContext context) { + JobDataMap data = context.getJobDetail().getJobDataMap(); + int counter = data.containsKey("counter") ? data.getInt("counter") : 0; + data.put("counter", counter + 1); //(1)! } } ``` + 1. Обновлённое значение сохраняется после выполнения и доступно при следующем запуске + === ":simple-kotlin: `Kotlin`" ```kotlin @Component class SomeService { + @DisallowConcurrentExecution @PersistJobDataAfterExecution - @ScheduleWithCron(config = "job") - fun schedule() { - // do something + @ScheduleWithCron(config = "scheduling.jobs.quartz") + fun schedule(context: JobExecutionContext) { + val data = context.jobDetail.jobDataMap + val counter = if (data.containsKey("counter")) data.getInt("counter") else 0 + data.put("counter", counter + 1) //(1)! + } + } + ``` + + 1. Обновлённое значение сохраняется после выполнения и доступно при следующем запуске + +### Плавная остановка { #graceful-shutdown-quartz } + +Во время [плавной остановки](container.md#component-lifecycle) параметр `scheduling.waitForJobComplete` управляет тем, как останавливается планировщик `Quartz`. +При `true` (по умолчанию) он вызывает `scheduler.shutdown(true)` и блокируется до завершения выполняющихся задач; при `false` он останавливается без ожидания. +Как и в случае собственного планировщика, длительно выполняющиеся задачи всё же должны кооперативно проверять +[Thread.currentThread().isInterrupted()](https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#isInterrupted--) и останавливать работу вручную. + +### Scheduler { #scheduler } + +Лежащий в основе `org.quartz.Scheduler` регистрируется как компонент и может быть внедрён для продвинутых сценариев, +таких как программная регистрация задач или инспекция состояния планировщика: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class SomeService { + + private final Scheduler scheduler; + + public SomeService(Scheduler scheduler) { + this.scheduler = scheduler; } } ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val scheduler: Scheduler) + ``` diff --git a/mkdocs/docs/ru/documentation/soap-client.md b/mkdocs/docs/ru/documentation/soap-client.md index 25fef92..d85ad5e 100644 --- a/mkdocs/docs/ru/documentation/soap-client.md +++ b/mkdocs/docs/ru/documentation/soap-client.md @@ -1,10 +1,14 @@ --- -description: "Explains Kora SOAP client setup, SOAP client configuration, usage patterns, generated clients, and wsdl2java Gradle plugin integration. Use when working with SoapClientModule, @SoapClient, wsdl2java, JAX-WS, SOAPAction, WebServiceClient." +description: "Explains Kora SOAP client setup, configuration, usage, generated clients, request customization and WS-Security, exception handling, testing, and the wsdl2java Gradle plugin. Use when working with SoapClientModule, wsdl2java, JAX-WS, SOAPAction, WebServiceClient, SoapFaultException, SoapEnvelopeProcessors." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora SOAP client setup, SOAP client configuration, usage patterns, generated clients, and wsdl2java Gradle plugin integration; key triggers include SoapClientModule, @SoapClient, wsdl2java, JAX-WS, SOAPAction, WebServiceClient." + use_when: "Use this file for Kora docs or implementation questions about Kora SOAP client setup, configuration, usage patterns, generated clients, envelope processors and WS-Security authorization, body-mapper logging, exception handling, testing, and wsdl2java Gradle plugin integration; key triggers include SoapClientModule, SoapServiceConfig, wsdl2java, JAX-WS, SOAPAction, WebServiceClient, SoapException, SoapFaultException, InvalidHttpResponseSoapException, SoapEnvelopeProcessors, wssAuth." --- -Модуль для создания и регистрации SOAP сервисов по классам аннотированным `javax.jws.WebService`/`jakarta.jws.WebService`. +`SOAP` — это протокол обмена сообщениями в формате `XML`, который часто используется для интеграции с внешними системами по `HTTP` и контракту `WSDL`. +Модуль `soap-client` создаёт реализации клиентов для интерфейсов, помеченных аннотацией `javax.jws.WebService` или `jakarta.jws.WebService`, и регистрирует их в графе приложения. + +Обычно такие интерфейсы и связанные с ними классы `JAXB` генерируются из `WSDL`, например с помощью `wsdl2java`. +После генерации Kora создаёт реализацию клиента и подключает её к `HTTP-клиенту`, преобразованию `XML` и телеметрии. ## Подключение { #dependency } @@ -34,26 +38,83 @@ agent: interface Application : SoapClientModule ``` -**Требуется** подключить реализацию [HTTP клиента](http-client.md). +**Требует** наличия в приложении реализации [`HTTP-клиента`](http-client.md) (например `http-client-jdk` или `http-client-ok`) +и модуля конфигурации ([HOCON](config.md#hocon) или [YAML](config.md#yaml)). + +Когда интерфейсы `SOAP` и классы `JAXB` генерируются [плагином `wsdl2java`](#wsdl2java-plugin) в режиме `jakarta`, +необходимая среда выполнения `jakarta.*` / `JAXB` уже предоставляется сгенерированными исходниками и `JDK`. +В этом случае транзитивные зависимости `jakarta` / `Glassfish` / `activation` модуля `soap-client` можно исключить, чтобы избежать конфликтов версий: + +===! ":fontawesome-brands-java: `Java`" + + `build.gradle`: + ```groovy + implementation("ru.tinkoff.kora:soap-client") { + exclude group: "jakarta.xml" + exclude group: "jakarta.jws" + exclude group: "jakarta.xml.ws" + exclude group: "jakarta.xml.bind" + exclude group: "org.glassfish.jaxb" + exclude group: "com.sun.activation" + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + `build.gradle.kts`: + ```groovy + implementation("ru.tinkoff.kora:soap-client") { + exclude(group = "jakarta.xml") + exclude(group = "jakarta.jws") + exclude(group = "jakarta.xml.ws") + exclude(group = "jakarta.xml.bind") + exclude(group = "org.glassfish.jaxb") + exclude(group = "com.sun.activation") + } + ``` ## Описание { #description } -Подразумевается что у нас есть классы аннотированные `javax.jws.WebService`/`jakarta.jws.WebService`, которые могут быть созданы другими средствами, -такими как [Gradle Plugin](#wsdl2java-plugin). +Предполагается, что в приложении уже есть интерфейсы, помеченные аннотацией `javax.jws.WebService` или `jakarta.jws.WebService` +(поддерживаются оба семейства аннотаций). Их можно написать вручную, но обычно они создаются из `WSDL` +отдельным инструментом, например [Gradle-плагином](#wsdl2java-plugin). -На основании таких классов с помощью Kora создаются реализации SOAP клиента с суффиксом Impl в том же пакете и регистрирует их как модуль с конфигурацей. +На основе таких интерфейсов процессор аннотаций (входящий в артефакт `annotation-processors`) создаёт в том же пакете: -Затем конфигурация и сам SOAP сервис становятся доступны для внедрения зависимостей автоматически. +- Реализацию клиента с именем `$_SoapClientImpl`, зарегистрированную как `@DefaultComponent` в графе приложения. +- Модуль с именем `$_SoapClientModule`, помеченный аннотацией `@Module`, который регистрирует `SoapServiceConfig` + (с тегом `@Tag(.class)`) и сам клиент. + +После этого конфигурация и `SOAP-клиент` автоматически становятся доступны для внедрения зависимостей. + +### Как это работает { #how-it-works } + +Во время работы сгенерированный клиент использует подключённый `HttpClient` и ведёт себя следующим образом: + +- Отправляет запрос `HTTP POST` с `Content-Type: text/xml` на адрес из параметра конфигурации `url`. +- Добавляет `HTTP`-заголовок `SOAPAction` только когда в аннотации `@WebMethod` метода задан `action`. +- Применяет значение конфигурации `timeout` как предельное время выполнения запроса. +- Трактует `HTTP 200` как успешный ответ и десериализует тело в тип возвращаемого значения метода. +- Трактует `HTTP 500` как `SOAP Fault` и преобразует его либо в [типизированное исключение ошибки WSDL](#exception-handling), либо в `SoapFaultException`. +- Выбрасывает `InvalidHttpResponseSoapException` для любого другого кода состояния `HTTP`. +- Автоматически разбирает ответы `multipart` (вложения `XOP` / `MTOM`). +- Для каждого `@WebMethod` генерирует синхронный метод и метод `Async`, возвращающий `CompletionStage` для [неблокирующих вызовов](#asynchronous). ## Конфигурация { #configuration } -Все конфигурации для SOAP клиентов создаются с префиксом `soapClient`, -а основная часть конфигурации клиента находится под именем клиента из WSDL аннотации `@WebService`, -который соответствует зачастую тегу `` в конфигурации WSDL. +Все конфигурации `SOAP-клиентов` создаются с префиксом `soapClient`. +Основная часть конфигурации клиента размещается под именем службы из аннотации `@WebService`. -Сервис SOAP с именем `SimpleService` будет иметь конфигурацию с путем `soapClient.SimpleService`. +Имя секции выбирается в следующем порядке: -Пример полной конфигурации, описанной в классе `SoapServiceConfig` (указаны примеры значений или значения по умолчанию): +1. `name` из `@WebService` +2. `serviceName` из `@WebService` +3. `portName` из `@WebService` +4. имя интерфейса + +У `SOAP-клиента` с именем `SimpleService` путь конфигурации будет `soapClient.SimpleService`. + +Основные параметры конфигурации: ===! ":material-code-json: `Hocon`" @@ -62,38 +123,12 @@ agent: SimpleService { url = "https://localhost:8090" //(1)! timeout = "60s" //(2)! - telemetry { - logging { - enabled = false //(3)! - } - metrics { - enabled = true //(4)! - slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(5)! - tags = { // (6)! - "key1" = "value1" - "key2" = "value2" - } - } - tracing { - enabled = true //(7)! - attributes = { // (8)! - "key1" = "value1" - "key2" = "value2" - } - } - } } } ``` - 1. URL сервиса куда будут отправляться запросы (**обязательный**) - 2. Максимальное время запроса - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Включает метрики модуля (по умолчанию `true`) - 5. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 6. Настройка тегов для метрик (опционально) - 7. Включает трассировку модуля (по умолчанию `true`) - 8. Настройка атрибутов для трассировки (опционально) + 1. `URL` службы, куда будут отправляться запросы (`обязательный`, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию не указано, опционально). === ":simple-yaml: `YAML`" @@ -102,36 +137,100 @@ agent: SimpleService: url: "https://localhost:8090" #(1)! timeout: "60s" #(2)! - telemetry: - logging: - enabled: false #(3)! - metrics: - enabled: true #(4)! - slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! - tags: #(6)! - key1: value1 - key2: value2 - tracing: - enabled: true #(7)! - attributes: #(8)! - key1: value1 - key2: value2 - ``` - - 1. URL сервиса куда будут отправляться запросы (**обязательный**) - 2. Максимальное время запроса - 3. Включает логгирование модуля (по умолчанию `false`) - 4. Включает метрики модуля (по умолчанию `true`) - 5. Настройка [SLO](https://www.atlassian.com/ru/incident-management/kpis/sla-vs-slo-vs-sli) для [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) метрики - 6. Настройка тегов для метрик (опционально) - 7. Включает трассировку модуля (по умолчанию `true`) - 8. Настройка атрибутов для трассировки (опционально) - -Предоставляемые метрики модуля описаны в разделе [Справочник метрик](metrics.md#soap-client). + ``` + + 1. `URL` службы, куда будут отправляться запросы (`обязательный`, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию не указано, опционально). + +??? note "Полная конфигурация" + + Пример полной конфигурации, описанной классом `SoapServiceConfig`: + + ===! ":material-code-json: `Hocon`" + + ```javascript + soapClient { + SimpleService { + url = "https://localhost:8090" //(1)! + timeout = "60s" //(2)! + telemetry { + logging { + enabled = false //(3)! + } + metrics { + enabled = true //(4)! + slo = [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] //(5)! + tags = { // (6)! + "key1" = "value1" + "key2" = "value2" + } + } + tracing { + enabled = true //(7)! + attributes = { // (8)! + "key1" = "value1" + "key2" = "value2" + } + } + } + } + } + ``` + + 1. `URL` службы, куда будут отправляться запросы (обязательная, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию: `60s`). + 3. Включает логирование модуля (по умолчанию: `false`). + 4. Включает метрики модуля (по умолчанию: `true`). + 5. Настройка [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 6. Дополнительные теги для метрик (по умолчанию: `{}`). + 7. Включает трассировку модуля (по умолчанию: `true`). + 8. Дополнительные атрибуты для трассировки (по умолчанию: `{}`). + + === ":simple-yaml: `YAML`" + + ```yaml + soapClient: + SimpleService: + url: "https://localhost:8090" #(1)! + timeout: "60s" #(2)! + telemetry: + logging: + enabled: false #(3)! + metrics: + enabled: true #(4)! + slo: [ 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 30000, 60000, 90000 ] #(5)! + tags: #(6)! + key1: value1 + key2: value2 + tracing: + enabled: true #(7)! + attributes: #(8)! + key1: value1 + key2: value2 + ``` + + 1. `URL` службы, куда будут отправляться запросы (обязательная, по умолчанию не указано). + 2. Максимальное время выполнения запроса (по умолчанию: `60s`). + 3. Включает логирование модуля (по умолчанию: `false`). + 4. Включает метрики модуля (по умолчанию: `true`). + 5. Настройка [SLO](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) для метрики [DistributionSummary](https://github.com/micrometer-metrics/micrometer-docs/blob/main/src/docs/concepts/distribution-summaries.adoc) (по умолчанию: `TelemetryConfig.MetricsConfig.DEFAULT_SLO`). + 6. Дополнительные теги для метрик (по умолчанию: `{}`). + 7. Включает трассировку модуля (по умолчанию: `true`). + 8. Дополнительные атрибуты для трассировки (по умолчанию: `{}`). + +Метрики модуля описаны в разделе [Справочник метрик](metrics.md#soap-client). + +Конфигурация описывается интерфейсом `SoapServiceConfig`. Параметр `url` **обязателен**: +если он отсутствует в конфигурации, граф приложения не собирается и выбрасывается `ConfigValueExtractionException` +(отсутствие значения после разбора). Параметр `timeout` по умолчанию равен `60s`. + +Конфигурация регистрируется в графе под `@Tag(.class)`, поэтому при +[ручном создании](#request-customization) клиента зависимость `SoapServiceConfig` должна разрешаться с тем же тегом. ## Использование { #usage } -После создания всех компонент созданный SOAP сервис становится доступен для внедрения, ниже показан пример для `SimpleService` сервиса: +После создания всех компонентов `SOAP-клиент` становится доступен для внедрения. +Ниже приведён пример для клиента `SimpleService`: ===! ":fontawesome-brands-java: `Java`" @@ -156,10 +255,392 @@ agent: } ``` -## Плагин wsdl2java { #wsdl2java-plugin } +### Вызов { #invocation } + +Сгенерированный метод принимает тип запроса и возвращает типизированный ответ. +Для клиента `SimpleService` с операцией `test`: + +===! ":fontawesome-brands-java: `Java`" -[Gradle Plugin](https://github.com/bjornvester/wsdl2java-gradle-plugin) может использоваться как один из вариантов для создания классов аннотированных `javax.jws.WebService`/`jakarta.jws.WebService` -на основании [WSDL](https://coderlessons.com/tutorials/xml-tekhnologii/uznaite-wsdl/wsdl-kratkoe-rukovodstvo). + ```java + @Component + public final class SomeService { + + private final SimpleService service; + + public SomeService(SimpleService service) { + this.service = service; + } + + public String call() throws Exception { + var request = new TestRequest(); + request.setVal1("1"); + request.setVal2("2"); + + TestResponse response = service.test(request); + return response.getVal1(); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val service: SimpleService) { + + fun call(): String? { + val request = TestRequest().apply { + val1 = "1" + val2 = "2" + } + + val response = service.test(request) + return response.val1 + } + } + ``` + +### Асинхронность { #asynchronous } + +Для каждого `@WebMethod` генератор также создаёт метод `Async`, возвращающий `CompletionStage` для неблокирующих вызовов. +Асинхронный метод объявляется в сгенерированном классе `$_SoapClientImpl`, а не в интерфейсе `WSDL`, +поэтому для его использования нужно привести внедрённый клиент к типу сгенерированной реализации: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class SomeService { + + private final SimpleService service; + + public SomeService(SimpleService service) { + this.service = service; + } + + public CompletionStage callAsync() { + var request = new TestRequest(); + request.setVal1("1"); + request.setVal2("2"); + + return (($SimpleService_SoapClientImpl) service).testAsync(request) + .thenApply(TestResponse::getVal1); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class SomeService(private val service: SimpleService) { + + fun callAsync(): CompletionStage { + val request = TestRequest().apply { + val1 = "1" + val2 = "2" + } + + return (service as `$SimpleService_SoapClientImpl`).testAsync(request) + .thenApply { it.val1 } + } + } + ``` + +## Настройка запроса { #request-customization } + +`SOAP`-клиенты не используют механизм `@InterceptWith` из [декларативных HTTP-клиентов](http-client.md#interceptors). +Вместо этого сгенерированный `$_SoapClientImpl` предоставляет **дополнительный конструктор**, принимающий обработчик +конверта `Function`. Обработчик применяется к конверту `SOAP` запроса +перед его сериализацией и отправкой — это точка расширения для добавления заголовков `SOAP` (авторизация, трассировка, +произвольные элементы) или иного преобразования исходящего конверта. + +У сгенерированной реализации два конструктора: + +- `(HttpClient, SoapClientTelemetryFactory, SoapServiceConfig)` — используется сгенерированным `@DefaultComponent`; применяет `Function.identity()` (без изменений). +- `(HttpClient, SoapClientTelemetryFactory, SoapServiceConfig, Function)` — позволяет задать собственный обработчик. + +Чтобы использовать собственный обработчик, зарегистрируйте свою фабрику, которая возвращает тип **интерфейса** клиента и создаёт +реализацию с обработчиком. Поскольку она предоставляет тот же тип интерфейса, ваша фабрика **переопределяет** сгенерированный +`@DefaultComponent`. Разрешайте `SoapServiceConfig` через `@Tag(.class)` — тег, под которым его регистрирует сгенерированный модуль: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Module + public interface SoapModule { + + default SimpleService simpleService(HttpClient httpClient, + SoapClientTelemetryFactory telemetryFactory, + @Tag(SimpleService.class) SoapServiceConfig config) { + var processor = SoapEnvelopeProcessors.wssAuth("username", "password"); //(1)! + try { + return new $SimpleService_SoapClientImpl(httpClient, telemetryFactory, config, processor); + } catch (Exception e) { + throw new IllegalStateException(e); + } + } + } + ``` + + 1. Здесь можно использовать любую `Function`; `SoapEnvelopeProcessors.wssAuth` — встроенная. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Module + interface SoapModule { + + fun simpleService(httpClient: HttpClient, + telemetryFactory: SoapClientTelemetryFactory, + @Tag(SimpleService::class) config: SoapServiceConfig): SimpleService { + val processor = SoapEnvelopeProcessors.wssAuth("username", "password") //(1)! + return `$SimpleService_SoapClientImpl`(httpClient, telemetryFactory, config, processor) + } + } + ``` + + 1. Здесь можно использовать любую `Function`; `SoapEnvelopeProcessors.wssAuth` — встроенная. + +Собственный обработчик может добавлять произвольные заголовки `SOAP`, дописывая их в `envelope.getHeader().getAny()`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + Function processor = envelope -> { + envelope.getHeader().getAny().add(myHeaderElement); // org.w3c.dom.Element или объект JAXB + return envelope; + }; + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val processor = Function { envelope -> + envelope.header.any.add(myHeaderElement) // org.w3c.dom.Element или объект JAXB + envelope + } + ``` + +### Авторизация { #authorization } + +`SoapEnvelopeProcessors.wssAuth(username, password)` — встроенный обработчик, который добавляет в каждый конверт запроса заголовок +`UsernameToken` стандарта [WS-Security](https://en.wikipedia.org/wiki/WS-Security) (`Username` и `Password` в открытом виде). +Подключите его ровно так, как показано выше, передав его в качестве обработчика конверта в конструктор клиента. + +## Логирование { #logging } + +Когда `telemetry.logging.enabled` равно `true`, клиент логирует полные конверты `SOAP` запроса и ответа (тела `XML`). +Чтобы замаскировать или преобразовать логируемые данные (например, скрыть чувствительные данные), переопределите компонент `SoapClientLogger.SoapClientLoggerBodyMapper`. +`SoapClientModule` предоставляет его как `@DefaultComponent`, поэтому пользовательская реализация `@Component` заменяет его: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class MaskingBodyMapper implements SoapClientLogger.SoapClientLoggerBodyMapper { + + @Override + public String mapRequest(String serviceName, String soapMethod, byte[] requestAsBytes) { + return ""; + } + + @Override + public String mapResponseSuccess(String serviceName, String soapMethod, byte[] responseAsBytes) { + return new String(responseAsBytes, StandardCharsets.UTF_8); + } + + @Override + public String mapResponseFailure(String serviceName, String soapMethod, byte[] responseAsBytes) { + return new String(responseAsBytes, StandardCharsets.UTF_8); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + class MaskingBodyMapper : SoapClientLogger.SoapClientLoggerBodyMapper { + + override fun mapRequest(serviceName: String, soapMethod: String, requestAsBytes: ByteArray): String { + return "" + } + + override fun mapResponseSuccess(serviceName: String, soapMethod: String, responseAsBytes: ByteArray): String { + return String(responseAsBytes, StandardCharsets.UTF_8) + } + + override fun mapResponseFailure(serviceName: String, soapMethod: String, responseAsBytes: ByteArray): String { + return String(responseAsBytes, StandardCharsets.UTF_8) + } + } + ``` + +## Обработка исключений { #exception-handling } + +Все ошибки `SOAP`-клиента являются непроверяемыми (unchecked). Транспортные и `HTTP`-ошибки наследуются от базового `SoapException` +(`RuntimeException`), поэтому их можно обработать одним `catch (SoapException e)` или перехватить конкретный подтип. +Исключения сериализации/десериализации `XML` наследуются напрямую от `RuntimeException` (а не от `SoapException`), поэтому их нужно +перехватывать отдельно. + +Основные типы исключений: + +- `SoapException` — базовое непроверяемое исключение (наследуется от `RuntimeException`) для транспортных и `HTTP`-ошибок `SOAP`-клиента. +- `SoapFaultException` (наследуется от `SoapException`) — сервер вернул `SOAP Fault`, который не соответствует типизированной ошибке `WSDL`. `getFault()` возвращает `SoapFault`, предоставляющий `getFaultcode()` (`QName`), `getFaultstring()`, `getFaultactor()` и `getDetail()`. +- `InvalidHttpResponseSoapException` (наследуется от `SoapException`) — сервер вернул неожиданный код состояния `HTTP` (любой, кроме `200` или `500`). +- `SoapRequestMarshallingException` (наследуется от `RuntimeException`, а **не** от `SoapException`) — конверт запроса не удалось сериализовать в `XML`. +- `SoapResponseUnmarshallingException` (наследуется от `RuntimeException`, а **не** от `SoapException`) — `XML` ответа не удалось десериализовать. + +Когда операция `WSDL` объявляет ошибки (``), генератор создаёт типизированные проверяемые исключения, помеченные аннотацией `@WebFault`, +и метод выбрасывает их напрямую, когда `detail` возвращённой ошибки соответствует одному из них. Если ошибка не соответствует ни одному +объявленному типу, вместо этого выбрасывается `SoapFaultException`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + try { + var response = service.test(request); + // ... использование ответа + } catch (MyServiceFault e) { //(1)! + // обработка конкретной объявленной ошибки WSDL + } catch (SoapFaultException e) { //(2)! + SoapFault fault = e.getFault(); + var code = fault.getFaultcode(); + var message = fault.getFaultstring(); + } catch (InvalidHttpResponseSoapException e) { + // неожиданный код состояния HTTP + } catch (SoapException e) { + // любая другая транспортная/HTTP-ошибка SOAP + } catch (SoapRequestMarshallingException | SoapResponseUnmarshallingException e) { + // ошибка (де)сериализации XML — наследуется от RuntimeException, а не от SoapException + } + ``` + + 1. Типизированное исключение `@WebFault`, сгенерированное из ``; конкретное имя класса берётся из `WSDL`. + 2. Любой `SOAP Fault`, не соответствующий объявленной типизированной ошибке. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + try { + val response = service.test(request) + // ... использование ответа + } catch (e: MyServiceFault) { //(1)! + // обработка конкретной объявленной ошибки WSDL + } catch (e: SoapFaultException) { //(2)! + val fault = e.fault + val code = fault.faultcode + val message = fault.faultstring + } catch (e: InvalidHttpResponseSoapException) { + // неожиданный код состояния HTTP + } catch (e: SoapException) { + // любая другая транспортная/HTTP-ошибка SOAP + } catch (e: SoapRequestMarshallingException) { + // ошибка сериализации XML запроса — наследуется от RuntimeException, а не от SoapException + } catch (e: SoapResponseUnmarshallingException) { + // ошибка десериализации XML ответа — наследуется от RuntimeException, а не от SoapException + } + ``` + + 1. Типизированное исключение `@WebFault`, сгенерированное из ``; конкретное имя класса берётся из `WSDL`. + 2. Любой `SOAP Fault`, не соответствующий объявленной типизированной ошибке. + +### Низкоуровневая модель результата { #result-model } + +Внутри движок запросов `SoapRequestExecutor` возвращает `SoapResult` — запечатанный (sealed) интерфейс с двумя record: +`SoapResult.Success(Object body)` и `SoapResult.Failure(SoapFault fault, String faultMessage)`. +Сгенерированный клиент отображает `Success` в типизированный ответ, а `Failure` — в типизированное исключение ошибки или `SoapFaultException`, +поэтому обычно с `SoapResult` напрямую работать не приходится. + +## Тестирование { #testing } + +Клиент можно протестировать с помощью [`@KoraAppTest`](junit5.md), внедрив его как `@TestComponent` и указав в `url` адрес мок-сервера. +В примере ниже `SOAP_CLIENT_URL` переопределяется на адрес мок-сервера, вызывается `service.test(request)` и проверяется типизированный ответ: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraAppTest(Application.class) + class SimpleServiceTests implements KoraAppTestConfigModifier { + + @TestComponent + private SimpleService service; + + @Override + public KoraConfigModification config() { + return KoraConfigModification.ofSystemProperty("SOAP_CLIENT_URL", "http://localhost:8080"); + } + + @Test + void testCall() throws Exception { + // мок-сервер отвечает конвертом TestResponse на запрос ниже + var request = new TestRequest(); + request.setVal1("1"); + request.setVal2("2"); + + var response = service.test(request); + assertEquals("1", response.getVal1()); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraAppTest(Application::class) + class SimpleServiceTests : KoraAppTestConfigModifier { + + @TestComponent + lateinit var service: SimpleService + + override fun config(): KoraConfigModification = + KoraConfigModification.ofSystemProperty("SOAP_CLIENT_URL", "http://localhost:8080") + + @Test + fun testCall() { + // мок-сервер отвечает конвертом TestResponse на запрос ниже + val request = TestRequest().apply { + val1 = "1" + val2 = "2" + } + + val response = service.test(request) + assertEquals("1", response.val1) + } + } + ``` + +Конверт запроса, отправляемый на сервер, и конверт ответа, который он возвращает, в передаче по сети выглядят так: + +```xml + + + + + + 1 + 2 + + + + + + + + + + 1 + + + +``` + +## Плагин `wsdl2java` { #wsdl2java-plugin } + +[Gradle-плагин](https://github.com/bjornvester/wsdl2java-gradle-plugin) можно использовать как один из вариантов для создания интерфейсов, помеченных аннотацией `javax.jws.WebService` или `jakarta.jws.WebService`, +а также классов `JAXB` на основе `WSDL`. ### Подключение { #dependency-2 } @@ -183,7 +664,8 @@ agent: ### Использование { #usage-2 } -Предположим что у нас есть WSDL, где объявлен сервис `SimpleService` то настройка плагина для `jakarta` аннотацией будет выглядить так: +Предположим, есть `WSDL`, в котором объявлена служба `SimpleService`. +Тогда конфигурация плагина для генерации с аннотациями `jakarta` будет выглядеть так: ===! ":fontawesome-brands-java: `Java`" @@ -225,3 +707,6 @@ agent: ) } ``` + +Опция `useJakarta = true` заставляет плагин генерировать интерфейсы с аннотациями `jakarta.jws`. +Установите её в `false` (или опустите), чтобы вместо этого генерировать аннотации `javax.jws` — процессор аннотаций поддерживает оба варианта. diff --git a/mkdocs/docs/ru/documentation/tracing.md b/mkdocs/docs/ru/documentation/tracing.md index bdd60aa..ca7a77b 100644 --- a/mkdocs/docs/ru/documentation/tracing.md +++ b/mkdocs/docs/ru/documentation/tracing.md @@ -1,17 +1,27 @@ --- -description: "Explains Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing. Use when working with TracingModule, OpenTelemetry, GrpcSender, OpentelemetryContext, Span, TraceContext, OTLP." +description: "Explains Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing. Use when working with OpentelemetryTracingModule, OpenTelemetry, OpentelemetryContext, Span, OTLP." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing; key triggers include TracingModule, OpenTelemetry, GrpcSender, OpentelemetryContext, Span, TraceContext, OTLP." + use_when: "Use this file for Kora docs or implementation questions about Kora OpenTelemetry tracing over gRPC and HTTP, tracing configuration, trace context propagation, synchronous tracing, and asynchronous tracing; key triggers include OpentelemetryTracingModule, OpenTelemetry, OpentelemetryContext, Span, OTLP." --- -Модуль для сбора трассировка приложения по стандарту [OpenTelemetry](https://opentelemetry.io/docs/what-is-opentelemetry/) -и экспорта трассировки по gRPC/HTTP в формате OTLP. +Трассировка помогает связать отдельные операции приложения в единую цепочку выполнения и понять, где запрос провел время или завершился ошибкой. +Kora использует [`OpenTelemetry`](https://opentelemetry.io/docs/what-is-opentelemetry/) для создания `Span`, хранения текущего контекста трассировки в `OpentelemetryContext` и экспорта данных в формате `OTLP`. -Если нужен пошаговый разбор перед справочным описанием, смотрите [Наблюдаемость](../guides/observability.md). +Текущий `Span` хранится в контексте Kora, поэтому его можно передавать между компонентами приложения и использовать при ручном создании вложенных `Span`. +Когда установлен `OpentelemetryContext`, Kora также добавляет `traceId` и `spanId` в `MDC`, чтобы эти идентификаторы появлялись в логах при использовании модуля логирования. + +Большинство `Span` создаются автоматически: модуль из коробки инструментирует HTTP-сервер и клиент, базу данных, потребителя и производителя `Kafka`, gRPC-сервер и клиент, а также другие подсистемы, +и распространяет контекст трассировки между сервисами по стандарту [W3C traceparent](https://www.w3.org/TR/trace-context/). + +Kora предоставляет два взаимоисключающих модуля экспортера, `OTLP/gRPC` и `OTLP/HTTP`; выберите ровно один в зависимости от протокола, который принимает ваш коллектор. +Любой из модулей экспортера транзитивно предоставляет базовую обвязку трассировки (`OpentelemetryTracingModule`) и автоматическую инструментацию (`OpentelemetryModule`), так что никакой другой зависимости для трассировки не требуется. + +Пошаговое руководство перед справочным описанием смотрите в разделе [Наблюдаемость](../guides/observability.md). ## gRPC { #grpc } -Модуль позволяет собирать трассировку с помощью [gRPC протокола](https://github.com/open-telemetry/oteps/blob/main/text/0035-opentelemetry-protocol.md#protocol-details) посредствам `GrpcSender`. +Модуль экспортирует данные трассировки в `OpenTelemetry Collector` через `OTLP/gRPC`. +Он строит `OtlpGrpcSpanExporter` поверх `BatchSpanProcessor`, а типичный адрес коллектора — `http://localhost:4317`. ===! ":fontawesome-brands-java: `Java`" @@ -41,7 +51,8 @@ agent: ## HTTP { #http } -Модуль позволяет собирать трассировку с помощью [HTTP протокола](https://github.com/open-telemetry/oteps/blob/main/text/0099-otlp-http.md) посредствам `HttpSender`. +Модуль экспортирует данные трассировки в `OpenTelemetry Collector` через `OTLP/HTTP`. +Он строит `OtlpHttpSpanExporter` поверх `BatchSpanProcessor`, а типичный адрес коллектора — `http://localhost:4318/v1/traces`. ===! ":fontawesome-brands-java: `Java`" @@ -71,9 +82,13 @@ agent: ## Конфигурация { #configuration } -Обязательным полем является только `endpoint`, аттрибуты из поля `attributes` будут отправляться с каждым спаном. +Параметры экспорта в секции `tracing.exporter` описываются классами `OpentelemetryGrpcExporterConfig` (для `OTLP/gRPC`) и `OpentelemetryHttpExporterConfig` (для `OTLP/HTTP`); оба класса имеют одинаковый набор полей. +Атрибуты ресурса в секции `tracing.attributes` описываются классом `OpentelemetryResourceConfig`. +Если `tracing.exporter.endpoint` не указан, экспортер не создается (конфигурация разрешается во внутреннее значение `Empty`, и используются пустые `SpanExporter`/`SpanProcessor`), и приложение запускается без отправки трассировок во внешний коллектор. -Пример полной конфигурации, описанной в классе `OpentelemetryGrpcExporterConfig` или `OpentelemetryHttpExporterConfig`, а также `OpentelemetryResourceConfig` (указаны примеры значений или значения по умолчанию): +Поле `tracing.attributes` задает атрибуты `OpenTelemetry Resource`, которые добавляются к **каждому** экспортируемому `Span` всего сервиса. +Обычно оно содержит имя и пространство имен сервиса, например `service.name` и `service.namespace`. +Эти общесервисные атрибуты `Resource` отличаются от атрибутов уровня отдельного span, настраиваемых в секции `.telemetry.tracing.attributes`, которые добавляются только к span конкретной подсистемы — смотрите [Конфигурация трассировки модуля](#module-config). ===! ":material-code-json: `Hocon`" @@ -89,7 +104,7 @@ agent: batchExportTimeout = "30s" //(7)! compression = "gzip" //(8)! exportUnsampledSpans = false //(9)! - retry { + retryPolicy { maxAttempts = 5 //(10)! initialBackoff = "1s" //(11)! maxBackoff = "5s" //(12)! @@ -103,20 +118,20 @@ agent: } ``` - 1. URL от [OpenTelemetry](https://opentelemetry.io/docs/collector/) коллектор сервиса (**обязательный**) - 2. Время ожидания соедининя с экспортером - 3. Максимальное время ожидания обработки телеметрии коллектором - 4. Время между экспортом телеметрии в коллектор - 5. Максимальная кол-во телеметрии в рамках одного экспорта - 6. Максимальный размер очереди неотправленной телеметрии - 7. Максимальное вреия ожидания экспорта - 8. Механизм сжатия телеметрии при экпорте - 9. Экспортировать ли не сэмплированную телеметрию - 10. Максимальное кол-во попыток на экспорт - 11. Начальное значение ожидания перед следующей попыткой экспорта - 12. Максимальное значение ожидания перед следующей попыткой экспорта - 13. Мультипликатор значения задержки ожидания - 14. Дополнительные атрибуты телеметрии + 1. Адрес `OpenTelemetry Collector` для экспорта трассировок (по умолчанию не указан, необязательный). Для `gRPC` обычно используется `http://localhost:4317`, а для `HTTP` — `http://localhost:4318/v1/traces`. + 2. Таймаут установки соединения с экспортером (по умолчанию не указан, необязательный). + 3. Максимальное время ожидания при отправке данных экспортером (по умолчанию `3s`). + 4. Задержка между отправками накопленных `Span` в коллектор (по умолчанию `2s`). + 5. Максимальное количество `Span` в одной партии экспорта (по умолчанию `512`). + 6. Максимальный размер очереди `Span`, ожидающих отправки (по умолчанию `2048`). + 7. Максимальное время, которое `BatchSpanProcessor` ждет экспорта одной накопленной партии; отличается от `exportTimeout`, ограничивающего один запрос `OTLP` (по умолчанию `30s`). + 8. Сжатие данных при экспорте, `gzip` или `none` (по умолчанию `gzip`). + 9. Экспортировать ли `Span`, которые не были выбраны `Sampler` (по умолчанию `false`). + 10. Максимальное количество повторных попыток (по умолчанию `5`). + 11. Начальная задержка перед повторной попыткой (по умолчанию `1s`). + 12. Максимальная задержка перед повторной попыткой (по умолчанию `5s`). + 13. Множитель задержки между повторными попытками (по умолчанию `1.5`). + 14. Атрибуты `OpenTelemetry Resource`, добавляемые к экспортируемым `Span` (по умолчанию `{}`). === ":simple-yaml: `YAML`" @@ -132,7 +147,7 @@ agent: batchExportTimeout: 30s #(7)! compression: gzip #(8)! exportUnsampledSpans: false #(9)! - retry: + retryPolicy: maxAttempts: 5 #(10)! initialBackoff: 1s #(11)! maxBackoff: 5s #(12)! @@ -142,26 +157,224 @@ agent: service.namespace: kora ``` - 1. URL от [OpenTelemetry](https://opentelemetry.io/docs/collector/) коллектор сервиса (**обязательный**) - 2. Время ожидания соедининя с экспортером - 3. Максимальное время ожидания обработки телеметрии коллектором - 4. Время между экспортом телеметрии в коллектор - 5. Максимальная кол-во телеметрии в рамках одного экспорта - 6. Максимальный размер очереди неотправленной телеметрии - 7. Максимальное вреия ожидания экспорта - 8. Механизм сжатия телеметрии при экпорте - 9. Экспортировать ли не сэмплированную телеметрию - 10. Максимальное кол-во попыток на экспорт - 11. Начальное значение ожидания перед следующей попыткой экспорта - 12. Максимальное значение ожидания перед следующей попыткой экспорта - 13. Мультипликатор значения задержки ожидания - 14. Дополнительные атрибуты телеметрии - -Параметры конфигурации сбора трассировки описываются в модулях в которых присутствует сбор трассировки, например [HTTP сервер](http-server.md), [HTTP клиент](http-client.md) и т.д. + 1. Адрес `OpenTelemetry Collector` для экспорта трассировок (по умолчанию не указан, необязательный). Для `gRPC` обычно используется `http://localhost:4317`, а для `HTTP` — `http://localhost:4318/v1/traces`. + 2. Таймаут установки соединения с экспортером (по умолчанию не указан, необязательный). + 3. Максимальное время ожидания при отправке данных экспортером (по умолчанию `3s`). + 4. Задержка между отправками накопленных `Span` в коллектор (по умолчанию `2s`). + 5. Максимальное количество `Span` в одной партии экспорта (по умолчанию `512`). + 6. Максимальный размер очереди `Span`, ожидающих отправки (по умолчанию `2048`). + 7. Максимальное время, которое `BatchSpanProcessor` ждет экспорта одной накопленной партии; отличается от `exportTimeout`, ограничивающего один запрос `OTLP` (по умолчанию `30s`). + 8. Сжатие данных при экспорте, `gzip` или `none` (по умолчанию `gzip`). + 9. Экспортировать ли `Span`, которые не были выбраны `Sampler` (по умолчанию `false`). + 10. Максимальное количество повторных попыток (по умолчанию `5`). + 11. Начальная задержка перед повторной попыткой (по умолчанию `1s`). + 12. Максимальная задержка перед повторной попыткой (по умолчанию `5s`). + 13. Множитель задержки между повторными попытками (по умолчанию `1.5`). + 14. Атрибуты `OpenTelemetry Resource`, добавляемые к экспортируемым `Span` (по умолчанию `{}`). + +Пример проекта использует подстановку переменных окружения для адреса и переопределяет несколько параметров экспорта: + +===! ":material-code-json: `Hocon`" + + ```javascript + tracing { + exporter { + endpoint = ${METRIC_COLLECTOR_ENDPOINT} //(1)! + exportTimeout = "250s" + scheduleDelay = "50ms" + maxExportBatchSize = 10000 + } + attributes { + "service.name" = "kora-java-telemetry" + "service.namespace" = "kora" + } + } + ``` + + 1. Разрешается из переменной окружения `METRIC_COLLECTOR_ENDPOINT`, смотрите [подстановку переменных окружения](config.md#environment-variables). + +=== ":simple-yaml: `YAML`" + + ```yaml + tracing: + exporter: + endpoint: ${METRIC_COLLECTOR_ENDPOINT} #(1)! + exportTimeout: "250s" + scheduleDelay: "50ms" + maxExportBatchSize: 10000 + attributes: + service.name: "kora-java-telemetry" + service.namespace: "kora" + ``` + + 1. Разрешается из переменной окружения `METRIC_COLLECTOR_ENDPOINT`, смотрите [подстановку переменных окружения](config.md#environment-variables). + +## Автоматическая трассировка { #automatic } + +Как только добавлен один модуль экспортера, Kora автоматически инструментирует свои подсистемы: для каждого входящего запроса, исходящего вызова, сообщения, запроса к базе данных или запланированного запуска она создает `Span`, привязывает его к текущему `OpentelemetryContext`, вкладывает его в текущий активный `Span` и распространяет контекст трассировки за границы сервиса. +Для этих `Span` не требуется ни аннотаций, ни ручного кода. + +Например, контроллер `GET /text` из примера телеметрии автоматически создает span типа `SERVER` с именем `GET /text`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + @HttpController + public final class SimpleController { + + @HttpRoute(method = HttpMethod.GET, path = "/text") + public HttpServerResponse get() { + return HttpServerResponse.of(200, HttpBody.plaintext("Hello world")); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + @HttpController + class SimpleController { + + @HttpRoute(method = HttpMethod.GET, path = "/text") + fun get(): HttpServerResponse { + return HttpServerResponse.of(200, HttpBody.plaintext("Hello world")) + } + } + ``` + +В таблице ниже перечислены подсистемы, инструментируемые `OpentelemetryModule`, итоговое имя `Span` и его [тип](https://opentelemetry.io/docs/specs/otel/trace/api/#spankind), а также основные атрибуты. +Имена атрибутов следуют [семантическим соглашениям OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/). + +| Подсистема | Имя span | Тип | Основные атрибуты | +|------------------------|------------------------------------------------|------------|-------------------------------------------------------------------------------------------------------------------------| +| HTTP-сервер | ` `, например `GET /text` | `SERVER` | `http.request.method`, `url.scheme`, `url.path`, `http.route`, `server.address`, `http.response.status_code` | +| HTTP-клиент | ` ` | `CLIENT` | `http.request.method`, `server.address`, `server.port`, `url.scheme`, `url.full`, `http.response.status_code` | +| База данных | имя операции запроса | `CLIENT` | `db.system`, `db.user`, `db.statement` | +| Потребитель Kafka | `kafka.poll`, ` receive`, ` process` | `CONSUMER` | `messaging.system` = `kafka`, `messaging.operation`, `messaging.destination.name`, `messaging.kafka.message.offset` | +| Производитель Kafka | ` send`, `producer transaction` | `PRODUCER` / `INTERNAL` | `messaging.system` = `kafka`, `messaging.operation` = `publish`, `messaging.destination.name` | +| gRPC-сервер | `/` | `SERVER` | `rpc.system` = `grpc`, `rpc.service`, `rpc.method`, `network.peer.address` | +| gRPC-клиент | `` | `CLIENT` | `rpc.system` = `grpc`, `rpc.service`, `rpc.method`, `server.address`, `server.port` | +| S3-клиент | `S3 ` | `CLIENT` | `client.name`, `http.request.method`, `aws.s3.bucket`, `aws.s3.key`, `http.response.status_code` | +| SOAP-клиент | `SOAP ` | `CLIENT` | `rpc.service`, `rpc.method`, `rpc.system` | +| Потребитель JMS | ` receive` | `CONSUMER` | `messaging.system` = `jms`, `messaging.destination.name`, `messaging.message.id` | +| Планирование | ` ` | `INTERNAL` | `code.function`, `code.filepath` | +| Кэш | `cache.call` | `INTERNAL` | `operation`, `cache`, `origin` | + +Подсистемы `Camunda` (движок BPMN, REST) и воркер `Zeebe` также инструментируются при наличии их модулей. + +При ошибке Kora устанавливает статус span в `ERROR` и записывает исключение через `Span#recordException`; при успехе статус устанавливается в `OK`. + +## Конфигурация трассировки модуля { #module-config } + +Трассировка каждой инструментируемой подсистемы настраивается в секции `telemetry.tracing` соответствующего модуля, описываемой классом `ru.tinkoff.kora.telemetry.common.TelemetryConfig.TracingConfig`. +Для каждой подсистемы доступны две опции: + +- `enabled` (по умолчанию `true`) — включает или выключает span подсистемы. Установите `false`, чтобы прекратить создание span для конкретного модуля, не удаляя экспортер. +- `attributes` (по умолчанию `{}`) — набор пар ключ/значение, добавляемых к каждому span, создаваемому **только этим модулем**. Эти атрибуты уровня span отличаются от общесервисных `tracing.attributes` (атрибутов `Resource`), применяемых ко всем span. + +Секция `telemetry.tracing` располагается по тому же пути, что и собственная конфигурация модуля, например `httpServer.telemetry.tracing`, `db.telemetry.tracing`, `grpcServer.telemetry.tracing` или `kafka..telemetry.tracing`. + +===! ":material-code-json: `Hocon`" + + ```javascript + httpServer { + telemetry { + tracing { + enabled = true //(1)! + attributes { //(2)! + "component" = "gateway" + } + } + } + } + db { + telemetry { + tracing { + enabled = false //(3)! + } + } + } + ``` + + 1. Включает трассировку HTTP-сервера (по умолчанию `true`). + 2. Атрибуты уровня span, добавляемые только к span HTTP-сервера (по умолчанию `{}`). + 3. Отключает трассировку запросов к базе данных (по умолчанию `true`). + +=== ":simple-yaml: `YAML`" + + ```yaml + httpServer: + telemetry: + tracing: + enabled: true #(1)! + attributes: #(2)! + component: "gateway" + db: + telemetry: + tracing: + enabled: false #(3)! + ``` + + 1. Включает трассировку HTTP-сервера (по умолчанию `true`). + 2. Атрибуты уровня span, добавляемые только к span HTTP-сервера (по умолчанию `{}`). + 3. Отключает трассировку запросов к базе данных (по умолчанию `true`). + +Специфичные для модуля параметры трассировки также описаны в документации соответствующих модулей, например [HTTP-сервер](http-server.md), [HTTP-клиент](http-client.md), [gRPC-сервер](grpc-server.md), [gRPC-клиент](grpc-client.md) и [Kafka](kafka.md). + +## Распространение контекста { #propagation } + +Kora сшивает распределенные трассировки по стандарту [W3C Trace Context](https://www.w3.org/TR/trace-context/): каждый инструментируемый клиент внедряет текущий `traceparent` в исходящий носитель, а каждый инструментируемый сервер извлекает его, чтобы установить родителя нового `Span`. +Это происходит автоматически и не требует настройки: + +- **HTTP** — `traceparent` внедряется в заголовки запроса HTTP-клиентом и извлекается из заголовков запроса HTTP-сервером. +- **Kafka** — `traceparent` внедряется в заголовки записи производителем и извлекается из заголовков записи потребителем (span `process` отдельной записи также связывается со span `receive` партии). +- **gRPC** — `traceparent` внедряется в метаданные вызова клиентом и извлекается из метаданных сервером. +- **JMS** — `traceparent` извлекается из свойств сообщения потребителем. + +Поскольку текущий `Span` живет в `Context` Kora, любой span, созданный вами вручную (смотрите [Синхронная трассировка](#tracing-sync)), автоматически подхватывается и распространяется инструментируемыми клиентами, вызываемыми в рамках того же контекста — вам не нужно передавать заголовки самостоятельно. + +## Сэмплирование { #sampling } + +Базовые компоненты трассировки предоставляются модулем `OpentelemetryTracingModule` как `@DefaultComponent`, а значит каждый из них можно переопределить, объявив собственный компонент того же типа: + +- `Sampler` — решает, какие `Span` записываются. По умолчанию используется `Sampler.parentBased(Sampler.alwaysOn())`, то есть записывается каждый корневой `Span`, а для дочерних `Span` следует решению родителя. +- `IdGenerator` — генерирует идентификаторы трассировки и span. По умолчанию используется `IdGenerator.random()`. +- `Supplier` — ограничения на количество атрибутов, событий и связей у одного `Span`. По умолчанию используется `SpanLimits.getDefault()`. + +Чтобы применить сэмплирование на стороне источника (head-based), переопределите фабричный метод `Sampler` в своем приложении, например для записи примерно 10% корневых трассировок: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends OpentelemetryGrpcExporterModule { + + @Override + default Sampler opentelemetryTracingSampler() { + return Sampler.parentBased(Sampler.traceIdRatioBased(0.1)); + } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : OpentelemetryGrpcExporterModule { + + override fun opentelemetryTracingSampler(): Sampler { + return Sampler.parentBased(Sampler.traceIdRatioBased(0.1)) + } + } + ``` + +Опция экспорта `exportUnsampledSpans` управляет тем, отправляются ли в коллектор `Span`, которые **не** были выбраны `Sampler`; по умолчанию она равна `false`, поэтому экспортируются только выбранные `Span`. ## Контекст трассировки { #tracing-context } -Чтобы получить текущий `Span` трассировки можно использовать метод `getSpan` у `OpentelemetryContext`: +Чтобы получить текущий `Span`, используйте метод `getSpan` у `OpentelemetryContext`: ===! ":fontawesome-brands-java: `Java`" @@ -172,10 +385,10 @@ agent: === ":simple-kotlin: `Kotlin`" ```kotlin - val span = OpentelemetryContext.getSpan(); + val span = OpentelemetryContext.getSpan() ``` -Для получения текущего идентификатора трассировки можно использовать метод `getTraceId()` у `OpentelemetryContext`: +Чтобы получить текущий идентификатор трассировки, используйте метод `getTraceId()` у `OpentelemetryContext`: ===! ":fontawesome-brands-java: `Java`" @@ -186,15 +399,30 @@ agent: === ":simple-kotlin: `Kotlin`" ```kotlin - val traceId = OpentelemetryContext.getTraceId(); + val traceId = OpentelemetryContext.getTraceId() ``` -## Синхронная трассировка { #tracing-sync } +Если текущего `Span` нет, оба метода возвращают `null`. +Если вам нужно недействительное значение-заглушка из `OpenTelemetry`, используйте `getSpanOrInvalid()` и `getTraceIdOrInvalid()`, которые вместо `null` возвращают `Span.getInvalid()` и его нулевой идентификатор трассировки. -Помимо автоматически создаваемых фреймворком спанов, можно пользоваться объектом `Tracer` -из контейнера для создания своих трассировок. +Для ручного управления span у `OpentelemetryContext` также есть объектный API, используемый вместе с `Context` Kora: -Создать трассировку синхронного кода от текущего родительского можно следующим образом: +- `OpentelemetryContext.get(ctx)` — возвращает `OpentelemetryContext`, хранящийся в переданном `Context` Kora (создавая пустой, если его нет). +- `OpentelemetryContext.set(ctx, otctx)` — сохраняет `OpentelemetryContext` в `Context` Kora и обновляет `traceId`/`spanId` в `MDC`. +- `otctx.add(span)` — возвращает новый `OpentelemetryContext` с переданным `Span` (или любым `ImplicitContextKeyed`), добавленным в качестве текущего. +- `otctx.getContext()` — возвращает лежащий в основе `io.opentelemetry.context.Context`, используемый как родитель при построении вложенного `Span`. + +## Корреляция логов { #mdc } + +Когда вызывается `OpentelemetryContext.set` (автоматической инструментацией или вашим кодом ручной трассировки), Kora записывает текущие `traceId` и `spanId` в [MDC](logging-slf4j.md). +Когда текущего `Span` нет, эти ключи снова удаляются из `MDC`. + +В результате, если используется [модуль логирования](logging-slf4j.md), каждая строка лога, порожденная в рамках трассируемой операции, несет `traceId` и `spanId`, что позволяет переходить от записи лога к соответствующей трассировке в системе наблюдаемости и обратно. + +## Синхронная трассировка { #tracing-sync } + +Помимо `Span`, автоматически создаваемых фреймворком, вы можете использовать объект `Tracer` из графа приложения и создавать собственные вложенные `Span`. +При ручной трассировке важно сохранить текущий `OpentelemetryContext`, установить новый контекст на время операции и восстановить исходный контекст в блоке `finally`. ===! ":fontawesome-brands-java: `Java`" @@ -212,8 +440,8 @@ agent: var ctx = ru.tinkoff.kora.common.Context.current(); var otctx = OpentelemetryContext.get(ctx); var span = tracer.spanBuilder("myOperation") - .setParent(otctx.getContext()) - .startSpan(); + .setParent(otctx.getContext()) + .startSpan(); OpentelemetryContext.set(ctx, otctx.add(span)); try { @@ -264,22 +492,21 @@ agent: } } - fun doWork(): String = // do some work + fun doWork(): String { + // do some work + } } ``` ## Асинхронная трассировка { #async-tracing } -Помимо автоматически создаваемых фреймворком спанов, можно пользоваться объектом `Tracer` -из контейнера для создания своих трассировок. Главная сложность заключается в прокидывании `Fork`'а контекста -в другой поток исполнения, для корректной работы трассировки. - -Создать трассировку асинхронного кода от текущего родительского можно следующим образом: +При переключении на другой поток выполнения передавайте не только `Span`, но и контекст Kora. +Используйте `Context.fork()` для `CompletionStage` и `Context.Kotlin.asCoroutineContext(ctx)` для `suspend`-кода. ===! ":fontawesome-brands-java: `Java`" - Пример показан для `CompletableStage` асинхронного подхода: - + Пример для асинхронного кода с `CompletionStage`: + ```java @Component public final class MyService { @@ -294,23 +521,23 @@ agent: var ctx = ru.tinkoff.kora.common.Context.current().fork(); var otctx = OpentelemetryContext.get(ctx); var span = tracer.spanBuilder("myOperation") - .setParent(otctx.getContext()) - .startSpan(); + .setParent(otctx.getContext()) + .startSpan(); return CompletableFuture.supplyAsync(() -> { - OpentelemetryContext.set(ctx, otctx.add(span)); - var result = doWork(); - return result; - }) - .whenComplete((r, e) -> { - if (e != null) { - span.recordException(e); - span.setStatus(StatusCode.ERROR, e.getMessage()); - } else { - span.setStatus(StatusCode.OK); - } - span.end(); - }); + OpentelemetryContext.set(ctx, otctx.add(span)); + return doWork(); + }) + .whenComplete((r, e) -> { + if (e != null) { + span.recordException(e); + span.setStatus(StatusCode.ERROR, e.getMessage()); + } else { + span.setStatus(StatusCode.OK); + } + span.end(); + OpentelemetryContext.set(ctx, otctx); + }); } public String doWork() { @@ -321,7 +548,7 @@ agent: === ":simple-kotlin: `Kotlin`" - Пример показан для `suspend` асинхронного подхода: + Пример для асинхронного `suspend`-кода: ```kotlin @Component @@ -351,6 +578,8 @@ agent: } } - fun doWork(): String = // do some work + fun doWork(): String { + // do some work + } } ``` diff --git a/mkdocs/docs/ru/documentation/validation.md b/mkdocs/docs/ru/documentation/validation.md index bfb96a6..9c6bff6 100644 --- a/mkdocs/docs/ru/documentation/validation.md +++ b/mkdocs/docs/ru/documentation/validation.md @@ -1,12 +1,16 @@ --- -description: "Explains Kora validation annotations, class and method validation, argument and result validation, custom validators, and supported validation signatures. Use when working with @Validate, @Valid, @NotNull, @NotEmpty, @Pattern, @Range, @Size, @Validator." +description: "Explains Kora validation annotations, class and method validation, argument and result validation, custom validators, mapping validation failures to HTTP 400, and supported validation signatures. Use when working with @Validate, @Valid, @NotBlank, @NotEmpty, @Pattern, @Range, @Size, @Validator, ValidatorModule, ValidationModule." agent: - use_when: "Use this file for Kora docs or implementation questions about Kora validation annotations, class and method validation, argument and result validation, custom validators, and supported validation signatures; key triggers include @Validate, @Valid, @NotNull, @NotEmpty, @Pattern, @Range, @Size, @Validator, ValidationModule." + use_when: "Use this file for Kora docs or implementation questions about Kora validation annotations, class and method validation, argument and result validation, custom validators, mapping ViolationException to HTTP 400, and supported validation signatures; key triggers include @Validate, @Valid, @NotBlank, @NotEmpty, @Pattern, @Range, @Size, @ValidatedBy, Validator, ValidatorFactory, ViolationException, ValidationHttpServerInterceptor, ValidatorModule, ValidationModule." --- -Модуль для валидации моделей и методов с помощью аннотаций аспектов. +Модуль валидации Kora проверяет модели, аргументы методов и результаты методов с помощью аннотаций. +Для моделей Kora генерирует `Validator` во время компиляции, а для методов применяет аспект `@Validate`, который вызывает нужные проверки до или после выполнения метода. -Если нужен пошаговый разбор перед справочным описанием, смотрите [Валидация](../guides/validation.md). +Валидация работает без использования `Reflection` во время выполнения приложения: структура объекта, вложенные поля, сигнатуры методов и доступные валидаторы проверяются процессорами аннотаций во время сборки. +Ошибки валидации возвращаются в виде списка `Violation` либо выбрасываются как `ViolationException`. + +Пошаговый разбор перед справочным описанием смотрите в разделе [Валидация](../guides/validation.md). ## Подключение { #dependency } @@ -26,7 +30,7 @@ agent: === ":simple-kotlin: `Kotlin`" [Зависимость](general.md#dependencies) `build.gradle.kts`: - ```groovy + ```kotlin implementation("ru.tinkoff.kora:validation-module") ``` @@ -36,37 +40,73 @@ agent: interface Application : ValidationModule ``` +Модуль поставляет два интерфейса-примеси, и вы выбираете один в зависимости от того, обслуживает ли приложение `HTTP`: + +| Модуль | Артефакт | Предоставляет | Когда использовать | +|--------|----------|---------------|--------------------| +| `ValidatorModule` | `validation-common` | Сгенерированные компоненты `Validator`, все встроенные фабрики ограничений и валидаторы элементов (`Validator>`, `Validator>`, `Validator>`) | Библиотеки и приложения без `HTTP`, либо когда вы обрабатываете `ViolationException` самостоятельно | +| `ValidationModule` | `validation-module` | Всё из `ValidatorModule` **плюс** `ValidationHttpServerInterceptor`, который отображает `ViolationException` в [ответ HTTP 400](#validation-response-http) | `HTTP`-сервисы, которые должны автоматически возвращать `400` клиентам | + +`ValidationModule` расширяет `ValidatorModule`, поэтому подключение `ValidationModule` даёт вам всё, что предоставляет базовый модуль. +Показанная выше зависимость (`validation-module`) — правильный выбор для `HTTP`-сервиса; библиотека, которой нужно только генерировать валидаторы, может зависеть от `validation-common` и подключать вместо этого `ValidatorModule`. + ## Аннотации валидации { #validation-annotations } -Специальные аннотации валидации используются Kora для проверки значений полей классов или аргументов метода. +Аннотации валидации указывают Kora, что нужно проверить в поле, аргументе метода или результате метода. +Их можно применять напрямую, либо вложенная валидация может запускаться через `@Valid`, когда у типа есть сгенерированный или предоставленный вручную `Validator`. + +!!! warning "Валидация Kora — это не Jakarta Bean Validation" + + Валидация Kora — это **не** [Jakarta Bean Validation (JSR-380)](https://jakarta.ee/specifications/bean-validation/). + Все аннотации ограничений Kora находятся в пакете `ru.tinkoff.kora.validation.common.annotation` и обрабатываются во время компиляции. + В частности, Kora **не** поставляет аннотацию ограничения `@NotNull`: значение по умолчанию является обязательным, а чтобы сделать его необязательным, вы помечаете его любой аннотацией `@Nullable` (см. [Необязательные поля](#optional-fields)). + Kora действительно распознаёт стандартный маркер `@Nonnull` / `@NotNull` (из `javax.annotation`, `jakarta.annotation` и подобных пакетов) как явное требование не-`null`, что важно главным образом для полей [`JsonNullable`](#json-nullable). -Доступные аннотации валидации: +Структурные аннотации, управляющие валидацией: -- `@NotEmpty` - Проверяет что строка не пустая -- `@NotBlank` - Проверяет что строка не состоит из пустых символов -- `@Pattern` - Проверяет соответствие Regular Expression (RegEx) -- `@Range` - Проверяет что число находится в заданном диапазоне -- `@Size` - Проверяет что коллекция (List, Set, Map) или `String` имеет размер в заданном диапазоне +- `@Valid` — на классе или `record` генерирует `Validator` для этого типа; на поле, аргументе или результате метода запускает вложенную валидацию через `Validator` соответствующего типа. Применима к типам, полям, параметрам и методам. +- `@Validate` — помечает метод, аргументы и/или результат которого должны быть провалидированы аспектом; параметр `failFast` управляет остановкой на первой ошибке (по умолчанию: `false`). Применима только к методам. +- `@ValidatedBy` — связывает пользовательскую аннотацию ограничения с `ValidatorFactory`, которая строит её `Validator` (см. [Пользовательские аннотации валидации](#custom-validation-annotations)). Применима только к типам аннотаций. + +Встроенные аннотации ограничений и их параметры: + +| Аннотация | Поддерживаемые типы | Параметры (значения по умолчанию) | Описание | +|-----------|---------------------|-----------------------------------|----------| +| `@NotBlank` | `String`, `CharSequence` | — | Значение не `null` и содержит хотя бы один непробельный символ. | +| `@NotEmpty` | `String`, `CharSequence`, `Iterable`, `Collection`, `List`, `Set`, `Map` | — | Значение не `null` и не пустое. | +| `@Pattern` | `String`, `CharSequence` | `value` (обязательный, без значения по умолчанию), `flags` (по умолчанию: `0`) | Значение соответствует регулярному выражению `value`; `flags` отображается на флаги [`java.util.regex.Pattern`](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/regex/Pattern.html#field.summary). | +| `@Range` | `Short`, `Integer`, `Long`, `Float`, `Double`, `BigInteger`, `BigDecimal` | `from` (обязательный, без значения по умолчанию), `to` (обязательный, без значения по умолчанию), `boundary` (по умолчанию: `INCLUSIVE_INCLUSIVE`) | Число лежит в пределах `[from, to]`; `boundary` управляет тем, включаются ли границы. | +| `@Size` | `String`, `CharSequence`, `Collection`, `List`, `Set`, `Map` | `min` (по умолчанию: `0`), `max` (обязательный, без значения по умолчанию) | Размер (длина) значения находится в пределах `min` и `max`. | + +!!! note + + Обращайте внимание на обязательные параметры: у `@Size.max` **нет значения по умолчанию**, поэтому его пропуск является ошибкой компиляции; `@Range.from` и `@Range.to` оба обязательны и объявлены как `double`. + Значение `@Range.boundary` — это перечисление `Range.Boundary` с вариантами `EXCLUSIVE_EXCLUSIVE`, `INCLUSIVE_EXCLUSIVE`, `EXCLUSIVE_INCLUSIVE` и `INCLUSIVE_INCLUSIVE`. ## Валидация класса { #class-validation } -Предлагается использовать аннотацию `@Valid` для маркировки класса которому требуется создать валидатор посредствам Kora. +Аннотация `@Valid` на классе или `record` указывает Kora создать `Validator` для этого типа. +Сгенерированный валидатор становится обычным компонентом графа зависимостей и может быть внедрён по сигнатуре `Validator`. ===! ":fontawesome-brands-java: `Java`" ```java @Valid - public record Foo(String number) { } + public record User(@NotBlank String id, + @Size(min = 3, max = 6) String name, + @Nullable String status) { } ``` === ":simple-kotlin: `Kotlin`" ```kotlin @Valid - data class Foo(val number: String) + data class User(@field:NotBlank val id: String, + @field:Size(min = 3, max = 6) val name: String, + val status: String?) ``` -Затем в контейнере зависимостей будет доступен валидатор такого класса: +После этого валидатор для данного класса будет доступен в контейнере зависимостей: ===! ":fontawesome-brands-java: `Java`" @@ -74,10 +114,10 @@ agent: @Component public final class Example { - private final Validator fooValidator; + private final Validator userValidator; - public Example(Validator fooValidator) { - this.fooValidator = fooValidator; + public Example(Validator userValidator) { + this.userValidator = userValidator; } } ``` @@ -86,20 +126,21 @@ agent: ```kotlin @Component - class Example(val fooValidator: Validator) + class Example(val userValidator: Validator) ``` -Созданнные валидаторы могут быть внедрены как зависимости в любой компонент, на примерах выше валидатор для класса `Foo`, -может быть внедрен по своей сигнатуре `Validator` как зависимость компонента и использовать вручную для валидации. +Сгенерированные валидаторы можно внедрять как зависимости в любой компонент. +В примере выше валидатор для `User` внедряется по сигнатуре `Validator` и может использоваться вручную. -Валидатор после валидации возвращает список нарушений, они могут использоваться для ручного составление ошибки либо -можно использовать метод `validateAndThrow` который бросит исключение `ViolationException` в случае ошибки валидации. +Метод `validate(...)` возвращает список `Violation`. +Вы можете обработать этот список самостоятельно или вызвать `validateAndThrow(...)`, который выбрасывает `ViolationException`, если есть нарушения. +Полный императивный API смотрите в разделе [Ручная валидация](#manual-validation). -### Валидация полей { #field-validation } +### Валидация поля { #field-validation } -Предполагается использовать для валидации полей специальный предоставляемый набор [аннотаций](#validation-annotations) валидации. +Валидация поля использует набор [аннотаций](#validation-annotations), предоставляемых модулем. -Пример размеченного для валидации объекта выглядит так: +Объект, помеченный для валидации, выглядит так: ===! ":fontawesome-brands-java: `Java`" @@ -108,11 +149,11 @@ agent: public record Foo(@NotEmpty String number) { } ``` - Для Record классов используется синтаксис доступа к полям через Record-like контракты геттеров, - в случае `Foo` и поля `code` будет использоваться *getter* `code()` в созданом `Validator`. + Для `record` доступ к полям осуществляется через методы самого `record`. + Для `Foo` и поля `number` сгенерированный `Validator` будет использовать метод `number()`. - Для обычного класса ожидается что будет использоваться синтаксис Java *Getters*, например для поля `id` будет использоваться *getter* `getId()`, - где *getter* должен иметь минимум *package-private* видимость. + Для обычного класса используется синтаксис `JavaBeans`: например, для поля `id` будет использоваться метод `getId()`. + Этот метод должен иметь как минимум видимость `package-private`. === ":simple-kotlin: `Kotlin`" @@ -123,25 +164,26 @@ agent: #### Обязательные поля { #required-fields } -Предполагается что все поля по умолчанию являются обязательными (`NotNull`), значит для всех них будут созданы `NotNull` проверки в `Validator`. +Все поля по умолчанию считаются обязательными, поэтому для них создаются проверки на `null`. #### Необязательные поля { #optional-fields } ===! ":fontawesome-brands-java: `Java`" - Чтобы указать поле как не обязательное, требуется пометить его любой `@Nullable` аннотацией, - для такого поля **не будет** создана проверка на *null*: + Чтобы пометить поле как необязательное, аннотируйте его любой аннотацией `@Nullable`. + Для такого поля проверка на `null` **не будет** создана: ```java @Valid public record Foo(@Nullable String number) { } //(1)! ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдёт любая аннотация `@Nullable`, например `javax.annotation.Nullable`, `jakarta.annotation.Nullable` или `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - Предполагается использовать [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис и помечать такое поле как Nullable: + Чтобы пометить поле как необязательное, используйте синтаксис [`Kotlin Nullability`](https://kotlinlang.org/docs/null-safety.html) и добавьте `?` к типу поля. + Для такого поля проверка на `null` **не будет** создана: ```kotlin @Valid @@ -150,9 +192,7 @@ agent: #### Вложенные поля { #embedded-fields } -Для валидации полей сложных объектов для которых созданы валидаторы (или предоставлены самостоятельно), -либо полей которые не поддерживаются стандартными средствами валидации, -предполагается использовать `@Valid` аннотацию: +Используйте `@Valid` для валидации вложенных объектов, у которых есть сгенерированные или предоставленные вручную валидаторы. ===! ":fontawesome-brands-java: `Java`" @@ -174,67 +214,225 @@ agent: data class Bar(val number: String) ``` -В примере выше для `Bar` будет создан валидатор `Validator` и для `Foo` будет создан `Validator`, -где при вызове валидатора `Validator` будет вызываться внутри валидатор для `Validator`. +В примере выше для `Bar` будет создан `Validator`, а для `Foo` будет создан `Validator`. +При вызове `Validator` он внутри себя вызовет `Validator`. + +#### Валидация коллекции { #collection-validation } + +`@Valid` на поле `List`, `Set` или `Collection` валидирует **каждый элемент** через `Validator` элемента. +`ValidatorModule` предоставляет эти валидаторы элементов из коробки (`Validator>`, `Validator>`, `Validator>`), поэтому дополнительной настройки не требуется. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Valid + public record Foo(@Valid List bars) { } + + @Valid + public record Bar(@NotBlank String number) { } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Valid + data class Foo(@field:Valid val bars: List) + + @Valid + data class Bar(@field:NotBlank val number: String) + ``` + +Каждый `Bar` в списке валидируется, а путь нарушения индексируется по позиции элемента, например `bars[0].number`. +Ограничения, такие как [`@Size`](#validation-annotations), можно комбинировать с `@Valid` на одной и той же коллекции, чтобы проверить и размер коллекции, и каждый элемент. + +#### Иерархии `Sealed` { #sealed-validation } + +Kora может создать `Validator` для `sealed`-иерархий. +Если `@Valid` помещена на `sealed`-тип, сгенерированный валидатор определяет фактический подтип и вызывает валидатор для соответствующей финальной реализации. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Valid + public sealed interface Command permits CreateCommand { + + @Valid + record CreateCommand(@NotBlank String name) implements Command { } + } + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Valid + sealed interface Command { + + @Valid + data class CreateCommand(@field:NotBlank val name: String) : Command + } + ``` + +#### `JsonNullable` { #json-nullable } + +Для `JsonNullable` Kora валидирует значение `T` внутри контейнера. +Если `JsonNullable` находится в состоянии `undefined`, обычные проверки значения не выполняются. +Используйте `@NotNull` или `@Nonnull`, чтобы запретить `undefined` или `null`. + +#### Параметры валидации { #validation-options } + +Существует два режима валидации, выбираемых через `ValidationContext`, передаваемый в `validate(...)`: + +- `Full` — проверяются все помеченные поля, собираются все возможные ошибки валидации, и только затем возвращается список нарушений или выбрасывается исключение. Это поведение по умолчанию. +- `FailFast` — валидация останавливается на первой найденной ошибке. + +`ValidationContext` можно построить несколькими эквивалентными способами: + +- `ValidationContext.builder().build()` — контекст `Full` по умолчанию (то же, что и вызов `validate(value)` без контекста). +- `ValidationContext.full()` — явный контекст `Full`. +- `ValidationContext.failFast()` — контекст `FailFast`. +- `ValidationContext.builder().failFast(true).build()` — форма `FailFast` через строитель. + +Пример валидации `FailFast`: + +===! ":fontawesome-brands-java: `Java`" + + ```java + ValidationContext context = ValidationContext.failFast(); + List violations = userValidator.validate(value, context); + ``` + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + val context = ValidationContext.failFast() + val violations = userValidator.validate(value, context) + ``` + +### Ручная валидация { #manual-validation } + +Сгенерированный `Validator` — это обычный компонент, поэтому его можно внедрить и вызвать напрямую — например, в сервисе, который не является `HTTP`-контроллером, или когда вы хотите изучить нарушения вместо выбрасывания исключения. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public final class UserService { + + private final Validator validator; + + public UserService(Validator validator) { + this.validator = validator; + } -#### Опции валидации { #validation-options } + public void process(User user) { + List violations = validator.validate(user); //(1)! + if (!violations.isEmpty()) { + Violation first = violations.get(0); + throw new IllegalStateException(first.path().full() + ": " + first.message()); //(2)! + } + } + } + ``` -Есть два вида валидации: + 1. `validate(value)` собирает **все** нарушения; используйте `validate(value, context)`, чтобы передать параметры валидации. + 2. Каждый `Violation` предоставляет `path()` и `message()`. -- `Full` - проверяются все поля которые только размечены, собираются все возможные ошибки валидации - и только потом бросается исключение. (**Поведение по умолчанию**) -- `FailFast` - исключение бросается на первой встреченной ошибке валидации. +=== ":simple-kotlin: `Kotlin`" -Пример FailFast валидации: -```java -ValidatorContext context = ValidationContext.builder().failFast(true).build(); -List violations = fooValidator.validate(value,context); -``` + ```kotlin + @Component + class UserService(private val validator: Validator) { + + fun process(user: User) { + val violations = validator.validate(user) //(1)! + if (violations.isNotEmpty()) { + val first = violations.first() + throw IllegalStateException("${first.path().full()}: ${first.message()}") //(2)! + } + } + } + ``` + + 1. `validate(value)` собирает **все** нарушения; используйте `validate(value, context)`, чтобы передать параметры валидации. + 2. Каждый `Violation` предоставляет `path()` и `message()`. + +Контракт `Validator` предлагает следующие методы: + +- `validate(value)` / `validate(value, context)` — возвращают `List`, который пуст, когда значение валидно (значение `null` завершается нарушением). +- `validateAndThrow(value)` / `validateAndThrow(value, context)` — выбрасывают `ViolationException` при возникновении любого нарушения и не делают ничего в противном случае. + +Когда `ViolationException` перехвачено, `getViolations()` возвращает агрегированный `List`, а `getMessage()` возвращает предварительно отформатированную многострочную сводку по каждому пути и сообщению нарушения. ## Валидация метода { #method-validation } -Предполагается использовать для валидации аргументов метода и результата специальный предоставляемый набор [аннотаций](#validation-annotations) валидации. +Валидация аргументов и результата метода использует аспект `@Validate` и набор [аннотаций](#validation-annotations), предоставляемых модулем. +Kora генерирует код аспекта во время компиляции, поэтому класс с такими методами должен поддерживать применение аспектов. -### Валидация аргументов { #argument-validation } +### Валидация аргумента { #argument-validation } -Чтобы провалидировать аргументы методы, требуется использовать аннотацию `@Validate` над методом: +Чтобы провалидировать аргументы метода, используйте аннотацию `@Validate` на методе и аннотируйте аргументы нужными [ограничениями](#validation-annotations). +Аргументы можно валидировать аннотациями ограничений напрямую или через `@Valid`, когда у типа аргумента есть собственный `Validator`: ===! ":fontawesome-brands-java: `Java`" ```java @Component - public class SomeService { + public class ArgumentValidator { + + @Valid + public record User(@NotBlank String id, + @Size(min = 3, max = 6) String name, + @Nullable String status) { } @Validate - public int validate(@NotEmpty String argument) { - return 1; + public int calculate(@Valid User user, //(1)! + @Range(from = 1, to = 900) int weight, //(2)! + @Pattern("ME\\d+") String code) { //(3)! + return Integer.parseInt(code.substring(2)); } } ``` + 1. Вложенная валидация через `Validator`. + 2. Ограничение числового диапазона, применённое напрямую к аргументу. + 3. Ограничение регулярного выражения, применённое напрямую к аргументу. + === ":simple-kotlin: `Kotlin`" ```kotlin @Component - open class SomeService { + open class ArgumentValidator { + + @Valid + data class User(@field:NotBlank val id: String, + @field:Size(min = 3, max = 6) val name: String, + val status: String?) @Validate - fun validate(@NotEmpty argument: String): Int { - return 1 + fun calculate(@Valid user: User, //(1)! + @Range(from = 1.0, to = 900.0) weight: Int, //(2)! + @Pattern("ME\\d+") code: String): Int { //(3)! + return code.substring(2).toInt() } } ``` + 1. Вложенная валидация через `Validator`. + 2. Ограничение числового диапазона, применённое напрямую к аргументу. + 3. Ограничение регулярного выражения, применённое напрямую к аргументу. + +Если какой-либо аргумент не проходит валидацию, аспект выбрасывает `ViolationException` **до** выполнения тела метода. + #### Обязательные аргументы { #required-arguments } -Предполагается что все аргументы по умолчанию являются обязательными (`NotNull`), значит для всех них будут созданы `NotNull` проверки. +Все аргументы по умолчанию считаются обязательными, поэтому для них создаются проверки на `null`. #### Необязательные аргументы { #optional-arguments } ===! ":fontawesome-brands-java: `Java`" - Чтобы указать аргумент как не обязательное, требуется пометить его любой `@Nullable` аннотацией, - для такого аргумента **не будет** создана проверка на *null*: + Чтобы пометить аргумент как необязательный, аннотируйте его любой аннотацией `@Nullable`. + Для такого аргумента проверка на `null` **не будет** создана: ```java @Component @@ -247,11 +445,12 @@ List violations = fooValidator.validate(value,context); } ``` - 1. Подойдет любая аннотация `@Nullable`, такие как `javax.annotation.Nullable` / `jakarta.annotation.Nullable` / `org.jetbrains.annotations.Nullable` / и т.д. + 1. Подойдёт любая аннотация `@Nullable`, например `javax.annotation.Nullable`, `jakarta.annotation.Nullable` или `org.jetbrains.annotations.Nullable`. === ":simple-kotlin: `Kotlin`" - Предполагается использовать [Kotlin Nullability](https://kotlinlang.ru/docs/null-safety.html) синтаксис и помечать такой аргумент как Nullable: + Чтобы пометить аргумент как необязательный, используйте синтаксис [`Kotlin Nullability`](https://kotlinlang.org/docs/null-safety.html) и добавьте `?` к типу аргумента. + Для такого аргумента проверка на `null` **не будет** создана: ```kotlin @Component @@ -266,9 +465,7 @@ List violations = fooValidator.validate(value,context); #### Вложенные аргументы { #embedded-arguments } -Для валидации полей сложных объектов для которых созданы валидаторы (или предоставлены самостоятельно), -либо полей которые не поддерживаются стандартными средствами валидации, -предполагается использовать `@Valid` аннотацию: +Используйте `@Valid` для валидации вложенных аргументов, у которых есть сгенерированные или предоставленные вручную валидаторы. ===! ":fontawesome-brands-java: `Java`" @@ -302,13 +499,64 @@ List violations = fooValidator.validate(value,context); } ``` -В примере выше для `Bar` будет создан валидатор `Validator` и для `Foo` будет создан `Validator`, -где при вызове валидатора `Validator` будет вызываться внутри валидатор для `Validator`. +В примере выше для `Foo` будет создан `Validator`. +При вызове метода аспект `@Validate` вызовет этот валидатор для аргумента `argument`. ### Валидация результата { #result-validation } -Чтобы провалидировать результат метода, требуется использовать аннотацию `@Validate` над методом и разметить его соответствующими [аннотациями](#validation-annotations), -для проверки, что значение не равно `null` требуется использовать любую `@NotNull/@Nonnull` аннотацию: +Чтобы провалидировать результат метода, используйте аннотацию `@Validate` на методе и аннотируйте результат соответствующими [аннотациями](#validation-annotations). +Поместите `@Valid` на метод, чтобы запустить вложенную валидацию через `Validator` возвращаемого типа. +Чтобы потребовать, чтобы результат был не `null`, используйте любую аннотацию `@Nonnull` или `@NotNull`. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Component + public class ResultValidator { + + @Valid + public record User(@NotBlank String id, + @Size(min = 3, max = 6) String name, + @Nullable @Size(min = 1, max = 10) String status) { } //(1)! + + @Valid //(3)! + @Validate //(2)! + public User create(String name, String status) { + return new User(UUID.randomUUID().toString(), name, status); + } + } + ``` + + 1. Ограничения можно накладывать друг на друга: `status` необязателен (`@Nullable`), но если он присутствует, его длина должна укладываться в `@Size`. + 2. Указывает, что метод требует валидации. + 3. Указывает, что результат должен быть провалидирован через `Validator` возвращаемого типа. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Component + open class ResultValidator { + + @Valid + data class User(@field:NotBlank val id: String, + @field:Size(min = 3, max = 6) val name: String, + @field:Size(min = 1, max = 10) val status: String?) //(1)! + + @Valid //(3)! + @Validate //(2)! + fun create(name: String, status: String): User { + return User(UUID.randomUUID().toString(), name, status) + } + } + ``` + + 1. Ограничения можно накладывать друг на друга: `status` необязателен (nullable), но если он присутствует, его длина должна укладываться в `@Size`. + 2. Указывает, что метод требует валидации. + 3. Указывает, что результат должен быть провалидирован через `Validator` возвращаемого типа. + +Валидация результата выполняется **после** тела метода, над его возвращаемым значением; если она не проходит, аспект выбрасывает `ViolationException` вместо возврата значения. + +Ограничения также можно применять к самому контейнеру результата. Например, у результата-коллекции можно одновременно проверить размер и провалидировать её элементы: ===! ":fontawesome-brands-java: `Java`" @@ -328,9 +576,9 @@ List violations = fooValidator.validate(value,context); } ``` - 1. Указывает что метод требует валидации - 2. Указывает что результат требуется валидировать валидатором с типа возвращаемого значения - 3. Стандартная аннотация валидации + 1. Указывает, что метод требует валидации. + 2. Указывает, что результат должен быть провалидирован через `Validator` возвращаемого типа. + 3. Стандартная аннотация валидации. === ":simple-kotlin: `Kotlin`" @@ -347,19 +595,18 @@ List violations = fooValidator.validate(value,context); } ``` - 1. Указывает что метод требует валидации - 2. Указывает что результат требуется валидировать валидатором с типа возвращаемого значения - 3. Стандартная аннотация валидации + 1. Указывает, что метод требует валидации. + 2. Указывает, что результат должен быть провалидирован через `Validator` возвращаемого типа. + 3. Стандартная аннотация валидации. -### Опции валидации { #validation-options-2 } +### Параметры валидации { #validation-options-2 } -Есть два вида валидации: +Существует два режима валидации: -- `Full` - проверяются все поля которые только размечены, собираются все возможные ошибки валидации - и только потом бросается исключение. (**Поведение по умолчанию**) -- `FailFast` - исключение бросается на первой встреченной ошибке валидации. +- `Full` — проверяются все помеченные аргументы и результат, собираются все возможные ошибки валидации, и только затем выбрасывается исключение. Это поведение по умолчанию. +- `FailFast` — исключение выбрасывается на первой найденной ошибке. -Пример FailFast валидации: +Пример валидации `FailFast`: ===! ":fontawesome-brands-java: `Java`" @@ -378,18 +625,193 @@ List violations = fooValidator.validate(value,context); ```kotlin @Component - class SomeService { + open class SomeService { @Validate(failFast = true) fun validate(@NotEmpty c2: String): Int = 1 } ``` -## Собственные аннотации валидации { #custom-validation-annotations } +## HTTP обработки ошибок { #validation-response-http } + +Когда `HTTP`-сервис Kora использует `ValidationModule` (из артефакта `validation-module`), неудачная валидация может быть автоматически превращена в ответ `HTTP` `400` вместо неперехваченной ошибки. + +Это обрабатывается `ValidationHttpServerInterceptor` — [перехватчиком HTTP-сервера](http-server.md#interceptors), который перехватывает `ViolationException`, выброшенное аспектом `@Validate` (включая исключение, обёрнутое в `CompletionException` для асинхронных сигнатур), и формирует ответ. +По умолчанию он возвращает статус `400` с [сообщением](#manual-validation) `ViolationException` в качестве тела в формате обычного текста; пользовательский [маппер ответа](#validation-response-custom) может заменить это. + +Глобальные перехватчики собираются по тегу `@Tag(HttpServerModule.class)` (см. [Перехватчики](http-server.md#interceptors)), поэтому перехватчик должен быть предоставлен **с этим тегом**, чтобы применяться к каждому маршруту: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @KoraApp + public interface Application extends + ValidationModule, //(1)! + UndertowHttpServerModule, + JsonModule { + + @Tag(HttpServerModule.class) //(2)! + default ValidationHttpServerInterceptor validationHttpServerInterceptor(@Nullable ViolationExceptionHttpServerResponseMapper mapper) { + return new ValidationHttpServerInterceptor(mapper); //(3)! + } + } + ``` + + 1. `ValidationModule` расширяет `ValidatorModule` и предоставляет связывание `ValidationHttpServerInterceptor` и `ViolationExceptionHttpServerResponseMapper`. + 2. Регистрирует перехватчик как **глобальный** перехватчик HTTP-сервера. + 3. Передача `null` в качестве маппера сохраняет ответ `400` по умолчанию в формате обычного текста. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @KoraApp + interface Application : ValidationModule, //(1)! + UndertowHttpServerModule, + JsonModule { + + @Tag(HttpServerModule::class) //(2)! + fun validationInterceptor(mapper: ViolationExceptionHttpServerResponseMapper?): ValidationHttpServerInterceptor { + return ValidationHttpServerInterceptor(mapper) //(3)! + } + } + ``` -Для создания собственной аннотации требуется: + 1. `ValidationModule` расширяет `ValidatorModule` и предоставляет связывание `ValidationHttpServerInterceptor` и `ViolationExceptionHttpServerResponseMapper`. + 2. Регистрирует перехватчик как **глобальный** перехватчик HTTP-сервера. + 3. Передача `null` в качестве маппера сохраняет ответ `400` по умолчанию в формате обычного текста. -1) Создать наследника `Validator`: +Метод контроллера, аннотированный `@Validate`, затем формирует `400` для клиента всякий раз, когда его аргументы или результат не проходят валидацию, без настройки для каждого контроллера: + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json + public record UserRequest(@NotBlank @Size(min = 2, max = 100) String name, + @NotBlank @Pattern("^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$") String email) { } + + @Component + @HttpController + public final class UserController { + + @HttpRoute(method = HttpMethod.POST, path = "/users") + @Validate //(1)! + @Json + public UserResponse createUser(@Valid @Json UserRequest request) { //(2)! + // request is already validated here + } + } + ``` + + 1. Включает валидацию аргументов (и результата) для этого маршрута. + 2. Вложенная валидация тела запроса; нарушение приводит к `HTTP` `400` до выполнения тела. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json + data class UserRequest(@field:NotBlank @field:Size(min = 2, max = 100) val name: String, + @field:NotBlank @field:Pattern("^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$") val email: String) + + @Component + @HttpController + class UserController { + + @HttpRoute(method = HttpMethod.POST, path = "/users") + @Validate //(1)! + @Json + fun createUser(@Valid @Json request: UserRequest): UserResponse { + // request is already validated here + } + } + ``` + + 1. Включает валидацию аргументов (и результата) для этого маршрута. + 2. Вложенная валидация тела запроса; нарушение приводит к `HTTP` `400` до выполнения тела. + +### Пользовательский ответ { #validation-response-custom } + +Чтобы управлять статусом, заголовками или телом ответа — например, чтобы вернуть структурированную ошибку `JSON` вместо обычного текста по умолчанию — предоставьте компонент `ViolationExceptionHttpServerResponseMapper`. +Его метод `apply(request, exception)` возвращает `HttpServerResponse` для отправки; возврат `null` откатывается к ответу `400` по умолчанию в формате обычного текста. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Json //(1)! + public record ValidationErrorResponse(String code, String message, List errors) { } + + @Json + public record ValidationErrorDetails(String field, String message) { } + + @KoraApp + public interface Application extends + ValidationModule, + UndertowHttpServerModule, + JsonModule { + + default ViolationExceptionHttpServerResponseMapper violationExceptionMapper(JsonWriter writer) { + return (request, exception) -> { + var errors = exception.getViolations().stream() //(2)! + .map(v -> new ValidationErrorDetails(v.path().full(), v.message())) + .toList(); + var body = new ValidationErrorResponse("VALIDATION_ERROR", "Validation failed", errors); + return HttpServerResponse.of(400, HttpBody.json(writer.toByteArrayUnchecked(body))); //(3)! + }; + } + + @Tag(HttpServerModule.class) + default ValidationHttpServerInterceptor validationHttpServerInterceptor(ViolationExceptionHttpServerResponseMapper mapper) { + return new ValidationHttpServerInterceptor(mapper); + } + } + ``` + + 1. Сериализуется с помощью [модуля JSON](json.md). + 2. `ViolationException.getViolations()` возвращает каждый `Violation`; `path().full()` — это путь через точку (например, `customer.address.city`). + 3. Может быть возвращён любой `HttpServerResponse`; возврат `null` откатился бы к `400` по умолчанию. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Json //(1)! + data class ValidationErrorResponse(val code: String, val message: String, val errors: List) + + @Json + data class ValidationErrorDetails(val field: String, val message: String) + + @KoraApp + interface Application : ValidationModule, + UndertowHttpServerModule, + JsonModule { + + fun violationExceptionMapper(writer: JsonWriter): ViolationExceptionHttpServerResponseMapper { + return ViolationExceptionHttpServerResponseMapper { request, exception -> + val errors = exception.violations.map { //(2)! + ValidationErrorDetails(it.path().full(), it.message()) + } + val body = ValidationErrorResponse("VALIDATION_ERROR", "Validation failed", errors) + HttpServerResponse.of(400, HttpBody.json(writer.toByteArrayUnchecked(body))) //(3)! + } + } + + @Tag(HttpServerModule::class) + fun validationInterceptor(mapper: ViolationExceptionHttpServerResponseMapper): ValidationHttpServerInterceptor { + return ValidationHttpServerInterceptor(mapper) + } + } + ``` + + 1. Сериализуется с помощью [модуля JSON](json.md). + 2. `ViolationException.getViolations()` возвращает каждый `Violation`; `path().full()` — это путь через точку (например, `customer.address.city`). + 3. Может быть возвращён любой `HttpServerResponse`; возврат `null` откатился бы к `400` по умолчанию. + +## Пользовательские аннотации валидации { #custom-validation-annotations } + +Пользовательская аннотация валидации нужна, когда стандартных проверок недостаточно. +Она связывает аннотацию с `ValidatorFactory`, и фабрика создаёт `Validator` для конкретного типа значения. + +Чтобы создать пользовательскую аннотацию: + +1. Создайте реализацию `Validator`: ===! ":fontawesome-brands-java: `Java`" @@ -415,7 +837,7 @@ List violations = fooValidator.validate(value,context); ```kotlin class MyValidStringValidator : Validator { - fun validate(value: String?, context: ValidationContext): List { + override fun validate(value: String?, context: ValidationContext): List { if (value == null) { return listOf(context.violates("Should be not empty, but was null")) } else if (value.isEmpty()) { @@ -426,7 +848,7 @@ List violations = fooValidator.validate(value,context); } ``` -2) Создать наследника `ValidatorFactory`: +2. Создайте подтип `ValidatorFactory`: ===! ":fontawesome-brands-java: `Java`" @@ -440,7 +862,7 @@ List violations = fooValidator.validate(value,context); interface MyValidValidatorFactory : ValidatorFactory ``` -3) Зарегистрировать наследника `ValidatorFactory` как компонент: +3. Зарегистрируйте `ValidatorFactory` как компонент: ===! ":fontawesome-brands-java: `Java`" @@ -471,7 +893,7 @@ List violations = fooValidator.validate(value,context); ``` -4) Создать аннотацию валидации и проаннотировать ее `@ValidatedBy` с ранее созданным наследником `ValidatorFactory`: +4. Создайте аннотацию валидации и пометьте её `@ValidatedBy`, используя ранее созданный подтип `ValidatorFactory`: ===! ":fontawesome-brands-java: `Java`" @@ -491,7 +913,7 @@ List violations = fooValidator.validate(value,context); annotation class MyValid ``` -5) Проаннотировать поле/аргумент/результат: +5. Пометьте поле, аргумент или результат новой аннотацией: ===! ":fontawesome-brands-java: `Java`" @@ -507,28 +929,89 @@ List violations = fooValidator.validate(value,context); data class Foo(@field:MyValid val number: String) ``` +### Параметризованные ограничения { #parameterized-constraints } + +Пользовательская аннотация ограничения может объявлять параметры. +Когда это так, её подтип `ValidatorFactory` должен объявить метод `create(...)`, список параметров которого совпадает с атрибутами аннотации (**то же количество параметров, в порядке объявления**). +Kora читает значения аннотации (с применёнными значениями по умолчанию) во время компиляции и передаёт их в этот метод `create(...)`; если подходящей перегрузки `create(...)` не существует, сборка завершается ошибкой. + +===! ":fontawesome-brands-java: `Java`" + + ```java + @Retention(RetentionPolicy.CLASS) + @Target({ElementType.FIELD, ElementType.PARAMETER}) + @ValidatedBy(PrefixedValidatorFactory.class) + public @interface Prefixed { + + String value(); //(1)! + } + + public interface PrefixedValidatorFactory extends ValidatorFactory { + + @Override + default Validator create() { //(2)! + throw new UnsupportedOperationException("Prefix is required"); + } + + Validator create(String prefix); //(3)! + } + ``` + + 1. Единственный атрибут аннотации. + 2. Унаследованный фабричный метод без аргументов непригоден для этого ограничения. + 3. Соответствующий `create(...)` с одним параметром; Kora передаёт `value()` в `prefix`. + +=== ":simple-kotlin: `Kotlin`" + + ```kotlin + @Retention(AnnotationRetention.RUNTIME) + @Target(AnnotationTarget.FIELD, AnnotationTarget.PROPERTY, AnnotationTarget.VALUE_PARAMETER) + @ValidatedBy(PrefixedValidatorFactory::class) + annotation class Prefixed(val value: String) //(1)! + + interface PrefixedValidatorFactory : ValidatorFactory { + + override fun create(): Validator = //(2)! + throw UnsupportedOperationException("Prefix is required") + + fun create(prefix: String): Validator //(3)! + } + ``` + + 1. Единственный атрибут аннотации. + 2. Унаследованный фабричный метод без аргументов непригоден для этого ограничения. + 3. Соответствующий `create(...)` с одним параметром; Kora передаёт `value` в `prefix`. + +Фабрика регистрируется как компонент точно так же, как и в случае без параметров (шаг 3 выше). +Это тот же механизм, который используют встроенные ограничения, и их публичные интерфейсы фабрик предоставляют переиспользуемые перегрузки, которым может делегировать пользовательская фабрика: + +- `RangeValidatorFactory` — `create(double from, double to)` и `create(double from, double to, Range.Boundary boundary)`. +- `SizeValidatorFactory` — `create(int to)` и `create(int from, int to)`. +- `PatternValidatorFactory` — `create(String pattern)` и `create(String pattern, int flags)`. +- `NotEmptyValidatorFactory` и `NotBlankValidatorFactory` — `create()` без параметров. + ## Сигнатуры { #signatures } -Доступные сигнатуры для методов которые поддерживают аннотации из коробки: +Сигнатуры методов, поддерживаемые аспектом `@Validate` из коробки: ===! ":fontawesome-brands-java: `Java`" Класс не должен быть `final`, чтобы аспекты работали. - Под `T` подразумевается тип возвращаемого значения. + `T` означает тип возвращаемого значения. - `T myMethod()` - `Optional myMethod()` - `CompletionStage myMethod()` [CompletionStage](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletionStage.html) - - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) - - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (надо подключить [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Mono myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (требует [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) + - `Flux myMethod()` [Project Reactor](https://projectreactor.io/docs/core/release/reference/) (требует [зависимость](https://mvnrepository.com/artifact/io.projectreactor/reactor-core)) === ":simple-kotlin: `Kotlin`" Класс должен быть `open`, чтобы аспекты работали. - Под `T` подразумевается тип возвращаемого значения, либо `T?`, либо `Unit`. + `T` означает тип возвращаемого значения, `T?` или `Unit`. - `myMethod(): T` - - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) - - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (надо подключить [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `suspend myMethod(): T` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требует [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) + - `myMethod(): Flow` [Kotlin Coroutine](https://kotlinlang.org/docs/coroutines-basics.html#your-first-coroutine) (требует [зависимость](https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core) как `implementation`) diff --git a/mkdocs/docs/ru/guides/database-cassandra.md b/mkdocs/docs/ru/guides/database-cassandra.md index 6eb4c1a..9d8cb79 100644 --- a/mkdocs/docs/ru/guides/database-cassandra.md +++ b/mkdocs/docs/ru/guides/database-cassandra.md @@ -101,7 +101,7 @@ Cassandra в Kora оставляют CQL явным и при этом гене Явный CQL остается видимым при проверке кода, а повторяющийся код драйвера генерируется. -### Сущности и сопоставление строк { #dao-models } +### Отображения и сопоставление строк { #dao-models } HTTP DTO и строки базы данных должны оставаться раздельными. `UserRequest` и `UserResponse` описывают вход и выход API. `UserDAO` описывает форму строки базы данных. @@ -247,7 +247,7 @@ Kora подключает Cassandra через `CassandraDatabaseModule`. Гра } ``` -## Сущность БД { #entity-db } +## Отображение { #view-db } Замените старую модель хранения в памяти на Cassandra DAO-модель, которую используют сопоставления репозитория. diff --git a/mkdocs/docs/ru/guides/database-jdbc.md b/mkdocs/docs/ru/guides/database-jdbc.md index 7a0e581..273df22 100644 --- a/mkdocs/docs/ru/guides/database-jdbc.md +++ b/mkdocs/docs/ru/guides/database-jdbc.md @@ -105,7 +105,7 @@ JDBC-репозитории Kora сохраняют явный SQL, но уби Явный SQL используется намеренно. Он делает контракт доступа к данным простым для чтения, проверки и оптимизации. Сгенерированная реализация берет на себя связующий код фреймворка, а сам запрос остается видимым в репозитории. -### Сущности и сопоставление строк { #dao-models } +### Отображения и сопоставление строк { #dao-models } HTTP DTO и строки базы данных не всегда одно и то же. DTO ответа описывает то, что возвращает API. DAO-модель описывает то, как данные хранятся и читаются из базы данных. В маленьких примерах они могут выглядеть похоже, но четкое разделение помогает по мере роста систем. @@ -253,7 +253,7 @@ JDBC также вводит инфраструктуру времени вып } ``` -## Сущность БД { #entity-db } +## Отображение { #view-db } Замените старую модель хранения `User` в памяти на JDBC DAO-модель, которую используют сопоставления репозитория. diff --git a/mkdocs/docs/ru/guides/dependency-injection-introduction.md b/mkdocs/docs/ru/guides/dependency-injection-introduction.md index f682f57..20a2976 100644 --- a/mkdocs/docs/ru/guides/dependency-injection-introduction.md +++ b/mkdocs/docs/ru/guides/dependency-injection-introduction.md @@ -1572,7 +1572,7 @@ Gradle-модулей, содержащих интерфейсы `@KoraApp` ил - класс с модификатором `final` (если не применены AOP-аспекты) - параметры конструктора становятся зависимостями -### Базовые фабричные методы { #basic-factory-methods } +### Базовые фабричные методы { #method-factory-basics } Методы по умолчанию в интерфейсах `@KoraApp` или `@Module`, которые возвращают компоненты: diff --git a/mkdocs/docs/ru/guides/dependency-injection.md b/mkdocs/docs/ru/guides/dependency-injection.md index 1cda0de..7da010d 100644 --- a/mkdocs/docs/ru/guides/dependency-injection.md +++ b/mkdocs/docs/ru/guides/dependency-injection.md @@ -1310,7 +1310,7 @@ java -version **Зачем это нужно**: библиотеки должны предоставлять надежные значения по умолчанию, но приложения должны сохранять окончательный контроль над поведением, видимым для предметной области. Это соответствует разделам [Внедрение зависимостей с Kora: стандартная фабрика](dependency-injection-introduction.md#defaultcomponent-factory), [@DefaultComponent](dependency-injection-introduction.md#defaultcomponent) -и [документация контейнера: стандартная фабрика](../documentation/container.md#standard-factory). +и [документация контейнера: стандартная фабрика](../documentation/container.md#default-factory). **Что мы имитируем**: настройку общего библиотечного уведомителя под конкретное приложение без ответвления или полного переписывания модуля.