From fceb9f3e08ac206ab4a540fce1546662ce2fbe9c Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 17:57:39 +0200 Subject: [PATCH 1/6] Document async replication enabled by default in v1.38 Async replication now runs automatically for any collection with a replication factor > 1; the per-collection asyncEnabled flag is removed. Update the async-rep, replication, env-var, runtime-config, collections config, multi-node-setup, horizontal-scaling and consistency pages, and the related includes. Replace the removed CLUSTER_MAX_WORKERS / ALIVE_NODES_CHECKING_FREQUENCY env vars with SCHEDULER_WORKERS / HASHTREE_INIT_CONCURRENCY, keeping the removed entries with notes. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...sync-replication-per-collection-config.mdx | 2 +- .../configuration/replication-consistency.mdx | 3 +- _includes/collection-mutable-parameters.mdx | 1 - docs/deploy/configuration/async-rep.md | 46 ++++++++++++------- docs/deploy/configuration/env-vars/index.md | 10 ++-- .../configuration/env-vars/runtime-config.md | 4 +- .../configuration/horizontal-scaling.mdx | 1 - docs/deploy/configuration/replication.md | 14 ++++-- .../replication-architecture/consistency.md | 8 ++-- docs/weaviate/config-refs/collections.mdx | 9 ++-- .../manage-collections/multi-node-setup.mdx | 2 - 11 files changed, 57 insertions(+), 43 deletions(-) diff --git a/_includes/async-replication-per-collection-config.mdx b/_includes/async-replication-per-collection-config.mdx index 14cb1dd7f..8f7c90444 100644 --- a/_includes/async-replication-per-collection-config.mdx +++ b/_includes/async-replication-per-collection-config.mdx @@ -1,3 +1,3 @@ :::info Collection-level configuration — Added in `v1.36` -Async replication parameters can also be set per-collection via the `asyncConfig` object in `replicationConfig`. Per-collection settings override the cluster-wide environment variable defaults. See [Collection `asyncConfig` parameters](/weaviate/config-refs/collections#async-config) for details. +Async replication runs by default for any collection with a replication factor greater than `1` (as of `v1.38`). To fine-tune its behavior for a specific collection, set the `asyncConfig` object in `replicationConfig`. Per-collection settings override the cluster-wide environment variable defaults. See [Collection `asyncConfig` parameters](/weaviate/config-refs/collections#async-config) for details. ::: diff --git a/_includes/code/configuration/replication-consistency.mdx b/_includes/code/configuration/replication-consistency.mdx index 9e1e6d0cc..1392a7afd 100644 --- a/_includes/code/configuration/replication-consistency.mdx +++ b/_includes/code/configuration/replication-consistency.mdx @@ -43,8 +43,7 @@ curl \ } ], "replicationConfig": { - "factor": 3, - "asyncEnabled": true + "factor": 3 } }' \ http://localhost:8080/v1/schema diff --git a/_includes/collection-mutable-parameters.mdx b/_includes/collection-mutable-parameters.mdx index 67fca22d9..9f177f235 100644 --- a/_includes/collection-mutable-parameters.mdx +++ b/_includes/collection-mutable-parameters.mdx @@ -14,7 +14,6 @@ - `autoTenantCreation` (introduced in `v1.25.0`) - `autoTenantActivation` (introduced in `v1.25.2`) - `replicationConfig` - - `asyncEnabled` (introduced in `v1.26.0`) - `factor` (not mutable in `v1.25` or higher) - `deletionStrategy` (introduced in `v1.27.0`) - `vectorIndexConfig` diff --git a/docs/deploy/configuration/async-rep.md b/docs/deploy/configuration/async-rep.md index cead8e7c6..1b109c410 100644 --- a/docs/deploy/configuration/async-rep.md +++ b/docs/deploy/configuration/async-rep.md @@ -16,7 +16,8 @@ This applies solely to data objects, as metadata consistency is treated differen ### Under the Hood - Async replication operates as a background process either per tenant (in a multi-tenant collection) or per shard (in a non-multi-tenant collection). -- It is disabled by default but can be enabled through collection configuration changes, similar to setting the replication factor. +- As of Weaviate `v1.38`, async replication is **enabled by default** for any collection with a replication factor greater than `1`. There is no longer a per-collection toggle to enable or disable it; the per-collection `asyncConfig` object is used only to fine-tune the behavior of an already-running process. +- To turn async replication off across the entire cluster, set the [`ASYNC_REPLICATION_DISABLED`](#async_replication_disabled) environment variable to `true`. ## Environment Variable Deep Dive @@ -48,13 +49,36 @@ Globally disables the entire async replication feature.
Cluster Worker Limits -#### `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` -Sets the maximum number of concurrent async replication workers across the entire cluster. +#### `ASYNC_REPLICATION_SCHEDULER_WORKERS` +Sets the number of workers in the cluster-wide pool that the async replication scheduler uses to run hashbeat work across all shards and tenants. -- Its default value is `30`. -- **Use case**: Limits the total number of concurrent replication workers to prevent resource exhaustion in large clusters with many collections or tenants. +- Its default value is `10`. The maximum is `100`. +- **Use case**: A single bounded worker pool replaces the previous per-shard goroutines, so this is the main lever for capping async replication's total concurrency and preventing resource exhaustion on clusters with many collections or tenants. - **Special Considerations**: - - This is a cluster-wide cap. Individual collections can set their own `maxWorkers` via the per-collection [`asyncConfig`](/weaviate/config-refs/collections#async-config), but the total across all collections will not exceed this cluster limit. + - This is a cluster-wide setting. There is no per-collection worker count; collections share this single pool. + +:::note Changed in `v1.38` +`ASYNC_REPLICATION_SCHEDULER_WORKERS` replaces the removed `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` environment variable, and the per-collection `maxWorkers` option has been removed. +::: + +#### `ASYNC_REPLICATION_HASHTREE_INIT_CONCURRENCY` +Sets how many shards may initialize (build) their hash tree concurrently when async replication starts up. + +- Its default value is `100`. +- **Use case**: Bounds the burst of work when many shards begin async replication at once, for example after a node restart or when many replicated collections exist. + +
+ +
+ Removed environment variables (v1.38) + +These variables were removed when async replication moved to a centralized scheduler in `v1.38`. They are listed here for reference and are no longer read by Weaviate. + +#### `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` +**Removed in `v1.38`.** Previously set the maximum number of concurrent async replication workers across the cluster (default `30`). Replaced by [`ASYNC_REPLICATION_SCHEDULER_WORKERS`](#async_replication_scheduler_workers). + +#### `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` +**Removed in `v1.38`.** Previously defined how often the background process checked for changes in node availability (default `5s`). The scheduler no longer uses a separate alive-nodes polling mechanism.
@@ -153,16 +177,6 @@ Defines a shorter frequency for subsequent comparison and propagation attempts w -
- Node Status Monitoring - -#### `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` -Defines the frequency at which the system checks for changes in the availability of nodes within the cluster. - - Its default value is `5s`. The value requires a time unit suffix (e.g. `5s`, `1m`). - - **Use Case(s)**: When a node rejoins the cluster after a period of downtime, it is highly likely to be out of sync. This setting ensures that the replication process is initiated promptly. - -
-
Timeout Management diff --git a/docs/deploy/configuration/env-vars/index.md b/docs/deploy/configuration/env-vars/index.md index 3db54c30e..579e9b22c 100644 --- a/docs/deploy/configuration/env-vars/index.md +++ b/docs/deploy/configuration/env-vars/index.md @@ -225,9 +225,7 @@ For more information on authentication and authorization, see the [Authenticatio | `RAFT_BOOTSTRAP_EXPECT` | The number of voter notes at bootstrapping time | `string - number` | `1` | | `RAFT_BOOTSTRAP_TIMEOUT` | The time in seconds to wait for the cluster to bootstrap | `string - number` | `90` | | `RAFT_DRAIN_SLEEP` | Grace period before shutdown to allow ongoing operations to complete. (Default: `200ms`) | `string - number` | `2s` | -| `RAFT_ENABLE_FQDN_RESOLVER` | If `true`, use DNS lookup instead of memberlist lookup for Raft. Removed in `v1.30`. ([Read more](/weaviate/concepts/cluster.md#node-discovery)) | `boolean` | `true` | | `RAFT_ENABLE_ONE_NODE_RECOVERY` | Enable running the single node recovery routine on restart. This is useful if the default hostname has changed and a single node cluster believes there are supposed to be two nodes. | `boolean` | `false` | -| `RAFT_FQDN_RESOLVER_TLD` | The top-level domain to use for DNS lookup, in `[node-id].[tld]` format. Removed in `v1.30`. ([Read more](/weaviate/concepts/cluster.md#node-discovery)) | `string` | `example.com` | | `RAFT_GRPC_MESSAGE_MAX_SIZE` | The maximum internal raft gRPC message size in bytes. Defaults to 1073741824 | `string - number` | `1073741824` | | `RAFT_JOIN` | Manually set Raft voter nodes. If set, RAFT_BOOTSTRAP_EXPECT needs to be adjusted manually to match the number of Raft voters. | `string` | `weaviate-0,weaviate-1` | | `RAFT_METADATA_ONLY_VOTERS` | If `true`, voter nodes only handle the schema. They do not accept any data. | `boolean` | `false` | @@ -250,12 +248,14 @@ For more information on authentication and authorization, see the [Authenticatio | Variable | Description | Type | Example Value | | --- | --- | --- | --- | -| `ASYNC_REPLICATION_DISABLED` | Disable async replication. Default: `false` | `boolean` | `false` | -| `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` | Maximum concurrent async replication workers across the cluster. Default: `30` | `string - number` | `10` | +| `ASYNC_REPLICATION_DISABLED` | Disable async replication cluster-wide. When `false` (default), async replication runs automatically for any collection with a replication factor greater than `1`. Default: `false` | `boolean` | `false` | +| `ASYNC_REPLICATION_SCHEDULER_WORKERS` | Number of workers in the cluster-wide pool that run async replication work across all shards and tenants. Added in `v1.38`, replacing `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS`. Default: `10`, Max: `100`
[Read more.](/deploy/configuration/async-rep.md#async_replication_scheduler_workers) | `string - number` | `10` | +| `ASYNC_REPLICATION_HASHTREE_INIT_CONCURRENCY` | Number of shards that may build their hash tree concurrently when async replication starts up. Added in `v1.38`. Default: `100`
[Read more.](/deploy/configuration/async-rep.md#async_replication_hashtree_init_concurrency) | `string - number` | `100` | +| `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` | **Removed in `v1.38`.** Previously set the maximum number of concurrent async replication workers across the cluster. Replaced by `ASYNC_REPLICATION_SCHEDULER_WORKERS`. | `string - number` | `30` | | `ASYNC_REPLICATION_HASHTREE_HEIGHT` | Height of the hash tree used for data comparison between nodes. If the height is `0` each node will store just one digest per shard. Default: `16` (single-tenant) / `10` (multi-tenant), Min: `0`, Max: `20`
[Read more about potentially increased memory consumption.](/weaviate/concepts/replication-architecture/consistency#memory-and-performance-considerations-for-async-replication) | `string - number` | `10` | | `ASYNC_REPLICATION_FREQUENCY` | Frequency of periodic data comparison between nodes. Default: `30s` | `string - duration` | `60s` | | `ASYNC_REPLICATION_FREQUENCY_WHILE_PROPAGATING` | Frequency of data comparison between nodes while propagation is active. Default: `3s` | `string - duration` | `5s` | -| `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` | Frequency of how often the background process checks for changes in the availability of nodes. Default: `5s` | `string - duration` | `20s` | +| `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` | **Removed in `v1.38`.** Previously set how often the background process checked for changes in node availability. No longer used by the async replication scheduler. | `string - duration` | `5s` | | `ASYNC_REPLICATION_LOGGING_FREQUENCY` | Frequency of how often the background process logs any events. Default: `60s` | `string - duration` | `7s` | | `ASYNC_REPLICATION_DIFF_BATCH_SIZE` | Specifies the batch size for comparing digest information between nodes. Default: `1000`, Min: `1`, Max: `10000` |`string - number` | `2000` | | `ASYNC_REPLICATION_DIFF_PER_NODE_TIMEOUT` | Defines the time limit a node has to provide a comparison response. Default: `10s` | `string - duration` | `30s` | diff --git a/docs/deploy/configuration/env-vars/runtime-config.md b/docs/deploy/configuration/env-vars/runtime-config.md index f0c294cdf..8333eff73 100644 --- a/docs/deploy/configuration/env-vars/runtime-config.md +++ b/docs/deploy/configuration/env-vars/runtime-config.md @@ -57,7 +57,9 @@ The following overrides are currently supported: | Runtime override name | Environment variable name | | :----------------------------------------------- | :------------------------------------------- | | `async_replication_disabled` | `ASYNC_REPLICATION_DISABLED` | -| `async_replication_cluster_max_workers` | `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` | +| `async_replication_scheduler_workers` | `ASYNC_REPLICATION_SCHEDULER_WORKERS` | +| `async_replication_hashtree_init_concurrency` | `ASYNC_REPLICATION_HASHTREE_INIT_CONCURRENCY`| +| `async_replication_cluster_max_workers` _(removed in `v1.38`)_ | `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` _(removed in `v1.38`)_ | | `autoschema_enabled` | `AUTOSCHEMA_ENABLED` | | `default_quantization` | `DEFAULT_QUANTIZATION` | | `default_sharding_count` | `DEFAULT_SHARDING_COUNT` | diff --git a/docs/deploy/configuration/horizontal-scaling.mdx b/docs/deploy/configuration/horizontal-scaling.mdx index 2b6b9c44b..e1e8a64c5 100644 --- a/docs/deploy/configuration/horizontal-scaling.mdx +++ b/docs/deploy/configuration/horizontal-scaling.mdx @@ -338,7 +338,6 @@ curl \ ], "replicationConfig": { "factor": 3, - "asyncEnabled": true, "deletionStrategy": "TimeBasedResolution" } }' \ diff --git a/docs/deploy/configuration/replication.md b/docs/deploy/configuration/replication.md index c1cfca1ba..989a1ed99 100644 --- a/docs/deploy/configuration/replication.md +++ b/docs/deploy/configuration/replication.md @@ -48,7 +48,7 @@ When Weaviate detects inconsistent data across nodes, it attempts to repair the Weaviate offers [async replication](/weaviate/concepts/replication-architecture/consistency.md#async-replication) to proactively detect inconsistencies. In earlier versions, Weaviate uses a [repair-on-read](/weaviate/concepts/replication-architecture/consistency.md#repair-on-read) strategy to repair inconsistencies at read time. -Repair-on-read is automatic. To activate async replication, set `asyncEnabled` to true in the `replicationConfig` section of your collection definition. +Repair-on-read is automatic. As of Weaviate `v1.38`, async replication is also **enabled by default** for any collection with a replication factor greater than `1` — there is no longer a per-collection flag to switch it on. To turn it off cluster-wide, set the [`ASYNC_REPLICATION_DISABLED`](/deploy/configuration/env-vars/index.md#async-replication) environment variable to `true`. The `replicationConfig` section is used to set the replication factor and to fine-tune async replication via `asyncConfig`: import ReplicationConfigWithAsyncRepair from '/\_includes/code/configuration/replication-consistency.mdx'; @@ -64,8 +64,10 @@ Update the following [environment variables](/deploy/configuration/env-vars/inde #### Worker limits -- **Set the cluster-wide worker cap:** `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` - Set the maximum number of concurrent async replication workers across the entire cluster. Default: `30`. Individual collections can set their own `maxWorkers` via `asyncConfig`, but the total across all collections will not exceed this cluster limit. +- **Set the scheduler worker pool size:** `ASYNC_REPLICATION_SCHEDULER_WORKERS` + Set the number of workers in the cluster-wide pool that run async replication work across all shards and tenants. Default: `10`, maximum: `100`. As of `v1.38` this replaces the removed `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` variable and the per-collection `maxWorkers` option; collections share this single pool. +- **Set hash tree init concurrency:** `ASYNC_REPLICATION_HASHTREE_INIT_CONCURRENCY` + Set how many shards may build their hash tree concurrently when async replication starts up. Default: `100`. #### Logging @@ -78,13 +80,15 @@ Update the following [environment variables](/deploy/configuration/env-vars/inde Define how often each node compares its local data with other nodes. - **Set comparison timeout:** `ASYNC_REPLICATION_DIFF_PER_NODE_TIMEOUT` Optionally configure a timeout for how long to wait during comparison when a node is unresponsive. -- **Monitor node availability:** `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` - Trigger comparisons whenever there’s a change in node availability. - **Configure hash tree height:** `ASYNC_REPLICATION_HASHTREE_HEIGHT` Specify the size of the hash tree, which helps narrow down data differences by comparing hash digests at multiple levels instead of scanning entire datasets. See [this page](/weaviate/concepts/replication-architecture/consistency.md#memory-and-performance-considerations-for-async-replication) for more information on the memory and performance considerations for async replication. - **Batch size for digest comparison:** `ASYNC_REPLICATION_DIFF_BATCH_SIZE` Define the number of objects whose digest (e.g., last update time) is compared between nodes before propagating actual objects. +:::note Removed in `v1.38` +The `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` (replaced by `ASYNC_REPLICATION_SCHEDULER_WORKERS`) and `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` environment variables were removed in `v1.38` when async replication moved to a centralized scheduler. +::: + #### Data synchronization Once differences between nodes are detected, Weaviate propagates outdated or missing data. Configure synchronization as follows: diff --git a/docs/weaviate/concepts/replication-architecture/consistency.md b/docs/weaviate/concepts/replication-architecture/consistency.md index afc3addc0..684e69b4e 100644 --- a/docs/weaviate/concepts/replication-architecture/consistency.md +++ b/docs/weaviate/concepts/replication-architecture/consistency.md @@ -90,7 +90,7 @@ Adding or changing data objects are **write** operations. Write operations are tunable starting with Weaviate v1.18, to `ONE`, `QUORUM` (default) or `ALL`. In v1.17, write operations are always set to `ALL` (highest consistency). ::: -The main reason for introducing configurable write consistency in v1.18 is because that is also when automatic repairs are introduced. A write will always be written to n (replication factor) nodes, regardless of the chosen consistency level. The coordinator node however waits for acknowledgments from `ONE`, `QUORUM` or `ALL` nodes before it returns. To guarantee that a write is applied everywhere without the availability of repairs on read requests, write consistency is set to `ALL` for now. Possible settings in v1.18+ are: +The main reason for introducing configurable write consistency in v1.18 is because that is also when automatic repairs are introduced. A write will always be written to n (replication factor) nodes, regardless of the chosen consistency level. The coordinator node however waits for acknowledgments from `ONE`, `QUORUM` or `ALL` nodes before it returns. To guarantee that a write is acknowledged everywhere before the request returns, without relying on read-time repairs, set write consistency to `ALL`. Possible settings in v1.18+ are: * **ONE** - a write must receive an acknowledgment from at least one replica node. This is the fastest (most available), but least consistent option. * **QUORUM** - a write must receive an acknowledgment from at least `QUORUM` replica nodes. `QUORUM` is calculated as _n / 2 + 1_, where _n_ is the number of replicas (replication factor). For example, using a replication factor of 6, the quorum is 4, which means the cluster can tolerate 2 replicas down. * **ALL** - a write must receive an acknowledgment from all replica nodes. This is the most consistent, but 'slowest' (least available) option. @@ -197,7 +197,7 @@ Repair-on-read works well with one or two isolated repairs. Async replication is Async replication supplements the repair-on-read mechanism. If a node becomes inconsistent between sync checks, the repair-on-read mechanism catches the problem at read time. -To activate async replication, set `asyncEnabled` to true in the [`replicationConfig` section of your collection definition](../../manage-collections/multi-node-setup.mdx#replication-settings). Visit the [How-to: Replication](/deploy/configuration/replication.md#async-replication-settings) page to learn more about the available async replication settings. +As of Weaviate `v1.38`, async replication is enabled by default for any collection with a replication factor greater than `1`, there is no per-collection flag to enable it. To turn it off cluster-wide, set the `ASYNC_REPLICATION_DISABLED` environment variable to `true`. Visit the [How-to: Replication](/deploy/configuration/replication.md#async-replication-settings) page to learn more about the available async replication settings. #### Memory and performance considerations for async replication @@ -262,9 +262,9 @@ A larger hash tree means less data for each leaf to hash, leading to faster comp - `Number of Leaves = 2^20 = 1,048,576` :::note Default settings -The default hash tree height is `16` for single-tenant collections and `10` for multi-tenant collections. These defaults balance memory consumption with replication performance. Similarly, the default `maxWorkers` is `3` for single-tenant collections and `30` for multi-tenant collections. +The default hash tree height is `16` for single-tenant collections and `10` for multi-tenant collections. These defaults balance memory consumption with replication performance. -As of `v1.36`, these parameters can be configured per-collection via the [`asyncConfig`](/weaviate/config-refs/collections#async-config) object in `replicationConfig`, overriding the cluster-wide environment variable defaults. +As of `v1.36`, these parameters can be configured per-collection via the [`asyncConfig`](/weaviate/config-refs/collections#async-config) object in `replicationConfig`, overriding the cluster-wide environment variable defaults. Worker concurrency is no longer a per-collection setting and as of `v1.38` the cluster shares a single async replication worker pool sized by [`ASYNC_REPLICATION_SCHEDULER_WORKERS`](/deploy/configuration/env-vars/index.md#async-replication). ::: ### Deletion resolution strategies diff --git a/docs/weaviate/config-refs/collections.mdx b/docs/weaviate/config-refs/collections.mdx index dad5f2226..1c233a1c7 100644 --- a/docs/weaviate/config-refs/collections.mdx +++ b/docs/weaviate/config-refs/collections.mdx @@ -677,10 +677,13 @@ More details about the `vectorIndexType` and `vectorIndexConfig` parameters can | Parameter | Type | Description | Default | Mutable | | :----------------- | :------ | :-------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------ | :-------------------------------------- | | `factor` | Integer | The number of copies (replicas) to maintain for each shard. A factor of `3` means one primary and two replicas. | `1` | No | -| `asyncEnabled` | Boolean | Enable asynchronous replication. | `false` | Yes | | `deletionStrategy` | String | Strategy for handling deletions in replication. Can be `NoAutomatedResolution`, `DeleteOnConflict` or `TimeBasedResolution`. | `"TimeBasedResolution"` | Yes | | `asyncConfig` | Object | Configuration for async replication tuning. See [`asyncConfig` parameters](#async-config) below. Added in `v1.36` | See below | Yes | +:::note Async replication is on by default (`v1.38`) +The `asyncEnabled` flag has been removed. As of Weaviate `v1.38`, async replication runs automatically for any collection with a `factor` greater than `1`. To turn it off, set the [`ASYNC_REPLICATION_DISABLED`](/deploy/configuration/env-vars/index.md#async-replication) environment variable to `true`. +::: +
Example replication configuration - JSON object @@ -693,10 +696,8 @@ An example of a complete `replicationConfig` object: // highlight-start "replicationConfig": { "factor": 3, - "asyncEnabled": true, "deletionStrategy": "TimeBasedResolution", "asyncConfig": { - "maxWorkers": 5, "hashtreeHeight": 16, "frequency": 30000 } @@ -719,11 +720,9 @@ Some `asyncConfig` parameters have different defaults depending on whether the c | Parameter | Type | Description | Default (single-tenant) | Default (multi-tenant) | | :--- | :--- | :--- | :--- | :--- | -| `maxWorkers` | Integer | Maximum number of concurrent async replication workers for this collection. | `3` | `30` | | `hashtreeHeight` | Integer | Height of the hash tree used for data comparison between nodes. Min: `0`, Max: `20` | `16` | `10` | | `frequency` | Integer | Frequency of periodic data comparison between nodes, in milliseconds. | `30000` | `30000` | | `frequencyWhilePropagating` | Integer | Frequency of data comparison while propagation is active, in milliseconds. | `3000` | `3000` | -| `aliveNodesCheckingFrequency` | Integer | Frequency of checking node availability, in milliseconds. | `5000` | `5000` | | `loggingFrequency` | Integer | How often the async replication process logs its activity, in seconds. | `60` | `60` | | `diffBatchSize` | Integer | Number of object keys fetched per request during comparison. Min: `1`, Max: `10000` | `1000` | `1000` | | `diffPerNodeTimeout` | Integer | Timeout for a comparison response from a remote node, in seconds. | `10` | `10` | diff --git a/docs/weaviate/manage-collections/multi-node-setup.mdx b/docs/weaviate/manage-collections/multi-node-setup.mdx index 82160978a..051d3d7e3 100644 --- a/docs/weaviate/manage-collections/multi-node-setup.mdx +++ b/docs/weaviate/manage-collections/multi-node-setup.mdx @@ -85,10 +85,8 @@ curl \ ], "replicationConfig": { "factor": 3, - "asyncEnabled": true, "deletionStrategy": "TimeBasedResolution", "asyncConfig": { - "maxWorkers": 5, "hashtreeHeight": 16, "frequency": 30000 } From 015254a682b90d95c945c4d487e4c961d2b2b1ec Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 17:57:49 +0200 Subject: [PATCH 2/6] Update code examples to drop removed asyncEnabled field Remove asyncEnabled/async_enabled arguments and assertions, and the removed per-collection maxWorkers option, from the Python, TypeScript, Java and Go replication snippets. Co-Authored-By: Claude Opus 4.8 (1M context) --- _includes/code/config-refs/reference.collections.py | 3 --- _includes/code/howto/go/docs/manage-data.classes_test.go | 2 +- _includes/code/howto/manage-data.collections.py | 6 +----- _includes/code/howto/manage-data.collections.ts | 5 +---- .../code/java-v6/src/test/java/ManageCollectionsTest.java | 7 +++---- 5 files changed, 6 insertions(+), 17 deletions(-) diff --git a/_includes/code/config-refs/reference.collections.py b/_includes/code/config-refs/reference.collections.py index 80d29c11c..5f1871bd9 100644 --- a/_includes/code/config-refs/reference.collections.py +++ b/_includes/code/config-refs/reference.collections.py @@ -57,7 +57,6 @@ ), replication_config=Configure.replication( factor=1, - async_enabled=False, deletion_strategy=ReplicationDeletionStrategy.TIME_BASED_RESOLUTION, ), ) @@ -733,7 +732,6 @@ # highlight-start replication_config=Configure.replication( factor=3, - async_enabled=True, deletion_strategy=ReplicationDeletionStrategy.TIME_BASED_RESOLUTION, ), # highlight-end @@ -743,7 +741,6 @@ # Test collection = client.collections.use("Article") config = collection.config.get() -assert config.replication_config.async_enabled == True assert ( config.replication_config.deletion_strategy == ReplicationDeletionStrategy.TIME_BASED_RESOLUTION diff --git a/_includes/code/howto/go/docs/manage-data.classes_test.go b/_includes/code/howto/go/docs/manage-data.classes_test.go index a6f07668b..cfc4209c2 100644 --- a/_includes/code/howto/go/docs/manage-data.classes_test.go +++ b/_includes/code/howto/go/docs/manage-data.classes_test.go @@ -585,8 +585,8 @@ func Test_ManageDataClasses(t *testing.T) { articleClass := &models.Class{ Class: "Article", Description: "Collection of articles", + // Async replication runs by default when the replication factor is greater than 1 ReplicationConfig: &models.ReplicationConfig{ - AsyncEnabled: true, Factor: 3, DeletionStrategy: models.ReplicationConfigDeletionStrategyTimeBasedResolution, }, diff --git a/_includes/code/howto/manage-data.collections.py b/_includes/code/howto/manage-data.collections.py index ffab3b269..994eac792 100644 --- a/_includes/code/howto/manage-data.collections.py +++ b/_includes/code/howto/manage-data.collections.py @@ -786,9 +786,9 @@ client.collections.create( "Article", # highlight-start + # Async replication runs by default when the replication factor is greater than 1 replication_config=Configure.replication( factor=3, - async_enabled=True, ), # highlight-end ) @@ -819,10 +819,8 @@ # highlight-start replication_config=Configure.replication( factor=3, - async_enabled=True, deletion_strategy=ReplicationDeletionStrategy.TIME_BASED_RESOLUTION, async_config=Configure.Replication.async_config( - max_workers=5, hashtree_height=16, frequency=30, ), @@ -840,7 +838,6 @@ collection.config.update( replication_config=Reconfigure.replication( async_config=Reconfigure.Replication.async_config( - max_workers=10, frequency=60, ), ), @@ -851,7 +848,6 @@ # Test collection = client.collections.use("Article") config = collection.config.get() -assert config.replication_config.async_enabled == True assert ( config.replication_config.deletion_strategy == ReplicationDeletionStrategy.TIME_BASED_RESOLUTION diff --git a/_includes/code/howto/manage-data.collections.ts b/_includes/code/howto/manage-data.collections.ts index b12ce8e1c..8418cf75a 100644 --- a/_includes/code/howto/manage-data.collections.ts +++ b/_includes/code/howto/manage-data.collections.ts @@ -677,9 +677,9 @@ import { configure } from 'weaviate-client'; await client.collections.create({ name: 'Article', // highlight-start + // Async replication runs by default when the replication factor is greater than 1 replication: configure.replication({ factor: 1, - asyncEnabled: true, }), // highlight-end }) @@ -710,10 +710,8 @@ await replicationClient.collections.create({ // highlight-start replication: configure.replication({ factor: 3, - asyncEnabled: true, deletionStrategy: 'TimeBasedResolution', asyncConfig: { - maxWorkers: 5, hashtreeHeight: 16, frequency: 30, }, @@ -729,7 +727,6 @@ const articleReplication = replicationClient.collections.use('Article') await articleReplication.config.update({ replication: reconfigure.replication({ asyncConfig: { - maxWorkers: 10, frequency: 60, }, }), diff --git a/_includes/code/java-v6/src/test/java/ManageCollectionsTest.java b/_includes/code/java-v6/src/test/java/ManageCollectionsTest.java index f54dd8cef..62cf717d4 100644 --- a/_includes/code/java-v6/src/test/java/ManageCollectionsTest.java +++ b/_includes/code/java-v6/src/test/java/ManageCollectionsTest.java @@ -408,12 +408,13 @@ void testReplicationSettings() throws IOException { @Test void testAsyncRepair() throws IOException { // START AsyncRepair + // Async replication runs by default when the replication factor is greater than 1 client.collections.create("Article", col -> col.replication( - Replication.of(rep -> rep.replicationFactor(1).asyncEnabled(true)))); + Replication.of(rep -> rep.replicationFactor(1)))); // END AsyncRepair var config = client.collections.getConfig("Article").get(); - assertThat(config.replication().asyncEnabled()).isTrue(); + assertThat(config.replication().replicationFactor()).isEqualTo(1); } @Test @@ -421,7 +422,6 @@ void testAllReplicationSettings() throws IOException { // START AllReplicationSettings threeNodeClient.collections.create("Article", col -> col.replication(Replication.of(rep -> rep.replicationFactor(3) - .asyncEnabled(true) .deletionStrategy(DeletionStrategy.TIME_BASED_RESOLUTION) .asyncReplication(AsyncReplicationConfig.of(async -> async .propagationConcurrency(5) @@ -431,7 +431,6 @@ void testAllReplicationSettings() throws IOException { var config = threeNodeClient.collections.getConfig("Article").get(); assertThat(config.replication().replicationFactor()).isEqualTo(3); - assertThat(config.replication().asyncEnabled()).isTrue(); // START UpdateReplicationSettings var collection = threeNodeClient.collections.use("Article"); From 29c6858d9d75283af81e79e9d0f84a03dd897d77 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 17:57:49 +0200 Subject: [PATCH 3/6] Update async replication metrics in monitoring reference Remove async_replication_goroutines_running (replaced by the scheduler) and add the new scheduler gauges and diff/deletion/reconcile counters introduced in v1.38. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/deploy/configuration/monitoring.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/deploy/configuration/monitoring.md b/docs/deploy/configuration/monitoring.md index a954c13f7..b434242de 100644 --- a/docs/deploy/configuration/monitoring.md +++ b/docs/deploy/configuration/monitoring.md @@ -438,9 +438,16 @@ These metrics track Write-Ahead Log (WAL) recovery operations during startup. These metrics track asynchronous replication operations for maintaining data consistency across replicas. +:::note Changed in `v1.38` +As of Weaviate `v1.38`, async replication runs through a centralized scheduler with a bounded worker pool (replacing the previous per-shard goroutines). The `async_replication_scheduler_*` metrics below are new in `v1.38`, and the `async_replication_goroutines_running` metric has been removed. +::: + | Metric | Description | Labels | Type | | -------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------- | ----------- | -| `async_replication_goroutines_running` | Number of currently running async replication goroutines | `type` (hashbeater, hashbeat_trigger) | `Gauge` | +| `async_replication_scheduler_worker_pool_size` | Current target size of the scheduler worker pool | None | `Gauge` | +| `async_replication_scheduler_workers_active` | Number of scheduler worker goroutines currently executing a hashbeat cycle | None | `Gauge` | +| `async_replication_scheduler_shards_registered` | Number of shards currently registered with the async replication scheduler | None | `Gauge` | +| `async_replication_scheduler_queue_depth` | Number of shards waiting in the scheduler heap (not in-flight) | None | `Gauge` | | `async_replication_hashtree_init_count` | Count of async replication hashtree initializations | None | `Counter` | | `async_replication_hashtree_init_running` | Number of currently running hashtree initializations | None | `Gauge` | | `async_replication_hashtree_init_failure_count` | Count of async replication hashtree initialization failures | None | `Counter` | @@ -451,10 +458,13 @@ These metrics track asynchronous replication operations for maintaining data con | `async_replication_iteration_running` | Number of currently running async replication iterations | None | `Gauge` | | `async_replication_hashtree_diff_duration_seconds` | Duration of async replication hashtree diff computation in seconds | None | `Histogram` | | `async_replication_object_digests_diff_duration_seconds` | Duration of async replication object digests diff computation in seconds | None | `Histogram` | +| `async_replication_objects_diff_total` | Total objects found in diff per hashbeat cycle (queued for propagation or locally deleted before propagation) | None | `Counter` | | `async_replication_propagation_count` | Count of async replication propagation executions | None | `Counter` | | `async_replication_propagation_failure_count` | Count of async replication propagation failures | None | `Counter` | | `async_replication_propagation_object_count` | Count of objects propagated by async replication | None | `Counter` | | `async_replication_propagation_duration_seconds` | Duration of async replication propagation in seconds | None | `Histogram` | +| `async_replication_local_deletions_total` | Total local object deletions applied due to remote-deleted verdicts or deletion-conflict responses during async replication | None | `Counter` | +| `async_replication_reconcile_failures_total` | Number of indices that failed to reconcile async replication with the global `ASYNC_REPLICATION_DISABLED` flag | None | `Counter` | #### Replication coordinator From 69d6921feca9907562682087cbec0bad018e2bab Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 17:57:59 +0200 Subject: [PATCH 4/6] Correct outdated FQDN node discovery info in cluster concepts The RAFT_ENABLE_FQDN_RESOLVER / RAFT_FQDN_RESOLVER_TLD resolver was removed in v1.30. Replace the cluster-architecture section with a brief removal note and drop the obsolete FQDN (and pre-v1.18.1) content from the cluster concepts page. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/weaviate/concepts/cluster.md | 42 ------------------- .../cluster-architecture.md | 27 ------------ 2 files changed, 69 deletions(-) diff --git a/docs/weaviate/concepts/cluster.md b/docs/weaviate/concepts/cluster.md index a57e34cc8..788efba8c 100644 --- a/docs/weaviate/concepts/cluster.md +++ b/docs/weaviate/concepts/cluster.md @@ -112,54 +112,12 @@ By default, Weaviate nodes in a cluster use a gossip-like protocol through [Hash Weaviate - especially when running as a cluster - is optimized to run on Kubernetes. The [Weaviate Helm chart](/deploy/installation-guides/k8s-installation.md#weaviate-helm-chart) makes use of a `StatefulSet` and a headless `Service` that automatically configures node discovery. All you have to do is specify the desired node count. -
- FQDN for node discovery - -:::caution Removed in `v1.30` - -This was an experimental feature. Use with caution. - -::: - -There can be a situation where IP-address based node discovery is not optimal. In such cases, you can set `RAFT_ENABLE_FQDN_RESOLVER` and `RAFT_FQDN_RESOLVER_TLD` [environment variables](/deploy/configuration/env-vars/index.md#multi-node-instances) to enable fully qualified domain name (FQDN) based node discovery. - -If this feature is enabled, Weaviate uses the FQDN resolver to resolve the node name to the node IP address for metadata (e.g., Raft) communication. - -:::info FQDN: For metadata changes only -This feature is only used for metadata changes which [use Raft as the consensus mechanism](./replication-architecture/cluster-architecture.md#metadata-replication-raft). It does not affect data read/write operations. -::: - -#### Examples of when to use FQDN for node discovery - -The use of FQDN can resolve a situation where if IP addresses are re-used across different clusters, the nodes in one cluster could mistakenly discover nodes in another cluster. - -It can also be useful when using services (for example, Kubernetes) where the IP of the services is different from the actual node IP, but it proxies the connection to the node. - -#### Environment variables for FQDN node discovery - -`RAFT_ENABLE_FQDN_RESOLVER` is a Boolean flag. This flag enables the FQDN resolver. If set to `true`, Weaviate uses the FQDN resolver to resolve the node name to the node IP address. If set to `false`, Weaviate uses the memberlist lookup to resolve the node name to the node IP address. The default value is `false`. - -`RAFT_FQDN_RESOLVER_TLD` is a string that is appended in the format `[node-id].[tld]` when resolving a node-id to an IP address, where `[tld]` is the top-level domain. - -To use this feature, set `RAFT_ENABLE_FQDN_RESOLVER` to `true`. - -
- ## Node affinity of shards and/or replication shards Weaviate tries to select the node with the most available disk space. This only applies when creating a new class, rather than when adding more data to an existing single class. -
- Pre-v1.18.1 behavior - -In versions `v1.8.0`-`v1.18.0`, users could not specify the node-affinity of a specific shard or replication shard. - -Shards were assigned to 'live' nodes in a round-robin fashion starting with a random node. - -
- ## Consistency and current limitations * Weaviate adopts the [Raft consensus algorithm](https://raft.github.io/) which is a log-based algorithm coordinated by an elected leader. This brings an additional benefit in that concurrent schema changes are now supported.
If you are a Kubernetes user, see the [`1.25 migration guide`](/deploy/migration/weaviate-1-25.md) before you upgrade. To upgrade, you have to delete your existing StatefulSet. diff --git a/docs/weaviate/concepts/replication-architecture/cluster-architecture.md b/docs/weaviate/concepts/replication-architecture/cluster-architecture.md index 53ac6f211..0a9b08d2d 100644 --- a/docs/weaviate/concepts/replication-architecture/cluster-architecture.md +++ b/docs/weaviate/concepts/replication-architecture/cluster-architecture.md @@ -17,33 +17,6 @@ By default, Weaviate nodes in a cluster use a gossip-like protocol through [Hash Weaviate is optimized to run on Kubernetes, especially when operating as a cluster. The [Weaviate Helm chart](/deploy/installation-guides/k8s-installation.md#weaviate-helm-chart) makes use of a `StatefulSet` and a headless `Service` that automatically configures node discovery. -
- FQDN for node discovery - -There can be a situation where IP-address based node discovery is not optimal. In such cases, you can set `RAFT_ENABLE_FQDN_RESOLVER` and `RAFT_FQDN_RESOLVER_TLD` [environment variables](/deploy/configuration/env-vars/index.md#multi-node-instances) to enable [fully qualified domain name (FQDN)](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) based node discovery. - -If this feature is enabled, Weaviate uses the FQDN resolver to resolve the node name to the node IP address for metadata (e.g., Raft) communication. - -:::info FQDN: For metadata changes only -This feature is only used for metadata changes which use Raft as the consensus mechanism. It does not affect data read/write operations. -::: - -#### Examples of when to use FQDN for node discovery - -The use of FQDN can resolve a situation where if IP addresses are re-used across different clusters, the nodes in one cluster could mistakenly discover nodes in another cluster. - -It can also be useful when using services (for example, Kubernetes) where the IP of the services is different from the actual node IP, but it proxies the connection to the node. - -#### Environment variables for FQDN node discovery - -`RAFT_ENABLE_FQDN_RESOLVER` is a Boolean flag. This flag enables the FQDN resolver. If set to `true`, Weaviate uses the FQDN resolver to resolve the node name to the node IP address. If set to `false`, Weaviate uses the memberlist lookup to resolve the node name to the node IP address. The default value is `false`. - -`RAFT_FQDN_RESOLVER_TLD` is a string that is appended in the format `[node-id].[tld]` when resolving a node-id to an IP address, where `[tld]` is the top-level domain. - -To use this feature, set `RAFT_ENABLE_FQDN_RESOLVER` to `true`. - -
- ## Metadata replication: Raft Weaviate uses the [Raft consensus algorithm](https://raft.github.io/) for metadata replication, implemented with Hashicorp's [raft library](https://pkg.go.dev/github.com/hashicorp/raft). Metadata in this context includes collection definition and shard/tenant states. From 831064dd3f336b191ccdc78ef5b3744a75dfb81b Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 17:57:59 +0200 Subject: [PATCH 5/6] Document cross-data-center (WAN) cluster configuration Add a WAN section to the Multi-DC page covering CLUSTER_ADVERTISE_ADDR (DefaultWANConfig) and related networking env vars, clearly separated from the unimplemented topology-aware Multi-DC replication feature. Also refresh the stale v1.17/v1.18 not-supported wording. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../replication-architecture/multi-dc.md | 50 +++++++++++++++++-- 1 file changed, 47 insertions(+), 3 deletions(-) diff --git a/docs/weaviate/concepts/replication-architecture/multi-dc.md b/docs/weaviate/concepts/replication-architecture/multi-dc.md index 1c1196a7c..0fa9c6425 100644 --- a/docs/weaviate/concepts/replication-architecture/multi-dc.md +++ b/docs/weaviate/concepts/replication-architecture/multi-dc.md @@ -1,22 +1,66 @@ --- title: Multi-Data center sidebar_position: 5 -description: "Future multi-data center replication capabilities for distributed deployments across geographic regions." +description: "Running a Weaviate cluster across data centers (WAN), and the roadmap for topology-aware multi-data center replication." image: og/docs/concepts.jpg # tags: ['architecture'] --- -Multi-Data center (Multi-DC) replication enables you to have multiple copies of the data on multiple servers across more than one data center. This form of replication is not yet supported in v1.17 and v1.18. The current version of replication is designed to support Multi-DC later on. If you are interested in this feature, upvote [this GitHub issue](https://github.com/weaviate/weaviate/issues/2436). +Multi-Data center (Multi-DC) replication enables you to have multiple copies of the data on multiple servers across more than one data center. This form of replication is not currently supported, but the existing replication design is intended to support Multi-DC in the future. If you are interested in this feature, upvote [this GitHub issue](https://github.com/weaviate/weaviate/issues/2436) and tell us about your use case so we can incorporate it into our roadmap. Multi-DC replication is beneficial if you have user groups spread over different geographical locations (e.g. Iceland and Australia). When you place nodes in different local regions of user groups, latency can be decreased. Multi-DC replication also comes with the additional benefit that data is redundant on more physical locations, which means that in the rare case of an entire data center going down, data can still be served from another location. -For now, you can work under the assumption that all replica nodes are in the same data center. The advantage of this is that network requests between nodes are cheap and fast. The downside is that if the entire data center fails, there is no redundancy. This will be solved with Multi-DC, [when implemented](https://github.com/weaviate/weaviate/issues/2436)! +In most deployments, you can work under the assumption that all replica nodes are in the same data center. The advantage of this is that network requests between nodes are cheap and fast. The downside is that if the entire data center fails, there is no redundancy. Topology-aware Multi-DC replication will solve this, [when implemented](https://github.com/weaviate/weaviate/issues/2436). + +It is, however, possible to run a single Weaviate cluster whose nodes are spread across data centers by tuning the cluster networking for high-latency links. This is a networking-layer capability only — Weaviate still treats every node as part of one flat cluster and is not yet aware of data center topology when placing replicas. See [Running a single cluster across data centers (WAN)](#running-a-single-cluster-across-data-centers-wan) below.

Replication multi-dc

+## Running a single cluster across data centers (WAN) + +Weaviate can operate a single cluster across data centers (a wide-area network, or WAN) by adjusting its inter-node communication for high-latency, lower-reliability links. + +:::info Networking only — not topology-aware replication +This configures how nodes discover and talk to each other across data centers. It does **not** make replication data center-aware (for example, keeping a replica in each region). The topology-aware Multi-DC replication feature described above remains on the [roadmap](https://github.com/weaviate/weaviate/issues/2436). +::: + +### Enabling WAN mode + +Set the `CLUSTER_ADVERTISE_ADDR` environment variable to enable WAN mode. When it is set, Weaviate switches its internal [memberlist](https://github.com/hashicorp/memberlist) configuration to `DefaultWANConfig`, which increases timeouts and relaxes failure-detection thresholds so they are suitable for cross-data center communication. + +### Key environment variables + +| Variable | Description | Required | +| --- | --- | --- | +| `CLUSTER_ADVERTISE_ADDR` | The public IP address that other nodes should use to reach this node. Setting this enables WAN mode. Must be a valid IP address (hostnames are rejected). | Yes (for WAN) | +| `CLUSTER_ADVERTISE_PORT` | The port to advertise to other nodes. If not set, it defaults to `CLUSTER_GOSSIP_BIND_PORT`. If set, it must be between `1024` and `65535`. | No | +| `CLUSTER_BIND_ADDR` | The local address to bind to. Defaults to `0.0.0.0`. | No | +| `CLUSTER_GOSSIP_BIND_PORT` | The port used for gossip (memberlist) traffic. Defaults to `7946`. | No | +| `CLUSTER_DATA_BIND_PORT` | The port used for data traffic. Defaults to `CLUSTER_GOSSIP_BIND_PORT + 1`. | No | +| `CLUSTER_JOIN` | A comma-separated list of `host:port` addresses of existing cluster members to join. | Yes (for joining) | + +### Example configuration + +Below is an example environment configuration for a node intended to participate in a cross-data center cluster: + +```yaml +environment: + - CLUSTER_HOSTNAME=weaviate-0 + - CLUSTER_ADVERTISE_ADDR=203.0.113.10 # Public IP of this node + - CLUSTER_JOIN=203.0.113.20:7946 # Public IP:Port of a node in another DC + - CLUSTER_BIND_ADDR=0.0.0.0 # Optional; this is already the default bind address + - CLUSTER_GOSSIP_BIND_PORT=7946 + - CLUSTER_DATA_BIND_PORT=7947 + - RAFT_BOOTSTRAP_EXPECT=3 # Use an odd number > 1 across DCs for faster, more reliable elections + - RAFT_JOIN=203.0.113.20:8300 # Join an existing Raft voter (default Raft port is 8300) +``` + +### Notes +- **Latency**: Cross-data center operations inherently have higher latency. Make sure your application timeouts account for this. +- **Security**: Gossip and data traffic are not encrypted by default. For cross-data center communication over the public internet, use a VPN or other secure tunnel, or make sure `CLUSTER_ADVERTISE_ADDR` is only reachable over a private network (for example, VPC peering). ## Questions and feedback From 6ddec7cd9ac35447c27803770ce5dec687af1967 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 17:57:59 +0200 Subject: [PATCH 6/6] Add tracking doc for removed content cleanup Track docs entries that describe removed features/env vars and are kept with a removal note, so they can be deleted in a later release. Co-Authored-By: Claude Opus 4.8 (1M context) --- REMOVED_CONTENT_CLEANUP.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 REMOVED_CONTENT_CLEANUP.md diff --git a/REMOVED_CONTENT_CLEANUP.md b/REMOVED_CONTENT_CLEANUP.md new file mode 100644 index 000000000..ecd50c81d --- /dev/null +++ b/REMOVED_CONTENT_CLEANUP.md @@ -0,0 +1,27 @@ +# Removed content — cleanup tracking + +This file tracks documentation content that describes **removed** Weaviate features, +environment variables, or configuration fields. Rather than deleting such content the +moment a feature is removed, we keep it in place with a short "Removed in `vX.Y`" note so +that users upgrading from an older version can still find the entry and understand what +happened to it. Once enough releases have passed that few users are upgrading across the +removal boundary, the noted content should be deleted. + +## Policy + +- When a feature/env var/config field is removed from Weaviate, **mark** the corresponding + docs entry as `Removed in vX.Y` instead of deleting it immediately. +- Add a row to the table below so the kept-but-stale content can be found and cleaned later. +- **Suggested cleanup window:** keep the note for roughly three minor releases after the + removal, then delete the entry and remove its row here. (Adjust per the supported-version + policy at cleanup time — these are guidelines, not hard commitments.) + +## Tracked entries + +| Page / file | Removed item | Removed in | Suggested cleanup | Notes | +| --- | --- | --- | --- | --- | +| `docs/deploy/configuration/env-vars/index.md` | `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` (table row) | `v1.38` | `v1.41`+ | Replaced by `ASYNC_REPLICATION_SCHEDULER_WORKERS`. | +| `docs/deploy/configuration/env-vars/index.md` | `ASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCY` (table row) | `v1.38` | `v1.41`+ | Scheduler no longer polls alive nodes separately. | +| `docs/deploy/configuration/env-vars/runtime-config.md` | `async_replication_cluster_max_workers` (override mapping row) | `v1.38` | `v1.41`+ | Runtime override removed alongside the env var. | +| `docs/deploy/configuration/async-rep.md` | "Removed environment variables (v1.38)" `
` block (both vars above) | `v1.38` | `v1.41`+ | Delete the whole block at cleanup. | +| `docs/deploy/configuration/replication.md` | "Removed in `v1.38`" `:::note` admonition | `v1.38` | `v1.41`+ | Mentions both removed env vars. |