[v1.38] Async replication on by default#434
Open
g-despot wants to merge 6 commits into
Open
Conversation
Async replication now runs automatically for any collection with a replication factor > 1; the per-collection asyncEnabled flag is removed. Update the async-rep, replication, env-var, runtime-config, collections config, multi-node-setup, horizontal-scaling and consistency pages, and the related includes. Replace the removed CLUSTER_MAX_WORKERS / ALIVE_NODES_CHECKING_FREQUENCY env vars with SCHEDULER_WORKERS / HASHTREE_INIT_CONCURRENCY, keeping the removed entries with notes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove asyncEnabled/async_enabled arguments and assertions, and the removed per-collection maxWorkers option, from the Python, TypeScript, Java and Go replication snippets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove async_replication_goroutines_running (replaced by the scheduler) and add the new scheduler gauges and diff/deletion/reconcile counters introduced in v1.38. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The RAFT_ENABLE_FQDN_RESOLVER / RAFT_FQDN_RESOLVER_TLD resolver was removed in v1.30. Replace the cluster-architecture section with a brief removal note and drop the obsolete FQDN (and pre-v1.18.1) content from the cluster concepts page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a WAN section to the Multi-DC page covering CLUSTER_ADVERTISE_ADDR (DefaultWANConfig) and related networking env vars, clearly separated from the unimplemented topology-aware Multi-DC replication feature. Also refresh the stale v1.17/v1.18 not-supported wording. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Track docs entries that describe removed features/env vars and are kept with a removal note, so they can be deleted in a later release. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![orca-security-eu[bot]](https://avatars.githubusercontent.com/in/276244?s=60&v=4){kind=small}
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
 Passed |
Secrets |  0  0  0  0 |
View in Orca |
g-despot
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Documents Weaviate v1.38 async replication defaults and removes/updates references to the removed per-collection asyncEnabled flag and related scheduler settings.
Changes:
- Updates async replication docs, env vars, runtime overrides, metrics, and collection configuration references for v1.38 behavior.
- Removes
asyncEnabled/maxWorkersfrom current code examples and snippets. - Adds WAN cluster configuration guidance and removes outdated FQDN node-discovery content.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
REMOVED_CONTENT_CLEANUP.md |
Adds tracking policy for removed docs content. |
docs/deploy/configuration/async-rep.md |
Updates async replication deep-dive for default-on scheduler behavior. |
docs/deploy/configuration/env-vars/index.md |
Updates async replication env vars and removes old FQDN env vars. |
docs/deploy/configuration/env-vars/runtime-config.md |
Updates runtime override mappings for new async replication settings. |
docs/deploy/configuration/horizontal-scaling.mdx |
Removes removed asyncEnabled field from example. |
docs/deploy/configuration/monitoring.md |
Updates async replication metrics reference. |
docs/deploy/configuration/replication.md |
Updates replication how-to for default-on async replication. |
docs/weaviate/concepts/cluster.md |
Removes outdated FQDN and old shard-affinity details. |
docs/weaviate/concepts/replication-architecture/cluster-architecture.md |
Removes outdated FQDN node-discovery details. |
docs/weaviate/concepts/replication-architecture/consistency.md |
Updates async replication and consistency wording. |
docs/weaviate/concepts/replication-architecture/multi-dc.md |
Adds WAN single-cluster configuration guidance. |
docs/weaviate/config-refs/collections.mdx |
Removes removed replication config fields and documents v1.38 defaults. |
docs/weaviate/manage-collections/multi-node-setup.mdx |
Removes removed async replication fields from cURL example. |
_includes/async-replication-per-collection-config.mdx |
Updates shared async replication include. |
_includes/collection-mutable-parameters.mdx |
Removes asyncEnabled from mutable parameter list. |
_includes/code/config-refs/reference.collections.py |
Updates Python collection reference snippet. |
_includes/code/configuration/replication-consistency.mdx |
Updates shared cURL replication snippet. |
_includes/code/howto/go/docs/manage-data.classes_test.go |
Updates Go replication example. |
_includes/code/howto/manage-data.collections.py |
Updates Python replication examples. |
_includes/code/howto/manage-data.collections.ts |
Updates TypeScript replication examples. |
_includes/code/java-v6/src/test/java/ManageCollectionsTest.java |
Updates Java replication examples/tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+411
to
+413
| // Async replication runs by default when the replication factor is greater than 1 | ||
| client.collections.create("Article", col -> col.replication( | ||
| Replication.of(rep -> rep.replicationFactor(1).asyncEnabled(true)))); | ||
| Replication.of(rep -> rep.replicationFactor(1)))); |
| | `async_replication_cluster_max_workers` | `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` | | ||
| | `async_replication_scheduler_workers` | `ASYNC_REPLICATION_SCHEDULER_WORKERS` | | ||
| | `async_replication_hashtree_init_concurrency` | `ASYNC_REPLICATION_HASHTREE_INIT_CONCURRENCY`| | ||
| | `async_replication_cluster_max_workers` _(removed in `v1.38`)_ | `ASYNC_REPLICATION_CLUSTER_MAX_WORKERS` _(removed in `v1.38`)_ | |
Comment on lines
+680
to
682
| // Async replication runs by default when the replication factor is greater than 1 | ||
| replication: configure.replication({ | ||
| factor: 1, |
| Async replication supplements the repair-on-read mechanism. If a node becomes inconsistent between sync checks, the repair-on-read mechanism catches the problem at read time. | ||
|
|
||
| To activate async replication, set `asyncEnabled` to true in the [`replicationConfig` section of your collection definition](../../manage-collections/multi-node-setup.mdx#replication-settings). Visit the [How-to: Replication](/deploy/configuration/replication.md#async-replication-settings) page to learn more about the available async replication settings. | ||
| As of Weaviate `v1.38`, async replication is enabled by default for any collection with a replication factor greater than `1`, there is no per-collection flag to enable it. To turn it off cluster-wide, set the `ASYNC_REPLICATION_DISABLED` environment variable to `true`. Visit the [How-to: Replication](/deploy/configuration/replication.md#async-replication-settings) page to learn more about the available async replication settings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documents the async replication on-by-default change in Weaviate v1.38 (PRs weaviate/weaviate#11155, #10926; weaviate-python-client#2047), plus related correctness fixes to the replication-architecture concept pages found while auditing them against the source.
As of v1.38, async replication runs automatically for any collection with a replication factor > 1; the per-collection
asyncEnabledflag is removed and disabling is done cluster-wide viaASYNC_REPLICATION_DISABLED.Commits
ASYNC_REPLICATION_CLUSTER_MAX_WORKERSandASYNC_REPLICATION_ALIVE_NODES_CHECKING_FREQUENCYfor the newASYNC_REPLICATION_SCHEDULER_WORKERS/ASYNC_REPLICATION_HASHTREE_INIT_CONCURRENCY, keeping the removed entries with "Removed in v1.38" notes.async_replication_goroutines_running, adds the new scheduler gauges and diff/deletion/reconcile counters.RAFT_ENABLE_FQDN_RESOLVER/RAFT_FQDN_RESOLVER_TLDresolver was removed in v1.30.CLUSTER_ADVERTISE_ADDR/DefaultWANConfignetworking, clearly separated from the unimplemented topology-aware Multi-DC replication feature.REMOVED_CONTENT_CLEANUP.mdlists kept-but-stale entries for deletion in a later release.Verification
v1.38.0-rc.0(env-var names/defaults, removed fields, metric names/types, WAN config behavior).yarn build-devpasses with no new broken links or anchors.Notes for reviewers
manage-data.collections-v2.ts/-v3.pysnippets still containasyncEnabled(intentionally left unchanged) — flag if they should also be updated.concepts/cluster.mdbut kept-with-note incluster-architecture.md+ env-vars table; can align to one approach on request.🤖 Generated with Claude Code