From 8c1e56c745f42bf97e6e8c6da9b064cb5454d211 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Tue, 26 May 2026 13:01:55 +0200 Subject: [PATCH 1/8] Add property reindex docs --- _includes/feature-notes/runtime-reindex.mdx | 5 + .../concepts/indexing/inverted-index.md | 2 +- .../config-refs/indexing/inverted-index.mdx | 14 + .../collection-operations.mdx | 5 +- .../manage-collections/inverted-index.mdx | 12 + .../manage-collections/reindex-property.mdx | 260 ++++++++++++++++++ sidebars.js | 1 + 7 files changed, 295 insertions(+), 4 deletions(-) create mode 100644 _includes/feature-notes/runtime-reindex.mdx create mode 100644 docs/weaviate/manage-collections/reindex-property.mdx diff --git a/_includes/feature-notes/runtime-reindex.mdx b/_includes/feature-notes/runtime-reindex.mdx new file mode 100644 index 00000000..3c68413e --- /dev/null +++ b/_includes/feature-notes/runtime-reindex.mdx @@ -0,0 +1,5 @@ +:::caution Preview — added in `v1.38` + +This is a preview feature. The REST shape and behavior may change before GA. Do not rely on backup/restore while a reindex is in flight or recently completed on a v1.38 Preview cluster — wait for all tasks to reach `ready` / `failed` / `cancelled` first. + +::: diff --git a/docs/weaviate/concepts/indexing/inverted-index.md b/docs/weaviate/concepts/indexing/inverted-index.md index b10375e5..6f5cdec0 100644 --- a/docs/weaviate/concepts/indexing/inverted-index.md +++ b/docs/weaviate/concepts/indexing/inverted-index.md @@ -50,7 +50,7 @@ import BlockmaxWand from '/_includes/feature-notes/blockmax-wand.mdx'; The BlockMax WAND algorithm is a variant of the WAND algorithm that is used to speed up BM25 and hybrid searches. It organizes the inverted index in blocks to enable skipping over blocks that are not relevant to the query. This can significantly reduce the number of documents that need to be scored, improving search performance. -If you are experiencing slow BM25 (or hybrid) searches and use a Weaviate version prior to `v1.30`, try migrating to a newer version that uses the BlockMax WAND algorithm to see if it improves performance. If you need to migrate existing data from a previous version of Weaviate, follow the [v1.30 migration guide](/deploy/migration/weaviate-1-30.md). +If you are experiencing slow BM25 (or hybrid) searches and use a Weaviate version prior to `v1.30`, try migrating to a newer version that uses the BlockMax WAND algorithm to see if it improves performance. If you need to migrate existing data from a previous version of Weaviate, follow the [v1.30 migration guide](/deploy/migration/weaviate-1-30.md) — or on `v1.38+`, use the live [Reindex a property](/weaviate/manage-collections/reindex-property.mdx#migrate-bm25-from-wand-to-blockmax) endpoint to migrate without restart. :::note Scoring changes with BlockMax WAND diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index a9da3d3a..d4fc368d 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -257,8 +257,22 @@ You can drop (delete) an inverted index from a property. This is a destructive o The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. +REST: `DELETE /v1/schema/{className}/properties/{propertyName}/index/{indexName}`. From `v1.38`, the drop is gated by the [runtime-reindex](../../manage-collections/reindex-property.mdx) MutationGuard — rejected while a reindex is in flight on the same property. + See [How-to: Drop an inverted index](../../manage-collections/inverted-index.mdx#drop-an-inverted-index) for code examples. +## Runtime reindex (v1.38 Preview) + +From `v1.38`, three REST endpoints let you alter a property's inverted-index configuration on a live collection without restart: + +| Method | Path | Purpose | +|---|---|---| +| `PUT` | `/v1/schema/{class}/indexes/{property}` | Add an inverted index, change tokenization, migrate BM25 WAND → BlockMax, rebuild a bucket, or cancel an in-flight task. The body shape selects the migration type. | +| `DELETE` | `/v1/schema/{class}/properties/{property}/index/{indexName}` | Drop a configured index. `indexName` ∈ `{filterable, searchable, rangeFilters}`. | +| `GET` | `/v1/schema/{class}/indexes` | Read per-property index status. | + +`PUT` / `DELETE` require `UPDATE` on `Collections`; `GET` requires `READ` on `CollectionsMetadata`. See [How-to: Reindex a property](../../manage-collections/reindex-property.mdx) for request bodies, response shapes, concurrency rules, and worked examples. + ## How Weaviate creates inverted indexes Weaviate creates **separate inverted indexes for each property and each index type**. For example, if you have a `title` property that is both searchable and filterable, diff --git a/docs/weaviate/manage-collections/collection-operations.mdx b/docs/weaviate/manage-collections/collection-operations.mdx index e920e093..87c12ee4 100644 --- a/docs/weaviate/manage-collections/collection-operations.mdx +++ b/docs/weaviate/manage-collections/collection-operations.mdx @@ -595,9 +595,8 @@ Property indexes are built at import time. If you add a new property after impor To create an index that includes all of the objects in a collection, do one of the following: - New collections: Add all of the collection's properties before importing objects. -- Existing collections: Export the existing data from the collection. Re-create it with the new property. Import the data into the updated collection. - -We are working on a re-indexing API to allow you to re-index the data after adding a property. This will be available in a future release. +- Existing collections (v1.38+): Use the runtime [Reindex a property](./reindex-property.mdx) endpoints to add or change inverted indexes on a live collection without restart. +- Existing collections (pre-v1.38): Export the existing data from the collection, recreate it with the new property, and re-import the data into the updated collection. diff --git a/docs/weaviate/manage-collections/inverted-index.mdx b/docs/weaviate/manage-collections/inverted-index.mdx index 76cd1155..c9ccf67a 100644 --- a/docs/weaviate/manage-collections/inverted-index.mdx +++ b/docs/weaviate/manage-collections/inverted-index.mdx @@ -16,6 +16,12 @@ import GoCode from "!!raw-loader!/_includes/code/howto/go/docs/manage-data.class An **inverted index** is a data structure in Weaviate that enables efficient text search and filtering operations. +:::tip Change inverted indexes on a live collection (v1.38+) + +You can add, change, rebuild, or drop inverted indexes on a populated collection without restart. See [Reindex a property](./reindex-property.mdx). + +::: +
Additional information @@ -155,6 +161,12 @@ Drop (delete) an inverted index from a property. This is a destructive operation The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. +:::note v1.38+ MutationGuard + +From `v1.38`, the drop is rejected with `409` while a [runtime reindex](./reindex-property.mdx) is in flight on the same property. Cancel the in-flight task first, or wait for it to reach `ready` / `failed` / `cancelled`. + +::: + + +Change a property's **inverted-index configuration** on a live collection without restarting the cluster and without losing writes. Reads stay available throughout the migration. This replaces the previous workaround of exporting, recreating the collection, and re-importing. + +This page covers **inverted indexes only** (`IndexFilterable`, `IndexSearchable`, `IndexRangeFilters`, BM25 algorithm, tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). + +## What you can do + +| Verb | Use when | +|---|---| +| **Add** a missing inverted index | The property was created without `IndexFilterable` / `IndexSearchable` / `IndexRangeFilters` and you now need to filter or search on it. | +| **Change** a property's tokenization | You picked `word` and want `trigram` (or vice versa) on a live `text` / `text[]` property. | +| **Migrate BM25** from WAND to BlockMax | One-way upgrade for the searchable bucket on existing collections. | +| **Rebuild / repair** a bucket | Refresh a `RoaringSet` / `RoaringSetRange` / BlockMax index after suspected corruption or heavy data churn. | +| **Cancel** an in-flight migration | You realised the call was wrong, or want to free the slot for another migration. | + +To **drop** a configured inverted index, use the existing [Drop an inverted index](./inverted-index.mdx#drop-an-inverted-index) flow — it has client-library support for Python, TypeScript, Go, and Java. From v1.38, the drop operation is gated by the same MutationGuard described on this page. + +## REST endpoints + +The feature is REST-only for v1.38 Preview. Two new endpoints (`PUT`, `GET`), plus a new MutationGuard contract on the existing drop endpoint: + +| Method | Path | Purpose | +|---|---|---| +| `PUT` | `/v1/schema/{class}/indexes/{property}` | **New in v1.38.** Submit a migration. The request body shape selects which migration type. | +| `GET` | `/v1/schema/{class}/indexes` | **New in v1.38.** Read per-property index status. | +| `DELETE` | `/v1/schema/{class}/properties/{property}/index/{indexName}` | Drop a configured index. Existed before v1.38 (see [Drop an inverted index](./inverted-index.mdx#drop-an-inverted-index)); from v1.38 it is now rejected by the MutationGuard while another reindex on the same property is in flight. | + +All examples below assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections` (`GET` only needs `READ` on `CollectionsMetadata`). + +## Add a missing inverted index + +To enable filtering on a property that was created without one: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"filterable":{"enabled":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/category" +``` + +Body shapes per index type: + +| Body | Effect | +|---|---| +| `{"filterable":{"enabled":true}}` | Creates a `RoaringSet` bucket and flips `IndexFilterable=true`. | +| `{"searchable":{"enabled":true,"tokenization":"word"}}` | Creates a BlockMax searchable bucket, sets `Tokenization`, flips `IndexSearchable=true`. Requires `text` / `text[]`. | +| `{"rangeable":{"enabled":true}}` | Creates a `RoaringSetRange` bucket and flips `IndexRangeFilters=true`. Numeric types only (`int`, `number`, `date`). | + +The server responds `202 Accepted` with a task ID: + +```json +{ + "status": "STARTED", + "taskId": "reindex/Article/category/enable-filterable/1700000000" +} +``` + +## Change tokenization + +To retokenize a populated `text` property: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"tokenization":"trigram"}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" +``` + +If the property has **both** a searchable and a filterable index, they are retokenized together in one coordinated migration. To retokenize only the filterable bucket: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"filterable":{"tokenization":"word"}}' \ + "$WEAVIATE/v1/schema/Article/indexes/category" +``` + +For the canonical list of tokenization values, see [Tokenization options](../config-refs/collections.mdx#tokenization). + +## Migrate BM25 from WAND to BlockMax + +[BlockMax WAND](../concepts/indexing/inverted-index.md) is a faster scoring algorithm that replaced the original WAND implementation. On collections created before BlockMax was the default, you can migrate every searchable property in one call: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"algorithm":"blockmax"}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" +``` + +The migration touches every searchable property on the class. The class-level `UsingBlockMaxWAND` flag flips to `true` after every property has been rebuilt. + +This is **one-way** — a request to flip `searchable.algorithm` back from `blockmax` to `wand` is rejected. WAND is deprecated. + +## Rebuild or repair an index + +If you suspect corruption or want to refresh an index after heavy data churn: + +```bash +# Rebuild the BlockMax searchable bucket from object storage +curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"rebuild":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" + +# Repair a filterable bucket +curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"filterable":{"rebuild":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/category" + +# Repair a rangeable bucket +curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"rangeable":{"rebuild":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/price" +``` + +`searchable.rebuild` is for BlockMax buckets only. To rebuild a property still on WAND, first migrate it to BlockMax (above) — `rebuild:true` on a WAND bucket is rejected. + +## Cancel an in-flight migration + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"cancel":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" +``` + +The response is one of: + +- `{"status": "CANCELLED", "taskId": ""}` — a `STARTED` task was found and cancelled. +- `{"status": "NO_OP"}` — nothing matched (already finished, never submitted, or already cancelled). Idempotent. + +Cancelling wipes partial on-disk state, so the next submit starts from a clean slate. + +## Check migration status + +```bash +curl -fsS -H "Authorization: Bearer $TOKEN" \ + "$WEAVIATE/v1/schema/Article/indexes" +``` + +```json +{ + "collection": "Article", + "properties": [{ + "name": "body", + "dataType": "text", + "indexes": [ + { "type": "filterable", "status": "ready", "tokenization": "word" }, + { "type": "searchable", "status": "indexing", "progress": 0.42, + "tokenization": "word", "targetTokenization": "trigram", + "algorithm": "blockmax" } + ] + }] +} +``` + +| `status` | Meaning | +|---|---| +| `ready` | Index is live and serving. No migration in flight. | +| `pending` | A task has been accepted; per-shard work has not started yet. | +| `indexing` | Per-shard work is running. `progress` is a 0..1 estimate. | +| `failed` | At least one node reported `Success=false`. The schema flag was **not** flipped — the property is still in its pre-migration state. Submit a new task to retry. | +| `cancelled` | Operator cancelled. Partial state has been scrubbed. | + +Readback is throttled (3 s), so polling in a tight loop is cheap: + +```bash +until curl -fsS "$WEAVIATE/v1/schema/Article/indexes" \ + | jq -e '.properties[] | select(.name=="body") + | .indexes[] | select(.type=="searchable") + | .status == "ready"' > /dev/null +do sleep 2; done +``` + +## Multi-tenancy + +Scope a task to specific tenants on a multi-tenant class with the `?tenants=` query parameter: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"filterable":{"rebuild":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/category?tenants=customerA,customerB" +``` + +Rules: + +| Class | `?tenants=` | Migration type | Result | +|---|---|---|---| +| Single-tenant | provided | any | `400` | +| Multi-tenant | omitted | format-only (`rebuild`, `repair`, `enable-rangeable`) | targets all tenants | +| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-*`) | targets all tenants (the schema flip is cluster-wide) | +| Multi-tenant | provided | format-only | targets the named subset | +| Multi-tenant | provided | semantic | `400` — semantic migrations cannot be sub-scoped | +| any | tenant in `OFFLOADED` / `FROZEN` | any | `400` — the offending tenant is named in the error | + +Each tenant's replicas form an independent barrier group: tenant A starts serving the new bucket as soon as its own replicas finish, even if tenant B is still reindexing. + +## Concurrency limits + +- **Per `(class, property)` exclusivity** — only one migration is allowed in flight on a given `(collection, property)` pair. A second submit on the same pair returns `409` with the offending task ID. +- **Per-class cap** — up to 32 concurrent migrations per class. The next submit returns `503` until the in-flight units drain. +- **Different properties on the same class run in parallel.** Different classes are fully independent. + +## What's blocked during a reindex + +While a reindex is in flight on `(class, property)`, the schema FSM rejects: + +- `UpdateProperty` on the same property. +- `DeleteClass` on the affected class. +- `DeleteTenants` / `UpdateTenants` on the targeted tenants when the transition makes shards locally unavailable (HOT → COLD / FROZEN / OFFLOADED). Transitions toward available (e.g. UNFREEZING) are not blocked. +- `DELETE /v1/schema/{class}/properties/{property}/index/{indexName}` on the same property. + +The reject message names the in-flight task. Submitting a reindex on a **different** property of the same class is **not** blocked. + +## Errors and recovery + +| Code | When | Resolution | +|---|---|---| +| `400` | Malformed body, wrong property type, missing prerequisite (e.g. `searchable.tokenization` on a property with no searchable index), `?tenants=` on a single-tenant class, `?tenants=` on a semantic migration, target tenant in `OFFLOADED` / `FROZEN`. | Error responses carry next-step hints — read them. | +| `404` | Class or property doesn't exist. | Verify the class + property names. | +| `409` | An in-flight task overlaps this `(collection, property)`. The error names the offending task ID and migration type. | Wait, or cancel the existing task first. | +| `429` / `503` | Per-class in-flight cap reached, or cluster temporarily unavailable. | Retry once in-flight migrations drain. | + +The schema flag is the source of truth: if a task ends in `failed`, the flag was not flipped and the property is still in its pre-migration state. A reindex is restart-safe at every phase — in-flight migrations are picked up automatically after a node restart. + +## Authentication and authorization + +| Endpoint | Required permission | +|---|---| +| `GET /v1/schema/{class}/indexes` | `READ` on `CollectionsMetadata` | +| `PUT /v1/schema/{class}/indexes/{property}` | `UPDATE` on `Collections` | +| `DELETE /v1/schema/{class}/properties/{property}/index/{indexName}` | `UPDATE` on `Collections` | + +`UPDATE` on `Collections` is the same permission that gates `UpdateClass` and replication-factor changes — there is no dedicated `reindex` role today. + +## Related pages + +- [Inverted index configuration](./inverted-index.mdx) — how to set inverted indexes at collection creation, and how to drop them (multi-language client support). +- [Tokenization options](../config-refs/collections.mdx#tokenization) — canonical list of tokenization values. +- [Vector configuration](./vector-config.mdx) — sibling page covering vector indexes. +- [BlockMax WAND](../concepts/indexing/inverted-index.md) — background on the BM25 algorithm migration. + +## Questions and feedback + +import DocsFeedback from '/\_includes/docs-feedback.mdx'; + + diff --git a/sidebars.js b/sidebars.js index b3200af6..eec3e37b 100644 --- a/sidebars.js +++ b/sidebars.js @@ -580,6 +580,7 @@ const sidebars = { "weaviate/manage-collections/vector-config", "weaviate/manage-collections/generative-reranker-models", "weaviate/manage-collections/inverted-index", + "weaviate/manage-collections/reindex-property", { type: "category", label: "Multi-tenancy", From e6635722e16eb453d3b6d915c76a5ac2f0e1d5c0 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 10:10:09 +0200 Subject: [PATCH 2/8] Refactor docs --- .../concepts/indexing/inverted-index.md | 10 +- .../config-refs/indexing/inverted-index.mdx | 88 +++++- .../collection-operations.mdx | 2 +- .../manage-collections/inverted-index.mdx | 143 +++++++++- .../manage-collections/reindex-property.mdx | 260 ------------------ docusaurus.config.js | 2 +- sidebars.js | 1 - 7 files changed, 232 insertions(+), 274 deletions(-) delete mode 100644 docs/weaviate/manage-collections/reindex-property.mdx diff --git a/docs/weaviate/concepts/indexing/inverted-index.md b/docs/weaviate/concepts/indexing/inverted-index.md index 6f5cdec0..0d4943d2 100644 --- a/docs/weaviate/concepts/indexing/inverted-index.md +++ b/docs/weaviate/concepts/indexing/inverted-index.md @@ -50,7 +50,7 @@ import BlockmaxWand from '/_includes/feature-notes/blockmax-wand.mdx'; The BlockMax WAND algorithm is a variant of the WAND algorithm that is used to speed up BM25 and hybrid searches. It organizes the inverted index in blocks to enable skipping over blocks that are not relevant to the query. This can significantly reduce the number of documents that need to be scored, improving search performance. -If you are experiencing slow BM25 (or hybrid) searches and use a Weaviate version prior to `v1.30`, try migrating to a newer version that uses the BlockMax WAND algorithm to see if it improves performance. If you need to migrate existing data from a previous version of Weaviate, follow the [v1.30 migration guide](/deploy/migration/weaviate-1-30.md) — or on `v1.38+`, use the live [Reindex a property](/weaviate/manage-collections/reindex-property.mdx#migrate-bm25-from-wand-to-blockmax) endpoint to migrate without restart. +If you are experiencing slow BM25 (or hybrid) searches and use a Weaviate version prior to `v1.30`, try migrating to a newer version that uses the BlockMax WAND algorithm to see if it improves performance. If you need to migrate existing data from a previous version of Weaviate, follow the [v1.30 migration guide](/deploy/migration/weaviate-1-30.md) — or on `v1.38+`, use the live [Reindex a property](/weaviate/manage-collections/inverted-index.mdx#migrate-bm25-from-wand-to-blockmax) endpoint to migrate without restart. :::note Scoring changes with BlockMax WAND @@ -168,6 +168,14 @@ An example of a complete collection object without inverted indexes:
+### Changing an index after import + +Because an inverted index is built at import time, a property created without one (or with the "wrong" tokenization or BM25 algorithm) historically required exporting the data, recreating the collection, and re-importing — an expensive, downtime-prone operation. + +From `v1.38`, Weaviate can **reindex a property on a collection** instead. A reindex builds the new bucket in the background from object storage while the existing index keeps serving reads, then atomically flips a single schema flag once every replica has finished. The schema flag is the source of truth: if the rebuild fails, the flag is never flipped and the property stays in its pre-migration state, and an interrupted reindex is picked up automatically after a node restart. This makes adding a missing index, changing tokenization, or migrating BM25 from WAND to BlockMax a non-destructive, restart-safe operation. + +For the operational steps, see [How-to: Reindex a property on a collection](/weaviate/manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection-v138); for the endpoint reference, see [References: Runtime reindex](/weaviate/config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). + ## Tokenization Tokenization is the process of breaking text into smaller units called tokens. This process is fundamental to how inverted indexes work - the tokens produced determine what can be searched and how matching occurs. diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index d4fc368d..4e32a60b 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -257,13 +257,19 @@ You can drop (delete) an inverted index from a property. This is a destructive o The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. -REST: `DELETE /v1/schema/{className}/properties/{propertyName}/index/{indexName}`. From `v1.38`, the drop is gated by the [runtime-reindex](../../manage-collections/reindex-property.mdx) MutationGuard — rejected while a reindex is in flight on the same property. +REST: `DELETE /v1/schema/{className}/properties/{propertyName}/index/{indexName}`. From `v1.38`, the drop is gated by the [runtime-reindex](#runtime-reindex-v138-preview) MutationGuard — rejected while a reindex is in flight on the same property. See [How-to: Drop an inverted index](../../manage-collections/inverted-index.mdx#drop-an-inverted-index) for code examples. ## Runtime reindex (v1.38 Preview) -From `v1.38`, three REST endpoints let you alter a property's inverted-index configuration on a live collection without restart: +import RuntimeReindexPreview from "/_includes/feature-notes/runtime-reindex.mdx"; + + + +From `v1.38`, three REST endpoints let you alter a property's inverted-index configuration on a collection without restart. For task-oriented walkthroughs with `curl` examples, see [How-to: Reindex a property on a collection](../../manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection-v138). This section is the endpoint and behavior reference. + +### Endpoints | Method | Path | Purpose | |---|---|---| @@ -271,7 +277,78 @@ From `v1.38`, three REST endpoints let you alter a property's inverted-index con | `DELETE` | `/v1/schema/{class}/properties/{property}/index/{indexName}` | Drop a configured index. `indexName` ∈ `{filterable, searchable, rangeFilters}`. | | `GET` | `/v1/schema/{class}/indexes` | Read per-property index status. | -`PUT` / `DELETE` require `UPDATE` on `Collections`; `GET` requires `READ` on `CollectionsMetadata`. See [How-to: Reindex a property](../../manage-collections/reindex-property.mdx) for request bodies, response shapes, concurrency rules, and worked examples. +### Request body shapes (`PUT`) + +| Body | Effect | +|---|---| +| `{"filterable":{"enabled":true}}` | Creates a `RoaringSet` bucket and flips `IndexFilterable=true`. | +| `{"searchable":{"enabled":true,"tokenization":"word"}}` | Creates a BlockMax searchable bucket, sets `Tokenization`, flips `IndexSearchable=true`. Requires `text` / `text[]`. | +| `{"rangeable":{"enabled":true}}` | Creates a `RoaringSetRange` bucket and flips `IndexRangeFilters=true`. Numeric types only (`int`, `number`, `date`). | +| `{"searchable":{"tokenization":"trigram"}}` | Retokenizes a populated `text` property. If a filterable bucket also exists, both are retokenized together. | +| `{"searchable":{"algorithm":"blockmax"}}` | Migrates every searchable property on the class from WAND to BlockMax. One-way; flipping back to `wand` is rejected. | +| `{"":{"rebuild":true}}` | Rebuilds the named bucket (`filterable` / `searchable` / `rangeable`) from object storage. `searchable.rebuild` requires the property to already be on BlockMax. | +| `{"":{"cancel":true}}` | Cancels an in-flight task on the property. Idempotent — returns `NO_OP` when nothing matches. | + +A successful submit returns `202 Accepted` with `{"status":"STARTED","taskId":""}`. + +### Status values (`GET`) + +`GET /v1/schema/{class}/indexes` returns per-property index status. Each index reports one of: + +| `status` | Meaning | +|---|---| +| `ready` | Index is live and serving. No migration in flight. | +| `pending` | A task has been accepted; per-shard work has not started yet. | +| `indexing` | Per-shard work is running. `progress` is a 0..1 estimate. | +| `failed` | At least one node reported `Success=false`. The schema flag was **not** flipped — the property is still in its pre-migration state. Submit a new task to retry. | +| `cancelled` | Operator cancelled. Partial state has been scrubbed. | + +Between "task FINISHED" and "schema flag flipped", the response keeps the index at `indexing@100%` for a 3–10-second finalize window so the status never blinks back to a pre-migration shape before the schema catches up. + +### Concurrency + +- **Per `(class, property)` exclusivity** — only one migration is allowed in flight on a given `(collection, property)` pair. A second submit on the same pair returns `409` with the offending task ID. +- **Per-class cap** — up to 32 concurrent migrations per class. The next submit returns `429 Too Many Requests` until the in-flight units drain. +- **Different properties on the same class run in parallel.** Different classes are fully independent. + +While a reindex is in flight on `(class, property)`, the schema FSM rejects `UpdateProperty` on the same property, `DeleteClass` on the affected class, `DeleteTenants` / `UpdateTenants` that make targeted shards locally unavailable (HOT → COLD / FROZEN / OFFLOADED), and `DELETE …/index/{indexName}` on the same property. The reject message names the in-flight task. A reindex on a **different** property of the same class is not blocked. + +### Multi-tenancy + +Scope a task to specific tenants on a multi-tenant class with the `?tenants=` query parameter (comma-separated). The rules: + +| Class | `?tenants=` | Migration type | Result | +|---|---|---|---| +| Single-tenant | provided | any | `400` | +| Multi-tenant | omitted | format-only (`rebuild`, `repair`, `enable-rangeable`) | targets all tenants | +| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-*`) | targets all tenants (the schema flip is cluster-wide) | +| Multi-tenant | provided | format-only | targets the named subset | +| Multi-tenant | provided | semantic | `400` — semantic migrations cannot be sub-scoped | +| any | tenant in `OFFLOADED` / `FROZEN` | any | `400` — the offending tenant is named in the error | + +Each tenant's replicas form an independent barrier group: tenant A starts serving the new bucket as soon as its own replicas finish, even if tenant B is still reindexing. + +### Errors and recovery + +| Code | When | Resolution | +|---|---|---| +| `400` | Malformed body, wrong property type, missing prerequisite (e.g. `searchable.tokenization` on a property with no searchable index), `?tenants=` on a single-tenant class, `?tenants=` on a semantic migration, target tenant in `OFFLOADED` / `FROZEN`. | Error responses carry next-step hints — read them. | +| `404` | Class or property doesn't exist. | Verify the class + property names. | +| `409` | An in-flight task overlaps this `(collection, property)`. The error names the offending task ID and migration type. | Wait, or cancel the existing task first. | +| `429` | Per-class in-flight cap reached (32 concurrent migrations). | Retry once in-flight migrations drain. | +| `503` | Cluster service temporarily unavailable. | Retry. | + +The schema flag is the source of truth: if a task ends in `failed`, the flag was not flipped and the property is still in its pre-migration state. A reindex is restart-safe at every phase — in-flight migrations are picked up automatically after a node restart. + +### Required permissions + +| Endpoint | Required permission | +|---|---| +| `GET /v1/schema/{class}/indexes` | `READ` on `CollectionsMetadata` | +| `PUT /v1/schema/{class}/indexes/{property}` | `UPDATE` on `Collections` | +| `DELETE /v1/schema/{class}/properties/{property}/index/{indexName}` | `UPDATE` on `CollectionsMetadata` | + +`PUT` is intentionally stricter than the other endpoints: submitting a reindex task rebuilds buckets on every replica and flips schema flags, so it requires `UPDATE` on `Collections` — the same permission that gates `UpdateClass` and replication-factor changes. `DELETE` shares the existing schema-metadata permission (`UPDATE` on `CollectionsMetadata`) used by the other property-management endpoints. There is no dedicated `reindex` role today. ## How Weaviate creates inverted indexes @@ -288,9 +365,8 @@ This is caused by the inverted index being built at import time. If you add a pr To avoid this, you can either: - Add the property before importing objects. -- Delete the collection, re-create it with the new property and then re-import the data. - -We are working on a re-indexing API to allow you to re-index the data after adding a property. This will be available in a future release. +- From `v1.38`, use the [runtime reindex](#runtime-reindex-v138-preview) endpoints to add or change an inverted index on the collection without restart. +- On versions before `v1.38`, delete the collection, re-create it with the new property, and then re-import the data. ## How tokenization affects inverted indexing diff --git a/docs/weaviate/manage-collections/collection-operations.mdx b/docs/weaviate/manage-collections/collection-operations.mdx index 87c12ee4..965e6c9f 100644 --- a/docs/weaviate/manage-collections/collection-operations.mdx +++ b/docs/weaviate/manage-collections/collection-operations.mdx @@ -595,7 +595,7 @@ Property indexes are built at import time. If you add a new property after impor To create an index that includes all of the objects in a collection, do one of the following: - New collections: Add all of the collection's properties before importing objects. -- Existing collections (v1.38+): Use the runtime [Reindex a property](./reindex-property.mdx) endpoints to add or change inverted indexes on a live collection without restart. +- Existing collections (v1.38+): Use the runtime [Reindex a property](./inverted-index.mdx#reindex-a-property-on-a-collection-v138) endpoints to add or change inverted indexes on a collection without restart. - Existing collections (pre-v1.38): Export the existing data from the collection, recreate it with the new property, and re-import the data into the updated collection. diff --git a/docs/weaviate/manage-collections/inverted-index.mdx b/docs/weaviate/manage-collections/inverted-index.mdx index c9ccf67a..5dad4b30 100644 --- a/docs/weaviate/manage-collections/inverted-index.mdx +++ b/docs/weaviate/manage-collections/inverted-index.mdx @@ -13,12 +13,13 @@ import TSCode from "!!raw-loader!/_includes/code/howto/manage-data.collections.t import JavaV6Code from "!!raw-loader!/_includes/code/java-v6/src/test/java/ManageCollectionsTest.java"; import CSharpCode from "!!raw-loader!/_includes/code/csharp/ManageCollectionsTest.cs"; import GoCode from "!!raw-loader!/_includes/code/howto/go/docs/manage-data.classes_test.go"; +import RuntimeReindexPreview from "/_includes/feature-notes/runtime-reindex.mdx"; An **inverted index** is a data structure in Weaviate that enables efficient text search and filtering operations. -:::tip Change inverted indexes on a live collection (v1.38+) +:::tip Change inverted indexes on a collection (v1.38+) -You can add, change, rebuild, or drop inverted indexes on a populated collection without restart. See [Reindex a property](./reindex-property.mdx). +You can add, change, rebuild, or drop inverted indexes on a populated collection without restart. See [Reindex a property on a collection](#reindex-a-property-on-a-collection-v138). ::: @@ -163,7 +164,7 @@ The following index types can be dropped: `searchable`, `filterable`, `rangeFilt :::note v1.38+ MutationGuard -From `v1.38`, the drop is rejected with `409` while a [runtime reindex](./reindex-property.mdx) is in flight on the same property. Cancel the in-flight task first, or wait for it to reach `ready` / `failed` / `cancelled`. +From `v1.38`, the drop is rejected with `409` while a [runtime reindex](#reindex-a-property-on-a-collection-v138) is in flight on the same property. Cancel the in-flight task first, or wait for it to reach `ready` / `failed` / `cancelled`. ::: @@ -270,10 +271,144 @@ For the full list of supported tokenizers — including `kagome_ja`, `kagome_kr` +## Reindex a property on a collection (v1.38+) + + + +From `v1.38`, you can change a property's inverted-index configuration on a **populated collection without restarting the cluster and without losing writes**. Reads stay available throughout the migration. This replaces the previous workaround of exporting the data, recreating the collection, and re-importing. + +These operations cover **inverted indexes only** (`IndexFilterable`, `IndexSearchable`, `IndexRangeFilters`, BM25 algorithm, tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). + +The feature is **REST-only** for the v1.38 Preview — there is no client-library support yet, so the examples below use `curl`. All examples assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections`. For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). + +### Add an inverted index to a collection + +To enable filtering on a property that was created without an index: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"filterable":{"enabled":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/category" +``` + +The body shape selects the index type — `filterable`, `searchable` (requires `text` / `text[]`), or `rangeable` (numeric types only). For the full body-shape reference, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). + +The server responds `202 Accepted` with a task ID: + +```json +{ + "status": "STARTED", + "taskId": "reindex/Article/category/enable-filterable/1700000000" +} +``` + +### Change tokenization on a collection + +To retokenize a populated `text` property: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"tokenization":"trigram"}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" +``` + +If the property has **both** a searchable and a filterable index, they are retokenized together in one coordinated migration. To retokenize only the filterable bucket, send `{"filterable":{"tokenization":"word"}}` instead. For the canonical list of tokenization values, see [Tokenization options](../config-refs/collections.mdx#tokenization). + +### Migrate BM25 from WAND to BlockMax + +[BlockMax WAND](../concepts/indexing/inverted-index.md#blockmax-wand-algorithm) is a faster scoring algorithm that replaced the original WAND implementation. On collections created before BlockMax was the default, you can migrate every searchable property in one call: + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"algorithm":"blockmax"}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" +``` + +The migration touches every searchable property on the class. The class-level `UsingBlockMaxWAND` flag flips to `true` after every property has been rebuilt. This is **one-way** — a request to flip `searchable.algorithm` back from `blockmax` to `wand` is rejected, because WAND is deprecated. + +### Rebuild or repair an index + +If you suspect corruption or want to refresh an index after heavy data churn, send `rebuild:true` for the bucket type: + +```bash +# Rebuild the BlockMax searchable bucket from object storage +curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"rebuild":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" + +# Repair a filterable bucket +curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"filterable":{"rebuild":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/category" + +# Repair a rangeable bucket +curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"rangeable":{"rebuild":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/price" +``` + +`searchable.rebuild` is for BlockMax buckets only. To rebuild a property still on WAND, first migrate it to BlockMax (above) — `rebuild:true` on a WAND bucket is rejected. `rangeable.rebuild` rebuilds from **object storage** (the same source enable-rangeable reads from), so it works even on numeric properties created with `IndexFilterable=false`. + +### Cancel an in-flight migration + +```bash +curl -X PUT \ + -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ + -d '{"searchable":{"cancel":true}}' \ + "$WEAVIATE/v1/schema/Article/indexes/body" +``` + +The response is one of: + +- `{"status": "CANCELLED", "taskId": ""}` — a `STARTED` task was found and cancelled. +- `{"status": "NO_OP"}` — nothing matched (already finished, never submitted, or already cancelled). Idempotent. + +Cancelling cleans up partial on-disk state so the next submit starts from a clean slate. The cleanup runs synchronously in the cancel request (with a 10-second drain timeout); if the drain times out, the next submit's pre-cleanup picks up the work. + +### Check migration status + +```bash +curl -fsS -H "Authorization: Bearer $TOKEN" \ + "$WEAVIATE/v1/schema/Article/indexes" +``` + +```json +{ + "collection": "Article", + "properties": [{ + "name": "body", + "dataType": "text", + "indexes": [ + { "type": "filterable", "status": "ready", "tokenization": "word" }, + { "type": "searchable", "status": "indexing", "progress": 0.42, + "tokenization": "word", "targetTokenization": "trigram", + "algorithm": "blockmax" } + ] + }] +} +``` + +The endpoint reads cluster task state directly, so polling in a tight loop is safe. For the meaning of each `status` value, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). To poll until an index is `ready`: + +```bash +until curl -fsS "$WEAVIATE/v1/schema/Article/indexes" \ + | jq -e '.properties[] | select(.name=="body") + | .indexes[] | select(.type=="searchable") + | .status == "ready"' > /dev/null +do sleep 2; done +``` + +To scope a migration to specific tenants on a multi-tenant class, add the `?tenants=` query parameter — see the [multi-tenancy rules](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview) in the reference. + ## Further resources - [References: Collection definition](/weaviate/config-refs/collections.mdx) -- [Concepts: Inverted index](../config-refs/indexing/inverted-index.mdx) +- [References: Inverted index](../config-refs/indexing/inverted-index.mdx) +- [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview) +- [Concepts: Inverted index](../concepts/indexing/inverted-index.md) ## Questions and feedback diff --git a/docs/weaviate/manage-collections/reindex-property.mdx b/docs/weaviate/manage-collections/reindex-property.mdx deleted file mode 100644 index c528e509..00000000 --- a/docs/weaviate/manage-collections/reindex-property.mdx +++ /dev/null @@ -1,260 +0,0 @@ ---- -title: Reindex a property -sidebar_label: Reindex a property -sidebar_position: 50 -image: og/docs/configuration.jpg -description: Change a property's inverted-index configuration on a live Weaviate collection without restart — add or drop filterable / searchable / range indexes, change tokenization, or migrate BM25 from WAND to BlockMax. Reads stay available throughout. -# tags: ['configuration', 'reindex', 'tokenization', 'bm25'] ---- - -import RuntimeReindexPreview from '/_includes/feature-notes/runtime-reindex.mdx'; - - - -Change a property's **inverted-index configuration** on a live collection without restarting the cluster and without losing writes. Reads stay available throughout the migration. This replaces the previous workaround of exporting, recreating the collection, and re-importing. - -This page covers **inverted indexes only** (`IndexFilterable`, `IndexSearchable`, `IndexRangeFilters`, BM25 algorithm, tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). - -## What you can do - -| Verb | Use when | -|---|---| -| **Add** a missing inverted index | The property was created without `IndexFilterable` / `IndexSearchable` / `IndexRangeFilters` and you now need to filter or search on it. | -| **Change** a property's tokenization | You picked `word` and want `trigram` (or vice versa) on a live `text` / `text[]` property. | -| **Migrate BM25** from WAND to BlockMax | One-way upgrade for the searchable bucket on existing collections. | -| **Rebuild / repair** a bucket | Refresh a `RoaringSet` / `RoaringSetRange` / BlockMax index after suspected corruption or heavy data churn. | -| **Cancel** an in-flight migration | You realised the call was wrong, or want to free the slot for another migration. | - -To **drop** a configured inverted index, use the existing [Drop an inverted index](./inverted-index.mdx#drop-an-inverted-index) flow — it has client-library support for Python, TypeScript, Go, and Java. From v1.38, the drop operation is gated by the same MutationGuard described on this page. - -## REST endpoints - -The feature is REST-only for v1.38 Preview. Two new endpoints (`PUT`, `GET`), plus a new MutationGuard contract on the existing drop endpoint: - -| Method | Path | Purpose | -|---|---|---| -| `PUT` | `/v1/schema/{class}/indexes/{property}` | **New in v1.38.** Submit a migration. The request body shape selects which migration type. | -| `GET` | `/v1/schema/{class}/indexes` | **New in v1.38.** Read per-property index status. | -| `DELETE` | `/v1/schema/{class}/properties/{property}/index/{indexName}` | Drop a configured index. Existed before v1.38 (see [Drop an inverted index](./inverted-index.mdx#drop-an-inverted-index)); from v1.38 it is now rejected by the MutationGuard while another reindex on the same property is in flight. | - -All examples below assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections` (`GET` only needs `READ` on `CollectionsMetadata`). - -## Add a missing inverted index - -To enable filtering on a property that was created without one: - -```bash -curl -X PUT \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"filterable":{"enabled":true}}' \ - "$WEAVIATE/v1/schema/Article/indexes/category" -``` - -Body shapes per index type: - -| Body | Effect | -|---|---| -| `{"filterable":{"enabled":true}}` | Creates a `RoaringSet` bucket and flips `IndexFilterable=true`. | -| `{"searchable":{"enabled":true,"tokenization":"word"}}` | Creates a BlockMax searchable bucket, sets `Tokenization`, flips `IndexSearchable=true`. Requires `text` / `text[]`. | -| `{"rangeable":{"enabled":true}}` | Creates a `RoaringSetRange` bucket and flips `IndexRangeFilters=true`. Numeric types only (`int`, `number`, `date`). | - -The server responds `202 Accepted` with a task ID: - -```json -{ - "status": "STARTED", - "taskId": "reindex/Article/category/enable-filterable/1700000000" -} -``` - -## Change tokenization - -To retokenize a populated `text` property: - -```bash -curl -X PUT \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"searchable":{"tokenization":"trigram"}}' \ - "$WEAVIATE/v1/schema/Article/indexes/body" -``` - -If the property has **both** a searchable and a filterable index, they are retokenized together in one coordinated migration. To retokenize only the filterable bucket: - -```bash -curl -X PUT \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"filterable":{"tokenization":"word"}}' \ - "$WEAVIATE/v1/schema/Article/indexes/category" -``` - -For the canonical list of tokenization values, see [Tokenization options](../config-refs/collections.mdx#tokenization). - -## Migrate BM25 from WAND to BlockMax - -[BlockMax WAND](../concepts/indexing/inverted-index.md) is a faster scoring algorithm that replaced the original WAND implementation. On collections created before BlockMax was the default, you can migrate every searchable property in one call: - -```bash -curl -X PUT \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"searchable":{"algorithm":"blockmax"}}' \ - "$WEAVIATE/v1/schema/Article/indexes/body" -``` - -The migration touches every searchable property on the class. The class-level `UsingBlockMaxWAND` flag flips to `true` after every property has been rebuilt. - -This is **one-way** — a request to flip `searchable.algorithm` back from `blockmax` to `wand` is rejected. WAND is deprecated. - -## Rebuild or repair an index - -If you suspect corruption or want to refresh an index after heavy data churn: - -```bash -# Rebuild the BlockMax searchable bucket from object storage -curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"searchable":{"rebuild":true}}' \ - "$WEAVIATE/v1/schema/Article/indexes/body" - -# Repair a filterable bucket -curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"filterable":{"rebuild":true}}' \ - "$WEAVIATE/v1/schema/Article/indexes/category" - -# Repair a rangeable bucket -curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"rangeable":{"rebuild":true}}' \ - "$WEAVIATE/v1/schema/Article/indexes/price" -``` - -`searchable.rebuild` is for BlockMax buckets only. To rebuild a property still on WAND, first migrate it to BlockMax (above) — `rebuild:true` on a WAND bucket is rejected. - -## Cancel an in-flight migration - -```bash -curl -X PUT \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"searchable":{"cancel":true}}' \ - "$WEAVIATE/v1/schema/Article/indexes/body" -``` - -The response is one of: - -- `{"status": "CANCELLED", "taskId": ""}` — a `STARTED` task was found and cancelled. -- `{"status": "NO_OP"}` — nothing matched (already finished, never submitted, or already cancelled). Idempotent. - -Cancelling wipes partial on-disk state, so the next submit starts from a clean slate. - -## Check migration status - -```bash -curl -fsS -H "Authorization: Bearer $TOKEN" \ - "$WEAVIATE/v1/schema/Article/indexes" -``` - -```json -{ - "collection": "Article", - "properties": [{ - "name": "body", - "dataType": "text", - "indexes": [ - { "type": "filterable", "status": "ready", "tokenization": "word" }, - { "type": "searchable", "status": "indexing", "progress": 0.42, - "tokenization": "word", "targetTokenization": "trigram", - "algorithm": "blockmax" } - ] - }] -} -``` - -| `status` | Meaning | -|---|---| -| `ready` | Index is live and serving. No migration in flight. | -| `pending` | A task has been accepted; per-shard work has not started yet. | -| `indexing` | Per-shard work is running. `progress` is a 0..1 estimate. | -| `failed` | At least one node reported `Success=false`. The schema flag was **not** flipped — the property is still in its pre-migration state. Submit a new task to retry. | -| `cancelled` | Operator cancelled. Partial state has been scrubbed. | - -Readback is throttled (3 s), so polling in a tight loop is cheap: - -```bash -until curl -fsS "$WEAVIATE/v1/schema/Article/indexes" \ - | jq -e '.properties[] | select(.name=="body") - | .indexes[] | select(.type=="searchable") - | .status == "ready"' > /dev/null -do sleep 2; done -``` - -## Multi-tenancy - -Scope a task to specific tenants on a multi-tenant class with the `?tenants=` query parameter: - -```bash -curl -X PUT \ - -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ - -d '{"filterable":{"rebuild":true}}' \ - "$WEAVIATE/v1/schema/Article/indexes/category?tenants=customerA,customerB" -``` - -Rules: - -| Class | `?tenants=` | Migration type | Result | -|---|---|---|---| -| Single-tenant | provided | any | `400` | -| Multi-tenant | omitted | format-only (`rebuild`, `repair`, `enable-rangeable`) | targets all tenants | -| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-*`) | targets all tenants (the schema flip is cluster-wide) | -| Multi-tenant | provided | format-only | targets the named subset | -| Multi-tenant | provided | semantic | `400` — semantic migrations cannot be sub-scoped | -| any | tenant in `OFFLOADED` / `FROZEN` | any | `400` — the offending tenant is named in the error | - -Each tenant's replicas form an independent barrier group: tenant A starts serving the new bucket as soon as its own replicas finish, even if tenant B is still reindexing. - -## Concurrency limits - -- **Per `(class, property)` exclusivity** — only one migration is allowed in flight on a given `(collection, property)` pair. A second submit on the same pair returns `409` with the offending task ID. -- **Per-class cap** — up to 32 concurrent migrations per class. The next submit returns `503` until the in-flight units drain. -- **Different properties on the same class run in parallel.** Different classes are fully independent. - -## What's blocked during a reindex - -While a reindex is in flight on `(class, property)`, the schema FSM rejects: - -- `UpdateProperty` on the same property. -- `DeleteClass` on the affected class. -- `DeleteTenants` / `UpdateTenants` on the targeted tenants when the transition makes shards locally unavailable (HOT → COLD / FROZEN / OFFLOADED). Transitions toward available (e.g. UNFREEZING) are not blocked. -- `DELETE /v1/schema/{class}/properties/{property}/index/{indexName}` on the same property. - -The reject message names the in-flight task. Submitting a reindex on a **different** property of the same class is **not** blocked. - -## Errors and recovery - -| Code | When | Resolution | -|---|---|---| -| `400` | Malformed body, wrong property type, missing prerequisite (e.g. `searchable.tokenization` on a property with no searchable index), `?tenants=` on a single-tenant class, `?tenants=` on a semantic migration, target tenant in `OFFLOADED` / `FROZEN`. | Error responses carry next-step hints — read them. | -| `404` | Class or property doesn't exist. | Verify the class + property names. | -| `409` | An in-flight task overlaps this `(collection, property)`. The error names the offending task ID and migration type. | Wait, or cancel the existing task first. | -| `429` / `503` | Per-class in-flight cap reached, or cluster temporarily unavailable. | Retry once in-flight migrations drain. | - -The schema flag is the source of truth: if a task ends in `failed`, the flag was not flipped and the property is still in its pre-migration state. A reindex is restart-safe at every phase — in-flight migrations are picked up automatically after a node restart. - -## Authentication and authorization - -| Endpoint | Required permission | -|---|---| -| `GET /v1/schema/{class}/indexes` | `READ` on `CollectionsMetadata` | -| `PUT /v1/schema/{class}/indexes/{property}` | `UPDATE` on `Collections` | -| `DELETE /v1/schema/{class}/properties/{property}/index/{indexName}` | `UPDATE` on `Collections` | - -`UPDATE` on `Collections` is the same permission that gates `UpdateClass` and replication-factor changes — there is no dedicated `reindex` role today. - -## Related pages - -- [Inverted index configuration](./inverted-index.mdx) — how to set inverted indexes at collection creation, and how to drop them (multi-language client support). -- [Tokenization options](../config-refs/collections.mdx#tokenization) — canonical list of tokenization values. -- [Vector configuration](./vector-config.mdx) — sibling page covering vector indexes. -- [BlockMax WAND](../concepts/indexing/inverted-index.md) — background on the BM25 algorithm migration. - -## Questions and feedback - -import DocsFeedback from '/\_includes/docs-feedback.mdx'; - - diff --git a/docusaurus.config.js b/docusaurus.config.js index 5bf32c41..f5242667 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -54,7 +54,7 @@ const config = { cdn: "https://cdn.jsdelivr.net/npm/@scalar/api-reference@1.49.0", configuration: { spec: { - url: "https://raw.githubusercontent.com/weaviate/weaviate/openapi-for-docs/openapi-specs/schema.json", + url: "https://raw.githubusercontent.com/weaviate/weaviate/v1-38/openapi-for-docs/openapi-specs/schema.json", }, hideModels: true, // showSidebar: true, diff --git a/sidebars.js b/sidebars.js index eec3e37b..b3200af6 100644 --- a/sidebars.js +++ b/sidebars.js @@ -580,7 +580,6 @@ const sidebars = { "weaviate/manage-collections/vector-config", "weaviate/manage-collections/generative-reranker-models", "weaviate/manage-collections/inverted-index", - "weaviate/manage-collections/reindex-property", { type: "category", label: "Multi-tenancy", From 9fa65f71145c4f0bae60908c103e9c469329a8f2 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 18:42:35 +0200 Subject: [PATCH 3/8] Address Copilot review on runtime-reindex docs - Note the status token also needs READ on CollectionsMetadata - Add Authorization header to the status poll-loop example - Disambiguate enable-rangeable as format-only in the multi-tenancy table - Update filtering.md: existing properties can gain a rangeable index from v1.38 Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/weaviate/concepts/filtering.md | 2 +- docs/weaviate/config-refs/indexing/inverted-index.mdx | 2 +- docs/weaviate/manage-collections/inverted-index.mdx | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/weaviate/concepts/filtering.md b/docs/weaviate/concepts/filtering.md index 0a369348..790df2a4 100644 --- a/docs/weaviate/concepts/filtering.md +++ b/docs/weaviate/concepts/filtering.md @@ -80,7 +80,7 @@ The `indexRangeFilters` index is a range-based index for filtering by numerical Internally, rangeable indexes are implemented as roaring bitmap slices. This data structure limits the index to values that can be stored as 64 bit integers. -`indexRangeFilters` is only available for new properties. Existing properties cannot be converted to use the rangeable index. +Before `v1.38`, `indexRangeFilters` was only available for new properties — existing properties could not be converted to use the rangeable index. From `v1.38`, you can add a rangeable index to an existing property on a populated collection without restart using the [runtime reindex](../manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection-v138) endpoints. ## Recall on Pre-Filtered Searches diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index 4e32a60b..2b81b517 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -321,7 +321,7 @@ Scope a task to specific tenants on a multi-tenant class with the `?tenants=` qu |---|---|---|---| | Single-tenant | provided | any | `400` | | Multi-tenant | omitted | format-only (`rebuild`, `repair`, `enable-rangeable`) | targets all tenants | -| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-*`) | targets all tenants (the schema flip is cluster-wide) | +| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-filterable`, `enable-searchable`) | targets all tenants (the schema flip is cluster-wide) | | Multi-tenant | provided | format-only | targets the named subset | | Multi-tenant | provided | semantic | `400` — semantic migrations cannot be sub-scoped | | any | tenant in `OFFLOADED` / `FROZEN` | any | `400` — the offending tenant is named in the error | diff --git a/docs/weaviate/manage-collections/inverted-index.mdx b/docs/weaviate/manage-collections/inverted-index.mdx index 5dad4b30..4ac890b2 100644 --- a/docs/weaviate/manage-collections/inverted-index.mdx +++ b/docs/weaviate/manage-collections/inverted-index.mdx @@ -279,7 +279,7 @@ From `v1.38`, you can change a property's inverted-index configuration on a **po These operations cover **inverted indexes only** (`IndexFilterable`, `IndexSearchable`, `IndexRangeFilters`, BM25 algorithm, tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). -The feature is **REST-only** for the v1.38 Preview — there is no client-library support yet, so the examples below use `curl`. All examples assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections`. For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). +The feature is **REST-only** for the v1.38 Preview — there is no client-library support yet, so the examples below use `curl`. All examples assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections` (required to submit a reindex) and `READ` on `CollectionsMetadata` (required to read the migration status). For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). ### Add an inverted index to a collection @@ -394,7 +394,7 @@ curl -fsS -H "Authorization: Bearer $TOKEN" \ The endpoint reads cluster task state directly, so polling in a tight loop is safe. For the meaning of each `status` value, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). To poll until an index is `ready`: ```bash -until curl -fsS "$WEAVIATE/v1/schema/Article/indexes" \ +until curl -fsS -H "Authorization: Bearer $TOKEN" "$WEAVIATE/v1/schema/Article/indexes" \ | jq -e '.properties[] | select(.name=="body") | .indexes[] | select(.type=="searchable") | .status == "ready"' > /dev/null From ea016557b1cd1c5c11d394d79cdfd3406a091718 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 20:36:46 +0200 Subject: [PATCH 4/8] Mark BM25 algorithm migration as semantic in MT table Confirmed against weaviate IsSemanticMigration(): change-algorithm (WAND -> BlockMax) is a semantic migration and cannot be tenant-scoped, while enable-rangeable is intentionally format-only and can be. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/weaviate/config-refs/indexing/inverted-index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index 2b81b517..5e0eadeb 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -321,7 +321,7 @@ Scope a task to specific tenants on a multi-tenant class with the `?tenants=` qu |---|---|---|---| | Single-tenant | provided | any | `400` | | Multi-tenant | omitted | format-only (`rebuild`, `repair`, `enable-rangeable`) | targets all tenants | -| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-filterable`, `enable-searchable`) | targets all tenants (the schema flip is cluster-wide) | +| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-filterable`, `enable-searchable`, `change-algorithm` (BM25 WAND → BlockMax)) | targets all tenants (the schema flip is cluster-wide) | | Multi-tenant | provided | format-only | targets the named subset | | Multi-tenant | provided | semantic | `400` — semantic migrations cannot be sub-scoped | | any | tenant in `OFFLOADED` / `FROZEN` | any | `400` — the offending tenant is named in the error | From d8f72bc9c43cb9312bda920300121cce3f5947a0 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 20:41:24 +0200 Subject: [PATCH 5/8] Remove internal implementation details from reindex docs Reword user-facing reindex content to drop internal concepts that users shouldn't need to know: storage buckets (RoaringSet/RoaringSetRange/ BlockMax buckets), internal schema-flag names, the schema FSM, replicas/ barrier groups, object storage, task-ID encoding, and finalize/drain timing. Describe observable behavior instead. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../concepts/indexing/inverted-index.md | 2 +- .../config-refs/indexing/inverted-index.mdx | 54 +++++++++---------- .../manage-collections/inverted-index.mdx | 34 ++++++------ 3 files changed, 45 insertions(+), 45 deletions(-) diff --git a/docs/weaviate/concepts/indexing/inverted-index.md b/docs/weaviate/concepts/indexing/inverted-index.md index 0d4943d2..f1123be6 100644 --- a/docs/weaviate/concepts/indexing/inverted-index.md +++ b/docs/weaviate/concepts/indexing/inverted-index.md @@ -172,7 +172,7 @@ An example of a complete collection object without inverted indexes: Because an inverted index is built at import time, a property created without one (or with the "wrong" tokenization or BM25 algorithm) historically required exporting the data, recreating the collection, and re-importing — an expensive, downtime-prone operation. -From `v1.38`, Weaviate can **reindex a property on a collection** instead. A reindex builds the new bucket in the background from object storage while the existing index keeps serving reads, then atomically flips a single schema flag once every replica has finished. The schema flag is the source of truth: if the rebuild fails, the flag is never flipped and the property stays in its pre-migration state, and an interrupted reindex is picked up automatically after a node restart. This makes adding a missing index, changing tokenization, or migrating BM25 from WAND to BlockMax a non-destructive, restart-safe operation. +From `v1.38`, Weaviate can **reindex a property on a collection** instead. A reindex builds the new index in the background from the stored objects while the existing index keeps serving reads, then switches over once the rebuild is complete. If the rebuild fails, the property is left in its pre-migration state — nothing is partially applied — and an interrupted reindex resumes automatically after a node restart. This makes adding a missing index, changing tokenization, or migrating BM25 from WAND to BlockMax a non-destructive, restart-safe operation. For the operational steps, see [How-to: Reindex a property on a collection](/weaviate/manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection-v138); for the endpoint reference, see [References: Runtime reindex](/weaviate/config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index 5e0eadeb..a8ace45e 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -257,7 +257,7 @@ You can drop (delete) an inverted index from a property. This is a destructive o The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. -REST: `DELETE /v1/schema/{className}/properties/{propertyName}/index/{indexName}`. From `v1.38`, the drop is gated by the [runtime-reindex](#runtime-reindex-v138-preview) MutationGuard — rejected while a reindex is in flight on the same property. +REST: `DELETE /v1/schema/{className}/properties/{propertyName}/index/{indexName}`. From `v1.38`, the drop is rejected while a [runtime reindex](#runtime-reindex-v138-preview) is in progress on the same property. See [How-to: Drop an inverted index](../../manage-collections/inverted-index.mdx#drop-an-inverted-index) for code examples. @@ -273,7 +273,7 @@ From `v1.38`, three REST endpoints let you alter a property's inverted-index con | Method | Path | Purpose | |---|---|---| -| `PUT` | `/v1/schema/{class}/indexes/{property}` | Add an inverted index, change tokenization, migrate BM25 WAND → BlockMax, rebuild a bucket, or cancel an in-flight task. The body shape selects the migration type. | +| `PUT` | `/v1/schema/{class}/indexes/{property}` | Add an inverted index, change tokenization, migrate BM25 WAND → BlockMax, rebuild an index, or cancel an in-progress migration. The body shape selects the migration type. | | `DELETE` | `/v1/schema/{class}/properties/{property}/index/{indexName}` | Drop a configured index. `indexName` ∈ `{filterable, searchable, rangeFilters}`. | | `GET` | `/v1/schema/{class}/indexes` | Read per-property index status. | @@ -281,15 +281,15 @@ From `v1.38`, three REST endpoints let you alter a property's inverted-index con | Body | Effect | |---|---| -| `{"filterable":{"enabled":true}}` | Creates a `RoaringSet` bucket and flips `IndexFilterable=true`. | -| `{"searchable":{"enabled":true,"tokenization":"word"}}` | Creates a BlockMax searchable bucket, sets `Tokenization`, flips `IndexSearchable=true`. Requires `text` / `text[]`. | -| `{"rangeable":{"enabled":true}}` | Creates a `RoaringSetRange` bucket and flips `IndexRangeFilters=true`. Numeric types only (`int`, `number`, `date`). | -| `{"searchable":{"tokenization":"trigram"}}` | Retokenizes a populated `text` property. If a filterable bucket also exists, both are retokenized together. | -| `{"searchable":{"algorithm":"blockmax"}}` | Migrates every searchable property on the class from WAND to BlockMax. One-way; flipping back to `wand` is rejected. | -| `{"":{"rebuild":true}}` | Rebuilds the named bucket (`filterable` / `searchable` / `rangeable`) from object storage. `searchable.rebuild` requires the property to already be on BlockMax. | -| `{"":{"cancel":true}}` | Cancels an in-flight task on the property. Idempotent — returns `NO_OP` when nothing matches. | +| `{"filterable":{"enabled":true}}` | Enables filtering on the property. | +| `{"searchable":{"enabled":true,"tokenization":"word"}}` | Enables keyword search on the property and sets its tokenization. Requires `text` / `text[]`. | +| `{"rangeable":{"enabled":true}}` | Enables range filtering on the property. Numeric types only (`int`, `number`, `date`). | +| `{"searchable":{"tokenization":"trigram"}}` | Retokenizes a populated `text` property. If the property also has a filterable index, both are retokenized together. | +| `{"searchable":{"algorithm":"blockmax"}}` | Migrates every searchable property on the collection from WAND to BlockMax. One-way; switching back to `wand` is rejected. | +| `{"":{"rebuild":true}}` | Rebuilds the named index (`filterable` / `searchable` / `rangeable`) from the stored objects. `searchable.rebuild` requires the property to already use BlockMax. | +| `{"":{"cancel":true}}` | Cancels an in-progress migration on the property. Idempotent — returns `NO_OP` when nothing matches. | -A successful submit returns `202 Accepted` with `{"status":"STARTED","taskId":""}`. +A successful submit returns `202 Accepted` with `{"status":"STARTED","taskId":""}`. Treat `taskId` as an opaque identifier. ### Status values (`GET`) @@ -297,21 +297,21 @@ A successful submit returns `202 Accepted` with `{"status":"STARTED","taskId":"< | `status` | Meaning | |---|---| -| `ready` | Index is live and serving. No migration in flight. | -| `pending` | A task has been accepted; per-shard work has not started yet. | -| `indexing` | Per-shard work is running. `progress` is a 0..1 estimate. | -| `failed` | At least one node reported `Success=false`. The schema flag was **not** flipped — the property is still in its pre-migration state. Submit a new task to retry. | -| `cancelled` | Operator cancelled. Partial state has been scrubbed. | +| `ready` | Index is live and serving. No migration in progress. | +| `pending` | A migration has been accepted but has not started yet. | +| `indexing` | Migration is running. `progress` is a 0..1 estimate. | +| `failed` | The migration did not complete. The property is left in its pre-migration state — submit it again to retry. | +| `cancelled` | The migration was cancelled. The property is left in its pre-migration state. | -Between "task FINISHED" and "schema flag flipped", the response keeps the index at `indexing@100%` for a 3–10-second finalize window so the status never blinks back to a pre-migration shape before the schema catches up. +The status stays at `indexing` until the change is fully applied, so it never briefly reports the pre-migration shape after the data is already rebuilt. ### Concurrency -- **Per `(class, property)` exclusivity** — only one migration is allowed in flight on a given `(collection, property)` pair. A second submit on the same pair returns `409` with the offending task ID. -- **Per-class cap** — up to 32 concurrent migrations per class. The next submit returns `429 Too Many Requests` until the in-flight units drain. -- **Different properties on the same class run in parallel.** Different classes are fully independent. +- **One migration per property** — only one migration can be in progress on a given property at a time. A second submit on the same property returns `409`. +- **Per-collection cap** — up to 32 migrations can run concurrently on a collection. Further submits return `429 Too Many Requests` until in-progress migrations finish. +- **Different properties on the same collection run in parallel.** Different collections are fully independent. -While a reindex is in flight on `(class, property)`, the schema FSM rejects `UpdateProperty` on the same property, `DeleteClass` on the affected class, `DeleteTenants` / `UpdateTenants` that make targeted shards locally unavailable (HOT → COLD / FROZEN / OFFLOADED), and `DELETE …/index/{indexName}` on the same property. The reject message names the in-flight task. A reindex on a **different** property of the same class is not blocked. +While a reindex is in progress on a property, Weaviate rejects updates to that property, deletion of the collection, dropping an index on that property, and tenant-status changes that would take the targeted tenants offline (for example moving them to `COLD`, `FROZEN`, or `OFFLOADED`). A reindex on a **different** property of the same collection is not blocked. ### Multi-tenancy @@ -321,12 +321,12 @@ Scope a task to specific tenants on a multi-tenant class with the `?tenants=` qu |---|---|---|---| | Single-tenant | provided | any | `400` | | Multi-tenant | omitted | format-only (`rebuild`, `repair`, `enable-rangeable`) | targets all tenants | -| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-filterable`, `enable-searchable`, `change-algorithm` (BM25 WAND → BlockMax)) | targets all tenants (the schema flip is cluster-wide) | +| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-filterable`, `enable-searchable`, `change-algorithm` (BM25 WAND → BlockMax)) | targets all tenants (the change is collection-wide) | | Multi-tenant | provided | format-only | targets the named subset | | Multi-tenant | provided | semantic | `400` — semantic migrations cannot be sub-scoped | | any | tenant in `OFFLOADED` / `FROZEN` | any | `400` — the offending tenant is named in the error | -Each tenant's replicas form an independent barrier group: tenant A starts serving the new bucket as soon as its own replicas finish, even if tenant B is still reindexing. +Each tenant is reindexed independently: a tenant starts serving its new index as soon as its own reindex finishes, even if other tenants are still in progress. ### Errors and recovery @@ -334,11 +334,11 @@ Each tenant's replicas form an independent barrier group: tenant A starts servin |---|---|---| | `400` | Malformed body, wrong property type, missing prerequisite (e.g. `searchable.tokenization` on a property with no searchable index), `?tenants=` on a single-tenant class, `?tenants=` on a semantic migration, target tenant in `OFFLOADED` / `FROZEN`. | Error responses carry next-step hints — read them. | | `404` | Class or property doesn't exist. | Verify the class + property names. | -| `409` | An in-flight task overlaps this `(collection, property)`. The error names the offending task ID and migration type. | Wait, or cancel the existing task first. | -| `429` | Per-class in-flight cap reached (32 concurrent migrations). | Retry once in-flight migrations drain. | -| `503` | Cluster service temporarily unavailable. | Retry. | +| `409` | A migration is already in progress on this property. | Wait, or cancel the existing migration first. | +| `429` | Per-collection concurrency cap reached (32 migrations). | Retry once in-progress migrations finish. | +| `503` | Service temporarily unavailable. | Retry. | -The schema flag is the source of truth: if a task ends in `failed`, the flag was not flipped and the property is still in its pre-migration state. A reindex is restart-safe at every phase — in-flight migrations are picked up automatically after a node restart. +If a migration ends in `failed`, the property is left in its pre-migration state — nothing is partially applied. Reindexing is also restart-safe: an interrupted migration resumes automatically after a node restart. ### Required permissions @@ -348,7 +348,7 @@ The schema flag is the source of truth: if a task ends in `failed`, the flag was | `PUT /v1/schema/{class}/indexes/{property}` | `UPDATE` on `Collections` | | `DELETE /v1/schema/{class}/properties/{property}/index/{indexName}` | `UPDATE` on `CollectionsMetadata` | -`PUT` is intentionally stricter than the other endpoints: submitting a reindex task rebuilds buckets on every replica and flips schema flags, so it requires `UPDATE` on `Collections` — the same permission that gates `UpdateClass` and replication-factor changes. `DELETE` shares the existing schema-metadata permission (`UPDATE` on `CollectionsMetadata`) used by the other property-management endpoints. There is no dedicated `reindex` role today. +`PUT` is intentionally stricter than the other endpoints: submitting a reindex rebuilds the index and changes the collection definition, so it requires `UPDATE` on `Collections` — the same permission that gates other collection-definition changes. `DELETE` shares the existing schema-metadata permission (`UPDATE` on `CollectionsMetadata`) used by the other property-management endpoints. There is no dedicated `reindex` role today. ## How Weaviate creates inverted indexes diff --git a/docs/weaviate/manage-collections/inverted-index.mdx b/docs/weaviate/manage-collections/inverted-index.mdx index 4ac890b2..d30ad358 100644 --- a/docs/weaviate/manage-collections/inverted-index.mdx +++ b/docs/weaviate/manage-collections/inverted-index.mdx @@ -162,9 +162,9 @@ Drop (delete) an inverted index from a property. This is a destructive operation The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. -:::note v1.38+ MutationGuard +:::note v1.38+ -From `v1.38`, the drop is rejected with `409` while a [runtime reindex](#reindex-a-property-on-a-collection-v138) is in flight on the same property. Cancel the in-flight task first, or wait for it to reach `ready` / `failed` / `cancelled`. +From `v1.38`, the drop is rejected with `409` while a [runtime reindex](#reindex-a-property-on-a-collection-v138) is in progress on the same property. Cancel the in-progress migration first, or wait for it to finish. ::: @@ -275,9 +275,9 @@ For the full list of supported tokenizers — including `kagome_ja`, `kagome_kr` -From `v1.38`, you can change a property's inverted-index configuration on a **populated collection without restarting the cluster and without losing writes**. Reads stay available throughout the migration. This replaces the previous workaround of exporting the data, recreating the collection, and re-importing. +From `v1.38`, you can change a property's inverted-index configuration on a **populated collection without restarting and without losing writes**. Reads stay available throughout the migration. This replaces the previous workaround of exporting the data, recreating the collection, and re-importing. -These operations cover **inverted indexes only** (`IndexFilterable`, `IndexSearchable`, `IndexRangeFilters`, BM25 algorithm, tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). +These operations cover **inverted indexes only** (filterable, searchable, and range indexes, the BM25 algorithm, and tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). The feature is **REST-only** for the v1.38 Preview — there is no client-library support yet, so the examples below use `curl`. All examples assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections` (required to submit a reindex) and `READ` on `CollectionsMetadata` (required to read the migration status). For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). @@ -294,12 +294,12 @@ curl -X PUT \ The body shape selects the index type — `filterable`, `searchable` (requires `text` / `text[]`), or `rangeable` (numeric types only). For the full body-shape reference, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). -The server responds `202 Accepted` with a task ID: +The server responds `202 Accepted` with a task ID, which you can treat as an opaque identifier: ```json { "status": "STARTED", - "taskId": "reindex/Article/category/enable-filterable/1700000000" + "taskId": "" } ``` @@ -314,7 +314,7 @@ curl -X PUT \ "$WEAVIATE/v1/schema/Article/indexes/body" ``` -If the property has **both** a searchable and a filterable index, they are retokenized together in one coordinated migration. To retokenize only the filterable bucket, send `{"filterable":{"tokenization":"word"}}` instead. For the canonical list of tokenization values, see [Tokenization options](../config-refs/collections.mdx#tokenization). +If the property has **both** a searchable and a filterable index, they are retokenized together in one coordinated migration. To retokenize only the filterable index, send `{"filterable":{"tokenization":"word"}}` instead. For the canonical list of tokenization values, see [Tokenization options](../config-refs/collections.mdx#tokenization). ### Migrate BM25 from WAND to BlockMax @@ -327,32 +327,32 @@ curl -X PUT \ "$WEAVIATE/v1/schema/Article/indexes/body" ``` -The migration touches every searchable property on the class. The class-level `UsingBlockMaxWAND` flag flips to `true` after every property has been rebuilt. This is **one-way** — a request to flip `searchable.algorithm` back from `blockmax` to `wand` is rejected, because WAND is deprecated. +The migration covers every searchable property on the collection, which then uses BlockMax once every property has been rebuilt. This is **one-way** — switching `searchable.algorithm` back from `blockmax` to `wand` is rejected, because WAND is deprecated. ### Rebuild or repair an index -If you suspect corruption or want to refresh an index after heavy data churn, send `rebuild:true` for the bucket type: +If you suspect corruption or want to refresh an index after heavy data churn, send `rebuild:true` for the index type: ```bash -# Rebuild the BlockMax searchable bucket from object storage +# Rebuild the searchable index curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{"searchable":{"rebuild":true}}' \ "$WEAVIATE/v1/schema/Article/indexes/body" -# Repair a filterable bucket +# Rebuild the filterable index curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{"filterable":{"rebuild":true}}' \ "$WEAVIATE/v1/schema/Article/indexes/category" -# Repair a rangeable bucket +# Rebuild the range index curl -X PUT -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{"rangeable":{"rebuild":true}}' \ "$WEAVIATE/v1/schema/Article/indexes/price" ``` -`searchable.rebuild` is for BlockMax buckets only. To rebuild a property still on WAND, first migrate it to BlockMax (above) — `rebuild:true` on a WAND bucket is rejected. `rangeable.rebuild` rebuilds from **object storage** (the same source enable-rangeable reads from), so it works even on numeric properties created with `IndexFilterable=false`. +Each index is rebuilt from the stored objects. `searchable.rebuild` requires the property to already use BlockMax — to rebuild a property still on WAND, first migrate it to BlockMax (above). -### Cancel an in-flight migration +### Cancel an in-progress migration ```bash curl -X PUT \ @@ -363,10 +363,10 @@ curl -X PUT \ The response is one of: -- `{"status": "CANCELLED", "taskId": ""}` — a `STARTED` task was found and cancelled. +- `{"status": "CANCELLED", "taskId": ""}` — an in-progress migration was found and cancelled. - `{"status": "NO_OP"}` — nothing matched (already finished, never submitted, or already cancelled). Idempotent. -Cancelling cleans up partial on-disk state so the next submit starts from a clean slate. The cleanup runs synchronously in the cancel request (with a 10-second drain timeout); if the drain times out, the next submit's pre-cleanup picks up the work. +Cancelling clears any partial state, so the next submit starts fresh. ### Check migration status @@ -391,7 +391,7 @@ curl -fsS -H "Authorization: Bearer $TOKEN" \ } ``` -The endpoint reads cluster task state directly, so polling in a tight loop is safe. For the meaning of each `status` value, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). To poll until an index is `ready`: +Polling this endpoint is cheap, so you can poll in a loop while a migration runs. For the meaning of each `status` value, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). To poll until an index is `ready`: ```bash until curl -fsS -H "Authorization: Bearer $TOKEN" "$WEAVIATE/v1/schema/Article/indexes" \ From bc39d8b8aa989c60e8927e60723528301cfe78ec Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Sun, 31 May 2026 20:44:12 +0200 Subject: [PATCH 6/8] Drop version suffix from reindex section headings Remove (v1.38+) / (v1.38 Preview) from the two reindex headings and update all cross-link anchors accordingly. The preview admonition at the start of each section already conveys the version. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/weaviate/concepts/filtering.md | 2 +- .../weaviate/concepts/indexing/inverted-index.md | 2 +- .../config-refs/indexing/inverted-index.mdx | 8 ++++---- .../manage-collections/collection-operations.mdx | 2 +- .../manage-collections/inverted-index.mdx | 16 ++++++++-------- 5 files changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/weaviate/concepts/filtering.md b/docs/weaviate/concepts/filtering.md index 790df2a4..bf19ff57 100644 --- a/docs/weaviate/concepts/filtering.md +++ b/docs/weaviate/concepts/filtering.md @@ -80,7 +80,7 @@ The `indexRangeFilters` index is a range-based index for filtering by numerical Internally, rangeable indexes are implemented as roaring bitmap slices. This data structure limits the index to values that can be stored as 64 bit integers. -Before `v1.38`, `indexRangeFilters` was only available for new properties — existing properties could not be converted to use the rangeable index. From `v1.38`, you can add a rangeable index to an existing property on a populated collection without restart using the [runtime reindex](../manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection-v138) endpoints. +Before `v1.38`, `indexRangeFilters` was only available for new properties — existing properties could not be converted to use the rangeable index. From `v1.38`, you can add a rangeable index to an existing property on a populated collection without restart using the [runtime reindex](../manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection) endpoints. ## Recall on Pre-Filtered Searches diff --git a/docs/weaviate/concepts/indexing/inverted-index.md b/docs/weaviate/concepts/indexing/inverted-index.md index f1123be6..4c9433ac 100644 --- a/docs/weaviate/concepts/indexing/inverted-index.md +++ b/docs/weaviate/concepts/indexing/inverted-index.md @@ -174,7 +174,7 @@ Because an inverted index is built at import time, a property created without on From `v1.38`, Weaviate can **reindex a property on a collection** instead. A reindex builds the new index in the background from the stored objects while the existing index keeps serving reads, then switches over once the rebuild is complete. If the rebuild fails, the property is left in its pre-migration state — nothing is partially applied — and an interrupted reindex resumes automatically after a node restart. This makes adding a missing index, changing tokenization, or migrating BM25 from WAND to BlockMax a non-destructive, restart-safe operation. -For the operational steps, see [How-to: Reindex a property on a collection](/weaviate/manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection-v138); for the endpoint reference, see [References: Runtime reindex](/weaviate/config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). +For the operational steps, see [How-to: Reindex a property on a collection](/weaviate/manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection); for the endpoint reference, see [References: Runtime reindex](/weaviate/config-refs/indexing/inverted-index.mdx#runtime-reindex). ## Tokenization diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index a8ace45e..8022daed 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -257,17 +257,17 @@ You can drop (delete) an inverted index from a property. This is a destructive o The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. -REST: `DELETE /v1/schema/{className}/properties/{propertyName}/index/{indexName}`. From `v1.38`, the drop is rejected while a [runtime reindex](#runtime-reindex-v138-preview) is in progress on the same property. +REST: `DELETE /v1/schema/{className}/properties/{propertyName}/index/{indexName}`. From `v1.38`, the drop is rejected while a [runtime reindex](#runtime-reindex) is in progress on the same property. See [How-to: Drop an inverted index](../../manage-collections/inverted-index.mdx#drop-an-inverted-index) for code examples. -## Runtime reindex (v1.38 Preview) +## Runtime reindex import RuntimeReindexPreview from "/_includes/feature-notes/runtime-reindex.mdx"; -From `v1.38`, three REST endpoints let you alter a property's inverted-index configuration on a collection without restart. For task-oriented walkthroughs with `curl` examples, see [How-to: Reindex a property on a collection](../../manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection-v138). This section is the endpoint and behavior reference. +From `v1.38`, three REST endpoints let you alter a property's inverted-index configuration on a collection without restart. For task-oriented walkthroughs with `curl` examples, see [How-to: Reindex a property on a collection](../../manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection). This section is the endpoint and behavior reference. ### Endpoints @@ -365,7 +365,7 @@ This is caused by the inverted index being built at import time. If you add a pr To avoid this, you can either: - Add the property before importing objects. -- From `v1.38`, use the [runtime reindex](#runtime-reindex-v138-preview) endpoints to add or change an inverted index on the collection without restart. +- From `v1.38`, use the [runtime reindex](#runtime-reindex) endpoints to add or change an inverted index on the collection without restart. - On versions before `v1.38`, delete the collection, re-create it with the new property, and then re-import the data. ## How tokenization affects inverted indexing diff --git a/docs/weaviate/manage-collections/collection-operations.mdx b/docs/weaviate/manage-collections/collection-operations.mdx index 965e6c9f..f6392efa 100644 --- a/docs/weaviate/manage-collections/collection-operations.mdx +++ b/docs/weaviate/manage-collections/collection-operations.mdx @@ -595,7 +595,7 @@ Property indexes are built at import time. If you add a new property after impor To create an index that includes all of the objects in a collection, do one of the following: - New collections: Add all of the collection's properties before importing objects. -- Existing collections (v1.38+): Use the runtime [Reindex a property](./inverted-index.mdx#reindex-a-property-on-a-collection-v138) endpoints to add or change inverted indexes on a collection without restart. +- Existing collections (v1.38+): Use the runtime [Reindex a property](./inverted-index.mdx#reindex-a-property-on-a-collection) endpoints to add or change inverted indexes on a collection without restart. - Existing collections (pre-v1.38): Export the existing data from the collection, recreate it with the new property, and re-import the data into the updated collection. diff --git a/docs/weaviate/manage-collections/inverted-index.mdx b/docs/weaviate/manage-collections/inverted-index.mdx index d30ad358..87872154 100644 --- a/docs/weaviate/manage-collections/inverted-index.mdx +++ b/docs/weaviate/manage-collections/inverted-index.mdx @@ -19,7 +19,7 @@ An **inverted index** is a data structure in Weaviate that enables efficient tex :::tip Change inverted indexes on a collection (v1.38+) -You can add, change, rebuild, or drop inverted indexes on a populated collection without restart. See [Reindex a property on a collection](#reindex-a-property-on-a-collection-v138). +You can add, change, rebuild, or drop inverted indexes on a populated collection without restart. See [Reindex a property on a collection](#reindex-a-property-on-a-collection). ::: @@ -164,7 +164,7 @@ The following index types can be dropped: `searchable`, `filterable`, `rangeFilt :::note v1.38+ -From `v1.38`, the drop is rejected with `409` while a [runtime reindex](#reindex-a-property-on-a-collection-v138) is in progress on the same property. Cancel the in-progress migration first, or wait for it to finish. +From `v1.38`, the drop is rejected with `409` while a [runtime reindex](#reindex-a-property-on-a-collection) is in progress on the same property. Cancel the in-progress migration first, or wait for it to finish. ::: @@ -271,7 +271,7 @@ For the full list of supported tokenizers — including `kagome_ja`, `kagome_kr` -## Reindex a property on a collection (v1.38+) +## Reindex a property on a collection @@ -279,7 +279,7 @@ From `v1.38`, you can change a property's inverted-index configuration on a **po These operations cover **inverted indexes only** (filterable, searchable, and range indexes, the BM25 algorithm, and tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). -The feature is **REST-only** for the v1.38 Preview — there is no client-library support yet, so the examples below use `curl`. All examples assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections` (required to submit a reindex) and `READ` on `CollectionsMetadata` (required to read the migration status). For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). +The feature is **REST-only** for the v1.38 Preview — there is no client-library support yet, so the examples below use `curl`. All examples assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections` (required to submit a reindex) and `READ` on `CollectionsMetadata` (required to read the migration status). For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex). ### Add an inverted index to a collection @@ -292,7 +292,7 @@ curl -X PUT \ "$WEAVIATE/v1/schema/Article/indexes/category" ``` -The body shape selects the index type — `filterable`, `searchable` (requires `text` / `text[]`), or `rangeable` (numeric types only). For the full body-shape reference, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). +The body shape selects the index type — `filterable`, `searchable` (requires `text` / `text[]`), or `rangeable` (numeric types only). For the full body-shape reference, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex). The server responds `202 Accepted` with a task ID, which you can treat as an opaque identifier: @@ -391,7 +391,7 @@ curl -fsS -H "Authorization: Bearer $TOKEN" \ } ``` -Polling this endpoint is cheap, so you can poll in a loop while a migration runs. For the meaning of each `status` value, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview). To poll until an index is `ready`: +Polling this endpoint is cheap, so you can poll in a loop while a migration runs. For the meaning of each `status` value, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex). To poll until an index is `ready`: ```bash until curl -fsS -H "Authorization: Bearer $TOKEN" "$WEAVIATE/v1/schema/Article/indexes" \ @@ -401,13 +401,13 @@ until curl -fsS -H "Authorization: Bearer $TOKEN" "$WEAVIATE/v1/schema/Article/i do sleep 2; done ``` -To scope a migration to specific tenants on a multi-tenant class, add the `?tenants=` query parameter — see the [multi-tenancy rules](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview) in the reference. +To scope a migration to specific tenants on a multi-tenant class, add the `?tenants=` query parameter — see the [multi-tenancy rules](../config-refs/indexing/inverted-index.mdx#runtime-reindex) in the reference. ## Further resources - [References: Collection definition](/weaviate/config-refs/collections.mdx) - [References: Inverted index](../config-refs/indexing/inverted-index.mdx) -- [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex-v138-preview) +- [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex) - [Concepts: Inverted index](../concepts/indexing/inverted-index.md) ## Questions and feedback From ea1a5e773b134427ede443c03bdb47a9df4d7a1b Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Mon, 1 Jun 2026 14:08:41 +0200 Subject: [PATCH 7/8] Trim request-body shapes from runtime-reindex reference Request/response body detail belongs in the REST API reference, not the config reference page. Replace the body-shapes table with a pointer to the REST API docs and the how-to guide. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../config-refs/indexing/inverted-index.mdx | 14 +------------- .../weaviate/manage-collections/inverted-index.mdx | 6 ++---- 2 files changed, 3 insertions(+), 17 deletions(-) diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index 8022daed..0c5ff753 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -277,19 +277,7 @@ From `v1.38`, three REST endpoints let you alter a property's inverted-index con | `DELETE` | `/v1/schema/{class}/properties/{property}/index/{indexName}` | Drop a configured index. `indexName` ∈ `{filterable, searchable, rangeFilters}`. | | `GET` | `/v1/schema/{class}/indexes` | Read per-property index status. | -### Request body shapes (`PUT`) - -| Body | Effect | -|---|---| -| `{"filterable":{"enabled":true}}` | Enables filtering on the property. | -| `{"searchable":{"enabled":true,"tokenization":"word"}}` | Enables keyword search on the property and sets its tokenization. Requires `text` / `text[]`. | -| `{"rangeable":{"enabled":true}}` | Enables range filtering on the property. Numeric types only (`int`, `number`, `date`). | -| `{"searchable":{"tokenization":"trigram"}}` | Retokenizes a populated `text` property. If the property also has a filterable index, both are retokenized together. | -| `{"searchable":{"algorithm":"blockmax"}}` | Migrates every searchable property on the collection from WAND to BlockMax. One-way; switching back to `wand` is rejected. | -| `{"":{"rebuild":true}}` | Rebuilds the named index (`filterable` / `searchable` / `rangeable`) from the stored objects. `searchable.rebuild` requires the property to already use BlockMax. | -| `{"":{"cancel":true}}` | Cancels an in-progress migration on the property. Idempotent — returns `NO_OP` when nothing matches. | - -A successful submit returns `202 Accepted` with `{"status":"STARTED","taskId":""}`. Treat `taskId` as an opaque identifier. +For the `PUT` request body shapes and response formats, see the REST API reference. For worked examples of each migration type, see the [how-to guide](../../manage-collections/inverted-index.mdx#reindex-a-property-on-a-collection). ### Status values (`GET`) diff --git a/docs/weaviate/manage-collections/inverted-index.mdx b/docs/weaviate/manage-collections/inverted-index.mdx index 87872154..7c3fdd5a 100644 --- a/docs/weaviate/manage-collections/inverted-index.mdx +++ b/docs/weaviate/manage-collections/inverted-index.mdx @@ -277,9 +277,7 @@ For the full list of supported tokenizers — including `kagome_ja`, `kagome_kr` From `v1.38`, you can change a property's inverted-index configuration on a **populated collection without restarting and without losing writes**. Reads stay available throughout the migration. This replaces the previous workaround of exporting the data, recreating the collection, and re-importing. -These operations cover **inverted indexes only** (filterable, searchable, and range indexes, the BM25 algorithm, and tokenization). Vector indexes are configured separately — see [Vector configuration](./vector-config.mdx). - -The feature is **REST-only** for the v1.38 Preview — there is no client-library support yet, so the examples below use `curl`. All examples assume `$WEAVIATE` is the cluster URL and `$TOKEN` is an API key with `UPDATE` on `Collections` (required to submit a reindex) and `READ` on `CollectionsMetadata` (required to read the migration status). For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex). +These operations cover **inverted indexes only** (filterable, searchable, and range indexes, the BM25 algorithm, and tokenization). For the endpoint list, request-body reference, status values, concurrency rules, error codes, and required permissions, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex). ### Add an inverted index to a collection @@ -292,7 +290,7 @@ curl -X PUT \ "$WEAVIATE/v1/schema/Article/indexes/category" ``` -The body shape selects the index type — `filterable`, `searchable` (requires `text` / `text[]`), or `rangeable` (numeric types only). For the full body-shape reference, see [References: Runtime reindex](../config-refs/indexing/inverted-index.mdx#runtime-reindex). +The body shape selects the index type — `filterable`, `searchable` (requires `text` / `text[]`), or `rangeable` (numeric types only). For the full request and response formats, see the REST API reference. The server responds `202 Accepted` with a task ID, which you can treat as an opaque identifier: From 3c3b073c30bf5bd615d4596256b3ea4e8f9f98a7 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Mon, 1 Jun 2026 14:23:51 +0200 Subject: [PATCH 8/8] Convert multi-tenancy reindex table to prose Replace the format-only vs semantic rules table with a bulleted explanation that's easier to read. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../config-refs/indexing/inverted-index.mdx | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/docs/weaviate/config-refs/indexing/inverted-index.mdx b/docs/weaviate/config-refs/indexing/inverted-index.mdx index 0c5ff753..20474c6b 100644 --- a/docs/weaviate/config-refs/indexing/inverted-index.mdx +++ b/docs/weaviate/config-refs/indexing/inverted-index.mdx @@ -303,16 +303,15 @@ While a reindex is in progress on a property, Weaviate rejects updates to that p ### Multi-tenancy -Scope a task to specific tenants on a multi-tenant class with the `?tenants=` query parameter (comma-separated). The rules: - -| Class | `?tenants=` | Migration type | Result | -|---|---|---|---| -| Single-tenant | provided | any | `400` | -| Multi-tenant | omitted | format-only (`rebuild`, `repair`, `enable-rangeable`) | targets all tenants | -| Multi-tenant | omitted | semantic (`change-tokenization*`, `enable-filterable`, `enable-searchable`, `change-algorithm` (BM25 WAND → BlockMax)) | targets all tenants (the change is collection-wide) | -| Multi-tenant | provided | format-only | targets the named subset | -| Multi-tenant | provided | semantic | `400` — semantic migrations cannot be sub-scoped | -| any | tenant in `OFFLOADED` / `FROZEN` | any | `400` — the offending tenant is named in the error | +On a multi-tenant collection, use the `?tenants=` query parameter (comma-separated) to scope a migration to specific tenants. Migrations fall into two groups, which behave differently: + +- **Format-only migrations** — rebuilding or repairing an index, and enabling a range index — can be scoped to a subset of tenants with `?tenants=`. If you omit the parameter, they apply to all tenants. +- **Semantic migrations** — enabling a filterable or searchable index, changing tokenization, and migrating BM25 from WAND to BlockMax — always apply to every tenant, because they change the collection definition. Supplying `?tenants=` for a semantic migration returns `400`. + +Other rules: + +- Supplying `?tenants=` on a single-tenant collection returns `400`. +- A migration that targets a tenant in the `OFFLOADED` or `FROZEN` state returns `400`, with the offending tenant named in the error. Each tenant is reindexed independently: a tenant starts serving its new index as soon as its own reindex finishes, even if other tenants are still in progress.