[1.19] Document language-neutral BM25 text processing#2445
Open
abdonpijpelink wants to merge 7 commits into
Open
[1.19] Document language-neutral BM25 text processing#2445abdonpijpelink wants to merge 7 commits into
abdonpijpelink wants to merge 7 commits into
Conversation
✅ Deploy Preview for condescending-goldwasser-91acf0 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
dancixx
approved these changes
Jun 24, 2026
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the deprecated `language: "none"` workaround with the new explicit
API: `stemmer: {"type": "none"}` to disable stemming and `stopwords: {}`
(empty StopwordsSet) to disable stopword removal.
Updates the language-neutral snippet for all seven client languages and
refreshes the Stemming and Stopwords + Language-neutral sections in
text-search.md with a deprecation callout for the old approach.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…or payload indexes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
abdonpijpelink
force-pushed
the
bm25-lang-neutral-processing
branch
from
7a11511 to
4c1b8b8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preview
Replaces the deprecated BM25
language: "none"workaround with the new explicit API:stemmer: {"type": "none"}to disable stemming andstopwords: {}(empty stopwords set) to disable stopword removal.Also added some language to the text payload indexing docs to make it explicit that, contrary to BM25, it already defaults to no stemming or stopwords by default, to avoid confusion.