Skip to content

Rewrite the Lakebase Search docs hub as an architecture-first page - #6781

Closed
bgrenon wants to merge 3 commits into
neondatabase:mainfrom
bgrenon:docs/lakebase-search-hub
Closed

bgrenon wants to merge 3 commits into
neondatabase:mainfrom
bgrenon:docs/lakebase-search-hub

Conversation

@bgrenon

@bgrenon bgrenon commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

What this changes

Rewrites the Lakebase Search hub page (content/docs/ai/lakebase-search.md) into an architecture-first, developer-facing overview.

Previously the page opened with a list of search types. It now leads with how Lakebase Search works and why it scales, which better matches how developers and AI coding tools look for this information.

New structure:

  • Hero: leads with "search built for agents," keeps the agent setup prompt.
  • Scaling search past pgvector: the limits teams hit with pgvector and HNSW at scale (memory that grows with data rather than usage, slow and disruptive index builds, the recall-versus-latency tradeoff, and the sync tax of a separate vector database), with data freshness as a first-class point.
  • How Lakebase Search works: separated storage and compute, IVF clustering, RaBitQ compression, stateless scale-to-zero compute, native BM25 with top-K pushdown, and hybrid search as plain SQL, with a first-pass architecture diagram.
  • What you get: lower cost at scale, scale to 100 million vectors and beyond, serverless with scale-to-zero, hybrid search with BM25, and data that is searchable immediately.
  • Benchmarks: a short qualitative pointer to the Databricks engineering blog instead of raw figures.
  • Search types: demoted to a reference section that links to the extension pages.

pgvector stays first-class throughout. lakebase_vector is positioned as a pgvector-compatible superset (same types, operators, and SQL), not a replacement.

Notes for reviewers

  • The architecture diagram is a first-pass Mermaid diagram so the page is complete and dark-mode safe. A polished hero illustration can replace it later.
  • Benchmark figures are intentionally not printed on the page; it links to the Databricks engineering blog and attributes the results to Databricks.

Pending confirmation before this leaves draft

  • Compression ratio wording: the page describes RaBitQ compression qualitatively rather than citing a single factor, pending confirmation of the right number to state.
  • Benchmark figures: pending publication clearance before any specific throughput or recall numbers are added.

This pull request and its description were written by Isaac.

Rework the Lakebase Search landing page to lead with search built for
agents instead of a list of search types. Add the problem framing
(scaling vector search in Postgres past pgvector), an architecture
section with a diagram, a "what you get" section, a qualitative
benchmarks pointer to the Databricks engineering blog, and a demoted
search-types reference. Keep pgvector first-class: lakebase_vector is
positioned as a pgvector-compatible superset.

Co-authored-by: Isaac <no-reply@databricks.com>
@vercel

vercel Bot commented Nov 17, 2026

Copy link
Copy Markdown

@bgrenon is attempting to deploy a commit to the neondatabase Team on Vercel.

A member of the Team first needs to authorize it.

bgrenon and others added 2 commits November 17, 2026 14:12
Expand HNSW on first use, describe Jev as a reranking model on Neon AI
Gateway, and reword the what-you-get opener to the pgvector-compatible
path to scale rather than a pgvector alternative.

Co-authored-by: Isaac <no-reply@databricks.com>
Lead the opener with the developer payoff, write to "you", add natural
contractions, drop an idiom and a corporate attribution phrasing, and
make a couple of abstractions concrete. No facts, structure, or
guardrails changed.

Co-authored-by: Isaac <no-reply@databricks.com>
@bgrenon

bgrenon commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #6782, which is the same change as a normal same-repo branch (this one was accidentally opened from a fork, so Vercel preview and the SDK check could not run).

@bgrenon bgrenon closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant