Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

</div>

Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~99% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, matching the retrieval quality of a code-specialized transformer while indexing ~220x faster and querying ~17x faster (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Use it as an MCP server, a CLI tool via AGENTS.md, or a dedicated sub-agent, and any coding agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo.
Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~99% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, matching the retrieval quality of a code-specialized transformer while indexing ~340x faster and querying ~17x faster (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Use it as an MCP server, a CLI tool via AGENTS.md, or a dedicated sub-agent, and any coding agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo.

## Quickstart

Expand Down Expand Up @@ -244,7 +244,7 @@ We benchmark quality and speed across ~1,250 queries over 63 repositories in 19
</tr>
</table>

The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency; semble matches the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) while indexing 220x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 99% fewer tokens on average and hits 97% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology.
The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency; semble matches the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) while indexing 340x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 99% fewer tokens on average and hits 97% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology.

## How it works

Expand Down
Binary file modified assets/images/speed_vs_ndcg_cold.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Quality and speed across all methods.

| Method | NDCG@10 | Index | Query p50 |
| -------------------- | --------: | ---------: | ----------: |
| **semble** | **0.854** | **518 ms** | **0.91 ms** |
| **semble** | **0.854** | **344 ms** | **0.91 ms** |
| CodeRankEmbed | 0.839 | 116 s | 16 ms |
| ColGREP | 0.693 | 5.4 s | 122 ms |
| BM25 | 0.673 | 47 ms | 0.17 ms |
Expand All @@ -32,7 +32,7 @@ Quality and speed across all methods.
| :-----------------------------------------------------------------: | :-----------------------------------------------------------------: |
| _Time to first result (index + query) vs NDCG@10_ | _Query latency on a warm index vs NDCG@10_ |

semble matches the NDCG@10 of the 137M-param CodeRankEmbed while winning index time by ~220x and query latency by ~17x.
semble matches the NDCG@10 of the 137M-param CodeRankEmbed while winning index time by ~340x and query latency by ~17x.

NDCG@10 is averaged across all queries. Speed numbers use one repo per language, CPU only: cold-start index time and warm query p50 (median across 5 consecutive runs).

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/plot.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ class _Method(TypedDict):
{
"name": "semble",
"ndcg10": 0.8544,
"index_ms": 518.2,
"index_ms": 343.5,
"query_p50_ms": 0.91,
"color": "#1a5fa8",
"params_m": 16,
Expand Down
Loading
Loading