Give your coding agents a memory they can actually search.
Every Codex, Claude, Cursor, Muse, OpenCode, DeepSeek Harness, ZCode and
Antigravity session lands in one DuckDB file — conversations only, no tool
noise — and becomes a search index your agent can query from the terminal.
Open a pull request for your favorite agent if it is not here.
What it does · Keeping sessions private · Install · Usage · How search works · Tray · Agent skill · Contributing
Your agents forget everything between sessions, and the raw logs they forget with are full of tool calls, environment dumps and half-finished reasoning. duck-mem turns those logs into something searchable:
- One store, eight agents. Codex, Claude, Cursor, Muse, OpenCode, DeepSeek Harness, ZCode and Antigravity sessions are parsed into a single DuckDB file. Nothing is copied out of the original agent directories, and the stores are read-only.
- Conversations, not transcripts. Tool calls, tool output, reasoning blocks and metadata are dropped. You keep the prompts, the answers and the text that actually explained the work.
- Search that returns the conversation.
queryprints whole sessions — the prompt that opened them, the matching messages, the project — not a wall of message IDs for the agent to go and read. - A topic graph, not just keywords. A
duckpgqproperty graph over the vocabulary finds what a project talks about together, so a query phrased differently still reaches the right session. - It keeps itself current. A systemd user service re-indexes every few minutes and skips unchanged files; a tray icon shows status and runs a full re-index with one click.
- Privacy at the prompt. Start a session with
private,noingestornoduckand duck-mem keeps that session out of memory entirely. - No LLM anywhere. Ingestion, indexing and search are deterministic. No message text ever leaves your machine, and no API key is needed.
Open a session by starting the first prompt with one of these tags:
| tag | example first prompt |
|---|---|
private |
private the payroll merge details |
noingest |
noingest: demo of the client login flow |
noduck |
noduck read my passwords |
On the next ingest cycle a tagged session is never stored: if an older
parser already put it in memory, duck-mem removes its messages, its
topic-graph contribution and its copy-detection fingerprint. The session
is then permanently marked in the no_ingest table — the mark outlives
the source log, so the session stays out of memory even if you edit the
tag away afterwards.
The match is on the first word of the first user message,
case-insensitively and ignoring surrounding punctuation, so Private:
blocks and privateer the ship does not. Marked files are checkpointed
and never re-parsed.
Requires Linux with Go 1.24+ and systemd --user (any GNOME, KDE, COSMIC or
Hyprland session). Extensions are installed on first run.
git clone https://github.com/jmarceno/duck-mem.git
cd duck-mem
go build -o duck-mem ./cmd/duck-mem
./duck-mem --install--install copies the binary to ~/.local/bin, adds that directory to your
PATH if needed, and enables and starts duck-mem.service (the indexer) and
duck-mem-tray.service (the tray icon). Open a new login shell to use
duck-mem by name.
To remove it again:
duck-mem --uninstall # asks whether to keep the database
duck-mem --uninstall --keep-data # noninteractive, keeps the database
duck-mem --uninstall --purge-data # also deletes duck-mem data and configUninstalling never touches your Codex, Claude, Cursor, Muse, OpenCode, DeepSeek Harness, ZCode or Antigravity session stores.
duck-mem ingest [--db PATH] [ROOT...] index new messages from session logs
duck-mem query [--db PATH] [--project P] [--source S] [--limit N] [--hits N] <text...>
best-matching sessions, with their hits
duck-mem show [--db PATH] [--context N] [--from A] [--to B] [--full] [--system] <session>[#seq]
read the messages around a hit
duck-mem related [--db PATH] [--project P] [--depth 1|2] [--limit N] <term>
what a project discusses together with <term>
duck-mem index [--db PATH] [--min-df N] [--full]
incremental (or full) topic-graph index
duck-mem daemon [--db PATH] [--interval 5m] [ROOT...]
the ingest loop the service runs
duck-mem repack [--db PATH] [--out PATH]
rebuild the file and reclaim unused space;
without --out it swaps in place
duck-mem relocate [--db PATH] <new-path>
copy the database to a new location, verify
the copy with a smoke run, then optionally move
the old file to the trash
duck-mem projects [--db PATH]
list stored projects, session counts, aliases
and captured git identity
Flags may come before or after the text. The database defaults to
~/.local/share/duck-mem/memory.duckdb; because it is a plain DuckDB file you
can also open it with the DuckDB CLI or any DuckDB client.
duck-mem relocate <new-path> chooses where the database lives — every
command and the installed services follow the recorded location, no --db
needed. On a machine with no database yet it only records the location; the
file is created there on the first ingest. When a database already exists it
is copied to the new path, the copy is verified table by table and opened for
a smoke run, and only then are you asked whether the old file should go to
the trash — never a permanent deletion, and n (the default) keeps it where
it was. If the copy or the smoke run fails, the old database stays exactly
where it was. The indexer is stopped for the move and restarted on the new
location.
$ duck-mem query "flaky test on darwin arm64" --project duck-mem --limit 2
2 sessions for "flaky test on darwin arm64" in projects matching "duck-mem", best first. Messages are numbered #seq.
[1] 0199f1c4-2b7a-7c31-9f0e-5d2a8b41c7de claude · 2026-08-14 → 2026-08-14 · 412 msgs · /home/you/Projects/duck-mem
opened with: "the arm64 build times out on the test runner, can you look at it?"
#12 user 14:02:11: the arm64 runner times out on ingest_test.go:184, flaky?
#41 assistant 14:19:03: TestRetryExponentialBackoff failed 3 of 5 runs on darwin-arm64
#58 assistant 14:41:57: the timeout is the -race flag, not the backoff; fixed in helper_test.go:22
ended with:
#410 user 15:02:40: commit it
#411 assistant 15:03:12: committed as 4f2a9c1; CI is green on darwin-arm64
[2] 0199f2d8-77aa-70c2-9d13-6b2f0aa9e410 codex · 2026-08-15 · 96 msgs · /home/you/Projects/duck-mem
(no query words here; found through the topic-graph neighbours below)
opened with: "re-run the CI matrix and report what broke"
#7 assistant 09:11:20: darwin-arm64 job failed again: flaky, retrying
Topic graph: often discussed with these words: race (4), timeout (3), retry (2), matrix (1)
Add one to the query to widen it, or map the area: duck-mem related --project duck-mem runner
Read a hit with the messages around it: duck-mem show 0199f1c4-2b7a-7c31-9f0e-5d2a8b41c7de#41Each result is a session: what it was, when, which project, how it started, the messages that matched (placed where the query terms are closest together) and how it ended: the last prompt, the last reply and, when the last turn got no reply, a note with the reason (usage limit, interrupted) and the log path. Sessions that match only through the topic graph are marked as such and appear after the direct hits, so the graph widens a search without reordering it; the topic-graph line is printed only when it explains such a session or the query left slots empty. A session ID (or unique prefix) in the query puts that session first, which is how an agent picks up another agent's thread.
show is the second step — it prints the messages around a hit so you can read
the reasoning in context. Host context (system prompts, AGENTS.md dumps, IDE
state) is collapsed to one line; --system prints it:
$ duck-mem show 0199f1c4#41 --context 2
session 0199f1c4-2b7a-7c31-9f0e-5d2a8b41c7de · claude · /home/you/Projects/duck-mem
2026-08-14 · 412 messages (#0–#411) · log /home/you/.claude/projects/<project>/<session>.jsonl
showing #39–#43; earlier: --from 0 --to 38; later: --from 44 --to 64; whole messages: --fullUnique ID prefixes are enough, and --project/--source narrow either command.
Every session carries a project value: the directory (or short name) the
harness reported. Values are normalized at ingest — trailing separators,
duplicate separators, ~ and backslash paths are canonicalized — so one
checkout cannot become two projects. --project accepts more than the exact
stored string. Resolution tries, in order: the exact stored value or path, a
name the project was renamed from, the directory basename (an agent's usual
repo-directory name), and a shared git identity — worktrees and clones of one
remote are the same project. Only then does the old substring filter apply.
--project is advisory: resolving a name narrows a read; it never merges or
rewrites data. Only duck-mem rename changes stored values, and it records an
alias so later queries under the old name keep resolving.
When a project cannot be resolved, the output says so instead of searching everything and offers up to three similar stored names with their session count, sources and date range:
$ duck-mem query --project celestial flaky test
no project matches "celestial"
similar names: /home/you/Projects/celeste (36 sessions, claude+codex+cursor, 2026-10-06 → 2026-10-19)
Similar name only, not verified — may be a different project, or an older name for the same one.A filter that names more than one project says so and lists each with its
session count, so Generic and generic are never silently blended:
$ duck-mem query --project generic ledger
--project "generic" matches 2 projects: /home/you/Projects/generic (79 sessions), /home/you/Projects/Generic (60 sessions)Git identity is captured lazily while changed sessions are re-parsed; there is
no backfill migration in the code. A previous data repair was done by hand
once, before this existed, and the alias map keeps it from reverting.
duck-mem projects shows everything the store knows:
$ duck-mem projects
60 stored projects, 4 with a captured git identity
/home/you/Projects/organizer 925 sessions claude,codex,cursor …
git github.com/you/organizer · /home/you/Projects/organizer
aliases: /home/you/projects/organizer-tiptap, …
/home/you/Projects/celeste 36 sessions claude,codex,cursor,dsh,opencode …$ duck-mem related runner --depth 2
matrix (14, via timeout)
race (11, via flaky)
flaky (9)
backoff (replaces: retry -> delay, 3)related answers "what is discussed together with X" by traversing the topic
graph. Nodes are Porter stems, so enemy and enemies are the same node and
your query words find their node directly. Co-mention edges are undirected and
weighted by how many messages mention both terms; typed relations recovered
from possessive and replacement phrasing (bastion --owns--> turret) keep
their direction and quote the sentence they came from. --depth 2 follows
neighbours-of-neighbours, expanding only the strongest direct links.
Two rankings are fused with reciprocal rank fusion, and the topic graph takes part in every query.
- BM25 over a posting list of Porter-stemmed terms, which gives recall: word forms match, and a long message that mentions the terms still ranks. Postings are written in the same transaction as their messages.
- Vector similarity over hashed term and phrase vectors (384 dimensions, exact cosine scan in DuckDB). It rewards messages that reuse the query's adjacent word pairs, and it only reorders the BM25 candidates, so a hash collision between two unrelated short messages cannot promote one of them.
- The topic graph contributes one hop of co-mention neighbours to the query, at a fraction of the weight of a direct term. If your words match fewer sessions than you asked for, those neighbours are searched again to fill the empty slots, and the results say so.
Ingestion and indexing are incremental by default: per-file checkpoints skip logs that have not changed, and appended messages are read from the byte offset where the previous version left off. Rewritten or truncated sessions are reconciled in place instead of rebuilt. Because incremental threshold decisions use counts as of each run, a periodic full rebuild heals the small drift:
0 4 * * 0 duck-mem index --full # weekly rebuild; daily ingest stays incremental
0 5 1 * * duck-mem repack # monthly repack reclaims space from rewritesDuckDB allows one read-write process at a time. The daemon is that process:
it opens the database at startup and keeps it. query, show, related,
ingest, and index send their work to the daemon and wait in one queue, so
a sync and a search no longer fail each other with a file-lock error. Time in
the queue is not limited. Once a query, show, or related starts, it has
30 seconds to finish; ingest, index, and checkpoint are not capped. With the
daemon stopped, those commands open the database themselves. repack also
works while the daemon runs: it closes the database for the duration, rebuilds
the file, and reopens.
A failed transaction (Failed to commit, Conflict on update) leaves DuckDB's
handle unable to commit again. The owner reopens the file after such an error,
and pauses writes when it happens twice without a clean write in between, so a
damaged file stops cleanly with the real error instead of failing every cycle
with a misleading conflict. A successful repack, or a restart after repair,
resumes the writes.
Vector search deliberately uses no approximate index. The vss HNSW index that earlier versions persisted kept every checkpoint's index blocks, so the file grew by roughly 100 MB per changed cycle with almost no new data. An exact scan of ~60k embeddings costs ~40 ms warm and finds every true neighbor (the HNSW index returned about 60–75% of them). Opening an older database drops that index; its blocks become reusable but the file only shrinks after a repack.
duck-mem repack rebuilds the database file to reclaim space left by rewrites
and dropped data, and drops a legacy HNSW index if one is still present. It
verifies every table's row count and complete-row hash, preserves sequence
positions, and rebuilds indexes before the new file becomes visible. With no
flags it replaces the live file in place; the source stays untouched when the
repack fails. --out NEW_FILE writes a separate copy instead and leaves the
live file alone; the destination must not exist. Repack does not need the
services stopped, and the tray menu offers the same in-place repack.
| Source | Read from | Project from |
|---|---|---|
| Codex | ~/.codex/sessions/**/*.jsonl, ~/.codex/archived_sessions/*.jsonl |
session metadata cwd |
| Claude | ~/.claude/projects/*/*.jsonl |
the record's cwd |
| Cursor | ~/.cursor/projects/*/agent-transcripts/*/*.jsonl, ~/.cursor/plans/*.plan.md |
workspace storage workspace.json |
| Muse | ~/.local/share/muse/sessions/**/session.jsonl |
route facts / workspace root |
| OpenCode | ~/.local/share/opencode/opencode.db (read-only, including WAL) |
session_v2.directory, else the worktree |
| ZCode | ~/.zcode/cli/db/db.sqlite (read-only, including WAL) |
session.directory |
| DeepSeek Harness | ~/.dsh/sessions/*/*/session.v4.jsonl.zstd (zstd, read-only) |
session header cwd |
| Antigravity | ~/.gemini/antigravity/brain/*/.system_generated/logs/transcript_full.jsonl |
conversation_summaries.db workspace_uris |
Project values are normalized when they are parsed (filepath.Clean-style:
trailing and repeated separators, ~, backslash paths), compared
case-insensitively, and stored with the source's casing.
Kept: user prompts, assistant text, system and developer text. Dropped
everywhere: tool calls, tool outputs, reasoning and thinking blocks,
environment snapshots, queue and metadata records. A Cursor plan file is
imported whole as a single note, and OpenCode child sessions are imported
alongside their parents. ZCode subagent sessions are imported the same way:
their reports reach the main conversation only as tool output, which is
dropped, so the subagent rows are the only place that content is kept. Cursor
transcripts whose project cannot be resolved stay under cursor:<slug> rather
than being given an invented path.
DeepSeek Harness session files are zstd-compressed event logs, decompressed as
they are read. The one source of system text it does not contribute is its own
system prompt: the harness writes that, plus the workspace instructions,
runtime state and skill catalog it injects, as user-role records carrying a
source kind other than user, and only records the user actually sent are
indexed. An unknown session format version is reported as an unexpected format
instead of being guessed at.
Antigravity conversations are append-only JSONL step logs. Each prompt is
stored as the text inside the <USER_REQUEST> wrapper, without the metadata
blocks the app appends for the model (local time, model-selection notices), and
only user input the user actually sent and the model's visible replies are
kept: tool output records and reasoning are dropped. transcript.jsonl beside
each log is the same conversation with long content clipped, and chunks/
holds partitions of both, so only the untruncated log is read and one
conversation can never be imported twice. The model id carries its reasoning
level (gemini-3.8-flash-high), which is split into the model and effort
columns; suffixes that are part of the model name (-tiered, -n, -lite)
stay in the model and leave the effort Unknown. A conversation outside any
project keeps an empty project rather than an invented path.
Three Antigravity behaviours are documented as temporary limitations, kept until there is data to design against:
- Failed and aborted turns leave no note. Every step observed reported
DONE, so the shape a failure or a user cancellation writes is unknown. A turn that ended without a reply looks like it stopped, unlike Codex and Cursor, which record a "turn failed" or "turn aborted" note. - Subagent conversations are not linked to their parent. The summaries
index carries
parent_conversation_idandnesting_depth, but no subagent brain directory has been observed, so it is unknown whether a subagent gets one of its own. A subagent that does is ingested as an ordinary unlinked session. - A conversation stored only as chunks is skipped. The full log and its chunk partitions have existed side by side in every sample. If a long conversation ever ships chunks without the full log, nothing is ingested rather than half-importing it or duplicating the conversation.
Every session also records the model it ran on, the reasoning effort it ran
at, and its source dates: created and last-updated, both taken from the
harness's own data (message timestamps, model-selection records, SQLite update
columns), never from ingest time. A model or effort switch mid-session keeps
the final value. Where a harness reports neither field the session stores
Unknown instead of an empty value; Cursor transcripts and plan files carry
neither dates nor a model, so they take the OS file birth and modification
times as dates and stay Unknown otherwise. A re-ingest that re-reads
identical content refreshes the metadata but never moves the dates, and
neither does a project rename: only genuinely new data does.
Imported cross-harness copies are suppressed when their entire conversation exactly matches the beginning of another retained session in the same project. Matching removes known timestamp/query wrappers, host context, and import markers; it preserves wording, internal whitespace, order, and failure notes. It requires at least three messages, 500 bytes, and a substantive assistant reply. Uncertain matches and shorter conversations remain imported.
Suppression removes the copy's messages, embeddings, search postings, and graph
contributions. Its identity, source path, fingerprints, and file checkpoint
remain. Changed copies are fully reparsed: distinct content restores the full
conversation as a separate session, including its shared history. Rewriting
an owner first restores any copies it would no longer cover; unavailable copy
logs prevent that rewrite rather than losing history. show accepts suppressed
IDs and resolves them to the retained session. Existing imports are checked
automatically on the first sync after upgrading; no parser-version bump or
full re-index is needed.
sessions(session_id TEXT PRIMARY KEY, source, project, started_at, path,
model, effort, updated_at)
messages(session_id, seq, source, project, role, text, created_at,
indexed, embedding FLOAT[384], PRIMARY KEY(session_id, seq))
msg_terms(session_id, seq, term, tf) -- BM25 postings, Porter-stemmed
term_df(project, term, msgs) -- per-project document frequencies
topics(topic_id, project, term, label) -- graph nodes
topic_edges(src_id, dst_id, term_a, term_b, weight, kind,
from_term, to_term, evidence)
msg_edges(session_id, seq, term_a, term_b, ...) -- per-message edge log
file_checkpoints(path, size, mtime_ns, tail_hash, ends_line, session_id, …)
session_copies(session_id, hashes, eligible, imported, duplicate_of, seen_at)
project_aliases(old_name TEXT PRIMARY KEY, new_name) -- rename map
project_identities(project, repo_root, remote_url, common_dir, updated_at)project is a value, not a column: DuckDB columns are schema, so one column
per project would need DDL for every new project and break every query. It is
an indexed TEXT on both tables; reads resolve a name to exact stored values
(see Projects and identity) and filter with
WHERE project IN (...), with the old substring match only as a fallback.
duck-mem-tray.service registers a StatusNotifierItem (and DBusMenu) item using
the project logo. The menu shows the indexer's state, the last completed
sync, and how many sessions are registered, and can start or stop the indexer,
sync immediately instead of waiting for the next cycle, force a full
re-index, rename a project, or repack the database to reclaim space. A dot
appears over the icon while an action is running or when the indexer is down.
Sync history is retained as monthly JSONL files in
~/.local/state/duck-mem/logs/. Each cycle records the harness, time, session
and extracted message counts (including zero when unchanged), and success or
failure. Failures include Unexpected format, Database error, or Other
and available error details; cycle-wide errors use harness all. No session
text is stored in these logs. Logs are retained without automatic deletion.
Open database folder opens ~/.local/share/duck-mem/ the same way.
View logs opens this directory in the default file manager and clears the
indicator for existing failures. Failures include the affected file path and
show a red dot drawn directly into the tray icon. The dot also clears when a
later cycle finishes without an error and without skipping a file, because a
skipped file is not checkpointed and is still missing. A later failure sets
the dot again. Busy and stopped states use amber and grey dots respectively.
The tray refreshes every ten seconds.
The tray is optional — duck-mem ingest, query and the rest work without it.
A read-only agent skill is included at
.agents/skills/duck-mem-query. Point
an agent at it and it will search your history before guessing, so it can answer
"we did this last month, here's the session" instead of reinventing it.
- From
grepover the logs: search returns the session that matters, with the surrounding conversation, instead of matching lines you have to reassemble. - From vector-only tools: BM25 keeps recall for names, error text and word forms; the graph supplies the vocabulary a plain embedding misses.
- From hosted memory services: everything stays in one local file. No account, no API key, no text leaving the machine.
Issues and pull requests are welcome. See CONTRIBUTING.md for the build, test and commit conventions, and please include a fixture with any parser change — the tests are the fastest way to agree on what a session log should produce.
MIT © 2026 Jardel G Marceno