Remote storage for local agent memory: only the core engram survives a machine

[yjc801]

Filed on behalf of @yjc801 (JC) from #buzz-dev.

Problem

Only one slice of an agent's memory is remote. Everything else lives on one machine's disk and dies with it.

What is already remote

• NIP-AE engrams: KIND_AGENT_ENGRAM = 30174 (crates/buzz-core/src/kind.rs:94), encrypted with the agent↔owner NIP-44 conversation key, CRUD via buzz mem ls/get/hash/set/patch/rm (crates/buzz-cli/src/commands/mem.rs), with owner-side recovery through --agent.
• buzz-acp fetches the core engram at session creation and injects it as [Agent Memory — core] (crates/buzz-acp/src/engram_fetch.rs). Non-core slugs are relay-backed too, but read on demand only.

What is local-only

• The whole agent workspace: RESEARCH/, PLANS/, GUIDES/, WORK_LOGS/, OUTBOX/, .scratch/, REPOS/, AGENTS.md.
• The underlying harness's own memory files (for a Claude Code-backed agent, the per-project memory directory and its index).

None of that is replicated anywhere. Consequences we have actually hit:

• Environment resets erase it silently. Worktrees and notes referenced in an agent's own core memory have simply been gone on a later session, with no signal other than the paths failing to ls. The memory then contains confident claims verified against directories that no longer exist.
• Remote wake starts blind. A buzz-waker-woken agent deploys into a fresh container. It gets core from the relay and nothing else — no plans, no work logs, no prior research. The same agent identity behaves like it has amnesia depending on where it happens to be running.
• No owner-side visibility or backup of anything except engrams.
• Sessions on different machines cannot see each other's notes, even though they share an identity.

Proposed direction

Sync local memory files into relay-backed storage, restored on session start. Two candidate layers, likely both:

1. Text notes → engrams. A directory-to-slug sync (buzz mem sync <dir>, hash-compared so unchanged files are a no-op) reusing the existing encryption, tombstones, and head-selection. Cheap, and it inherits owner-side recovery for free.
2. Bulk/binary → Blossom blob + addressable index event. For anything past a sane inline size (repo checkouts explicitly out of scope; snapshot artifacts and large research maybe).

Restore must follow engram_fetch.rs's discipline: never block session creation, and never let a transport failure present as "empty" — that is the failure mode that invites an agent to overwrite real, momentarily unreachable memory.

Acceptance criteria

• A documented set of workspace paths that are synced, and an explicit exclude list (REPOS/, .scratch/, anything with credentials).
• Push path: content-hash comparison so a quiet session costs no writes.
• Restore path on session start / on fresh container, idempotent, non-blocking, failure-distinguishable from empty.
• Size caps enforced with a clear error rather than a relay rejection.
• Concurrent-session conflict behaviour defined (two sessions of the same agent on different machines both writing).
• Tests: round-trip, no-op-on-unchanged, restore-into-empty-container, relay-outage-is-not-empty.

Open questions for JC

• Is the target "an agent's workspace follows it anywhere it runs" (full sync), or narrower — just durable notes, with the machine-local scratch staying disposable? These need very different size budgets.
• Retention/history: keep versions (engram heads already give some of this) or last-write-wins?
• Does the owner get a way to browse/prune an agent's synced memory from Buzz Desktop, or is CLI-only fine for now?

[yjc801]

Related: #86 (skills), #88 (self-evolve — depends on this one).