Skip to content

SDD completeness: 002–007 artifacts + 009–011 fleet protocols #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
derekclair merged 1 commit into from
Aug 20, 2026
Merged

SDD completeness: 002–007 artifacts + 009–011 fleet protocols #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions specs/002-multi-user-support/plan.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Plan: Multi-user support (002)

**Feature**: 002-multi-user-support
**Spec**: [spec.md](./spec.md)
**Date**: 2025-05-21 (design); recorded 2026-08-19

Honest status: **designed**, not a shipped tenant model. The tree has an identity
seam. It does not identify speakers.

## 1. Architecture

Identity is a string. Memory isolation is that string used as a Supermemory
`container_tag`. Session isolation is `thread_id` (LangGraph checkpointer is
spec 004 — not required to call this design done).

```
household user (voice or text)
│
▼
identify → user_id v1: explicit only
│ (--user, /user <id>, "this is <user_id>")
▼
get_agent(user_id) ──► create_memory_tools(user_id)
│ profile / add / search
│ container_tag = user_id
▼
AgentState.user_id + thread_id
│
├── long-term: Supermemory container_tag
└── short-term: LangGraph thread (004, not in tree)
```

| Piece | Owner |
|-------|--------|
| `DEFAULT_USER_ID`, `get_agent(user_id)`, MemoryChat `/user` | **this repo** (`thelab_langchain`) |
| Supermemory `container_tag` scoping | `create_memory_tools` / `MemoryChat` |
| Speaker diarization / voice embeddings | **out of scope for v1** |
| Checkpointer per `thread_id` | spec 004 |

Live voice I/O (sibling package) must pass the bound `user_id` into `get_agent()`.
It must not invent a second identity model.

## 2. Tech choices (locked for v1)

| Concern | Choice | Why |
|---------|--------|-----|
| Identity | Opaque `user_id` string | No household roster in code or SDD |
| Long-term isolation | Supermemory `container_tag = user_id` | Already the memory API’s tenant key |
| Session isolation | `thread_id`, later `{user_id}::{thread_id}` | Prevents short-term mix when 004 lands |
| Who is talking (v1) | Explicit identification first | `/user`, `--user`, spoken declaration |
| Who is talking (not v1) | Speaker diarization | Out of scope; do not block v1 on it |
| Default session | `DEFAULT_USER_ID` | Single-user path stays one flag |
| Shared facts | Optional household `container_tag` | Only if explicitly stored as shared |
| Brain factory | Existing `get_agent(user_id)` | Do not fork the graph per person |

## 3. Phases

### Phase 0 — Identity seam (this repo; in the tree)

- `DEFAULT_USER_ID` from settings.
- `get_agent(user_id)` / `build_agent_graph(user_id)` bind tools to that id.
- `MemoryChat(user_id)` uses `container_tag=self.user_id`.
- CLI `--user` and `/user <id>` rebuild the chat for a different container.
- `AgentState` already has `user_id` and `thread_id` fields.

This is a **container switch**, not speaker ID.

### Phase 1 — Explicit identification (not shipped)

- Bind a session to a `user_id` at start, or parse an explicit declaration.
- If unbound / unknown, ask which `user_id` to use; do not guess.
- Voice I/O passes the bound id into `get_agent(user_id)` on every turn.
- Known ids come from config, not from a coded household list.

### Phase 2 — Isolation completeness (design; not a tenant product)

- Namespace threads `{user_id}::{thread_id}` once spec 004 has a checkpointer.
- Tests: container A must not recall container B.
- Optional shared household `container_tag` with an explicit write path.
- Still no diarization.

Phase 2 is finishing **this** isolation design. It is not multi-tenant SaaS.

## 4. Risks

| Risk | Mitigation |
|------|------------|
| Calling this “multi-tenant” because `/user` exists | Spec and tasks state: seam only, not speaker ID |
| Cross-container recall via a shared client | Always pass `container_tag=user_id`; never a global search |
| Default `user_id` silently used for the wrong person | Unknown speaker → ask; do not fall back without saying so |
| Inventing a household roster in docs or config | Opaque `user_id` / `container_tag` only |
| Checkpointer mixing threads across users | Spec 004 namespacing; this plan does not fake persistence |
| Diarization scope creep | Keep v1 explicit-only; diarization stays a non-goal |

## 5. Success metrics

- Two `user_id` values, two containers: facts stored under A never appear in B’s profile/search.
- `/user <id>` (or `--user`) changes the container for subsequent turns.
- Unbound session asks for a `user_id` instead of guessing.
- Adding a user is a new id in config, not a code change.
- No speaker-ID model required for the above to be true.

## 6. What this plan is not

It is not a shipped tenant model. It is not speaker identification. It is not
spec 004 (checkpointers). It does not define a household of named people. It
does not replace spec 001.
136 changes: 80 additions & 56 deletions specs/002-multi-user-support/spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,98 +1,122 @@
# Feature Spec: Multi-User Support for the Voice Agent

**Feature ID**: 002-multi-user-support
**Status**: Draft / Future
**Related to**: [001-voice-dgx-spark-agent](../001-voice-dgx-spark-agent/spec.md)
**Feature ID**: 002-multi-user-support
**Status**: Designed; not a shipped tenant model
**Related to**: [001-voice-dgx-spark-agent](../001-voice-dgx-spark-agent/spec.md)
**Created**: 2025-05-21
**Recorded here**: 2026-08-19
**Owner**: Derek Clair

## Current state (honest)

This spec is the **design** for per-user isolation. It is not a product multi-tenant system.

Code in this repo today is an identity **seam**, not speaker ID:

- `DEFAULT_USER_ID` in settings
- `user_id` on `get_agent()` / `build_agent_graph()` and `AgentState`
- Supermemory calls scoped with `container_tag=user_id`
- MemoryChat CLI `--user` and `/user <id>` (rebuilds the chat for that container)

There is no speaker diarization, no voice fingerprint, and no automatic “who is talking” path. A `/user` switch is an explicit container change.

## Overview

The voice agent should gracefully support multiple users within the same household (initially: Derek, wife, two daughters, son, and occasional "other" guests or family members).
The voice agent should support multiple **household users** on the same deployment.

Each person should have their own persistent identity and long-term memory context. When someone speaks to the agent, it should correctly identify who they are (or be told) and recall the right history, preferences, ongoing projects, and relationships.
Each user is an opaque `user_id`. Long-term memory is isolated by a Supermemory `container_tag` (the same string as `user_id`). Short-term conversation state is isolated by `thread_id`. When someone speaks, the session must already be bound to a `user_id`, or the speaker must **declare** it.

This is a **cross-cutting concern** that affects user identification, Supermemory container isolation, session/thread management, the LangGraph state, and the overall voice experience.
This is a **cross-cutting concern**: identification, Supermemory container isolation, session/thread management, LangGraph state, and the voice loop.

## Goals

- Natural multi-user experience in a family setting.
- Strong long-term memory isolation per person (via Supermemory `container_tag`).
- Reasonable accuracy in knowing "who is talking" without constant re-identification.
- Future-proof for adding more family members or occasional guests.
- Maintain privacy boundaries between users.
- Natural multi-user experience for household users.
- Strong long-term memory isolation per `user_id` (via Supermemory `container_tag`).
- Low-friction identification: explicit declaration first; do not require a login ritual every turn once the session is bound.
- Adding another `user_id` is configuration, not a rewrite.
- Privacy boundaries between users: no cross-container recall.

## Non-Goals (for initial version)
## Non-Goals (for v1)

- Full biometric voice fingerprinting / speaker diarization (nice to have later).
- Remote multi-user access from outside the home.
- Complex household roles/permissions system.
- Guest accounts with temporary memory.
- Speaker diarization / biometric voice fingerprinting (out of scope for v1; possible later).
- Remote multi-user access from outside the deployment.
- Roles, permissions, or an admin/RBAC model.
- Guest accounts with temporary memory (open question, not v1).
- Inferring household relationships from names or stories.

## User Stories

1. **As Derek**, I want the agent to remember my ongoing projects, preferences, and conversations even when other family members have spoken to it recently.
2. **As my wife**, I want the agent to remember things that are important to me (kids' schedules, our shared tasks, etc.) without mixing them up with Derek's work stuff.
3. **As a kid**, I want the agent to know who I am when I talk to it and remember things like my homework, favorite games, or ongoing stories.
4. **As a parent**, I want to be able to say "Hey Lab, this is Sarah talking" or have the system figure it out reasonably well.
5. **As the household**, we want the agent to understand family relationships ("my sister", "Dad", "the kids") when context is relevant.
1. **As a household user**, I want memories, preferences, and ongoing work scoped to my `user_id` even if another household user spoke to the agent recently.
2. **As a household user**, I want my Supermemory `container_tag` isolated so another user’s facts are not injected into my turns.
3. **As a household user**, I want to identify myself explicitly (or start a session already bound to my `user_id`) so the agent uses the right container.
4. **As the operator**, I want adding or switching a `user_id` to be a low-effort config / command, not a new deployment.
5. **As a household user**, if the agent does not know which `user_id` is speaking, I want it to ask rather than guess.

## Functional Requirements

### FR-1: User Identity & Routing
- The system must be able to associate a voice interaction with a specific user identity.
- Supported identification methods (in rough priority order):
1. Explicit declaration ("Hey Lab, it's Derek")
2. Wake-word + name patterns
3. Heuristic / voice characteristics (future)
4. Device or room context (if multiple microphones are added later)

- Every voice or text interaction must be associated with a specific `user_id`.
- Identification methods for v1, in priority order:
1. Explicit declaration (e.g. “this is `<user_id>`”) or session start with `--user` / `/user <id>`
2. Wake-word + declared-name patterns (same explicit idea; not voice biometrics)
3. Heuristic / voice characteristics — **out of scope for v1**
4. Device or room context — later, if multiple capture devices exist
- Speaker diarization is **out of scope for v1**.

### FR-2: Memory Isolation (Supermemory)
- Every user must have their own `container_tag` in Supermemory.
- All `profile()`, `add()`, and `search` calls must be correctly scoped to the identified user.
- Cross-user leakage must be prevented (the agent should not accidentally recall one person's private facts to another).

- Every `user_id` has its own `container_tag`.
- All `profile()`, `add()`, and `search` calls must be scoped to the identified `user_id`.
- Cross-user leakage must be prevented (no accidental recall of one container’s facts into another).

### FR-3: Session & Thread Management
- Each user should have their own conversation threads (`thread_id`).
- Short-term memory (LangGraph checkpointer) must be isolated per user.
- It should be possible to have parallel conversations with different family members.

### FR-4: Relationship & Household Context
- The agent should be able to reason about family relationships when given the right context ("Tell my wife...", "What does Dad usually say about this?").
- There may be a lightweight "household" or "family" memory layer in addition to individual profiles.
- Each user has their own conversation threads (`thread_id`).
- Short-term memory (LangGraph checkpointer, when spec 004 lands) must be isolated per user.
- Parallel conversations for different household users must not share thread state.

### FR-4: Shared household container (optional)

- Individual profiles stay per `user_id` / `container_tag`.
- There may be a lightweight **shared** household `container_tag` for facts that are explicitly stored as shared — not a substitute for per-user isolation.
- The agent must not invent a household roster or infer private relationships.

### FR-5: Unknown speakers

### FR-5: Graceful Handling of Unknown Speakers
- If the agent cannot confidently identify the speaker, it should ask for clarification in a friendly way ("Sorry, I didn't catch who I'm speaking with — is this Derek, Sarah, or one of the kids?").
- If the agent cannot bind a turn to a `user_id`, it asks for clarification (e.g. “Which `user_id` should I use for this session?”).
- It must not guess a container.

## Non-Functional Requirements

- **Privacy**: One family member's private memories or conversations must never leak to another.
- **Low Friction**: Identification should feel natural, not like logging into a system every time.
- **Scalability**: Design should support adding more users without major rewrites.
- **Auditability** (future): It should be possible to see which user a memory belongs to.
- **Privacy**: One user’s private memories or conversations must never leak to another `user_id`.
- **Low friction**: Identification should feel like a one-time bind for the session, not a login on every turn.
- **Scalability**: Design supports additional `user_id` values without major rewrites.
- **Auditability** (future): It should be possible to see which `user_id` a memory belongs to.

## Open Questions

- How do we initially bootstrap user identities? (Manual config file? First-time "register yourself" flow?)
- Should there be a concept of a "primary user" (Derek) who has elevated capabilities?
- Do we want speaker diarization / voice embedding models running locally on the DGX for passive identification?
- How do we handle "other" / guests? Temporary containers? A generic "guest" profile?
- Should the agent proactively learn voices over time ("You sound like Maya today")?
- How do we bootstrap known `user_id` values? (Config file vs first-time “register this id” flow.)
- Is there a default `user_id` (`DEFAULT_USER_ID`) for single-user sessions, or must every session declare one?
- Do we want speaker diarization / voice embeddings later (local, on-box)? Not v1.
- How do we handle unknown / guest speakers? A generic `guest` container vs refuse until identified.
- Optional shared household `container_tag`: what is allowed to be written there, and who can read it?

## Relationship to Feature 001

This feature is a natural evolution of the single-user voice agent defined in 001.
This feature extends the single-user voice agent in 001.

The core architecture decisions made in 001 (Supermemory `container_tag` per user, `thread_id` per session, LangGraph state) were intentionally designed to be multi-user friendly. This spec captures the additional work needed to make the experience truly multi-user in a family context.
001 already assumed `container_tag` per user and `thread_id` per session. This spec is the extra work to make that a real multi-user experience: explicit identification, isolation guarantees, and unknown-speaker handling. It does not replace 001 and does not implement spec 004 (checkpointers).

## Success Criteria (for when we eventually implement)
## Success Criteria (when implemented)

- Derek, his wife, and both kids can have natural, separate ongoing conversations with the agent over weeks/months with correct memory recall.
- The agent rarely confuses one person's context with another's.
- Adding a new family member is a low-effort configuration task.
- The experience feels personal and "knows" each person without feeling creepy or overly technical.
- Distinct household users can keep separate ongoing conversations with correct memory recall.
- The agent does not mix one `user_id`’s context into another’s.
- Adding a new `user_id` is a low-effort configuration task.
- Identification is explicit first; no biometric path is required for v1.

---

**Status**: This spec is captured for future planning. It is **not** in scope for the current implementation wave.
**Status**: Design captured for planning. **Not** a shipped tenant model. The identity seam (`DEFAULT_USER_ID`, `get_agent(user_id)`, MemoryChat `/user`) is in the tree; speaker ID and product isolation are not.

Next time we pick up multi-user work, we should create a `plan.md` and `tasks.md` under this directory following the established SDD process.
See [plan.md](./plan.md) and [tasks.md](./tasks.md) for the SDD record of what exists vs what remains.
45 changes: 45 additions & 0 deletions specs/002-multi-user-support/tasks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Tasks: Multi-user support (002)

**Feature**: 002-multi-user-support
**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md)

Checkboxes record what is in this tree today versus what remains design-only.
This file was filled in when the SDD record was completed, not when the
identity seam was first written. This is **not** a shipped tenant model.

## Phase 0 — Identity seam (this repo)

- [x] `DEFAULT_USER_ID` in settings (`thelab_langchain.config`)
- [x] `user_id` on `get_agent()` / `build_agent_graph()`
- [x] `AgentState.user_id` and `AgentState.thread_id` fields
- [x] `create_memory_tools(user_id)` scopes `profile` / `add` / `search` with `container_tag=user_id`
- [x] `MemoryChat(user_id)` uses the same `container_tag`
- [x] CLI `--user` and `/user <id>` (rebuilds MemoryChat for that container)
- [x] Voice orchestrator accepts `user_id` and passes it into `get_agent()`
- [ ] LangGraph checkpointer per thread (spec 004 — not required to call 002’s seam done)
- [ ] Speaker identification (not in tree; `/user` is not speaker ID)

## Phase 1 — Explicit identification (not shipped)

- [ ] Bind a session to a `user_id` at start (flag, config, or spoken declaration)
- [ ] Parse explicit “this is `<user_id>`” (or equivalent) and switch the bound id
- [ ] If the turn cannot be bound, ask which `user_id` to use; do not guess
- [ ] Known ids from config only — no coded household roster
- [ ] Voice I/O (sibling package) must pass the bound `user_id` into `get_agent()` every turn
- [ ] Document the bind/switch commands next to `/user` without treating them as speaker ID

## Phase 2 — Isolation completeness (design)

- [ ] Thread namespacing `{user_id}::{thread_id}` when spec 004 has a checkpointer
- [ ] Test: memories stored under `user_a` are not returned for `user_b`
- [ ] Test: `/user` (or equivalent) actually changes `container_tag` for the next turn
- [ ] Optional shared household `container_tag` with an explicit write path
- [ ] Guest / unknown policy (refuse vs generic `guest` container) — decide, then implement
- [ ] Speaker diarization / voice embeddings — out of scope for v1

## Traceability

Phase 0 lives in this package (`thelab_langchain.config`, `agent.graph.get_agent`,
`agent.tools.memory.create_memory_tools`, `chat.MemoryChat`, `cli` `/user`).
Phases 1–2 are not done. This tasks file is only the checklist view of the
design plus the identity seam that already exists.
Loading
Loading