From 2b49964f63612100786088b25f73bc0516bdd39a Mon Sep 17 00:00:00 2001 From: Derek Clair Date: Thu, 20 Aug 2026 00:38:08 -0600 Subject: [PATCH 1/3] =?UTF-8?q?Complete=20SDD=20trail:=20002=E2=80=93007?= =?UTF-8?q?=20artifacts,=20009=E2=80=93011=20fleet=20protocols.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add plan.md and tasks.md for 002–006. Scrub 002 of household PII. Correct 005 current state (tests + CPU CI exist). Extend 007 with the live inference slot policy (one local LLM; 120B+Riva is experimental). Add 009 STE architect↔coder handoff, 010 complete-or-block worker protocol, and 011 voice reply contract (wanted, not a shipped filter). Hermes operating files stay in Hermes; this is the protocol record only. --- specs/002-multi-user-support/plan.md | 107 ++++++++++ specs/002-multi-user-support/spec.md | 136 +++++++------ specs/002-multi-user-support/tasks.md | 45 +++++ specs/003-deployment-infrastructure/plan.md | 113 +++++++++++ specs/003-deployment-infrastructure/tasks.md | 65 ++++++ specs/004-persistence-checkpointers/plan.md | 103 ++++++++++ specs/004-persistence-checkpointers/tasks.md | 47 +++++ specs/005-testing-and-cicd/plan.md | 127 ++++++++++++ specs/005-testing-and-cicd/spec.md | 22 +- specs/005-testing-and-cicd/tasks.md | 59 ++++++ specs/006-alternative-memory-systems/plan.md | 93 +++++++++ specs/006-alternative-memory-systems/tasks.md | 47 +++++ specs/007-dgx-hardware-optimization/plan.md | 91 ++++++--- specs/007-dgx-hardware-optimization/spec.md | 189 ++++++++++++----- specs/007-dgx-hardware-optimization/tasks.md | 90 ++++++--- specs/009-architect-coder-handoff/plan.md | 115 +++++++++++ specs/009-architect-coder-handoff/spec.md | 173 ++++++++++++++++ specs/009-architect-coder-handoff/tasks.md | 44 ++++ specs/010-worker-completion-protocol/plan.md | 105 ++++++++++ specs/010-worker-completion-protocol/spec.md | 162 +++++++++++++++ specs/010-worker-completion-protocol/tasks.md | 47 +++++ specs/011-voice-reply-contract/plan.md | 149 ++++++++++++++ specs/011-voice-reply-contract/spec.md | 190 ++++++++++++++++++ specs/011-voice-reply-contract/tasks.md | 73 +++++++ specs/README.md | 30 +-- 25 files changed, 2239 insertions(+), 183 deletions(-) create mode 100644 specs/002-multi-user-support/plan.md create mode 100644 specs/002-multi-user-support/tasks.md create mode 100644 specs/003-deployment-infrastructure/plan.md create mode 100644 specs/003-deployment-infrastructure/tasks.md create mode 100644 specs/004-persistence-checkpointers/plan.md create mode 100644 specs/004-persistence-checkpointers/tasks.md create mode 100644 specs/005-testing-and-cicd/plan.md create mode 100644 specs/005-testing-and-cicd/tasks.md create mode 100644 specs/006-alternative-memory-systems/plan.md create mode 100644 specs/006-alternative-memory-systems/tasks.md create mode 100644 specs/009-architect-coder-handoff/plan.md create mode 100644 specs/009-architect-coder-handoff/spec.md create mode 100644 specs/009-architect-coder-handoff/tasks.md create mode 100644 specs/010-worker-completion-protocol/plan.md create mode 100644 specs/010-worker-completion-protocol/spec.md create mode 100644 specs/010-worker-completion-protocol/tasks.md create mode 100644 specs/011-voice-reply-contract/plan.md create mode 100644 specs/011-voice-reply-contract/spec.md create mode 100644 specs/011-voice-reply-contract/tasks.md diff --git a/specs/002-multi-user-support/plan.md b/specs/002-multi-user-support/plan.md new file mode 100644 index 0000000..f22fa8e --- /dev/null +++ b/specs/002-multi-user-support/plan.md @@ -0,0 +1,107 @@ +# Plan: Multi-user support (002) + +**Feature**: 002-multi-user-support +**Spec**: [spec.md](./spec.md) +**Date**: 2025-05-21 (design); recorded 2026-08-19 + +Honest status: **designed**, not a shipped tenant model. The tree has an identity +seam. It does not identify speakers. + +## 1. Architecture + +Identity is a string. Memory isolation is that string used as a Supermemory +`container_tag`. Session isolation is `thread_id` (LangGraph checkpointer is +spec 004 — not required to call this design done). + +``` +household user (voice or text) + │ + ▼ + identify → user_id v1: explicit only + │ (--user, /user , "this is ") + ▼ + get_agent(user_id) ──► create_memory_tools(user_id) + │ profile / add / search + │ container_tag = user_id + ▼ + AgentState.user_id + thread_id + │ + ├── long-term: Supermemory container_tag + └── short-term: LangGraph thread (004, not in tree) +``` + +| Piece | Owner | +|-------|--------| +| `DEFAULT_USER_ID`, `get_agent(user_id)`, MemoryChat `/user` | **this repo** (`thelab_langchain`) | +| Supermemory `container_tag` scoping | `create_memory_tools` / `MemoryChat` | +| Speaker diarization / voice embeddings | **out of scope for v1** | +| Checkpointer per `thread_id` | spec 004 | + +Live voice I/O (sibling package) must pass the bound `user_id` into `get_agent()`. +It must not invent a second identity model. + +## 2. Tech choices (locked for v1) + +| Concern | Choice | Why | +|---------|--------|-----| +| Identity | Opaque `user_id` string | No household roster in code or SDD | +| Long-term isolation | Supermemory `container_tag = user_id` | Already the memory API’s tenant key | +| Session isolation | `thread_id`, later `{user_id}::{thread_id}` | Prevents short-term mix when 004 lands | +| Who is talking (v1) | Explicit identification first | `/user`, `--user`, spoken declaration | +| Who is talking (not v1) | Speaker diarization | Out of scope; do not block v1 on it | +| Default session | `DEFAULT_USER_ID` | Single-user path stays one flag | +| Shared facts | Optional household `container_tag` | Only if explicitly stored as shared | +| Brain factory | Existing `get_agent(user_id)` | Do not fork the graph per person | + +## 3. Phases + +### Phase 0 — Identity seam (this repo; in the tree) + +- `DEFAULT_USER_ID` from settings. +- `get_agent(user_id)` / `build_agent_graph(user_id)` bind tools to that id. +- `MemoryChat(user_id)` uses `container_tag=self.user_id`. +- CLI `--user` and `/user ` rebuild the chat for a different container. +- `AgentState` already has `user_id` and `thread_id` fields. + +This is a **container switch**, not speaker ID. + +### Phase 1 — Explicit identification (not shipped) + +- Bind a session to a `user_id` at start, or parse an explicit declaration. +- If unbound / unknown, ask which `user_id` to use; do not guess. +- Voice I/O passes the bound id into `get_agent(user_id)` on every turn. +- Known ids come from config, not from a coded household list. + +### Phase 2 — Isolation completeness (design; not a tenant product) + +- Namespace threads `{user_id}::{thread_id}` once spec 004 has a checkpointer. +- Tests: container A must not recall container B. +- Optional shared household `container_tag` with an explicit write path. +- Still no diarization. + +Phase 2 is finishing **this** isolation design. It is not multi-tenant SaaS. + +## 4. Risks + +| Risk | Mitigation | +|------|------------| +| Calling this “multi-tenant” because `/user` exists | Spec and tasks state: seam only, not speaker ID | +| Cross-container recall via a shared client | Always pass `container_tag=user_id`; never a global search | +| Default `user_id` silently used for the wrong person | Unknown speaker → ask; do not fall back without saying so | +| Inventing a household roster in docs or config | Opaque `user_id` / `container_tag` only | +| Checkpointer mixing threads across users | Spec 004 namespacing; this plan does not fake persistence | +| Diarization scope creep | Keep v1 explicit-only; diarization stays a non-goal | + +## 5. Success metrics + +- Two `user_id` values, two containers: facts stored under A never appear in B’s profile/search. +- `/user ` (or `--user`) changes the container for subsequent turns. +- Unbound session asks for a `user_id` instead of guessing. +- Adding a user is a new id in config, not a code change. +- No speaker-ID model required for the above to be true. + +## 6. What this plan is not + +It is not a shipped tenant model. It is not speaker identification. It is not +spec 004 (checkpointers). It does not define a household of named people. It +does not replace spec 001. diff --git a/specs/002-multi-user-support/spec.md b/specs/002-multi-user-support/spec.md index 6af56d1..043ffd1 100644 --- a/specs/002-multi-user-support/spec.md +++ b/specs/002-multi-user-support/spec.md @@ -1,98 +1,122 @@ # Feature Spec: Multi-User Support for the Voice Agent -**Feature ID**: 002-multi-user-support -**Status**: Draft / Future -**Related to**: [001-voice-dgx-spark-agent](../001-voice-dgx-spark-agent/spec.md) +**Feature ID**: 002-multi-user-support +**Status**: Designed; not a shipped tenant model +**Related to**: [001-voice-dgx-spark-agent](../001-voice-dgx-spark-agent/spec.md) **Created**: 2025-05-21 +**Recorded here**: 2026-08-19 +**Owner**: Derek Clair + +## Current state (honest) + +This spec is the **design** for per-user isolation. It is not a product multi-tenant system. + +Code in this repo today is an identity **seam**, not speaker ID: + +- `DEFAULT_USER_ID` in settings +- `user_id` on `get_agent()` / `build_agent_graph()` and `AgentState` +- Supermemory calls scoped with `container_tag=user_id` +- MemoryChat CLI `--user` and `/user ` (rebuilds the chat for that container) + +There is no speaker diarization, no voice fingerprint, and no automatic “who is talking” path. A `/user` switch is an explicit container change. ## Overview -The voice agent should gracefully support multiple users within the same household (initially: Derek, wife, two daughters, son, and occasional "other" guests or family members). +The voice agent should support multiple **household users** on the same deployment. -Each person should have their own persistent identity and long-term memory context. When someone speaks to the agent, it should correctly identify who they are (or be told) and recall the right history, preferences, ongoing projects, and relationships. +Each user is an opaque `user_id`. Long-term memory is isolated by a Supermemory `container_tag` (the same string as `user_id`). Short-term conversation state is isolated by `thread_id`. When someone speaks, the session must already be bound to a `user_id`, or the speaker must **declare** it. -This is a **cross-cutting concern** that affects user identification, Supermemory container isolation, session/thread management, the LangGraph state, and the overall voice experience. +This is a **cross-cutting concern**: identification, Supermemory container isolation, session/thread management, LangGraph state, and the voice loop. ## Goals -- Natural multi-user experience in a family setting. -- Strong long-term memory isolation per person (via Supermemory `container_tag`). -- Reasonable accuracy in knowing "who is talking" without constant re-identification. -- Future-proof for adding more family members or occasional guests. -- Maintain privacy boundaries between users. +- Natural multi-user experience for household users. +- Strong long-term memory isolation per `user_id` (via Supermemory `container_tag`). +- Low-friction identification: explicit declaration first; do not require a login ritual every turn once the session is bound. +- Adding another `user_id` is configuration, not a rewrite. +- Privacy boundaries between users: no cross-container recall. -## Non-Goals (for initial version) +## Non-Goals (for v1) -- Full biometric voice fingerprinting / speaker diarization (nice to have later). -- Remote multi-user access from outside the home. -- Complex household roles/permissions system. -- Guest accounts with temporary memory. +- Speaker diarization / biometric voice fingerprinting (out of scope for v1; possible later). +- Remote multi-user access from outside the deployment. +- Roles, permissions, or an admin/RBAC model. +- Guest accounts with temporary memory (open question, not v1). +- Inferring household relationships from names or stories. ## User Stories -1. **As Derek**, I want the agent to remember my ongoing projects, preferences, and conversations even when other family members have spoken to it recently. -2. **As my wife**, I want the agent to remember things that are important to me (kids' schedules, our shared tasks, etc.) without mixing them up with Derek's work stuff. -3. **As a kid**, I want the agent to know who I am when I talk to it and remember things like my homework, favorite games, or ongoing stories. -4. **As a parent**, I want to be able to say "Hey Lab, this is Sarah talking" or have the system figure it out reasonably well. -5. **As the household**, we want the agent to understand family relationships ("my sister", "Dad", "the kids") when context is relevant. +1. **As a household user**, I want memories, preferences, and ongoing work scoped to my `user_id` even if another household user spoke to the agent recently. +2. **As a household user**, I want my Supermemory `container_tag` isolated so another user’s facts are not injected into my turns. +3. **As a household user**, I want to identify myself explicitly (or start a session already bound to my `user_id`) so the agent uses the right container. +4. **As the operator**, I want adding or switching a `user_id` to be a low-effort config / command, not a new deployment. +5. **As a household user**, if the agent does not know which `user_id` is speaking, I want it to ask rather than guess. ## Functional Requirements ### FR-1: User Identity & Routing -- The system must be able to associate a voice interaction with a specific user identity. -- Supported identification methods (in rough priority order): - 1. Explicit declaration ("Hey Lab, it's Derek") - 2. Wake-word + name patterns - 3. Heuristic / voice characteristics (future) - 4. Device or room context (if multiple microphones are added later) + +- Every voice or text interaction must be associated with a specific `user_id`. +- Identification methods for v1, in priority order: + 1. Explicit declaration (e.g. “this is ``”) or session start with `--user` / `/user ` + 2. Wake-word + declared-name patterns (same explicit idea; not voice biometrics) + 3. Heuristic / voice characteristics — **out of scope for v1** + 4. Device or room context — later, if multiple capture devices exist +- Speaker diarization is **out of scope for v1**. ### FR-2: Memory Isolation (Supermemory) -- Every user must have their own `container_tag` in Supermemory. -- All `profile()`, `add()`, and `search` calls must be correctly scoped to the identified user. -- Cross-user leakage must be prevented (the agent should not accidentally recall one person's private facts to another). + +- Every `user_id` has its own `container_tag`. +- All `profile()`, `add()`, and `search` calls must be scoped to the identified `user_id`. +- Cross-user leakage must be prevented (no accidental recall of one container’s facts into another). ### FR-3: Session & Thread Management -- Each user should have their own conversation threads (`thread_id`). -- Short-term memory (LangGraph checkpointer) must be isolated per user. -- It should be possible to have parallel conversations with different family members. -### FR-4: Relationship & Household Context -- The agent should be able to reason about family relationships when given the right context ("Tell my wife...", "What does Dad usually say about this?"). -- There may be a lightweight "household" or "family" memory layer in addition to individual profiles. +- Each user has their own conversation threads (`thread_id`). +- Short-term memory (LangGraph checkpointer, when spec 004 lands) must be isolated per user. +- Parallel conversations for different household users must not share thread state. + +### FR-4: Shared household container (optional) + +- Individual profiles stay per `user_id` / `container_tag`. +- There may be a lightweight **shared** household `container_tag` for facts that are explicitly stored as shared — not a substitute for per-user isolation. +- The agent must not invent a household roster or infer private relationships. + +### FR-5: Unknown speakers -### FR-5: Graceful Handling of Unknown Speakers -- If the agent cannot confidently identify the speaker, it should ask for clarification in a friendly way ("Sorry, I didn't catch who I'm speaking with — is this Derek, Sarah, or one of the kids?"). +- If the agent cannot bind a turn to a `user_id`, it asks for clarification (e.g. “Which `user_id` should I use for this session?”). +- It must not guess a container. ## Non-Functional Requirements -- **Privacy**: One family member's private memories or conversations must never leak to another. -- **Low Friction**: Identification should feel natural, not like logging into a system every time. -- **Scalability**: Design should support adding more users without major rewrites. -- **Auditability** (future): It should be possible to see which user a memory belongs to. +- **Privacy**: One user’s private memories or conversations must never leak to another `user_id`. +- **Low friction**: Identification should feel like a one-time bind for the session, not a login on every turn. +- **Scalability**: Design supports additional `user_id` values without major rewrites. +- **Auditability** (future): It should be possible to see which `user_id` a memory belongs to. ## Open Questions -- How do we initially bootstrap user identities? (Manual config file? First-time "register yourself" flow?) -- Should there be a concept of a "primary user" (Derek) who has elevated capabilities? -- Do we want speaker diarization / voice embedding models running locally on the DGX for passive identification? -- How do we handle "other" / guests? Temporary containers? A generic "guest" profile? -- Should the agent proactively learn voices over time ("You sound like Maya today")? +- How do we bootstrap known `user_id` values? (Config file vs first-time “register this id” flow.) +- Is there a default `user_id` (`DEFAULT_USER_ID`) for single-user sessions, or must every session declare one? +- Do we want speaker diarization / voice embeddings later (local, on-box)? Not v1. +- How do we handle unknown / guest speakers? A generic `guest` container vs refuse until identified. +- Optional shared household `container_tag`: what is allowed to be written there, and who can read it? ## Relationship to Feature 001 -This feature is a natural evolution of the single-user voice agent defined in 001. +This feature extends the single-user voice agent in 001. -The core architecture decisions made in 001 (Supermemory `container_tag` per user, `thread_id` per session, LangGraph state) were intentionally designed to be multi-user friendly. This spec captures the additional work needed to make the experience truly multi-user in a family context. +001 already assumed `container_tag` per user and `thread_id` per session. This spec is the extra work to make that a real multi-user experience: explicit identification, isolation guarantees, and unknown-speaker handling. It does not replace 001 and does not implement spec 004 (checkpointers). -## Success Criteria (for when we eventually implement) +## Success Criteria (when implemented) -- Derek, his wife, and both kids can have natural, separate ongoing conversations with the agent over weeks/months with correct memory recall. -- The agent rarely confuses one person's context with another's. -- Adding a new family member is a low-effort configuration task. -- The experience feels personal and "knows" each person without feeling creepy or overly technical. +- Distinct household users can keep separate ongoing conversations with correct memory recall. +- The agent does not mix one `user_id`’s context into another’s. +- Adding a new `user_id` is a low-effort configuration task. +- Identification is explicit first; no biometric path is required for v1. --- -**Status**: This spec is captured for future planning. It is **not** in scope for the current implementation wave. +**Status**: Design captured for planning. **Not** a shipped tenant model. The identity seam (`DEFAULT_USER_ID`, `get_agent(user_id)`, MemoryChat `/user`) is in the tree; speaker ID and product isolation are not. -Next time we pick up multi-user work, we should create a `plan.md` and `tasks.md` under this directory following the established SDD process. \ No newline at end of file +See [plan.md](./plan.md) and [tasks.md](./tasks.md) for the SDD record of what exists vs what remains. diff --git a/specs/002-multi-user-support/tasks.md b/specs/002-multi-user-support/tasks.md new file mode 100644 index 0000000..0b18e20 --- /dev/null +++ b/specs/002-multi-user-support/tasks.md @@ -0,0 +1,45 @@ +# Tasks: Multi-user support (002) + +**Feature**: 002-multi-user-support +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +Checkboxes record what is in this tree today versus what remains design-only. +This file was filled in when the SDD record was completed, not when the +identity seam was first written. This is **not** a shipped tenant model. + +## Phase 0 — Identity seam (this repo) + +- [x] `DEFAULT_USER_ID` in settings (`thelab_langchain.config`) +- [x] `user_id` on `get_agent()` / `build_agent_graph()` +- [x] `AgentState.user_id` and `AgentState.thread_id` fields +- [x] `create_memory_tools(user_id)` scopes `profile` / `add` / `search` with `container_tag=user_id` +- [x] `MemoryChat(user_id)` uses the same `container_tag` +- [x] CLI `--user` and `/user ` (rebuilds MemoryChat for that container) +- [x] Voice orchestrator accepts `user_id` and passes it into `get_agent()` +- [ ] LangGraph checkpointer per thread (spec 004 — not required to call 002’s seam done) +- [ ] Speaker identification (not in tree; `/user` is not speaker ID) + +## Phase 1 — Explicit identification (not shipped) + +- [ ] Bind a session to a `user_id` at start (flag, config, or spoken declaration) +- [ ] Parse explicit “this is ``” (or equivalent) and switch the bound id +- [ ] If the turn cannot be bound, ask which `user_id` to use; do not guess +- [ ] Known ids from config only — no coded household roster +- [ ] Voice I/O (sibling package) must pass the bound `user_id` into `get_agent()` every turn +- [ ] Document the bind/switch commands next to `/user` without treating them as speaker ID + +## Phase 2 — Isolation completeness (design) + +- [ ] Thread namespacing `{user_id}::{thread_id}` when spec 004 has a checkpointer +- [ ] Test: memories stored under `user_a` are not returned for `user_b` +- [ ] Test: `/user` (or equivalent) actually changes `container_tag` for the next turn +- [ ] Optional shared household `container_tag` with an explicit write path +- [ ] Guest / unknown policy (refuse vs generic `guest` container) — decide, then implement +- [ ] Speaker diarization / voice embeddings — out of scope for v1 + +## Traceability + +Phase 0 lives in this package (`thelab_langchain.config`, `agent.graph.get_agent`, +`agent.tools.memory.create_memory_tools`, `chat.MemoryChat`, `cli` `/user`). +Phases 1–2 are not done. This tasks file is only the checklist view of the +design plus the identity seam that already exists. diff --git a/specs/003-deployment-infrastructure/plan.md b/specs/003-deployment-infrastructure/plan.md new file mode 100644 index 0000000..c5d6884 --- /dev/null +++ b/specs/003-deployment-infrastructure/plan.md @@ -0,0 +1,113 @@ +# Plan: Deployment infrastructure & Dockerization (003) + +**Feature**: 003-deployment-infrastructure +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-19 (SDD record; spec drafted 2025-05-21) + +## 1. Architecture + +Two deployment stories. Only one of them is the live spoken path. + +``` +Live desk voice (NOT this compose) + Lenovo Go (ALSA / HID) + │ + ▼ + conversational-voice-agent ← STT / TTS / button / LED + │ get_agent() + ▼ + this package (thelab_langchain) + +Experimental compose in *this* tree + agent ──gRPC──► riva + ──HTTP──► nemotron NIM +``` + +| Piece | Owner | +|-------|--------| +| Live STT, TTS, USB I/O | [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) ([spec 008](../008-local-tts-lenovo-go-spike/plan.md)) | +| LangGraph brain, provider factory | **this repo** | +| Experimental `agent` + `riva` + `nemotron` Compose | `docker-compose.yml` here — **not** production voice | +| Riva wrappers in `src/thelab_langchain/voice/` | Spike / Phase 2; streaming still `NotImplementedError` | + +`docker compose up` of this file is an experiment toward spec 001's Riva/NIM stack. It is not how the desk currently talks. + +## 2. What is already in the tree + +Honest inventory — the May 2025 spec "current state" is stale. + +| Artifact | What it actually is | +|----------|---------------------| +| `Dockerfile` | Multi-stage slim Python image, non-root `appuser`, PortAudio for `sounddevice`. Exists. | +| `docker-compose.yml` | Three services (`agent`, `riva`, `nemotron`) with Compose **profiles**, `restart: unless-stopped`, named model volumes. Experimental. | +| YAML `healthcheck:` blocks | Present on all three services. Agent probe is `import thelab_langchain` (import-ok, not readiness). Riva/NIM HTTP probes are unproven on this hardware. **Still a gap.** | +| YAML `deploy.resources` GPU reservations | Present. Swarm-style `deploy.devices` is not a verified `docker compose up` GPU story. No CPU/memory limits. No light-vs-full GPU profiles. **Still a gap.** | +| `.env.example` | Documents Python/app keys and provider flags. Not compose-time validation. | +| Makefile `docker-build` / `docker-push` | Local helpers. Tag and registry come from the operator's environment. This SDD does not name a registry host. | + +There is no `docker-compose.override.yml`, no model-downloader service, no secrets driver, and no CI image build (see [005](../005-testing-and-cicd/plan.md)). + +## 3. Tech choices (locked for this spec) + +| Concern | Choice | Why | +|---------|--------|-----| +| Live voice I/O | Sibling `conversational-voice-agent` | Spec 008 already runs on the desk; do not pretend Compose is that path | +| Experimental GPU stack | Compose on a single DGX Spark | Spec out of scope: Kubernetes / multi-node | +| Agent image | Existing multi-stage `Dockerfile` | Same image should run on a laptop CI runner *or* Spark; no Mac-only layers | +| GPU in Compose | Compose-native device requests (`gpus` / device_requests), not Swarm-only `deploy` | `docker compose up` is the intended command | +| Secrets | Host `.env` (gitignored); never bake keys into compose YAML or this SDD | No secrets in compose docs | +| Private registry | Operator sets `REGISTRY` + `TAG`; push/pull is a documented workflow, not a hostname in git | Do not commit a registry URL | +| LLM switch | Existing `LLM_PROVIDER` / `openai_compatible` | One-line switch; Compose must not fork the factory | +| Observability | Deferred (spec Phase C) | Structured logs later; no metrics stack in this wave | + +Do not put API keys, NGC tokens, or example secret values in compose comments or this directory. + +## 4. Phases + +### Phase A — Reliability & DX (high value) + +Close the gaps that make the *experimental* stack start and fail loudly: + +- Healthchecks that mean "ready for traffic", not "Python import succeeded". Agent should wait on real Riva/NIM readiness when those profiles are used. +- Compose-native GPU reservations that `docker compose up` honors; CPU/memory limits so one service cannot starve the box. +- `.env.example` fields the compose stack actually reads, with required vs optional called out (names only — no values). +- Compose profiles that match the comments (`full`, `agent`, `riva-only`, `nemotron-only`). Today every service has a profile, so a bare `docker compose up` starts nothing. +- A documented **hack-on-the-brain** path: venv + mocked / host LLM, no Riva container required. Live spoken testing stays in the sibling repo. + +Dockerfile multi-stage + non-root is already done; do not redo it unless a probe or user change requires it. + +### Phase B — Volumes, registry, restarts + +- First-run model download / cache story for Riva and NIM volumes (script or one-shot service). Version the cache layout; do not copy weights into the agent image. +- Private registry workflow: build → tag (`git-sha` and optional semver) → push → pull on Spark. Registry hostname stays in the operator's environment, not in SDD. +- Restart backoff that survives model-load and GPU OOM without a tight crash loop. +- Basic structured logging (no Prometheus/Grafana yet). + +### Phase C — Later + +- Secrets management (Docker secrets / a vault) instead of plain env. +- Observability stack (when the team is ready). +- Automated image builds in CI — owned with spec 005; this spec only requires the image to *be* buildable. + +## 5. Risks + +| Risk | Mitigation | +|------|------------| +| Treating `docker-compose.yml` as the live voice stack | This plan + README: live path is the sibling I/O repo; Compose is experimental | +| Riva `2.15.0` image vs current Spark GPU | Compose already notes GB10 may not run that tag; do not block desk voice on it | +| Large NIM as compose default vs spec 007 budget | Do not bless 120B-class agent loops on one Spark; keep NIM as an experiment | +| Swarm `deploy.resources` ignored by Compose | Move GPU requests to a Compose-native key and verify with `nvidia-smi` in-container | +| Import-only agent healthcheck | Replace with a real ready check; `depends_on: service_healthy` is useless until then | +| Secrets or a registry hostname landing in git | `.env` gitignored; SDD and compose comments stay hostname-free and key-free | + +## 6. Success metrics + +- A new person can follow a runbook and bring up the **experimental** full profile on a Spark in under two hours when models are cached (spec success criterion). This is not "desk voice works." +- Agent image rebuilds independently of Riva/NIM images. +- `LLM_PROVIDER` remains a one-line switch (already true in code). +- Individual experimental services can restart without taking the others down permanently. +- Live spoken sessions still go through [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent). + +## 7. What this plan is not + +It is not a rewrite of spec 001. It is not the Lenovo Go spike (008). It is not Kubernetes, canary deploys, or cost work across multiple Sparks (spec out of scope). It does not document a private registry hostname or any credentials. diff --git a/specs/003-deployment-infrastructure/tasks.md b/specs/003-deployment-infrastructure/tasks.md new file mode 100644 index 0000000..4f3762e --- /dev/null +++ b/specs/003-deployment-infrastructure/tasks.md @@ -0,0 +1,65 @@ +# Tasks: Deployment infrastructure & Dockerization (003) + +**Feature**: 003-deployment-infrastructure +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +Checkboxes are the honest tree as of this SDD record, not the May 2025 spec +"current state". Compose here is **experimental**. Live voice I/O is +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent). + +Do not commit secrets, serials, or a registry hostname in any of this work. + +## Already in tree (do not redo as if missing) + +- [x] Multi-stage `Dockerfile` (builder wheel → slim runtime, non-root user) +- [x] Root `docker-compose.yml` with `agent`, `riva`, `nemotron` services +- [x] Compose profiles keys (`full`, `agent`, `riva` / `riva-only`, `nemotron` / `nemotron-only`) +- [x] `restart: unless-stopped` on those services +- [x] Named volumes declared for Riva / NIM caches +- [x] `.env.example` for the Python app (provider + key *names*) +- [x] Makefile `docker-build` / `docker-push` taking `REGISTRY` + `TAG` from the environment + +## Phase A — Reliability & DX + +YAML stubs exist for health and GPU; they are **not** done. + +- [ ] Real healthchecks: agent ready for traffic (not `import thelab_langchain`) +- [ ] Real healthchecks: Riva and NIM probes verified on the images we actually run +- [ ] Agent `depends_on` / startup order that waits on those probes when using `full` +- [ ] Compose-native GPU device requests that `docker compose up` honors (not Swarm-only `deploy.resources`) +- [ ] CPU and memory limits per service +- [ ] Light vs full GPU allocation profiles (documented, not just a comment) +- [ ] Bare `docker compose up` vs `--profile` behavior matches the file comments (today a bare up starts nothing) +- [ ] `.env.example` lists compose-relevant variables; required vs optional; **no secret values** +- [ ] Document hack-on-the-brain: `make install` / `make chat` with host or mocked LLM, no Riva +- [ ] Document that spoken I/O is the sibling repo, not this compose file + +## Phase B — Volumes, registry, restarts + +- [ ] First-run model download / cache helper (script or one-shot service) +- [ ] Document volume layout and how caches are shared; do not copy weights into the agent image +- [ ] Registry workflow in a runbook: build → tag `git-sha` (and optional semver) → push → pull on Spark +- [ ] Keep registry hostname out of git and out of this SDD (operator env only) +- [ ] Restart policy / backoff that survives model load and GPU OOM +- [ ] Basic structured logging configuration (no metrics stack) + +## Phase C — Later + +- [ ] Secrets mechanism other than plain env (Docker secrets or a vault) +- [ ] Observability stack (deferred with the spec) +- [ ] Automated image build in CI (tracked in [005](../005-testing-and-cicd/tasks.md); this spec only needs the image to stay buildable) + +## Out of scope (leave unchecked on purpose) + +- [ ] Kubernetes / Spark-specific cluster orchestration +- [ ] Canary / blue-green +- [ ] Multi-node cost / placement +- [ ] Replacing the live 008 voice path with Riva Compose + +## Traceability + +Implementation of the live spoken path is +[`derekclair/conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent). +This tasks file is the checklist for *this* repo's experimental Compose/Docker +gaps. Dockerfile + compose skeleton are already here; health, Compose GPU, and +registry hygiene are not. diff --git a/specs/004-persistence-checkpointers/plan.md b/specs/004-persistence-checkpointers/plan.md new file mode 100644 index 0000000..9444fbc --- /dev/null +++ b/specs/004-persistence-checkpointers/plan.md @@ -0,0 +1,103 @@ +# Plan: Persistence & checkpointers (004) + +**Feature**: 004-persistence-checkpointers +**Spec**: [spec.md](./spec.md) +**Date**: 2025-05-21 (spec); recorded 2026-08-19 + +## 1. Architecture (as shipped) + +Short-term conversation state is **not** a LangGraph checkpoint. `get_agent()` +compiles with no `checkpointer` argument. Turns that survive a process only +do so because the **caller** keeps a message list. + +``` +Live voice I/O (sibling) this package + session Human/AI list get_agent(user_id) + │ │ + └── graph.invoke({messages}) ──► graph.compile() # no checkpointer + │ + memory_injection → call_llm → execute_tools + │ + ▼ + Supermemory (long-term only) +``` + +| Piece | Owner today | +|-------|-------------| +| Per-turn history | Caller. Live path is [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent): accumulate `HumanMessage` / `AIMessage` for the session. | +| LangGraph checkpoint | **None.** `get_agent()` is `return graph.compile()`. | +| `thread_id` on state | Data field / log tag. Not `config["configurable"]["thread_id"]`. | +| In-tree `VoiceOrchestrator` | Invokes with **this turn only** (`[HumanMessage(text)]`). Does not accumulate. | +| Long-term facts | Supermemory tools (`create_memory_tools`). Out of scope for 004. | + +`MemorySaver` is **not** implicit. Omitting `checkpointer` means no thread +memory inside the graph. Each `invoke` sees only the `messages` the caller +passed. + +## 2. Tech choices + +### Locked now (honest) + +| Concern | Choice | Why | +|---------|--------|-----| +| Checkpointer | Not wired | Spec 008 does not require it; nothing here survives a restart | +| Session history | Caller-side list | Sibling already does this; do not double-store | +| Durable short-term | None | Process death / reboot drops in-flight turns | +| Long-term | Unchanged Supermemory | Spec 006 | + +### If this spec is picked up later (not started) + +| Concern | Intended choice | Why | +|---------|-----------------|-----| +| Dev | `MemorySaver` | In-process only | +| Default durable | SQLite (`langgraph-checkpoint-sqlite`) + volume | Single host, low ops | +| Upgrade | Postgres via env | Same factory | +| Factory | `get_checkpointer()` + `CHECKPOINTER_BACKEND` | `memory` / `sqlite` / `postgres` | +| Isolation | `{user_id}::{thread_id}` as LangGraph `thread_id` | Spec 002; do not confuse with `AgentState.thread_id` | + +## 3. Phases + +### Phase 0 — Document current in-memory behavior (this SDD) + +- Record that `compile()` has no checkpointer. +- Record caller-side accumulation on the live voice path. +- Record that the in-tree orchestrator is single-turn per invoke. + +### Phase 1 — Factory + wire (not started) + +- `get_checkpointer()` in the agent package. +- Pass it into `graph.compile(checkpointer=...)`. +- Invoke with `configurable.thread_id` (namespaced). +- Decide whether the sibling still accumulates, or the graph becomes the source of history (do not do both blindly). + +### Phase 2 — Durable backend (not started) + +- SQLite file on a persistent volume. +- Postgres as a config change, not a second graph. +- Optional last-N checkpoint cleanup. + +Phase 1–2 are **not** in this tree. Do not treat this plan as a claim they shipped. + +## 4. Risks + +| Risk | Mitigation | +|------|------------| +| Double history (checkpointer + caller list) | Pick one owner of short-term turns before wiring | +| `AgentState.thread_id` vs LangGraph config `thread_id` | Namespacing lives in `configurable`; state field is not a checkpoint key | +| Spec text that called MemorySaver “implicit default” | This plan supersedes that: it is opt-in | +| Backend swap later | Factory + one env var; no graph fork | + +## 5. Success metrics (only after Phase 1–2) + +- Restarting the agent process does not drop an active thread. +- Two `user_id` values cannot read each other’s short-term state. +- SQLite → Postgres is env + volume, not a rewrite. + +None of these hold today. + +## 6. What this plan is not + +It is not an implementation of `get_checkpointer()`. It is not a SQLite volume +in compose. It is not a requirement to call spec 008 done. It is not +persistence of audio / LED / VAD state. It is not a second long-term memory +store (that is 006, and 006 is also not building adapters). diff --git a/specs/004-persistence-checkpointers/tasks.md b/specs/004-persistence-checkpointers/tasks.md new file mode 100644 index 0000000..2fccdb4 --- /dev/null +++ b/specs/004-persistence-checkpointers/tasks.md @@ -0,0 +1,47 @@ +# Tasks: Persistence & checkpointers (004) + +**Feature**: 004-persistence-checkpointers +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +Checkboxes record what is actually in the tree. A LangGraph checkpointer is +**not** wired. Phase 0 is this SDD record. Leave Phase 1–2 unchecked until +code ships. + +## Phase 0 — Document current in-memory behavior + +- [x] Record that `get_agent()` is `graph.compile()` with no `checkpointer` +- [x] Record that live voice I/O accumulates `HumanMessage` / `AIMessage` caller-side +- [x] Record that in-tree `VoiceOrchestrator` invokes with the current turn only +- [x] Record that `AgentState.thread_id` is not LangGraph `configurable.thread_id` +- [x] Record that `MemorySaver` is opt-in, not an implicit default + +## Phase 1 — Factory + wire (not started) + +- [ ] Add `get_checkpointer()` in the agent package +- [ ] `CHECKPOINTER_BACKEND` env (`memory` / `sqlite` / `postgres`) +- [ ] Pass the checkpointer into `graph.compile(...)` +- [ ] Namespace LangGraph `thread_id` as `{user_id}::{thread_id}` +- [ ] Invoke with `config={"configurable": {"thread_id": ...}}` +- [ ] Choose one owner of short-term history (graph vs caller); do not double-store +- [ ] Tests that two thread ids do not share checkpoint state (in-memory backend) + +## Phase 2 — Durable backend (not started) + +- [ ] SQLite checkpointer + persistent volume +- [ ] Postgres path as the same factory, different env +- [ ] Document backend swap (env + volume; no graph fork) +- [ ] Optional last-N checkpoint cleanup per thread + +## Out of scope (do not check as 004 done) + +- [ ] LangGraph checkpointer required for spec 008 +- [ ] Persist voice / audio / LED state +- [ ] Second long-term memory backend (spec 006) +- [ ] Multi-tenant product isolation (spec 002 is design-only) + +## Traceability + +`src/thelab_langchain/agent/graph.py` — `get_agent()` compiles with no +checkpointer. Live session lists live in +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent), +not in this graph. diff --git a/specs/005-testing-and-cicd/plan.md b/specs/005-testing-and-cicd/plan.md new file mode 100644 index 0000000..61a611c --- /dev/null +++ b/specs/005-testing-and-cicd/plan.md @@ -0,0 +1,127 @@ +# Plan: Testing strategy, CI/CD, and coverage (005) + +**Feature**: 005-testing-and-cicd +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-19 (SDD record; spec drafted 2025-05-21) + +## 1. Architecture + +Tests live in this repo. CI is GitHub Actions on a CPU runner. Live spoken I/O +and GPU voice loops are **not** in this workflow. + +``` +PR / push + │ + ▼ +.github/workflows/ci.yml ubuntu-latest, no GPU + ├── ruff check . + └── pytest -q tests/ only, mocked services +``` + +| Layer | Where | What it covers | +|-------|--------|----------------| +| Unit (shipped) | `tests/` | Graph routing, memory injection, config keys, LLM factory, prompt block | +| CI (shipped) | `.github/workflows/ci.yml` | Ruff + pytest, CPU-only install (`pip install -e . --no-deps` + lightweight deps) | +| Coverage gates | **not shipped** | spec target 60% overall / 80%+ on `agent/` and `voice/` | +| Docker image CI | **not shipped** | build (and later push) of the agent image | +| GPU / hardware e2e | **not shipped** | out of scope for every PR (spec) | + +The May 2025 spec "current state" (`No tests/ directory`, `No GitHub Actions`) is +**wrong today**. Do not plan as if those are missing. + +## 2. What is already in the tree + +| Artifact | Notes | +|----------|--------| +| `tests/test_agent_graph.py` | `_should_continue` routing; `_memory_injection` with mocked tools; no extra summarization LLM | +| `tests/test_chat.py` | `MemoryContext.to_prompt_block` | +| `tests/test_config.py` | `Settings.validate_keys` per provider | +| `tests/test_llm.py` | `get_chat_model` routing with fake provider modules | +| `pyproject.toml` | `pytest` + `pytest-asyncio`; `[tool.pytest.ini_options]` `testpaths = ["tests"]`, `asyncio_mode = auto` | +| `ruff` / `mypy` | Dev deps. `make lint` runs both. **CI runs ruff only**, not mypy. | +| CI install | Skips `sounddevice` / `nvidia-riva-client` so a plain runner can import the brain | + +There is **no** `make test` target (Makefile has `lint`, not pytest). There is +**no** `pytest-cov` / coverage config. There is **no** image-build job. There is +**no** GPU job. + +## 3. Tech choices (locked for this spec) + +| Concern | Choice | Why | +|---------|--------|-----| +| Runner | GitHub Actions `ubuntu-latest` | Matches the existing workflow | +| Unit suite | pytest, mocked Supermemory / LLM | Fast, no keys, no GPU, no PortAudio | +| Lint on PR | ruff (already) | Keep the current job; add mypy later, do not drop ruff | +| Coverage | `pytest-cov` / coverage.py when we add gates | Spec names these; do not invent a hosted-coverage vendor requirement | +| Image CI | Separate job or workflow, not on the CPU unit job | Unit job must stay lightweight | +| GPU e2e | Gated / manual / self-hosted later | Spec: not on every PR | +| Live voice | Sibling [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) | Do not put ALSA / Parakeet / Piper in this repo's CI | + +No API keys in workflow files. No registry hostname in workflow YAML committed +to this repo; if a later push job needs a registry, it reads from Actions +secrets / env that are not documented as literals here. + +## 4. Phases + +### Phase 0 — Unit + CPU CI (done) + +Keep: + +- `tests/` as the example for new tests (spec success: contributors copy these). +- CPU-only CI install so audio/GPU deps do not break the runner. +- Ruff + pytest on push and pull_request. + +Do not delete or "bootstrap" a tests directory that already exists. + +### Phase 1 — Coverage, DX, types in CI + +- Add coverage measurement (`pytest-cov` or coverage.py) and a **gate** that + matches the spec's intent: fail on significant drops / below the initial + floor (60% overall; 80%+ on `agent/` and `voice/` once those packages are + measured honestly — `voice/` is mostly untested Riva spike code). +- `make test` (and optionally `make test-cov`) so local DX matches CI. +- Add mypy to CI if we want the spec's "lint + type check" line; local + `make lint` already runs it. +- Cache pip in Actions. + +### Phase 2 — Docker image CI + +- Build the agent image from the existing `Dockerfile` on tags and/or main. +- Tag with `git-sha` (and semver when we cut tags). +- Push is optional and operator-configured. Do not bake a registry URL into + the workflow file in git. +- Multi-platform only if we prove we need it (Spark is aarch64; CI is amd64). + +Compose stack smoke (`docker compose` healthy) is integration, not this unit +job. It stays gated. + +### Phase 3 — Heavier tests (gated) + +- Graph integration with real-ish tools (still no live Supermemory account in CI). +- Voice loop with mocked audio / Riva — this repo's `voice/` module, not 008. +- GPU e2e on a Spark or a GPU runner: **not** every PR. +- Hardware-in-the-loop with the Lenovo Go: out of scope for this package's CI. + +## 5. Risks + +| Risk | Mitigation | +|------|------------| +| Planning as if `tests/` or CI do not exist | This plan; mark those tasks `[x]` | +| Coverage gate that punishes the untested `voice/` spike | Measure `agent/` first; do not fail the repo for Riva `NotImplementedError` paths until we test them | +| Pulling audio/GPU wheels on `ubuntu-latest` | Keep the CPU-only `--no-deps` install in CI | +| Image push leaking a registry hostname or credentials | Secrets only; SDD stays hostname-free | +| Treating Compose e2e as desk voice | Compose is experimental (003); live I/O is the sibling repo | + +## 6. Success metrics + +- `pytest` locally (ideally `make test`) runs the unit suite in well under 30s. +- Every PR gets ruff + unit results (already true). +- Coverage gate exists before we claim "CI enforces coverage." +- Image builds in CI before we claim "we can cut a Spark image from git." +- New tests follow `tests/test_*.py` patterns (mocked services, no keys). + +## 7. What this plan is not + +It is not a claim that the May 2025 spec current-state bullets are still true. +It is not GPU voice CI. It is not mutation testing or a performance bench in +Actions (spec out of scope). It is not the 008 hardware loop. diff --git a/specs/005-testing-and-cicd/spec.md b/specs/005-testing-and-cicd/spec.md index da780d4..f1c7f98 100644 --- a/specs/005-testing-and-cicd/spec.md +++ b/specs/005-testing-and-cicd/spec.md @@ -1,23 +1,23 @@ # Feature Spec: Testing Strategy, CI/CD, and Coverage **Feature ID**: 005-testing-and-cicd -**Status**: Draft +**Status**: Partial (unit tests + CPU CI exist; coverage gates and image CI do not) **Related**: 001-voice-dgx-spark-agent, 003-deployment-infrastructure -**Date**: 2025-05-21 +**Date**: 2025-05-21 +**Updated**: 2026-08-20 ## Overview -The repository currently has almost no automated tests, no CI pipeline, and no coverage measurement. As the system grows (especially the agent brain, voice layer, and multi-service Docker stack), this becomes a major risk. +The agent graph, config, and LLM factory need automated tests so changes do not rely on desk smoke only. This spec is the testing and delivery target. Some of it has shipped; some has not. -This spec defines the target testing and delivery infrastructure. +## Current State (2026-08-20) -## Current State - -- No `tests/` directory with meaningful coverage. -- No `pytest` configuration beyond a stub in `pyproject.toml`. -- No GitHub Actions or other CI workflow. -- No coverage reporting (codecov, etc.). -- Manual testing is the primary validation method. +- `tests/` exists: `test_agent_graph`, `test_chat`, `test_config`, `test_llm` (CPU, mocked externals). +- `pytest` + `pytest-asyncio` and `testpaths` are in `pyproject.toml`. +- GitHub Actions `.github/workflows/ci.yml` runs ruff + pytest with a CPU-only `--no-deps` install. +- No coverage measurement or coverage gate. +- No Docker image build in CI. +- No hardware-in-the-loop tests (and they stay out of every-PR CI). ## Goals diff --git a/specs/005-testing-and-cicd/tasks.md b/specs/005-testing-and-cicd/tasks.md new file mode 100644 index 0000000..12792ce --- /dev/null +++ b/specs/005-testing-and-cicd/tasks.md @@ -0,0 +1,59 @@ +# Tasks: Testing strategy, CI/CD, and coverage (005) + +**Feature**: 005-testing-and-cicd +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +The May 2025 spec said there was no `tests/` directory and no GitHub Actions. +That is outdated. Phase 0 is **done**. Coverage gates, GPU e2e, and Docker +image CI are **not**. + +## Phase 0 — Unit tests + CPU CI (done) + +- [x] `tests/` directory with meaningful unit tests +- [x] `tests/test_agent_graph.py` — graph routing + memory injection (mocked tools) +- [x] `tests/test_chat.py` — `MemoryContext.to_prompt_block` +- [x] `tests/test_config.py` — `Settings.validate_keys` +- [x] `tests/test_llm.py` — `get_chat_model` provider routing +- [x] pytest + pytest-asyncio in `[project.optional-dependencies] dev` +- [x] `[tool.pytest.ini_options]` (`testpaths = ["tests"]`, `asyncio_mode = auto`) +- [x] `.github/workflows/ci.yml` on push and pull_request +- [x] CI: ruff check +- [x] CI: pytest -q on ubuntu-latest +- [x] CI: CPU-only install (`pip install -e . --no-deps` + lightweight brain deps; no Riva / PortAudio) + +## Phase 1 — Coverage, DX, types + +- [ ] `pytest-cov` or coverage.py wired so `pytest` can emit a report +- [ ] Coverage **gate** (spec floor: 60% overall; 80%+ on `agent/` and `voice/` once measured honestly) +- [ ] Fail PRs on the gate (or on a significant drop) — not "coverage is printed but ignored" +- [ ] `make test` (Makefile currently has `lint`, not pytest) +- [ ] mypy in CI (local `make lint` already runs ruff + mypy; Actions does not) +- [ ] pip cache on the CI job + +## Phase 2 — Docker image CI + +- [ ] CI job that builds the agent image from the root `Dockerfile` +- [ ] Tag with `git-sha` (semver when tags exist) +- [ ] Optional push to a private registry via operator secrets — **no registry hostname in git** +- [ ] Multi-platform build only if we need amd64 CI → aarch64 Spark; do not assume it + +## Phase 3 — Gated / heavier tests + +- [ ] Integration: graph execution with lightly mocked tools beyond the current unit file +- [ ] Voice loop with mocked audio / Riva (this repo's `voice/` package) +- [ ] Compose smoke: stack starts and reports healthy (depends on [003](../003-deployment-infrastructure/tasks.md) probes actually meaning ready) +- [ ] GPU e2e on a Spark or GPU runner — **not** on every PR + +## Out of scope (leave unchecked on purpose) + +- [ ] Hardware-in-the-loop on the Lenovo Go in this repo's CI +- [ ] Mutation testing +- [ ] Performance benchmarking in CI +- [ ] Hosted coverage SaaS (optional; not required to close the gate) + +## Traceability + +Unit tests and the CPU workflow are in this repo today +(`.github/workflows/ci.yml`, `tests/`). Live spoken e2e belongs to +[`derekclair/conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) +(spec 008), not this checklist. diff --git a/specs/006-alternative-memory-systems/plan.md b/specs/006-alternative-memory-systems/plan.md new file mode 100644 index 0000000..bb9b691 --- /dev/null +++ b/specs/006-alternative-memory-systems/plan.md @@ -0,0 +1,93 @@ +# Plan: Alternative long-term memory systems (006) + +**Feature**: 006-alternative-memory-systems +**Spec**: [spec.md](./spec.md) +**Date**: 2025-05-21 (spec); recorded 2026-08-19 + +## 1. Architecture (escape hatch, not a second store) + +Long-term memory already has a narrow seam. The graph and voice layer do not +import a second backend. Keep it that way until a **real** need appears. + +``` +get_agent(user_id) + │ + ├── memory_injection ──► create_memory_tools(user_id) + ├── call_llm (tools bound) │ + └── execute_tools ▼ + get_user_profile + recall_memories(query, limit) + store_memory(content, metadata) + │ + ▼ + Supermemory +``` + +| Piece | Owner | +|-------|--------| +| Tool names + signatures | `src/thelab_langchain/agent/tools/memory.py` | +| Per-user scope | `user_id` → Supermemory `container_tag` | +| Graph / voice | Call the three tools only | +| Short-term turns | Spec 004 (caller-side list; no checkpointer). Not this spec. | + +## 2. Tech choices (locked until a trigger fires) + +| Concern | Choice | Why | +|---------|--------|-----| +| Long-term store | Supermemory | Already delivering profile + recall | +| Seam | The three tools above | Cheap swap later; no ABC yet | +| `MemoryBackend` protocol | **Do not add** | Protocol-for-one-impl is noise | +| Second adapter (Mem0, Zep, local vectors, …) | **Do not build** | No air-gap / cost / quality trigger yet | +| Default if we ever swap | Keep Supermemory as default | Household path stays the known UX | + +Candidates in the spec (Zep, Mem0, LangGraph store, custom vector+graph, +SQLite+embeddings) stay a table of options. They are not a backlog to +implement in order. + +## 3. Phases + +### Phase 0 — Keep the interface narrow (now) + +- Leave `create_memory_tools(user_id)` as the only factory. +- Do not introduce a protocol, registry, or dual-write. +- New graph nodes must not import a memory SDK except through those tools. + +### Phase 1 — Adapter, only after a real need (not started) + +Triggers that would justify Phase 1 (any one is enough; none are true today): + +- Fully air-gapped deploy (no outbound memory calls). +- Cost of the current store is material at this scale. +- Measured recall/profile gap another system actually fixes. +- Need graph-shaped queries the current store cannot do. + +Then, and only then: + +1. Pick **one** second backend for that need (not a portfolio). +2. Extract a small protocol that matches the three methods we already use. +3. Make `create_memory_tools` pluggable; default remains Supermemory. + +Do not start Phase 1 “so it will be ready.” + +## 4. Risks + +| Risk | Mitigation | +|------|------------| +| Premature ABC | No protocol until a second impl is chosen | +| Dual-write / split brain | One store per deploy; no silent fan-out | +| Backend types leaking into graph nodes | Tools stay the only import surface | +| Confusing 004 checkpointers with 006 stores | Short-term ≠ long-term; do not merge them | + +## 5. Success metrics + +- Still one factory and three tool names. +- `graph.py` does not grow a second memory client. +- A second backend appears only after a trigger above is written down. + +There is no success metric for “we have N adapters.” + +## 6. What this plan is not + +It is not a Mem0 or Zep port. It is not a local vector store. It is not +leaving Supermemory. It is not a LangGraph checkpointer (004). It is not +permission to add `MemoryBackend` “for cleanliness.” diff --git a/specs/006-alternative-memory-systems/tasks.md b/specs/006-alternative-memory-systems/tasks.md new file mode 100644 index 0000000..dd7a472 --- /dev/null +++ b/specs/006-alternative-memory-systems/tasks.md @@ -0,0 +1,47 @@ +# Tasks: Alternative long-term memory systems (006) + +**Feature**: 006-alternative-memory-systems +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +Checkboxes record what is actually in the tree. This spec is an escape hatch. +Do **not** build adapters or a second backend until a real need is written +down. + +## Phase 0 — Keep the interface narrow + +- [x] Long-term access only via `get_user_profile`, `recall_memories`, `store_memory` +- [x] Keep `create_memory_tools(user_id)` as the only factory +- [x] Scope tools with `user_id` (`container_tag`); graph does not pick a backend +- [x] Document this as an escape hatch, not a multi-backend project +- [x] Do not add a `MemoryBackend` protocol / ABC for a single implementation +- [x] Do not add a second memory client in `graph.py` or the voice layer + +Standing rule: new code talks to long-term memory only through those three +tools. + +## Phase 1 — One adapter, only after a real need (not started) + +Do not schedule these. They unlock when a trigger in the plan is real. + +- [ ] Write the trigger (air-gap, cost, measured recall gap, or graph queries) +- [ ] Choose **one** second store for that trigger +- [ ] Define a protocol that matches the three methods we already use +- [ ] Implement one adapter; keep Supermemory the default +- [ ] Make `create_memory_tools` pluggable without changing graph node shape +- [ ] Tests: graph still compiles when the default backend is the only one configured + +## Out of scope (no second backend until a real need) + +- [ ] Zep adapter +- [ ] Mem0 adapter +- [ ] LangGraph long-term memory store as a parallel backend +- [ ] Custom vector + graph store +- [ ] SQLite + embeddings as a second production path +- [ ] Dual-write to two stores +- [ ] Swap motivated only by “we might want it later” + +## Traceability + +`src/thelab_langchain/agent/tools/memory.py` is the seam. +`src/thelab_langchain/agent/graph.py` calls `create_memory_tools` for injection +and tool-calling. Short-term turns are spec 004 (still no checkpointer). diff --git a/specs/007-dgx-hardware-optimization/plan.md b/specs/007-dgx-hardware-optimization/plan.md index 4b2e6f5..0498c21 100644 --- a/specs/007-dgx-hardware-optimization/plan.md +++ b/specs/007-dgx-hardware-optimization/plan.md @@ -2,49 +2,85 @@ **Feature**: 007-dgx-hardware-optimization **Related Spec**: [spec.md](./spec.md) -**Date**: 2025-05-22 +**Date**: 2025-05-22 (living slot policy notes added 2026-08-19) **Implementation Branch**: `feat/007-dgx-hardware-optimization-impl` ## 1. Goal -Execute the strategy defined in the spec with rigorous measurement: +Execute the strategy defined in the spec with rigorous measurement, **without** treating the 001 compose stack as live production. -- Capture an accurate **baseline** on the current production configuration (120B + full Riva on single DGX Spark). -- Run controlled experiments for the highest-leverage changes (lightweight English audio stack, 49B model swap). +- Keep the **living inference slot policy** (spec.md) aligned with the desk: one local generative LLM, CPU STT/TTS, hosted Grok for quality-critical fleet roles. +- Capture an accurate **live-path baseline** (spec 008 I/O + `get_agent()` + hosted Grok and/or one ~30B-class `openai_compatible` / Ollama worker). This is what actually runs. +- Treat **120B NIM + full Riva** (`docker-compose` in this repo) as an **optional experimental** capture — not “current production.” Do not invent GB figures; those runs stay unmeasured until a report exists. +- Run controlled experiments for high-leverage changes (lightweight English audio on the compose path if revived; ~30B vs 49B as the *single* occupied slot). **No 120B+ agent loops on one Spark** as a daily driver. - Quantify headroom, voice turn latency, concurrency limits, and memory behavior. -- Make a data-driven decision on the **sweet-spot configuration**. +- Make a data-driven decision on the **sweet-spot configuration**, or keep the living policy as practice-without-numbers, labeled unmeasured. - Document everything so future changes (including 2× node work) have a clear before/after reference. -Success = we have reproducible numbers and a locked "recommended daily driver" profile for the family voice agent. +Success = the slot policy matches reality, and we have reproducible numbers (or an explicit “unmeasured” label) for a locked daily-driver profile for the household voice agent. + +## 1.1 Inference slot policy (plan notes) + +Matches [spec.md — Inference slot policy (living)](./spec.md#inference-slot-policy-living). This is how we schedule work on the GB10; the phases below are how we *measure*. + +| Rule | Practice | +|------|----------| +| GB10 budget | ~128 GB unified, ~273 GB/s, **one serious local LLM at a time** | +| Live voice I/O | `conversational-voice-agent` (spec 008): Parakeet TDT 0.6B **CPU** STT + Piper **CPU** TTS | +| Brain | this repo `get_agent()` | +| Default LLM | hosted Grok | +| Local option | `openai_compatible` / Ollama ~30B-class, hosted fallback | +| This repo compose (`agent` + `riva` + `nemotron` 120b) | experimental; **not** the live spoken path | +| Quality-critical fleet | orchestrator, architect, reviewer, design → hosted Grok (do not fight the slot) | +| Local workers | coder, researcher → ~30B-class with hosted fallback | +| Forbidden on one Spark | 120B+ agent loops; a second large local LLM next to the occupied slot | +| Speech vs GPU | Prefer CPU STT/TTS so the unified/GPU slot stays with at most one generative LLM | + +Harness work must label every report **live** vs **experimental-compose**. Phase 0 originally assumed compose 120B + Riva was the as-is stack; that assumption is **retired**. ## 2. High-Level Phases ### Phase 0 – Baseline Capture (Must Do First) -Establish the "as-is" numbers on the exact current stack before touching anything. +Establish numbers **before** changing the *experimental* compose stack — and, separately, sample the **live** path that already runs. + +**0a. Live path (priority; this is as-is):** +- Spec 008 I/O (Parakeet CPU + Piper CPU) + `get_agent()` + hosted Grok, then the same with one ~30B-class local worker occupying the slot. +- Instrument or manually sample unified memory / CPU / (if a local LLM is up) GPU. +- Produce a live-path report. Do not invent GB figures if the sampler is not in place — leave TBD. + +**0b. Experimental compose (optional; not production):** +- Clean DGX Spark with this repo’s `docker-compose.yml` + nemotron-3-super-120b-a12b + full Riva. +- Same metrics. Label the report experimental. NIM + Riva GB numbers remain **unmeasured** until this run exists. -- Run on clean DGX Spark with current `docker-compose.yml` + nemotron-3-super-120b-a12b + full Riva. -- Instrument or manually measure the core metrics from the spec. -- Produce a `baseline-report.md` (or JSON + human summary) committed in the repo. +Do not present 0b as “what we run today.” -### Phase 1 – Audio Stack Reduction (Highest Leverage Quick Win) -Replace full Riva with a minimal English-only path (Parakeet CTC + high-quality English TTS NIM or equivalent). +### Phase 1 – Audio Stack Reduction (Compose experiment; live path already on CPU) + +**Already practice (not a 007 deliverable):** live desk voice is Parakeet TDT 0.6B CPU + Piper CPU (spec 008). That was the high-leverage win for the spoken path. Do not plan this phase as if Riva is the live ASR. + +**If compose/Riva is revived:** replace full Riva with a minimal English-only path (Parakeet CTC + high-quality English TTS NIM or equivalent). - Create a lightweight audio service profile (new container or slimmed Riva config). - Update `docker-compose` with profiles or separate override files. - Re-run the benchmark harness. -- Compare delta vs baseline (memory saved, latency change, perceived voice quality). +- Compare delta vs the *experimental* compose baseline (memory saved, latency change, perceived voice quality). **Unmeasured** until that run exists. +- Prefer keeping speech off the GPU/unified generative slot. + +**Decision gate**: Live default stays 008 CPU speech. Compose light-audio becomes the experimental default only if measured quality and headroom justify it. + +### Phase 2 – Model A/B Testing (single occupied slot) + +Living policy: **one** local generative LLM. Quality-critical fleet stays on hosted Grok. Local workers are ~30B-class with hosted fallback. Do **not** stand up 49B *alongside* 120B on one Spark. -**Decision gate**: If quality is acceptable and headroom improves significantly → adopt as new default. +A/B the **single** slot: -### Phase 2 – Model A/B Testing (49B vs Current 120B) -Stand up `llama-3.3-nemotron-super-49b-v1.5` alongside the 120B. +- Live option already: `openai_compatible` / Ollama ~30B-class vs hosted Grok (agent already routes via `LLM_PROVIDER` / `LLM_BASE_URL`). +- Optional experiment: `llama-3.3-nemotron-super-49b-v1.5` as the one loaded model (compose profile or override) — not a second concurrent NIM. +- 120B remains an optional labeled experiment, **not** a daily “deep mode” on one Spark (that *is* occupying the only slot with a forbidden-size loop). -- Add a second LLM service in compose (different port or profile). -- Make the agent configurable (env var or CLI flag) to point at different NIM endpoints. -- Run identical benchmark scenarios on both models (with the winning audio stack from Phase 1). -- Measure: latency (especially TTFT + full turn), memory headroom, subjective quality on memory-recall + household prompts, tool-calling reliability. +Measure: latency (TTFT + full turn), memory headroom (**unmeasured** until sampled), subjective quality on memory-recall + household prompts, tool-calling reliability. -**Decision gate**: Choose primary model (likely 49B for daily use, 120B as optional "deep" mode). +**Decision gate**: Confirm ~30B as the local-worker class, or promote 49B as the single-slot experiment winner. Do not lock 120B as optional always-on deep mode on one node. ### Phase 3 – Context & Memory Efficiency Tuning With the chosen model + audio, optimize how we use the remaining headroom. @@ -55,10 +91,11 @@ With the chosen model + audio, optimize how we use the remaining headroom. - Validate multi-user (2–4 concurrent simulated family members) stability. ### Phase 4 – Sweet-Spot Lock + Operationalization -- Update default `docker-compose.yml`, `.env` examples, and Makefile targets for the chosen configuration. +- Do **not** make 120B + Riva the default compose “production” profile. Defaults must match the living slot policy (CPU speech lives in the I/O repo; this package is `get_agent()`; local LLM is optional ~30B-class). +- Update default `docker-compose.yml`, `.env` examples, and Makefile targets only for configurations we actually intend to run, and mark experimental profiles as such. - Add documented "benchmark" and "profile" make targets. -- Update architecture docs and the 001 spec references. -- Create a "current sweet spot" section in the 007 directory with the final numbers and rationale. +- Update architecture docs and the 001 spec references so they do not re-introduce the old production framing. +- Create a "current sweet spot" section in the 007 directory with the final numbers and rationale — or an explicit unmeasured label. ### Phase 5 – 2× DGX Spark Preparation (Future, After Phase 4) - Design multi-node compose / orchestration approach (tensor-parallel for 340B or service separation). @@ -103,7 +140,7 @@ These will be thin wrappers that set the right compose profiles + env and invoke ## 4. Docker & Deployment Changes -- Keep the existing `docker-compose.yml` as the "current baseline" reference. +- Keep the existing `docker-compose.yml` as the **experimental compose** reference (agent + riva + nemotron 120b). It is **not** the live spoken path and **not** the production baseline. - Introduce compose profiles or override files: - `docker-compose.light-audio.yml` - `docker-compose.49b.yml` @@ -121,7 +158,7 @@ After each major phase we will: 3. Update the decision matrix in the spec (or a living `decision-log.md`). 4. Hold a quick "gate" discussion (even async via PR comment or the issue tracker) before proceeding to the next phase. -No optimization change lands in the default compose without passing through this measured gate. +No optimization change lands in the default compose without passing through this measured gate. Do not land a 120B+ daily loop on one Spark even if a report looks flattering — that violates the living slot policy. ## 6. Risks & Mitigations @@ -149,4 +186,4 @@ This keeps the loop tight and the excitement high. --- -**Ready to cut.** Once the spec PR is reviewed/merged, we will land the first pieces of the harness and capture the all-important baseline numbers on the actual DGX Spark hardware. \ No newline at end of file +**Ready to measure the live slot.** Harness work should start from the desk path (008 + `get_agent()`), not from a fictional 120B + Riva production stack. Experimental compose numbers stay optional and labeled unmeasured until captured. \ No newline at end of file diff --git a/specs/007-dgx-hardware-optimization/spec.md b/specs/007-dgx-hardware-optimization/spec.md index c140f9c..2b95880 100644 --- a/specs/007-dgx-hardware-optimization/spec.md +++ b/specs/007-dgx-hardware-optimization/spec.md @@ -1,8 +1,8 @@ # Spec: DGX Spark Hardware Optimization & Sweet-Spot Strategy **Feature ID**: 007-dgx-hardware-optimization -**Status**: Draft / Strategy & Benchmarking Spec -**Related to**: 001-voice-dgx-spark-agent, 002-multi-user-support, 003-deployment-infrastructure +**Status**: Draft / Strategy & Benchmarking Spec (living slot policy added 2026-08-19) +**Related to**: 001-voice-dgx-spark-agent, 002-multi-user-support, 003-deployment-infrastructure, 008-local-tts-lenovo-go-spike **Created**: 2025-05-22 **Branch**: `feat/007-dgx-hardware-optimization` @@ -10,7 +10,13 @@ We have reached the point where we need a deliberate, measurable optimization strategy for the voice-first LangGraph + Supermemory agent running on NVIDIA DGX Spark hardware (single node today, with an eye toward 2× DGX Spark). -Current production configuration (as of Feature 001): +**Honest live path (what actually runs on the desk today — not the 001 compose hypothesis):** +- Desk voice I/O: [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) (spec 008) — Parakeet TDT 0.6B CPU STT + Piper CPU TTS +- Brain: this repo’s `get_agent()`; default LLM is hosted Grok (xAI); local option is `openai_compatible` / Ollama ~30B-class with hosted fallback +- `docker-compose` in this repo (`agent` + `riva` + `nemotron` 120b) is **experimental**, not the live spoken path +- See [Inference slot policy (living)](#inference-slot-policy-living) for the one-local-LLM budget and fleet-role split + +**Original 001 compose hypothesis (experimental / unmeasured on this Spark — do not read as “what we run today”):** - LLM: `nvcr.io/nim/nvidia/nemotron-3-super-120b-a12b:latest` (120B hybrid MoE/Mamba, ~12B active params, 1M native context) - Voice: Full NVIDIA Riva (NeMo ASR + TTS) via gRPC sidecar - Agent: LangGraph StateGraph with proactive Supermemory injection + reactive memory tools @@ -37,35 +43,99 @@ Key characteristics that drive every optimization decision: - Networking: 10 GbE + dual 100/200 GbE ConnectX-7 (RDMA capable) - Power/thermals: ~140 W SoC, ~240 W PSU, designed for quiet/home-lab operation -**Implication**: Every added service (Riva, larger model, longer context, multiple concurrent family conversations) directly reduces headroom for the "rest of the app" (LangGraph execution, Supermemory client calls, VAD, playback, future tools). +**Implication**: Every added service (Riva, larger model, longer context, multiple concurrent household conversations) directly reduces headroom for the "rest of the app" (LangGraph execution, Supermemory client calls, VAD, playback, future tools). **One serious local generative LLM at a time** on this chip; see the living slot policy below. + +## Inference slot policy (living) + +This section is the operating policy for the single GB10. It **corrects** earlier 007/001 wording that treated 120B NIM + full Riva as the live spoken path. That stack remains a valid *experiment* if we ever want numbers for it; it is not what we run today. + +### Hardware budget (qualitative — no invented GB figures) + +- GB10: ~128 GB unified LPDDR5X, ~273 GB/s, coherent CPU + GPU. There is no separate VRAM. +- **One serious local LLM at a time.** Do not run 120B+ agent loops on one Spark. +- Prefer keeping the GPU / unified slot for **at most one** local generative LLM. +- STT/TTS on CPU is a **deliberate** budget choice: Parakeet TDT 0.6B (CPU) and Piper (CPU) leave the slot free for a ~30B-class worker, or empty while hosted Grok does the turn. + +Measured NIM + Riva footprints on this Spark are **still unmeasured**. Ranges elsewhere in this spec (40–70 GB, 10–25 GB, etc.) stay **engineering estimates / TBD**, not inventory. + +### Honest live path + +| Layer | What actually runs | +|-------|-------------------| +| Voice I/O | spec 008 / `conversational-voice-agent`: Parakeet TDT 0.6B CPU STT + Piper CPU TTS | +| Brain | this repo `get_agent()` | +| Default LLM | hosted Grok (xAI) | +| Local LLM option | `openai_compatible` / Ollama, ~30B-class, hosted fallback if the slot is busy or the local endpoint is down | +| This repo `docker-compose` (`agent` + `riva` + `nemotron` 120b) | experimental; **not** the live spoken path | + +### Fleet roles vs the slot + +Quality-critical roles **do not** occupy the local slot — they use hosted Grok so they never fight a worker for unified memory: + +- orchestrator, architect, reviewer, design → hosted Grok + +Local workers **may** occupy the single slot, with hosted fallback: + +- coder, researcher → ~30B-class local (`openai_compatible` / Ollama) + +When the slot is occupied, other work uses hosted models. Do not co-schedule a second large local LLM. Workstation fleet ops live in Hermes (`~/.hermes/docs/agentic-workflow.md`); this spec only owns the **memory-budget** rule. + +### Policy rules -## Current Baseline (What We Are Running Today) +1. At most one local generative LLM loaded on the Spark. +2. No 120B+ (or 340B) agent loops on a single Spark. +3. Keep STT/TTS on CPU unless a measured experiment shows GPU speech still leaves the generative slot intact. +4. Treat compose 120B + full Riva as an optional harness target, not production. +5. Do not publish GB “we use X GB today” numbers until a 007 report lands them. -- **LLM**: nemotron-3-super-120b-a12b (120B total / ~12–12.7B active per token via hybrid MoE + Mamba). Excellent agentic/tool-calling and long-context reasoning — ideal for our Supermemory injection + multi-turn household conversations. +### What this spec still measures + +The rest of 007 (baseline harness, light-audio vs full Riva, 49B vs 120B A/B, 2× Spark) remains useful **experiment design**. Those runs are gated on real hardware numbers. They are **not** a claim that the experimental stack is the daily driver. + +## Current baseline (live vs experimental) + +### Live desk path (practice today) + +- **Voice**: Parakeet TDT 0.6B via NeMo on CPU + Piper CPU TTS (spec 008). +- **Brain**: `get_agent()` in this package. +- **LLM**: hosted Grok by default; optional local ~30B-class via `openai_compatible` / Ollama. +- **Slot**: CPU speech; GPU/unified reserved for at most one ~30B-class worker (or idle). + +We do **not** yet have a committed 007 harness report for this live path’s unified-memory samples either. Latency and quality notes belong in 008 / the I/O repo until a 007 report exists. + +### Experimental compose stack (001 hypothesis — unmeasured) + +If we stand up this repo’s compose on the Spark, the intended services are: + +- **LLM**: nemotron-3-super-120b-a12b (120B total / ~12–12.7B active per token via hybrid MoE + Mamba). - **Context**: Native 1M tokens (practical NIM limits often 128K–256K depending on profile and KV precision). - **Voice**: Full Riva stack (Parakeet-class ASR + high-quality TTS, multilingual capable). -- **Expected characteristics** (to be validated on hardware): - - Model load + idle memory: Significant fraction of 128 GB (exact TBD via NIM profile). +- **Expected characteristics** (**unmeasured** on this Spark; exact TBD via NIM profile + `nvidia-smi` / container stats): + - Model load + idle memory: Significant fraction of 128 GB (exact TBD). - Real-time voice turn latency (end-of-speech → first audio out): Target sub-second natural feel. - - Concurrent family users: Currently designed for single primary user; multi-user will increase memory pressure. + - Concurrent household users: Originally designed for a single primary user; multi-user will increase memory pressure. -We do **not** yet have hard numbers on this exact DGX Spark + Docker + Riva + 120B combination. This spec exists to create those numbers systematically. +We do **not** have hard numbers on this exact DGX Spark + Docker + Riva + 120B combination. This spec still exists to create those numbers **if** we run that experiment. Do not treat the 120B + Riva row as the current daily driver. ## Model Comparison | Model | Params (Total / Active) | Context | Architecture | Expected Footprint (Single Spark) | Strengths for Our Use Case | Weaknesses / Risks | Voice Latency Impact | |-------|--------------------------|---------|--------------|-----------------------------------|----------------------------|--------------------|----------------------| -| **nemotron-3-super-120b-a12b** (current) | 120B / ~12B active | 1M native (NIM ~128–256K practical) | Hybrid Mamba + Transformer MoE | High (but runnable per community reports; tight with Riva + long ctx) | Best agentic reasoning, tool use, long-horizon memory recall, retains large Supermemory context without constant re-fetch | Highest memory/latency of the three practical options; risk of swapping under load | Higher TTFT + decode latency vs lighter models | +| **nemotron-3-super-120b-a12b** (experimental compose; **not** the live daily driver) | 120B / ~12B active | 1M native (NIM ~128–256K practical) | Hybrid Mamba + Transformer MoE | High (community reports; tight with Riva + long ctx). **Unmeasured** on this Spark. **Forbidden as a daily agent loop** on one Spark under the living slot policy. | Best agentic reasoning, tool use, long-horizon memory recall, retains large Supermemory context without constant re-fetch | Occupies the only local slot; fights voice/fleet workers; risk of swapping under load | Higher TTFT + decode latency vs lighter models | | **llama-3.3-nemotron-super-49b-v1.5** (strong candidate) | 49B dense | 128K | NAS-optimized dense Transformer | Medium (comfortable headroom on single Spark) | Excellent accuracy/efficiency; fast tokens/s; proven on H100-class; lower latency, more room for Riva or concurrent users | Smaller context than 120B (may require more aggressive summarization) | Best-in-class for its size; fastest turn times of the three | | **nemotron-4-340b-instruct** ("big boi") | 340B dense | 4K native (extendable) | Dense Transformer | Impractical on single node even heavily quantized; feasible on 2× via tensor-parallel | Maximum raw intelligence and instruction following; potential "reasoning brain" for hardest queries | Enormous memory (hundreds of GB raw); high latency even sharded; overkill for most voice turns | Significantly higher latency; best used selectively or for offline tasks | -**Recommendation for primary inference path**: Start with the 49B v1.5 as the default "daily driver" for voice responsiveness while keeping the 120B as an optional "deep thinker" that can be swapped in for complex multi-step planning or heavy memory synthesis. +**Recommendation for primary inference path (updated by the living slot policy)**: Live daily driver is **hosted Grok** for quality-critical work and **at most one ~30B-class local worker** (coder / researcher) when we want on-box generation. The 49B v1.5 remains a strong *experiment* if we measure a single-slot local voice/brain. The 120B is **not** an optional always-on “deep thinker” on one Spark — swapping it in *is* occupying the only local slot, and the policy forbids 120B+ agent loops on a single node. 340B stays multi-node-only. ## Audio Stack: Can We Axe Riva? -Current: Full Riva (enterprise-grade, multi-language, multiple models for ASR + TTS). +**Live path already did, on CPU.** Spec 008 / `conversational-voice-agent` uses Parakeet TDT 0.6B (NeMo, CPU) + Piper CPU TTS. That is the desk spoken loop. Full Riva in this repo’s compose is an experimental sidecar, not production audio. + +The remainder of this section is still useful as an experiment design **if** we ever bring Riva (or a GPU speech NIM) onto the Spark next to a local LLM. -For an **English-only household** (Derek + family), the multilingual enterprise features are mostly wasted. +**Experimental compose “current”**: Full Riva (enterprise-grade, multi-language, multiple models for ASR + TTS). + +For an **English-only household**, the multilingual enterprise features are mostly wasted. **Lighter English-only alternatives**: - Pin to **Parakeet 1.1B CTC English** (or smaller 0.6B variants) via dedicated lightweight ASR NIM or direct NeMo inference — typically 2–8 GB for real-time streaming. @@ -73,7 +143,7 @@ For an **English-only household** (Derek + family), the multilingual enterprise - Total audio stack: **4–12 GB** instead of 10–25+ GB for full Riva. **Benefits of dropping full Riva**: -- Reclaim 8–15+ GB of unified memory → directly usable for larger KV cache (longer effective context), higher quality model, or concurrent family sessions. +- Reclaim 8–15+ GB of unified memory (**estimate, unmeasured**) → directly usable for larger KV cache (longer effective context), a higher-quality single local model, or leaving the slot free. - Simpler deployment (fewer sidecars, smaller attack surface, faster startup). - Lower CPU/GPU contention during voice turns. @@ -82,11 +152,11 @@ For an **English-only household** (Derek + family), the multilingual enterprise - Must validate English quality and latency of the lighter path (Parakeet CTC is already very strong for English). - Potential future desire for "voice cloning" or multiple family voices — still doable with lighter dedicated voices. -**Conclusion**: Yes — for the English-only family voice agent we can (and probably should) replace full Riva with a minimal English Parakeet + English TTS profile (or emerging smaller NVIDIA speech NIMs). This is one of the highest-leverage single changes available today. +**Conclusion**: For the English-only household voice agent we already run a minimal CPU Parakeet + Piper path (spec 008). If the compose experiment is revived, prefer that same “light English audio” posture over full Riva so the unified slot stays with at most one generative LLM. GPU speech NIMs are an experiment, not a default, until measured. ## Headroom Analysis (Single DGX Spark, 128 GB Unified) -Rough engineering estimates (to be replaced by measured data): +Rough engineering estimates (**unmeasured** on this Spark — to be replaced by harness reports; do not treat as live inventory): **Always-present baseline**: - OS + Docker + non-root agent container + Python + sounddevice + VAD + Supermemory client + LangGraph overhead + checkpointers: **8–15 GB** @@ -101,11 +171,11 @@ Rough engineering estimates (to be replaced by measured data): - English-only Parakeet + TTS: **4–12 GB** **Headroom for "the rest of the app"** (reactive tool calls, memory injection, future vision/tools, burst concurrency): -- Current 120B + full Riva: **Very tight** (often <10–15 GB free under load). Risk of OOM, swapping, or forced context truncation during long family conversations. -- 49B + lighter audio: **Healthy headroom** (20–40+ GB free) → room for 2–4 concurrent family members, longer context windows, or future capabilities. -- 120B + lighter audio: **Recoverable** — may be the pragmatic sweet spot for intelligence + responsiveness. +- Experimental 120B + full Riva: **Very tight** (estimate, often <10–15 GB free under load — **unmeasured**). Risk of OOM, swapping, or forced context truncation during long household conversations. **Not** the live daily driver. +- 49B + lighter audio: **Healthy headroom** (20–40+ GB free, **unmeasured**) → more room for longer context or future capabilities if that single slot is occupied by 49B rather than 120B. +- 120B + lighter audio: **Recoverable** vs full Riva (**unmeasured**) — still a 120B+ loop on one Spark, so **not** a living-policy daily driver even if headroom improves. -**Key insight**: The biggest single lever for headroom today is **replacing full Riva with an English-only lightweight audio path**. The second biggest is **model choice** (49B vs 120B MoE). +**Key insight (living policy)**: The biggest lever already in practice is **not occupying the GPU/unified slot with speech** (CPU Parakeet + Piper) and **not loading a second local LLM**. The next lever, if we revive compose experiments, is still **not running full Riva next to a large NIM**, then **model class** (~30B worker vs 49B experiment vs 120B — the last is forbidden as a daily loop on one Spark). ## Multi-Node (2× DGX Spark) Projections @@ -144,11 +214,19 @@ We will establish a repeatable benchmark harness before making major changes. - **Thermals / power / noise**: Important for a living-room/home device. ### Baseline Capture (First Experiment on Current Stack) -1. Single DGX Spark, current 120B + full Riva Docker Compose. -2. Clean boot, measure idle memory. -3. Run scripted voice sessions (single user, then 2–3 overlapping). -4. Capture all metrics above + full `nvidia-smi` / container memory + system logs. -5. Document exact NIM profiles, quantization settings, Riva config, and context management strategy used. + +Two different “baselines” — do not collapse them: + +**A. Live path (practice; still needs a 007 report):** spec 008 I/O + `get_agent()` + hosted Grok and/or one ~30B-class `openai_compatible` worker. CPU STT/TTS. This is what the desk actually runs. + +**B. Experimental compose (001 hypothesis; unmeasured):** single DGX Spark, this repo’s `docker-compose.yml` with 120B NIM + full Riva. Optional harness target only. + +If we run **B**: +1. Clean boot, measure idle memory. +2. Run scripted voice sessions (single user, then 2–3 overlapping). +3. Capture all metrics above + full `nvidia-smi` / container memory + system logs. +4. Document exact NIM profiles, quantization settings, Riva config, and context management strategy used. +5. Label the report experimental — not “production baseline.” ### Subsequent Experiments (Compare Against Baseline) - 120B + lighter English audio only @@ -162,35 +240,41 @@ Every change must be accompanied by before/after numbers against the baseline. " Primary levers (ranked by expected impact on single-node headroom + latency): -1. **Audio stack reduction** (full Riva → English Parakeet + TTS): Highest immediate win. -2. **Model swap** (120B MoE → 49B dense): Large win on latency and headroom; acceptable quality trade for most turns. -3. **Context strategy** (aggressive summarization + proactive injection vs. raw long context): Reduces KV pressure and improves recall quality. -4. **Quantization / NIM profile tuning**: FP8, NVFP4, lower KV precision where quality allows. -5. **Concurrency limits & backpressure**: Limit parallel family sessions or queue intelligently. -6. **Process placement** (future): Move audio to a dedicated lightweight container or even separate node. -7. **2× node scaling**: When single-node sweet spot is exhausted. +1. **Do not fight the slot** (living policy, already practice): one local generative LLM; STT/TTS on CPU; quality-critical fleet on hosted Grok. +2. **Audio stack reduction** (full Riva → English Parakeet + TTS): Highest immediate win *if* compose/Riva is revived; live path already uses CPU Parakeet + Piper. +3. **Model class** (~30B local worker vs 49B experiment vs 120B): 120B is not a daily loop on one Spark. 49B remains an A/B candidate for a *single* occupied slot. +4. **Context strategy** (aggressive summarization + proactive injection vs. raw long context): Reduces KV pressure and improves recall quality. +5. **Quantization / NIM profile tuning**: FP8, NVFP4, lower KV precision where quality allows. +6. **Concurrency limits & backpressure**: Limit parallel household sessions or queue intelligently. +7. **Process placement** (future): Move audio to a dedicated lightweight container or even separate node. +8. **2× node scaling**: When single-node sweet spot is exhausted — including when we want a second local LLM. ### Decision Matrix (Example) | Configuration | Expected Headroom | Expected Voice Latency | Intelligence Level | Multi-User Comfort | Recommendation | |---------------|-------------------|------------------------|--------------------|--------------------|----------------| -| 120B + Full Riva | Low | Medium-High | Highest | Poor | Baseline only; optimize away | -| 120B + Light Audio | Medium | Medium | Highest | Good | Strong candidate if quality holds | -| 49B + Light Audio | High | Lowest | Very High | Excellent | Default daily driver target | -| 340B (2× sharded) | N/A (multi-node) | High | Maximum | Excellent | On-demand specialist brain | +| Hosted Grok + CPU Parakeet/Piper (live) | Slot free or idle (**unmeasured** GB) | Dominated by network LLM + CPU speech | Highest for quality-critical roles | N/A (hosted) | **Live default** for orchestrator / architect / reviewer / design and for voice when no local worker is loaded | +| ~30B local worker + CPU Parakeet/Piper (live option) | Occupies the one local slot (**unmeasured** GB) | Local TTFT + CPU speech | Good for coder / researcher | One local LLM only | **Live local option** with hosted fallback; do not co-schedule a second LLM | +| 120B + Full Riva (experimental compose) | Low (**unmeasured**) | Medium-High | Highest | Poor | Experiment only; **not** live production; forbidden as a daily agent loop on one Spark | +| 120B + Light Audio | Medium (**unmeasured**) | Medium | Highest | Good | Experiment only; still a 120B+ loop on one Spark — policy says no | +| 49B + Light Audio | High (**unmeasured**) | Lowest | Very High | Excellent | Strong *single-slot* experiment; not claimed as measured sweet spot | +| 340B (2× sharded) | N/A (multi-node) | High | Maximum | Excellent | On-demand specialist brain; never a single-Spark daily loop | ## Recommended Path to Sweet Spot (Single Node First) -1. **Immediate (this branch / next sprint)**: Capture rigorous baseline on current 120B + Riva. -2. **High-leverage experiment**: Replace Riva with minimal English audio stack; re-benchmark. -3. **Model A/B**: Stand up 49B v1.5 side-by-side; measure latency + memory + subjective quality on household-style prompts + memory recall tasks. -4. **Context tuning**: Implement or improve summarization + injection strategy; measure impact on effective memory quality vs. KV usage. -5. **Decision gate**: Choose primary model + audio stack for the family deployment based on data. -6. **2× node phase**: Once single-node sweet spot is locked and we need more (concurrency, 340B, or future capabilities), move to multi-node architecture. +1. **Immediate (living policy — already practice, document it)**: Treat the live path as 008 CPU speech + `get_agent()` + hosted Grok / one ~30B local worker. Do not call 120B + Riva “production.” +2. **Measure the live slot** (still open): idle vs one ~30B worker vs CPU STT/TTS; no invented GB figures until a report exists. +3. **Optional compose experiment**: If we want numbers, capture a labeled *experimental* baseline on 120B + Riva — not a production baseline. +4. **High-leverage experiment (compose only)**: Replace Riva with minimal English audio; re-benchmark. Live path already did the CPU version. +5. **Model A/B**: ~30B worker vs 49B v1.5 as the *single* occupied slot; 120B is not a daily-driver candidate on one Spark. +6. **Context tuning**: summarization + injection; measure recall quality vs. KV usage on the chosen single local model. +7. **Decision gate**: confirm the living slot policy with measured numbers (or keep it as practice-without-numbers, labeled unmeasured). +8. **2× node phase**: Once the single-node slot is understood and we need more (concurrency, 340B, or a second local LLM), move to multi-node architecture. ## Success Criteria -- We have a documented, reproducible benchmark baseline for the current stack. +- The living slot policy is written down and matches the desk (CPU speech, one local LLM, hosted Grok for quality-critical roles). +- We have a documented, reproducible benchmark baseline for the **live** stack, and (optionally) a clearly labeled experimental 120B + Riva report. Neither is claimed without a report. - We have measured data (not guesses) for at least two alternative configurations (light audio, 49B model). - We can articulate "the sweet spot" with numbers: model choice, audio stack, max comfortable context, max concurrent users, and expected voice turn latency. - The chosen configuration leaves **measurable, comfortable headroom** (>15–20 GB) for the agent, memory system, and future features under typical household load. @@ -198,7 +282,8 @@ Primary levers (ranked by expected impact on single-node headroom + latency): ## Open Questions & Risks -- Exact real-world memory footprint of the 120B NIM on DGX Spark unified memory (with our Docker setup) — highest priority unknown. +- Exact real-world memory footprint of the 120B NIM on DGX Spark unified memory (with our Docker setup) — **unmeasured**; optional experiment, not a live-path unknown. +- Exact real-world footprint of one ~30B-class Ollama / `openai_compatible` worker plus CPU Parakeet + Piper — **unmeasured** (practice exists; 007 report does not). - Quality delta between full Riva voices and lighter English TTS options for family members (subjective but important). - Whether 128K context on the 49B is "enough" given our Supermemory + summarization strategy, or whether we will miss the 1M capability. - Practical limits of 2-node RDMA tensor-parallel for the 340B in a home setting (latency, stability, complexity). @@ -207,14 +292,14 @@ Primary levers (ranked by expected impact on single-node headroom + latency): ## Next Steps -1. Review and approve this spec (user + team). -2. Create `plan.md` and `tasks.md` under `specs/007-dgx-hardware-optimization/` following the established SDD process. -3. Implement the benchmark harness + first baseline capture. -4. Run the high-leverage experiments (audio reduction, model comparison). -5. Iterate to the sweet spot with data in hand. +1. Keep the living slot policy in sync with the desk (this section is the source of truth for “what occupies the Spark”). +2. `plan.md` / `tasks.md` in this folder already exist; extend them when adding harness work — do not re-open 120B + Riva as the implied production path. +3. Implement the benchmark harness and capture a **live-path** report (CPU speech + hosted Grok and/or one ~30B worker). NIM + Riva GB numbers stay `[ ]` until measured. +4. Optional: labeled experimental compose runs (audio reduction, 49B vs 30B). No 120B+ daily loops. +5. Iterate to a measured sweet spot; until then, practice follows the living policy and estimates stay labeled unmeasured. --- -**We are getting there.** This spec gives us the map, the measuring stick, and the decision framework so we can move from "it works" to "it is optimal under our real constraints" with confidence and excitement. +**We are getting there.** This spec gives us the map, the measuring stick, and the decision framework so we can move from "it works" to "it is optimal under our real constraints" with confidence and excitement. The living slot policy is how we use the one local generative slot **today**; the rest of the document is how we measure experiments without pretending they are production. -**Status**: Ready for review. Once approved, we will commit the spec and proceed to planning the concrete benchmarking and optimization work. \ No newline at end of file +**Status**: Living policy in effect. Benchmark numbers for NIM + Riva (and for the live 30B worker) remain unmeasured until a 007 report lands. \ No newline at end of file diff --git a/specs/007-dgx-hardware-optimization/tasks.md b/specs/007-dgx-hardware-optimization/tasks.md index a20bf90..b4a947c 100644 --- a/specs/007-dgx-hardware-optimization/tasks.md +++ b/specs/007-dgx-hardware-optimization/tasks.md @@ -3,12 +3,42 @@ **Feature**: 007-dgx-hardware-optimization **Related Spec**: [spec.md](./spec.md) **Related Plan**: [plan.md](./plan.md) -**Status**: Ready for implementation +**Status**: Ready for implementation (living slot policy recorded 2026-08-19) **Branch**: `feat/007-dgx-hardware-optimization-impl` -This document breaks the work into small, dependency-ordered, checkable tasks. The first priority is always **capturing a trustworthy baseline** before any changes. +This document breaks the work into small, dependency-ordered, checkable tasks. The first priority is always **capturing a trustworthy baseline** before any changes — and labeling **live** vs **experimental-compose**. Do not treat 120B NIM + full Riva as the production spoken path. -Mark tasks complete only after the work is committed and (where applicable) the corresponding benchmark report is added. +Mark tasks complete only after the work is committed and (where applicable) the corresponding benchmark report is added. Policy-practice items below may be `[x]` without a 007 report when they already match the desk; **memory-number** items stay `[ ]` until measured. + +--- + +## Inference slot policy (living) — practice vs unmeasured + +Canonical text: [spec.md — Inference slot policy (living)](./spec.md#inference-slot-policy-living). + +### T-SLOT.1 – Honest live path (already practice) + +- [x] Desk voice I/O is spec 008 / `conversational-voice-agent`: Parakeet TDT 0.6B CPU STT + Piper CPU TTS (not this repo’s Riva orchestrator). +- [x] Brain is this repo `get_agent()`; default LLM is hosted Grok; local option is `openai_compatible` / Ollama ~30B-class with hosted fallback. +- [x] This repo’s `docker-compose` (`agent` + `riva` + `nemotron` 120b) is documented as experimental, not the live spoken path. +- [x] Policy recorded: GB10 ~128 GB unified / ~273 GB/s; **one serious local LLM at a time**; no 120B+ agent loops on one Spark. +- [x] STT/TTS on CPU is an explicit budget choice so the GPU/unified slot stays with at most one generative LLM. + +### T-SLOT.2 – Fleet vs slot (already practice) + +- [x] Quality-critical roles (orchestrator, architect, reviewer, design) use hosted Grok and do not occupy the local slot. +- [x] Local workers (coder, researcher) use ~30B-class with hosted fallback. +- [x] Do not co-schedule a second large local LLM next to an occupied slot. + +### T-SLOT.3 – Unmeasured NIM + Riva (and live-slot) memory numbers + +Do **not** invent GB figures. Leave TBD until a harness report exists. + +- [ ] Measured idle + load unified-memory footprint of the 120B NIM on this Spark (experimental compose). +- [ ] Measured full Riva ASR+TTS unified-memory footprint on this Spark (experimental compose). +- [ ] Measured peak for 120B NIM + full Riva + agent during a voice turn (experimental compose). +- [ ] Measured idle vs occupied-slot samples for one ~30B-class `openai_compatible` / Ollama worker plus CPU Parakeet + Piper (live path). +- [ ] Written 007 report that labels live vs experimental-compose and does not call 120B + Riva “production.” --- @@ -37,23 +67,32 @@ Mark tasks complete only after the work is committed and (where applicable) the - System RAM / swap. - [ ] Integrate sampling into the benchmark runner for the duration of a run. -### T0.5 – Baseline Run on DGX (Current Stack) -- [ ] On clean DGX Spark, pull the exact current images (120B + full Riva). -- [ ] Run the benchmark harness against the existing `docker-compose.yml`. +### T0.5a – Live-path baseline (as-is desk; priority) +- [ ] On the Spark, sample the spec 008 I/O loop + `get_agent()` with hosted Grok (slot idle). +- [ ] Repeat with one ~30B-class local worker occupying the slot (hosted fallback still configured). +- [ ] Execute at least one short-turns session and one longer household conversation on the live voice path. +- [ ] Capture whatever metrics the harness can take (do not invent GB figures). +- [ ] Commit as `benchmarks/reports/YYYY-MM-DD-baseline-live-cpu-speech/` (with `summary.md` + `raw/`), labeled **live**. + +### T0.5b – Experimental compose run (120B + Riva; not production) +- [ ] On clean DGX Spark, pull the compose images (120B NIM + full Riva). Optional experiment only. +- [ ] Run the benchmark harness against this repo’s `docker-compose.yml`. - [ ] Execute at least one "short turns" session and one "long household conversation" session. - [ ] Capture full report (metrics, logs, nvidia-smi samples, container stats). -- [ ] Commit the report as `benchmarks/reports/2025-05-XX-baseline-120b-riva/` (with `summary.md` + `raw/`). +- [ ] Commit as `benchmarks/reports/YYYY-MM-DD-experimental-120b-riva/` (with `summary.md` + `raw/`), labeled **experimental-compose**, never “production baseline.” ### T0.6 – Baseline Documentation -- [ ] Write `specs/007-dgx-hardware-optimization/results/phase-0-baseline.md` summarizing the measured numbers against the expectations in the spec. -- [ ] Update the decision matrix in the spec (or a living `decision-log.md`) with actual data. +- [ ] Write `specs/007-dgx-hardware-optimization/results/phase-0-baseline.md` summarizing measured numbers against the spec. Separate live vs experimental. Leave NIM + Riva GB as **unmeasured** if T0.5b has not run. +- [ ] Update the decision matrix in the spec (or a living `decision-log.md`) with actual data, or keep estimates labeled unmeasured. --- -## Phase 1 – Audio Stack Reduction +## Phase 1 – Audio Stack Reduction (compose experiment) + +Live path already uses CPU Parakeet + Piper (T-SLOT.1). Phase 1 is only if we revive Riva/NIM speech in compose. ### T1.1 – Lightweight English Audio Service Definition -- [ ] Research and select the exact lighter image(s): Parakeet English CTC NIM (or equivalent small ASR) + English TTS. +- [ ] Research and select the exact lighter image(s): Parakeet English CTC NIM (or equivalent small ASR) + English TTS. Prefer not occupying the generative GPU slot. - [ ] Create `docker-compose.light-audio.yml` (or profile) that replaces the full Riva service with the slimmed version. - [ ] Document exact tags, ports, and healthcheck expectations in the compose file and a small `audio-profiles.md`. @@ -67,18 +106,20 @@ Mark tasks complete only after the work is committed and (where applicable) the - [ ] Produce report `benchmarks/reports/...-light-audio/`. ### T1.4 – Phase 1 Gate & Decision -- [ ] Compare memory headroom, voice turn latency (p50/p95), and subjective quality. +- [ ] Compare memory headroom, voice turn latency (p50/p95), and subjective quality (**unmeasured** until T1.3). - [ ] Write `results/phase-1-audio-reduction.md`. -- [ ] Decision recorded: adopt light audio as new default (or keep full Riva). +- [ ] Decision recorded: live default remains 008 CPU speech; compose light-audio vs full Riva is experimental only. --- -## Phase 2 – Model Comparison (49B Candidate) +## Phase 2 – Model Comparison (single occupied slot) + +Do not load 49B *and* 120B at once. Living policy: one local generative LLM. -### T2.1 – Second LLM Service in Compose -- [ ] Add support for `llama-3.3-nemotron-super-49b-v1.5` (new service definition or override). -- [ ] Make the agent LLM endpoint configurable (`LLM_BASE_URL`, `LLM_MODEL` or similar) so we can point at different NIMs without code changes. -- [ ] Create `docker-compose.49b.yml` profile. +### T2.1 – Alternate LLM as the one slot occupant +- [x] Agent LLM endpoint already configurable (`LLM_PROVIDER`, `LLM_BASE_URL`, `LLM_MODEL`) for hosted Grok vs `openai_compatible` / Ollama. +- [ ] Add support for `llama-3.3-nemotron-super-49b-v1.5` as an **alternate** single-slot occupant (new service definition or override — not a concurrent second NIM). +- [ ] Create `docker-compose.49b.yml` profile, marked experimental. ### T2.2 – 49B Benchmark Runs - [ ] With the winning audio stack from Phase 1, run identical scenarios on the 49B model. @@ -86,8 +127,8 @@ Mark tasks complete only after the work is committed and (where applicable) the - [ ] Produce report and `results/phase-2-49b-comparison.md`. ### T2.3 – Phase 2 Gate -- [ ] Update decision matrix with real latency + headroom numbers. -- [ ] Lock primary model recommendation (49B daily driver + 120B optional deep mode is the current hypothesis). +- [ ] Update decision matrix with real latency + headroom numbers, or keep **unmeasured**. +- [ ] Lock local-worker class (~30B with hosted fallback is the living-policy hypothesis). Do **not** lock 120B as optional always-on deep mode on one Spark. --- @@ -108,9 +149,10 @@ Mark tasks complete only after the work is committed and (where applicable) the ## Phase 4 – Sweet-Spot Operationalization ### T4.1 – Default Configuration Update -- [ ] Update the main `docker-compose.yml` (or make the winning profiles the easy defaults via env). +- [ ] Do not default compose to 120B + Riva as “production.” Experimental profiles stay named experimental. +- [ ] Update the main `docker-compose.yml` (or make the winning profiles the easy defaults via env) only for stacks we intend to run. - [ ] Add high-level `make` targets: `make benchmark`, `make profile-sweet-spot`, etc. -- [ ] Update `docs/development.md` and any DGX runbooks with the new recommended command sequence. +- [ ] Update `docs/development.md` and any DGX runbooks with the new recommended command sequence (live path = 008 I/O + `get_agent()`). ### T4.2 – Final Results Package - [ ] Write `results/final-sweet-spot.md` with the locked configuration, all key metrics, and rationale. @@ -139,6 +181,6 @@ Mark tasks complete only after the work is committed and (where applicable) the --- -**First actionable tasks**: T0.1 – T0.3 (get the harness skeleton + instrumentation in place) so that the very first DGX run (T0.5) produces trustworthy, comparable numbers. +**First actionable tasks**: T-SLOT is recorded. Harness skeleton remains T0.1 – T0.3 so the first DGX run (T0.5a live path, optional T0.5b experimental compose) produces trustworthy, comparable numbers. NIM + Riva GB items in T-SLOT.3 stay `[ ]` until measured. -Let's go get those baseline numbers! \ No newline at end of file +Let's go get those baseline numbers — and keep calling the live path the live path. \ No newline at end of file diff --git a/specs/009-architect-coder-handoff/plan.md b/specs/009-architect-coder-handoff/plan.md new file mode 100644 index 0000000..663d221 --- /dev/null +++ b/specs/009-architect-coder-handoff/plan.md @@ -0,0 +1,115 @@ +# Plan: Architect ↔ coder inter-agent handoff (009) + +**Feature**: 009-architect-coder-handoff +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-19 + +## 1. What this plan is + +A map of **who writes what, in which language, and in which order**. It is not a plan to add modules to `thelab-langchain`. Success is a practiced loop, not a merged feature flag. + +Living practice sits in Hermes profiles `dgx-architect` and `dgx-coder`. This repo only records the contract. + +## 2. Handoff flow + +``` +architect writes packet + │ + ▼ + human accept ──reject──► architect revises packet + │ accept + ▼ + coder implements accepted spec only + │ + ▼ + review against acceptance criteria + │ + ├── pass → close work; residual risks stay documented + └── fail or new blocker → architect (redesign) or coder (fix), + only inside the accepted spec +``` + +### Step 1 — Spec (architect) + +The architect writes the design. Human prose for goals, non-goals, and trade-offs. STE for any procedure the coder must run. The architect does not open an implementation branch as the architect. + +### Step 2 — Human accept + +A person reads the five artifacts. Accept means: the coder may implement **this** packet. Reject means: the architect revises. Chat agreement without the packet is not accept. + +### Step 3 — Implement (coder) + +The coder follows the kanban body and the spec. The coder does not add architecture. If the packet is wrong, the coder files a blocker and stops. The coder does not “fix the spec in the PR.” + +### Step 4 — Review + +A human (and optionally a reviewer agent) ticks acceptance criteria. Residual risks are not automatic fail. Unstated work is fail (scope creep) or a new spec, not a silent extra commit. + +## 3. Where text is STE vs human prose + +| Text | Language | Why | +|------|----------|-----| +| This SDD folder (`spec.md` / `plan.md` / `tasks.md`) | Human prose | Humans read rationale. STE is the wrong register for “why.” | +| Spec overview, goals, non-goals, relationships | Human prose | Design argument. | +| Spec procedures the coder must execute | STE | Agent-consumed. Skill `asd-ste100`. | +| Acceptance criteria | STE (one check per line) | Reviewer ticks; no synonyms. | +| Kanban body | STE | Primary coder input. Numbered lists for 3+ steps. | +| Blockers | STE facts | “X is missing.” Not “we should maybe wait.” | +| Residual risks | Human prose is allowed; names stay one-meaning | Risk needs context; do not hide it in hedges. | +| README / marketing / user docs | Human prose | STE is **not** marketing copy. | +| Review pass/fail lines | STE | “Criterion FR-2 fails. The factory is missing.” | +| Review narrative | Human prose | What was tried, what was out of scope. | +| Chat between humans | Human prose | Chat is not the packet. | + +The skill **`asd-ste100`** (agents skill `~/.agents/skills/asd-ste100`, Grok skill `asd-ste100`) is the procedure reference. Do not paste the skill body or the ASD dictionary into the packet. + +## 4. Packet shape (architect output) + +The architect produces, in one place the coder can fetch: + +1. **Spec** — design. Link or body. Seams named (`get_agent()`, editable install, env examples) without host home paths as required layout. +2. **Acceptance criteria** — tick list. No invented latency numbers. +3. **Kanban body** — STE steps. Points at (1) and (2). +4. **Blockers** — empty list is allowed if explicitly written as “none.” +5. **Residual risks** — empty list is allowed if explicitly written as “none.” + +“None” written is complete. A missing section is not. + +## 5. Coder constraints + +- Implement **accepted** specs only. +- Do not implement from an architect draft, a voice transcript, or a chat summary. +- Do not copy Hermes profile files into the worktree. +- Stop at the spec’s edge. Out-of-tree work (as in spec 008) stays out of tree; this package keeps the brain seam only. + +## 6. What this repo does and does not run + +| Mechanism | Status | +|-----------|--------| +| Hermes profiles `dgx-architect` / `dgx-coder` | Practiced on the workstation (outside this repo) | +| SDD record in `specs/009-architect-coder-handoff/` | This folder | +| CI lint of STE | **Not done** | +| CI that requires an accepted packet | **Not done** | +| Runtime enforcement in `thelab-langchain` | Out of scope | + +Do not add a STE linter, pre-commit hook, or GitHub Action under this spec. That would be a later spec, likely after 005, and it is not claimed here. + +## 7. Relationship to 001 and 008 + +- Work **toward** spec 001 still uses this handoff. 009 does not pick Riva vs Parakeet vs Piper. +- Spec 008 already ran out of tree. New I/O-repo work should arrive as an accepted packet, not as a paste of a Hermes wiki. + +## 8. Risks + +| Risk | Mitigation | +|------|------------| +| Chat replaces the packet | Human accept looks for the five artifacts. No packet → no implement. | +| Architect implements “a small fix” | Role rule: architect never implements. Small fixes still need a coder (or a human who is not wearing the architect role). | +| Coder redesigns in the PR | Review fails on unstated work. New design → architect + new accept. | +| STE used for README voice | Plan table: marketing and rationale stay prose. | +| Skill or dictionary copied into git | Spec forbids it. Reference `asd-ste100` by name. | +| This spec treated as a product feature | Status line: living practice only. | + +## 9. What this plan is not + +It is not a rewrite of spec 001. It is not the fleet operating manual. It is not a promise that CI will catch a missing handoff. diff --git a/specs/009-architect-coder-handoff/spec.md b/specs/009-architect-coder-handoff/spec.md new file mode 100644 index 0000000..ce4a4d9 --- /dev/null +++ b/specs/009-architect-coder-handoff/spec.md @@ -0,0 +1,173 @@ +# Feature Spec: Architect ↔ coder inter-agent handoff (STE) + +**Feature ID**: 009-architect-coder-handoff +**Status**: Living practice (documented here; **not** a product feature of `thelab-langchain`) +**Created**: 2026-08-19 +**Owner**: Derek +**Related**: [001-voice-dgx-spark-agent](../001-voice-dgx-spark-agent/spec.md), [008-local-tts-lenovo-go-spike](../008-local-tts-lenovo-go-spike/spec.md) + +## Record-keeping note + +This spec records a **workstation practice**: two Hermes profiles (`dgx-architect` and `dgx-coder`) hand work to each other through written artifacts. The practice is already in use. This folder is the SDD trail in the brain repo so later specs (and humans) can see the contract. + +It is **not** a library, CLI flag, LangGraph node, or CI gate in this package. Spec 008’s spike ran out of tree; this protocol is how design and implementation stay split when that kind of work happens again. Spec 001 remains the long-term voice-agent goal. This spec does not change 001’s stack. + +Do **not** treat Hermes profile files, skill bodies, or an ASD dictionary as part of this repo. Those live outside git. This document **distills** the design. + +## Overview + +An **architect** agent designs. A **coder** agent implements. A **human** accepts the design before code is written. The packet that crosses the gap is a small, named set of artifacts. Procedure text that an agent must follow is written in Simplified Technical English (ASD-STE100 **principles** — see rules of thumb below). Rationale, trade-offs, and user-facing prose stay in ordinary English. + +The full controlled-language skill is **`asd-ste100`**. It lives outside this repo (`~/.agents/skills/asd-ste100`, and the Grok skill of the same name). Reference the skill by name. Do not paste the skill body or the ASD dictionary into specs, PRs, or kanban text. + +## Goals + +- Keep design and implementation in different roles so the coder does not invent architecture. +- Make the handoff packet complete enough that a coder can start without a chat transcript. +- Put agent-consumed procedures in STE so instructions have one meaning. +- Leave a human accept gate between “designed” and “implement this.” +- Document residual risk instead of hiding it in chat. + +## Non-goals + +- A product feature, API, or runtime mode inside `thelab-langchain`. +- CI that lints STE, blocks merges, or assigns Hermes profiles (see [tasks.md](./tasks.md) — **not done**). +- The full workstation fleet manual (orchestrator, researcher, reviewer, kanban-vs-chat). That stays in Hermes docs, not this spec. +- Copying profile `SOUL.md`, Hermes env files, or the ASD-STE100 dictionary into git. +- Marketing copy, README voice, or human design rationale written as STE. +- Telephony, hosted voice, or unpublished-company product framing. + +## Roles + +| Role | Does | Does not | +|------|------|----------| +| **Architect** (`dgx-architect`) | Designs. Writes the spec, acceptance criteria, kanban body, blockers, and residual risks. Names seams and out-of-scope work. | Implement. Open an implementation PR. “Just quickly” patch production code. Expand into researcher or reviewer work. | +| **Coder** (`dgx-coder`) | Implements **accepted** specs only. Follows the kanban body and acceptance criteria. Reports new blockers. Stops at the spec’s edge. | Redesign. Implement from chat only. Widen scope. Silently drop acceptance checks. | +| **Human** | Accepts or rejects the design packet. Resolves product calls the architect flagged. Reviews the result against acceptance criteria (alone or with a reviewer). | Skip the accept gate “because the architect was confident.” | + +The architect never implements. The coder never treats an unaccepted draft as a build order. + +## Domain terms (define once) + +Use these words with one meaning in handoff artifacts: + +| Term | Meaning | +|------|---------| +| **Spec** | The design document the architect writes. It states what to build, what not to build, and which seams stay stable. | +| **Acceptance criteria** | Binary checks. The implementation passes or it fails. No “should feel faster.” | +| **Kanban body** | The work-item text the coder **consumes**. Procedure, not a status emoji. | +| **Blocker** | A condition that prevents start or completion. Named, owned, and either cleared or carried. | +| **Residual risk** | A known remaining risk after the human accepts the design. Not a surprise at review. | +| **Handoff** | The packet: spec + acceptance criteria + kanban body + blockers + residual risks, in the accepted state. | +| **Human accept** | The gate. A person marks the packet accepted. Only then may the coder implement. | +| **STE** | Simplified Technical English using ASD-STE100 principles (rules of thumb in this spec). The `asd-ste100` skill is the procedure reference. | + +Do not reuse these words for other meanings in the same packet (for example, do not call a brainstorm a “spec”). + +## Handoff artifacts + +Every architect → coder handoff includes all five. If one is missing, the packet is not ready for human accept. + +### 1. Spec + +- States goals, non-goals, seams, and the smallest change that meets the goal. +- Human prose is allowed for *why*. +- If the spec contains a procedure the coder must execute, that procedure is STE. + +### 2. Acceptance criteria + +- Written as checks a reviewer can tick. +- Each criterion is one testable outcome. +- No latent numbers, no invented latency targets, no “as before unless it is better.” + +### 3. Kanban body + +- The instruction the coder follows. +- STE. Numbered steps when there are three or more. +- Points at the spec and the acceptance criteria. Does not replace them. +- Does not embed secrets, tokens, host layout paths, or board/issue identifiers as required reading. + +### 4. Blockers + +- What must be true before implementation starts, or what will stop it mid-flight. +- Each blocker is a fact (missing seam, unaccepted dependency, out-of-tree repo not ready), not a mood. + +### 5. Residual risks + +- What remains wrong or fragile if the coder meets every acceptance criterion. +- The reviewer reads this list. The coder does not “fix” residual risk unless the accepted spec says so. + +## STE rules of thumb + +These are **principles** for agent-consumed procedures. They are not a substitute for the `asd-ste100` skill and not a copy of the ASD dictionary. + +1. **One meaning per word.** Pick a term from the table above (or define a new domain term **once**) and keep it. +2. **Active voice.** “The coder writes the factory.” Not “the factory should be written.” +3. **Simple tense.** Give instructions in the present or imperative. Do not stack conditionals. +4. **One instruction per sentence.** +5. **Short sentences.** Split a long sentence. +6. **Numbered lists for 3+ steps.** Do not hide a sequence in a paragraph. +7. **Noun clusters ≤ 3 words.** Prefer “checkpointer factory” to “optional session persistence checkpointer factory helper.” +8. **Define domain terms once.** Then use the defined word. + +STE is for **agent-consumed procedures** (kanban bodies, implementation steps, acceptance checks). It is **not** for marketing copy, README tone, or the “why we chose this” sections of a spec. + +## Functional requirements + +### FR-1 Role split + +- Architect output is design artifacts only. +- Coder input is an **accepted** handoff packet. +- Unaccepted drafts are not implementation tasks. + +### FR-2 Packet completeness + +- Human accept is refused if any of the five artifacts is missing or is a placeholder. +- The kanban body must not be the only copy of the spec. + +### FR-3 Language split + +- Procedures the coder or another agent must follow: STE, skill `asd-ste100`. +- Human rationale, status notes, and this SDD folder: ordinary prose. + +### FR-4 Stop conditions + +- The coder stops when acceptance criteria are met or a new blocker appears. +- Scope not in the spec is out of scope, including “obvious” refactors. + +### FR-5 Secrets and layout + +- Handoff text must not require API keys, tokens, hardware serials, board/chat identifiers, or absolute home-directory paths as the layout of record. +- Point to gitignored env examples and documented seams (`get_agent()`, provider factory) instead. + +## Non-functional requirements + +- The protocol is practiced in Hermes profiles `dgx-architect` and `dgx-coder` on the workstation. +- This repository **does not** enforce the protocol in CI. +- Specs in `specs/` remain human-readable SDD. They may *describe* STE; they need not be written entirely in STE. +- No latency or throughput numbers unless a later spec measures them. + +## User stories + +1. As architect, I hand a complete packet to a human so the coder never has to reconstruct the design from chat. +2. As human, I accept or reject before anyone writes production code. +3. As coder, I implement only what the accepted spec and kanban body say, in STE steps I can follow without guessing synonyms. +4. As reviewer, I tick acceptance criteria and read residual risks instead of rediscovering them. + +## Acceptance criteria (for this SDD record) + +- [x] This folder contains `spec.md`, `plan.md`, and `tasks.md` that name the two roles, the five artifacts, the STE rules of thumb, and the `asd-ste100` skill **by name only**. +- [x] Status is “living practice,” not a `thelab-langchain` feature. +- [ ] CI in this repo lint-checks STE or blocks coder PRs that lack an accepted packet — **not done** (out of scope until a later spec). +- [x] No Hermes profile file, skill body, or ASD dictionary is copied into this repo. + +## Relationship to other specs + +- **001** — long-term voice agent. Handoffs for work *on* 001 follow this protocol. This spec does not revise 001’s ASR/TTS/NIM choices. +- **008** — Lenovo Go spike, **executed out of tree**. The spike is the example of implementation living outside this package while the brain seam stays here. Future out-of-tree work should still cross this handoff, not a chat paste. +- **004 / 005** — checkpointers and CI remain their own specs. 009 does not implement them and does not claim CI enforcement. + +## Open questions + +- Whether a later spec should add a lightweight “packet complete?” checklist in PR templates (still not CI). +- Whether reviewer-agent output must also be STE. Default until decided: **acceptance write-up in human prose; fail/pass lines in STE.** diff --git a/specs/009-architect-coder-handoff/tasks.md b/specs/009-architect-coder-handoff/tasks.md new file mode 100644 index 0000000..8bd9993 --- /dev/null +++ b/specs/009-architect-coder-handoff/tasks.md @@ -0,0 +1,44 @@ +# Tasks: Architect ↔ coder inter-agent handoff (009) + +**Feature**: 009-architect-coder-handoff +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +Checkboxes are honest. The protocol is **practiced in Hermes**. This repo **does not** enforce it in CI. This folder is the SDD record, not a `thelab-langchain` feature. + +## Phase 0 — Record the practice (this repo) + +- [x] Write `spec.md` with roles, five handoff artifacts, STE rules of thumb, and skill name `asd-ste100` (no skill body, no ASD dictionary) +- [x] Write `plan.md` with flow spec → human accept → implement → review, and STE vs human prose +- [x] Write `tasks.md` (this file) +- [x] State status as living practice, not a product feature +- [x] Point at related specs 001 and 008 without revising their stack choices + +## Phase 1 — Workstation practice (Hermes, outside this repo) + +- [x] Architect profile `dgx-architect` designs and does not implement (living practice) +- [x] Coder profile `dgx-coder` implements accepted specs only (living practice) +- [x] Handoff packet in use: spec, acceptance criteria, kanban body, blockers, residual risks +- [x] STE reserved for agent-consumed procedures via skill `asd-ste100` (agents skill and Grok skill; not vendored here) + +Do not copy profile files, Hermes env files, or skill bodies into this tree to “complete” a checkbox. + +## Phase 2 — Enforcement in this repo (**not done**) + +- [ ] CI job that lints STE in kanban/spec procedures +- [ ] CI job that blocks implementation PRs without an accepted packet +- [ ] Pre-commit or ruff-like hook for noun-cluster / sentence rules +- [ ] Runtime or library support in `thelab-langchain` for architect/coder roles + +Phase 2 is **out of scope** for 009. Leave the boxes empty. Do not implement them under this spec. + +## Phase 3 — Optional later SDD hygiene (not required to call 009 done) + +- [ ] PR template checklist that names the five artifacts (docs only; still not CI) +- [ ] Decision on reviewer-agent output language (plan default: fail/pass in STE, narrative in prose) + +## Traceability + +- Practice: Hermes profiles `dgx-architect` and `dgx-coder` on the workstation. +- Skill: `asd-ste100` outside this repo. +- Spike that already ran out of tree: [008](../008-local-tts-lenovo-go-spike/tasks.md). +- This tasks file is only the checklist view. It does not claim CI or package enforcement. diff --git a/specs/010-worker-completion-protocol/plan.md b/specs/010-worker-completion-protocol/plan.md new file mode 100644 index 0000000..98e431e --- /dev/null +++ b/specs/010-worker-completion-protocol/plan.md @@ -0,0 +1,105 @@ +# Plan: Durable-board worker completion protocol (010) + +**Feature**: 010-worker-completion-protocol +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-19 + +## 1. Protocol vs runtime + +The spec is the durable contract. **Hermes Kanban is the current runtime** — an implementation detail, not the protocol. + +``` +Human + └─ orchestrator (front door + dispatcher) + └─ durable board + ├─ architect / researcher / designer (durable directory) + ├─ coder (git worktree → PR) + └─ reviewer (same tree as the change) +``` + +This Python package does not own that board, spawn workers, or persist cards. Do not add a board module here to “implement 010.” + +## 2. Mapping table (Hermes today) + +Abstract spec terms map onto the workstation fleet as follows. Profile *models* and how to start a gateway are out of scope; they live in the local Hermes operating manual and will change. + +| Spec term | Hermes Kanban today | +|-----------|---------------------| +| Board | Durable Kanban (SQLite-backed). Cards outlive a chat turn. | +| **complete** | Worker terminal `kanban_complete` (or equivalent complete action). | +| **block** | Worker terminal `kanban_block`. Human later unblocks. | +| Protocol violation | Clean worker exit with neither action. Circuit breaker auto-blocks the card. Not success. | +| **scratch** | `scratch` workspace — deleted on complete. | +| **durable directory** | `dir:` workspace rooted at the repo or docs tree. | +| **git worktree** | `worktree` / `worktree:` workspace. | +| **orchestrator** | Default profile; owns dispatch (`kanban.orchestrator_profile`). | +| **architect** | Architect specialist profile (specs only). | +| **researcher** | Researcher specialist profile. | +| **coder** | Coder specialist profile. | +| **designer** | Designer profile (visual only). | +| **reviewer** | Reviewer profile (never implements). | +| Unknown assignee | Card stays **ready**; dispatcher does not spawn. | +| Done (code) | Human merge on the host Git forge. Worker complete ≠ merge. | + +If the board product changes, keep the spec terms and rewrite this table. Do not fork a second protocol. + +## 3. How a card is supposed to finish (runtime) + +1. Dispatcher assigns a **roster** profile and a **workspace kind** that matches the deliverable. +2. Worker does the work in that workspace. +3. Worker calls **complete** (artifacts in the summary) or **block** (human question / gate). +4. Dependent cards stay unstarted until parents are **done** in board terms. For code, “done” in the *product* sense is still human merge (spec FR-5); parent-complete is only the board edge that unblocks the next specialist. + +Standard shape, not a command sheet: + +``` +architect (durable directory, spec) + → human accepts + → coder (worktree, PR + tests) → reviewer (block or approve) + → human merges +``` + +Research or design lanes may feed architect; they still complete or block, and they still do not land product code. + +## 4. What Hermes is responsible for (not this repo) + +- Persisting cards and comments. +- Spawning the assigned profile into the chosen workspace. +- Treating missing complete/block as a violation (circuit breaker). +- Leaving unknown assignees in ready. +- Deleting scratch on complete. +- Not running a second dispatcher beside the orchestrator’s. + +Operator recovery (reclaim, reassign, unblock) is Hermes operations. This plan does not catalog those commands. + +## 5. What this repo is responsible for + +- Keep this SDD folder as the protocol source of truth in git. +- When fleet workers touch **this** tree, they obey the spec: durable directory for `specs/` and `docs/`, worktree for `src/` / tests, complete/block, no secrets on the card, human merge for code. +- Do **not** encode the protocol in pytest or GitHub Actions. Spec 005 CI is for this package’s Python, not for Hermes worker exits. + +## 6. Risks + +| Risk | Mitigation | +|------|------------| +| Spec lives only in a local how-to and drifts | This folder; revise when the runtime mapping changes | +| Worker “succeeds” by exiting | Runtime circuit breaker; treat as violation; fix the worker, then reclaim | +| Spec/code written on scratch | Ban scratch for durable deliverables (spec FR workspace table) | +| Invented assignee | Closed roster; idle-in-ready is the failure mode | +| Coder complete treated as ship | Spec FR-5: human merge | +| Secrets in card comments | Spec FR-3; redact and re-complete/block if it happens | +| Building a board in `thelab_langchain` | Explicit non-goal | + +## 7. Success (qualitative) + +No invented metrics. The protocol is working when: + +- Completed cards name artifacts a human can open. +- Blocked cards name the human action required. +- Silent exits are treated as violations, not green cards. +- Specs still exist after architect complete (durable directory). +- Merged PRs, not completed coder cards, are what landed in `main`. + +## 8. What this plan is not + +It is not a Hermes CLI cheat sheet. It is not Slack (or any messenger) delivery. It is not a model-routing or GPU-budget plan (see 007). It is not a request to vendor `~/.hermes/docs/agentic-workflow.md` into this tree — that file stays where Hermes expects it. diff --git a/specs/010-worker-completion-protocol/spec.md b/specs/010-worker-completion-protocol/spec.md new file mode 100644 index 0000000..9b22d50 --- /dev/null +++ b/specs/010-worker-completion-protocol/spec.md @@ -0,0 +1,162 @@ +# Feature Spec: Durable-board worker completion protocol + +**Feature ID**: 010-worker-completion-protocol +**Status**: Living practice +**Created**: 2026-08-19 +**Owner**: Derek Clair +**Related**: [009-architect-coder-handoff](../009-architect-coder-handoff/spec.md), [008-local-tts-lenovo-go-spike](../008-local-tts-lenovo-go-spike/spec.md) + +## Record-keeping note + +This is the workstation **fleet protocol**: how a durable-board worker is allowed to finish. It is already practiced on the desk. This folder is the SDD record in the brain repo so the rule is not only a local Hermes how-to. + +It does **not** add a board to `thelab_langchain`. The current runtime is Hermes Kanban (see [plan.md](./plan.md)). The protocol outlives that runtime. + +## Overview + +Specialist workers (architect, researcher, coder, designer, reviewer) take durable cards from a board. Each run has exactly one legal finish: + +1. **complete** — the card’s acceptance criteria are met, and the summary names concrete artifacts a human can open; or +2. **block** — the worker cannot proceed without a human (missing decision, failed gate, unsafe change). + +A clean process exit with neither is a **protocol violation**. The card is not done. Downstream work must not treat silence as success. + +## Goals + +- Make terminal state unambiguous: complete or block, never “the process returned 0.” +- Make completed work inspectable: paths, PR URLs, test counts — not vibes. +- Keep secrets off the board (summaries, comments, metadata, artifact fields). +- Put durable work on durable workspaces; delete-on-complete scratch is only for throwaway probes. +- Dispatch only the real roster. Invented role names must not look like they are queued to run. +- Keep **human merge** as the definition of done for code. + +## Non-goals + +- A Kanban (or any board) implementation inside this Python package. +- CI in this repo that asserts complete/block (the fleet is not a thelab unit test). +- Vendoring the Hermes operating manual, CLI recipes, or chat/notification plumbing. +- Short in-conversation subagents (`delegate_task` and the like). Those die with the parent turn and are not this protocol. +- Changing spec 008’s hardware spike or this package’s `get_agent()` contract. + +## User stories + +1. As a worker, I finish by completing or blocking so the board never confuses a quiet exit with success. +2. As a human, I open a completed card and find artifacts I can verify (a spec path, a PR, a test count). +3. As a human, I never find keys, tokens, or env dumps in board fields. +4. As an architect, my spec still exists after the card completes because it was not on scratch. +5. As a dispatcher, I only assign names on the roster; a typo sits in ready instead of spawning a ghost worker. +6. As a coder, “I opened a PR” is not done — a human merges. + +## Roster (closed) + +Board dispatch uses **only** these roles: + +| Role | Owns | Must not | +|------|------|----------| +| **orchestrator** | Decompose work, assign the roster, keep the board moving | Invent assignees; flood the board without a task graph | +| **architect** | Specs, architecture, plans | Product implementation | +| **researcher** | Sources, findings, comparisons | Product implementation | +| **coder** | Implementation from an accepted spec; branch, tests, PR | Merge; treat unreviewed work as done | +| **designer** | Visual / UI deliverables | Backend ownership | +| **reviewer** | Review only; approve or block with comments | Implement the fix | + +Unknown role names are **not** dispatched. They remain in **ready** forever. That idle state is a routing bug, not a running worker. + +Other chat profiles may exist on the workstation. They are not board assignees unless they are added to this table in a spec revision. + +## Workspace kinds + +| Kind | Use for | After **complete** | +|------|---------|-------------------| +| **scratch** | Throwaway probes only | **Deleted**. Never for durable specs, docs, or product code. | +| **durable directory** | Specs, plans, docs packages | Survives. This is where SDD lives. | +| **git worktree** | Code changes | Survives as a worktree / branch. Not a substitute for a PR + human merge. | + +A card whose deliverable must be read later **must not** use scratch. Completing a spec card on scratch is a failed card even if the worker called complete. + +## Functional requirements + +### FR-1 Terminal action (non-negotiable) + +- Every worker run **MUST** end with **complete** or **block**. +- Clean exit without either is a **protocol violation**. +- A violation MUST NOT be recorded as success. The runtime SHOULD trip a circuit breaker / auto-block so the card cannot look healthy. +- Reclaim and retry are operator actions after a violation; they do not rewrite history into “completed.” + +### FR-2 Complete payload + +On **complete**, the board-visible summary (and any artifact metadata) MUST include concrete, checkable items as they apply: + +- Filesystem paths for specs/docs (durable directory). +- PR URL (and branch name if useful) for code. +- Test counts actually observed (e.g. `N passed` / `N failed`) — do not invent numbers. + +Optional but useful: what was *not* done, if the accepted spec scoped it out. + +### FR-3 Secrets stay off the board + +Board fields (title, body, comments, complete summary, metadata, artifact lists) MUST NOT contain: + +- API keys, tokens, OAuth material, `.env` contents +- Serials, phone numbers, personal IPs +- Chat/channel/DM identifiers +- Issue-tracker deep links that are private coordination, when a repo path or PR URL suffices + +Secrets belong in the worker’s private environment, never in the card. + +### FR-4 Block is a first-class finish + +**block** is a legal, expected terminal action. The worker MUST say: + +- what is blocked, +- what a human must decide or provide, +- what was already tried, if that is needed to unblock. + +A reviewer who will not approve **blocks**. An architect who lacks a decision **blocks**. Stalling in-process hoping the parent notices is not a finish. + +### FR-5 Definition of done (code) + +For product code: + +1. Architect spec accepted by a human. +2. Coder implements on a worktree / branch and opens a PR; coder **completes** with that PR and test evidence. +3. Reviewer **completes** (approve) or **blocks** (comments). Reviewer complete is not merge. +4. **A human merges.** That merge is the definition of done. + +A completed coder card with an unmerged PR is *ready for review / merge*, not done. + +For specs and docs on a durable directory, **complete** means the files are on disk at the named paths. Human review of the spec is still the gate before implementation (see FR-6). + +### FR-6 Spec-first for non-trivial code + +Non-trivial implementation is not assigned to coder until a human has accepted the spec. Reviewer does not write the product patch. + +### FR-7 Handoff language + +*What* to write in the card (tone, how to name artifacts, how to ask a human) is spec **009**. This spec is the *terminal action* and the workspace/roster rules. A well-worded silent exit still violates FR-1. + +## Non-functional requirements + +- Protocol is role- and runtime-agnostic: complete/block, workspace kinds, closed roster, human merge. +- Honest SDD: this package does not run the board; do not write tasks as if it will. +- No metrics theater: do not invent pass rates, latency, or fleet health numbers in this spec. + +## Acceptance criteria + +- [x] A worker that exits without complete or block is a protocol violation, not a successful card. +- [x] Complete summaries name artifacts (paths, PR URLs, and/or real test counts) and contain no secrets. +- [x] Scratch is never used for specs, docs, or product code that must survive the card. +- [x] Durable directory is the workspace for SDD; git worktree is the workspace for code. +- [x] Dispatch uses only orchestrator, architect, researcher, coder, designer, reviewer. Unknown names sit in ready. +- [x] Architect / researcher / reviewer / designer do not implement product code on their cards. +- [x] Code is done when a human merges, not when a worker completes. + +## Relationship to other specs + +- **009** — handoff *language* (how a worker talks on the card). This spec is the *protocol* (how a worker is allowed to stop). +- **008** — Lenovo Go voice I/O spike, executed in another repo. Same honesty rule: record where work actually lives; do not pretend this package owns it. +- **001 / 007** — product/hardware goals. This spec does not change them. + +## What this spec is not + +It is not a Hermes CLI manual. It is not a request to build `thelab_langchain.kanban`. It is not permission to treat chat-profile names as board roles. diff --git a/specs/010-worker-completion-protocol/tasks.md b/specs/010-worker-completion-protocol/tasks.md new file mode 100644 index 0000000..8d8d500 --- /dev/null +++ b/specs/010-worker-completion-protocol/tasks.md @@ -0,0 +1,47 @@ +# Tasks: Durable-board worker completion protocol (010) + +**Feature**: 010-worker-completion-protocol +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +This protocol is **practiced on the Hermes Kanban runtime**. This Python package +does not implement a board. There is **no CI** in thelab for complete/block. +Checkboxes below are a record of that split, not a backlog to build Kanban here. + +## Phase 0 — Protocol (living practice, not this package) + +Practiced on the workstation fleet; not code in `src/thelab_langchain/`. + +- [x] Workers end with **complete** or **block** +- [x] Clean exit without either is a protocol violation (not success) +- [x] Complete carries concrete artifacts (paths, PR URLs, observed test counts) +- [x] No secrets in board fields +- [x] Workspace: scratch deleted on complete; never for durable specs/code +- [x] Workspace: durable directory for specs/docs +- [x] Workspace: git worktree for product code +- [x] Closed roster only (orchestrator, architect, researcher, coder, designer, reviewer) +- [x] Unknown role names sit in ready; not dispatched +- [x] Human merge is definition of done for code +- [x] Architect / researcher / designer / reviewer do not implement product code on their cards + +## Phase 1 — SDD record (this repo) + +- [x] `specs/010-worker-completion-protocol/spec.md` +- [x] `specs/010-worker-completion-protocol/plan.md` (Hermes Kanban as current runtime) +- [x] `specs/010-worker-completion-protocol/tasks.md` (this file) + +## Explicitly not tasks in thelab + +Do not open work in this package for: + +- A board, dispatcher, or worker runner under `thelab_langchain` +- Pytest or GitHub Actions that assert Hermes complete/block +- Copying the local Hermes operating manual into git +- Chat/notification integration as part of 010 + +If the board runtime is replaced, update [plan.md](./plan.md) mapping — do not add a board here to “finish” 010. + +## Traceability + +Runtime and recovery procedure: local Hermes docs (not vendored). +Handoff wording: spec 009. +This folder is only the protocol SDD. diff --git a/specs/011-voice-reply-contract/plan.md b/specs/011-voice-reply-contract/plan.md new file mode 100644 index 0000000..7c58ec4 --- /dev/null +++ b/specs/011-voice-reply-contract/plan.md @@ -0,0 +1,149 @@ +# Plan: Voice-facing reply contract (011) + +**Feature**: 011-voice-reply-contract +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-19 +**Status**: Specified / not fully enforced in code + +## 1. Where the text comes from + +``` +utterance ─► I/O repo (Parakeet) + │ + ▼ + get_agent() this repo + memory_injection → LLM (+ optional memory tools) + │ + ▼ + AIMessage.content ← contract applies here + │ + ▼ + I/O repo: sentence-chunk → Piper → aplay + (button stop_event cancels playback) +``` + +The I/O process does not re-implement the agent (spec 008). It also does not +rewrite the reply for speakability. Chunking splits on `.` `!` `?` so first +audio can start before the full reply is synthesized. That is playback +scheduling, not a markdown/code filter. + +This plan is only about making `AIMessage.content` safe to speak. Latency +tables stay in the sibling repo README; this plan does not copy them. + +## 2. Three ways to enforce (none locked) + +### A. Prompt / system message on the graph (possible now) + +Add a short voice-facing `SystemMessage` (or prepend to the memory-injection +message) so `get_agent()` asks for short speakable prose, no tables/code unless +asked, and escalation of multi-step work. + +Facts that make this cheap: + +- Live speakerphone already uses `get_agent()`. `thelab-chat` uses `MemoryChat`, + not the graph. A graph-only instruction would not change the CLI path. +- No extra LLM call. Same turn, different instruction. +- 001 T4.2 already named this; it was never done. + +Limits: + +- Models ignore style instructions under tool-use or “be thorough” pressure. +- No unit-testable guarantee. A table can still come out. +- Must keep the text short so it does not fight memory context for attention. + +### B. Thin deterministic formatter (not built) + +A pure function on the reply string after the graph returns, before the I/O +process speaks it. Examples of mechanical rules (illustrative, not a shipped +list): + +- Drop fenced code blocks or replace with “I have a code block; say if you want + it read.” +- Drop markdown tables or summarize as “that is a table of N rows.” +- Strip heading hashes and collapse bullet markers into commas. +- Cap length (e.g. first N sentences) unless the user asked for more. + +Where it could live: + +- **This repo** — I/O keeps calling `get_agent()` / `invoke` and speaking + whatever comes back. Better seam: one brain, one content policy. +- **I/O repo** — this package stays format-agnostic. Worse: every consumer + reimplements the contract. + +Limits: + +- Easy to over-strip when the user *did* ask for a snippet. +- Heuristics are English-and-markdown-shaped; they will miss clever formatting. +- Still not a second model. Must not add an LLM rewrite pass (spec 007 / the + existing “no extra round-trip per voice turn” rule in `graph.py`). + +Status: **not built**. No module, no tests, no hook in `get_agent()`. + +### C. Leave it to the voice-profile SOUL in Hermes + +A Hermes voice profile already carries persona and tone for some sessions. +That file stays in Hermes. **Do not copy `SOUL.md` into this tree.** + +Limits: + +- The Lenovo Go loop (spec 008) invokes `get_agent()` with session + `HumanMessage` / `AIMessage` history plus this package’s memory injection. + It does not load a Hermes SOUL. Relying on SOUL alone does **not** cover the + live speakerphone path. +- Same “models can ignore it” limit as option A, plus an extra repo to keep + in sync. +- Useful as *additional* flavor if a Hermes-hosted session injects it; not a + substitute for A or B on the 008 path. + +## 3. Suggested sequence (if we implement) + +Not a commitment; a default order if someone picks this up: + +1. **Prompt-level on `get_agent()`** (option A). Smallest change, possible now, + matches 001 T4.2. Keep the instruction to a handful of lines. +2. **Listen to real sessions.** If markdown/code still hits Piper, add option B + in *this* repo as a post-`invoke` helper the I/O process can call — or fold + it into `get_agent()` so the I/O import surface stays one function. +3. **Do not vendor SOUL.** If Hermes sessions need the same rules, point them + at this spec rather than duplicating a second policy file here. + +Mixes are allowed (A + B). C is optional flavor, not the desk-loop control. + +## 4. Escalation (spoken turn vs board) + +The spoken turn is the wrong place to implement a multi-step coding or research +job. Implementation of *how* work reaches the orchestrator / board is Hermes +fleet operations, not this package. + +Until that wiring exists, option A can still say: if the ask is a multi-file +change or a research spike, reply with a short ack and do not dump the work +product. That is a content rule we can state now even if the handoff is +manual. + +Do not add tools to `get_agent()` whose only job is “file a ticket” unless the +fleet spec asks for it. Scope creep. + +## 5. What we will not do in this plan + +- Copy or paraphrase Hermes `SOUL.md`. +- Move Piper/ALSA/button interrupt into this repo. +- Claim a formatter exists. +- Put measured TTS timings in this tree. +- Add an LLM-as-judge or rewrite node on the voice path. + +## 6. Risks + +| Risk | Mitigation | +|------|------------| +| Prompt ignored; Piper reads a table | Option B later; do not mark A as “enforced” | +| Formatter strips a requested snippet | Opt-in exception when the user asked this turn; keep rules dumb | +| SOUL assumed to cover 008 | Document that the Go loop does not load SOUL | +| Extra LLM rewrite “to be safe” | Forbidden: extra round-trip per turn | +| I/O and brain both grow formatters | Prefer one helper in this package | + +## 7. Success + +- Developers reading this folder know the contract and that it is not a filter. +- If A ships: graph tests or a fixture show a voice system message exists. +- If B ships: unit tests on the formatter, no hardware required. +- I/O still owns stop-on-button and chunked playback. diff --git a/specs/011-voice-reply-contract/spec.md b/specs/011-voice-reply-contract/spec.md new file mode 100644 index 0000000..f956670 --- /dev/null +++ b/specs/011-voice-reply-contract/spec.md @@ -0,0 +1,190 @@ +# Feature Spec: Voice-facing reply contract + +**Feature ID**: 011-voice-reply-contract +**Status**: Specified / not fully enforced in code +**Created**: 2026-08-19 +**Owner**: Derek Clair +**Related**: [008-local-tts-lenovo-go-spike](../008-local-tts-lenovo-go-spike/spec.md), +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) +**Parent**: [001-voice-dgx-spark-agent](../001-voice-dgx-spark-agent/spec.md) (T4.2 +voice-aware behaviors) + +## Honest current state + +This is a **contract we want**, not a shipped filter. + +`get_agent()` in this repo produces the text that Piper speaks. The live I/O +loop ([008](../008-local-tts-lenovo-go-spike/spec.md), executed in +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent)) +takes the last `AIMessage.content` and synthesizes it. Sentence chunking in that +repo is for time-to-first-audio, not for speakability. + +This package does **not** post-process replies for speakability. There is no +graph node, wrapper, or test that strips markdown, caps length, or blocks +unspeakable formatting. `MemoryChat` (CLI text) has a generic “be concise but +warm” system prompt; the graph used by voice does not. Memory injection is a +`SystemMessage` of profile + recall, not a spoken-UX policy. + +Button interrupt of TTS is an I/O concern (spec 008 / the sibling repo). This +spec is about **what** the model is allowed to emit, not how playback is +cancelled. + +## Overview + +Spoken UX constraints belong in the brain repo even though ALSA, Piper, and +the Teams button live next door. + +When the consumer is a speakerphone, a good reply is short, speakable prose. +Markdown tables, fenced code, heading hashes, and other unspeakable formatting +are a failure of the contract unless the person at the desk asked for them in +that voice session. + +Multi-step coding and research do not belong in the spoken turn. The desk loop +should acknowledge and escalate to the workstation orchestrator / board (Hermes +fleet operating manual, not vendored here). It should not narrate a long plan +or dump a patch through Piper. + +## Goals + +- Define what a voice-facing `get_agent()` reply may contain. +- Keep spoken answers short and listenable by default. +- Forbid unspeakable formatting unless the user asked for it in the voice session. +- Send multi-step coding / research work to the orchestrator / board instead of + doing it in the spoken turn. +- Stay honest: specify the contract without claiming a filter exists in code. + +## Non-goals + +- ALSA, Piper, Parakeet, VAD, LED, USB hotplug, or button interrupt of playback + (spec 008 / I/O repo). +- Inventing or copying TTS latency numbers. Measured stage times, if any, live + in the sibling repo README; they are not this contract. +- Vendoring Hermes voice-profile SOUL or copying `SOUL.md` into this tree. +- Specifying the orchestrator / board / Kanban workflow (stays in Hermes). +- Changing `MemoryChat` / `thelab-chat` text UX, except to note it is a + different consumer. +- Streaming barge-in, Riva, or rewriting spec 001’s long-term voice stack. +- A product “customer service” tone guide. + +## User stories + +1. As the person at the desk, I ask a short question over the speakerphone and + hear a short spoken answer, not a markdown document. +2. As that person, I ask for a multi-file change or a research spike and hear + that it is handed to the board, not a spoken walkthrough of the work. +3. As that person, I can still say “read me that snippet” or “say the table” + and get what I asked for in that turn. +4. As a developer, I know this repo owns the reply *content* contract, and the + I/O repo owns *playback* (chunking, stop-on-button). +5. As a developer, I can tell “prompt guidance we could add now” from “a + deterministic speakability filter we have not built.” + +## Functional requirements + +### FR-1 Default spoken shape + +- Default voice replies are short, speakable prose (a few sentences, one + thought-group). Not an essay, not a blog post, not a README. +- Prefer words Piper can say. Avoid layout that only makes sense on a screen. +- Warm and direct is fine. Padding, recap-the-question, and “as an AI” throat-clearing + are not. + +### FR-2 Unspeakable formatting (opt-in, not default) + +Unless the user **asked for it in this voice session**, do not emit: + +- Markdown tables +- Fenced or indented code dumps +- Heading-hash outlines (`##`, `###`) +- Long bullet forests, numbered runbooks, or checkbox lists meant for a ticket +- Raw JSON / YAML / diff dumps +- Bare URLs or path dumps read aloud as punctuation soup + +If the user did ask (e.g. “read the function”, “say the rows”), the model may +emit that content. The I/O layer still sentence-chunks for playback; that is +not a license to dump an unbounded file. + +### FR-3 Escalate instead of doing the work in the spoken turn + +- Multi-step coding, multi-file edits, and open-ended research are **out of + band** for a voice turn. +- The spoken reply should confirm the ask and say it is going to the + orchestrator / board. It should not start implementing, paste a patch, or + narrate a long investigation. +- What “going to the board” means operationally lives in Hermes, not here. + This spec only forbids doing that work *as the spoken answer*. + +### FR-4 This spec vs I/O + +| Concern | Owner | +|---------|--------| +| What text the model may emit | **this repo** (`get_agent()` output) | +| Sentence-chunked Piper / TTFA | I/O repo | +| Button interrupt of TTS | I/O repo (spec 008) | +| Half-duplex ALSA, STT, LED | I/O repo (spec 008) | +| Prompt / optional formatter that enforces FR-1–FR-3 | this repo (not built; see plan) | +| Voice-profile SOUL | Hermes (not copied here) | + +### FR-5 Honesty in code + +- Until a formatter or voice system message exists, consumers must assume + `AIMessage.content` is unconstrained LLM text. +- Docs and tasks must not mark a speakability filter as done. +- Text CLI (`MemoryChat`) is out of this contract’s enforcement path; do not + pretend a CLI “be concise” line covers the speakerphone. + +## Non-functional requirements + +- No secrets, serials, or household identifiers in this spec or in example + utterances used for the contract. +- Do not bake measured TTS timings into this package. Point at the sibling + README if someone needs hardware numbers. +- Same `get_agent(user_id)` seam as spec 008. Do not fork the graph for “voice + vs text” unless the plan explicitly chooses a voice-only wrapper. +- Optional enforcement (prompt or formatter) must not add an extra LLM + round-trip per turn. Memory injection already skipped summarization for that + reason. + +## Acceptance criteria + +- [ ] Spec reviewed: voice replies are defined as short speakable prose with + unspeakable formatting opt-in, not default. +- [ ] Escalation of multi-step coding/research is written as a requirement, not + a suggestion. +- [ ] Boundary with spec 008 is explicit (content here, interrupt/playback there). +- [ ] Code in this repo still has **no** speakability post-process (honest until + a later task ships one). +- [ ] Prompt-level guidance is identified as possible now; a deterministic + filter is identified as not built (see [tasks.md](./tasks.md)). + +## Seams this package must keep stable + +| Seam | Contract | +|------|----------| +| `get_agent(user_id)` | Compiled graph. Voice I/O invokes this; reply text is last AI content. | +| `graph.invoke({"messages": ...})` | Unchanged call shape from spec 008. | +| Reply string | Today: raw model text. Wanted: FR-1–FR-3. Not filtered. | +| Hermes SOUL | Optional persona for some sessions. Not an API of this package. | + +## Relationship to other specs + +- **001** — long-term desktop voice. T4.2 (“voice-aware behaviors / shorter + responses”) is the historical checkbox; this folder is the actual contract. +- **008** — Lenovo Go I/O spike. Consumes the text this spec governs. Does not + define speakability. +- **004** — checkpointers. Orthogonal; session memory is not reply shape. +- **007** — Spark budget. A speakability filter, if built, stays CPU-cheap + (no second model call). +- Workstation fleet (orchestrator / architect / researcher / coder / reviewer) + is not specified here. + +## Open questions + +- Enforce via prompt on the graph, a thin deterministic formatter, Hermes SOUL, + or some mix? Options are in [plan.md](./plan.md); none is locked. +- How does the spoken turn *signal* escalation (a sentence of ack vs a tool vs + a convention the orchestrator already watches)? Out of scope until fleet + wiring is specified in Hermes. +- Should `get_agent()` always be voice-shaped, given the CLI uses `MemoryChat`? + Probably yes if the only live graph consumer is the speakerphone — confirm + before adding a system message. diff --git a/specs/011-voice-reply-contract/tasks.md b/specs/011-voice-reply-contract/tasks.md new file mode 100644 index 0000000..7a63b27 --- /dev/null +++ b/specs/011-voice-reply-contract/tasks.md @@ -0,0 +1,73 @@ +# Tasks: Voice-facing reply contract (011) + +**Feature**: 011-voice-reply-contract +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) +**Status**: Specified / not fully enforced in code + +Checkboxes are honest. Spec-only work can be marked done; enforcement is not. + +## Phase 0 — Specify the contract (this folder) + +- [x] Write `spec.md` (spoken shape, unspeakable formatting, escalate vs + implement, I/O vs brain boundary) +- [x] Write `plan.md` (prompt vs thin formatter vs Hermes SOUL; SOUL not copied) +- [x] Write `tasks.md` (this file) +- [x] Record that this package does **not** post-process replies today + +## Phase 1 — Prompt-level guidance (possible now) + +Not done. Possible now because `get_agent()` is the live voice consumer and +`MemoryChat` is a separate CLI path. No new service, no extra LLM call. + +- [ ] Add a short voice-facing system instruction on the graph (`get_agent()` / + memory injection or a dedicated preamble) +- [ ] Cover: short speakable prose; no tables / code dumps / heading hashes + unless the user asked this turn; escalate multi-step coding/research + rather than doing it in the spoken answer +- [ ] Keep the instruction small so it does not drown memory context +- [ ] Unit test or fixture: compiled graph (or injection helper) includes the + voice instruction — does **not** prove the model obeys it +- [ ] Do not treat this checkbox as “enforced speakability” + +## Phase 2 — Deterministic speakability filter (not built) + +Not built. Do not check these off until a pure helper exists and is wired. + +- [ ] Pure formatter: strip or replace unspeakable markdown (tables, fences, + heading hashes) without a second model call +- [ ] Length / sentence cap with an exception when the user asked for a dump +- [ ] Unit tests on strings only (no ALSA, no Piper, no keys) +- [ ] Wire through this package so I/O can keep a single brain import +- [ ] Decide with a review whether formatter lives inside `get_agent()` or as a + sibling helper the I/O process calls +- [ ] Explicit non-goal until then: claiming Piper is “safe” because of chunking + +## Phase 3 — Escalation seam (Hermes, not this repo) + +Out of band. Listed so it is not silently implemented as a spoken dump. + +- [ ] Spoken ack of “handed to the board” once fleet wiring exists (Hermes + operating manual, not vendored) +- [ ] No `get_agent()` ticket-filing tool unless that fleet spec asks for it +- [ ] Do not copy voice-profile `SOUL.md` into this tree + +## Out of scope (stay unchecked here) + +- [ ] Button interrupt of TTS — I/O repo / spec 008 +- [ ] Sentence-chunked Piper / TTFA — I/O repo +- [ ] Voice barge-in — out of scope for 008 and for this contract +- [ ] Copying sibling-repo latency tables into this package + +## Traceability + +| Want | Code today | +|------|------------| +| Speakable default | Unconstrained `AIMessage.content` | +| Prompt-level guidance | Possible now; **not** in `graph.py` | +| Deterministic filter | **Not built** | +| Button stop of playback | Sibling I/O repo | +| Hermes SOUL | Hermes only | + +Live consume path: [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) +calls `thelab_langchain.agent.graph.get_agent`. Historical 001 checkbox: T4.2 +in [001/tasks.md](../001-voice-dgx-spark-agent/tasks.md). diff --git a/specs/README.md b/specs/README.md index 5a3e00b..052d333 100644 --- a/specs/README.md +++ b/specs/README.md @@ -1,19 +1,23 @@ # Specs -Design and planning for this package. Status in each file is honest: several -are drafts or future work, not a claim that every spec is implemented. +Each numbered folder has **spec.md**, **plan.md**, and **tasks.md**. Status in +the spec is honest: living practice, executed out of tree, designed-not-built, +or partial. | ID | Title | What it is | |----|-------|------------| -| [001](001-voice-dgx-spark-agent/spec.md) | Voice-enabled agent on DGX Spark | Original desktop-voice goal; live I/O now lives in [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) | -| [002](002-multi-user-support/spec.md) | Multi-user support | Per-user `container_tag` / `thread_id` — designed, not a product multi-tenant system | -| [003](003-deployment-infrastructure/spec.md) | Deployment / Docker | Compose + NIM path; gaps called out in the spec | -| [004](004-persistence-checkpointers/spec.md) | Persistence & checkpointers | Planned LangGraph checkpointer; conversation state is still in-memory | -| [005](005-testing-and-cicd/spec.md) | Testing & CI | Direction; a CPU-only pytest + ruff workflow is in `.github/workflows/ci.yml` | -| [006](006-alternative-memory-systems/spec.md) | Alternative memory backends | Future consideration; Supermemory is the current store | -| [007](007-dgx-hardware-optimization/spec.md) | DGX Spark hardware budget | ~30B-class local models; no 120B+ agent loops on one Spark | -| [008](008-local-tts-lenovo-go-spike/spec.md) | Local-tts Lenovo Go spike | Button → VAD → Parakeet → `get_agent()` → Piper. **Executed in** [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) | +| [001](001-voice-dgx-spark-agent/spec.md) | Voice-enabled agent on DGX Spark | Broader desktop-voice goal (Riva/NIM compose). Not the live path. | +| [002](002-multi-user-support/spec.md) | Multi-user support | Per-user `container_tag` / `thread_id`. Designed; not speaker ID. | +| [003](003-deployment-infrastructure/spec.md) | Deployment / Docker | Experimental compose; healthchecks and GPU limits still open. | +| [004](004-persistence-checkpointers/spec.md) | Persistence & checkpointers | Not wired. Caller (or in-memory process) holds turns. | +| [005](005-testing-and-cicd/spec.md) | Testing & CI | Unit tests + CPU GitHub Actions exist; no coverage gate or image CI. | +| [006](006-alternative-memory-systems/spec.md) | Alternative memory backends | Escape hatch. Supermemory stays default; do not build adapters yet. | +| [007](007-dgx-hardware-optimization/spec.md) | Spark hardware + inference slot | One local LLM; CPU STT/TTS; Grok for quality-critical roles. | +| [008](008-local-tts-lenovo-go-spike/spec.md) | Local-tts Lenovo Go spike | **Executed in** [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent). | +| [009](009-architect-coder-handoff/spec.md) | Architect ↔ coder handoff (STE) | Living practice. Not a library feature; not CI-enforced. | +| [010](010-worker-completion-protocol/spec.md) | Worker complete-or-block | Living practice. This package does not implement a board. | +| [011](011-voice-reply-contract/spec.md) | Voice-facing reply contract | Wanted speakability rules. No filter in code yet. | -Workstation **fleet operations** (orchestrator / architect / researcher / -coder / reviewer, Kanban vs chat) are not specified here. That operating -manual stays in Hermes at `~/.hermes/docs/agentic-workflow.md`. +Hermes **operating manual** (CLI, gateway, profile files) stays at +`~/.hermes/docs/agentic-workflow.md`. Specs 009–010 record the *protocol*, +not that file. From 52f9077228c0d0c49eead6a85b1ff0bf686a2b87 Mon Sep 17 00:00:00 2001 From: Derek Clair Date: Thu, 20 Aug 2026 01:37:41 -0600 Subject: [PATCH 2/3] =?UTF-8?q?Add=20SDD=20012=E2=80=93015:=20fleet=20disp?= =?UTF-8?q?atch,=20reviewer=20gate,=20memory=20graph,=20telemetry.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 012 records conversation vs subagent vs durable board (not a dispatcher). 013 records the reviewer gate (never implements). 014 documents the shipped get_agent() memory-injection graph; the 500–1000ms comment stays unmeasured. 015 is the content-free telemetry contract; the graph does not export OTEL; the hub is derekclair/lan-agent-otel by pointer only. No Hermes file copies. --- README.md | 1 + specs/012-fleet-dispatch-model/plan.md | 153 +++++++++++ specs/012-fleet-dispatch-model/spec.md | 317 ++++++++++++++++++++++ specs/012-fleet-dispatch-model/tasks.md | 60 ++++ specs/013-reviewer-quality-gate/plan.md | 164 +++++++++++ specs/013-reviewer-quality-gate/spec.md | 288 ++++++++++++++++++++ specs/013-reviewer-quality-gate/tasks.md | 71 +++++ specs/014-memory-injection-graph/plan.md | 138 ++++++++++ specs/014-memory-injection-graph/spec.md | 235 ++++++++++++++++ specs/014-memory-injection-graph/tasks.md | 84 ++++++ specs/015-content-free-telemetry/plan.md | 140 ++++++++++ specs/015-content-free-telemetry/spec.md | 184 +++++++++++++ specs/015-content-free-telemetry/tasks.md | 84 ++++++ specs/README.md | 8 +- 14 files changed, 1925 insertions(+), 2 deletions(-) create mode 100644 specs/012-fleet-dispatch-model/plan.md create mode 100644 specs/012-fleet-dispatch-model/spec.md create mode 100644 specs/012-fleet-dispatch-model/tasks.md create mode 100644 specs/013-reviewer-quality-gate/plan.md create mode 100644 specs/013-reviewer-quality-gate/spec.md create mode 100644 specs/013-reviewer-quality-gate/tasks.md create mode 100644 specs/014-memory-injection-graph/plan.md create mode 100644 specs/014-memory-injection-graph/spec.md create mode 100644 specs/014-memory-injection-graph/tasks.md create mode 100644 specs/015-content-free-telemetry/plan.md create mode 100644 specs/015-content-free-telemetry/spec.md create mode 100644 specs/015-content-free-telemetry/tasks.md diff --git a/README.md b/README.md index 9768917..3434e3c 100644 --- a/README.md +++ b/README.md @@ -163,6 +163,7 @@ examples/ Non-interactive snippets - **[Development](docs/development.md)** — venv, chat, common commands - **[Specs index](specs/README.md)** — design and planning already in this repo - **[Spec 008](specs/008-local-tts-lenovo-go-spike/spec.md)** — local-tts spike; implemented in [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) +- **[Spec 014](specs/014-memory-injection-graph/spec.md)** — the shipped LangGraph (`get_agent()`) - Workstation fleet operating manual (Hermes, not vendored): `~/.hermes/docs/agentic-workflow.md` diff --git a/specs/012-fleet-dispatch-model/plan.md b/specs/012-fleet-dispatch-model/plan.md new file mode 100644 index 0000000..c1ec88e --- /dev/null +++ b/specs/012-fleet-dispatch-model/plan.md @@ -0,0 +1,153 @@ +# Plan: Workstation fleet dispatch model (012) + +**Feature**: 012-fleet-dispatch-model +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-20 + +## 1. What this plan is + +A map of **which layer, which role, and which repo owns the runtime**. It is +not a plan to add a dispatcher to `thelab-langchain`. Success is a practiced +loop, not a merged feature flag. + +Living practice sits in Hermes (profiles, board, gateway). This repo only +records the contract. Runtime CLI and gateway stay in Hermes. Do not vendor +that how-to here. + +## 2. Protocol vs runtime + +The spec is the durable contract (layers, roster, spec-first, human merge). +**Hermes is the current runtime** — an implementation detail, not the model. + +``` +Human + └─ orchestrator (front door + dispatcher) + ├─ conversation (this turn) + ├─ in-process subagent (dies with this turn) + └─ durable board + ├─ architect / researcher / designer (durable directory) + ├─ coder (git worktree → PR) + └─ reviewer (same tree as the change) +``` + +This Python package does not own that board, spawn workers, or persist cards. +Do not add a dispatch module here to “implement 012.” + +## 3. Mapping table (Hermes today) + +Abstract spec terms map onto the workstation as follows. How to start a +gateway, which binary flags exist, and profile *filenames* will change. +Keep the spec terms; rewrite this table if the runtime changes. + +| Spec term | Hermes today | +|-----------|----------------| +| Conversation / front door | Default profile; human talks to one orchestrator | +| In-process subagent | Short-lived helper inside the parent turn; discarded when the turn ends | +| Durable board | Persistent Kanban; cards outlive a chat turn | +| **orchestrator** | Default profile; owns board dispatch | +| **architect** / **researcher** / **coder** / **designer** / **reviewer** | Specialist profiles spawned by the dispatcher (see [010 plan](../010-worker-completion-protocol/plan.md) §2 for the current filename map) | +| Unknown assignee | Card stays unstarted; dispatcher does not spawn | +| Done (code) | Human merge on the host Git forge | + +Filenames in that 010 table are **runtime wiring**, not 012’s product names. +Do not treat a profile path as a roster role. + +Workspace kinds, **complete** / **block**, and scratch-vs-durable stay in +010. This plan does not reprint them. + +## 4. Dispatch flow + +1. Human speaks to the orchestrator (conversation layer). +2. Orchestrator classifies: + - small enough for this turn → answer (or a dying-with-the-turn helper); + - must survive or needs a specialist gate → **task graph**, then board cards. +3. Board cards use roster roles only. Architect / researcher / designer get a + durable directory. Coder gets a git worktree. Reviewer shares the change’s + tree. +4. Non-trivial code waits on **human accept** of the architect packet (009). +5. Board workers **complete** or **block** (010). +6. Human merges. That is done for code. + +The orchestrator proposes the graph **before** flooding the board. Children +stay unstarted until parents are done *on the board*. For product code, board +“done” is still not ship; ship is human merge (spec FR-6). + +## 5. Who uses which model class (007) + +Do not invent footprints. Slot policy lives in 007. + +| Roles | Slot | +|-------|------| +| Orchestrator, architect, reviewer, designer | Hosted. Do not occupy the Spark’s single local generative LLM. | +| Coder, researcher | May use the one local ~30B-class slot, with hosted fallback if the slot is busy or the endpoint is down. | + +One serious local LLM at a time. No 120B+ agent loops on one Spark. When the +slot is occupied, other work uses hosted models. 012 does not pick weights. + +## 6. What Hermes is responsible for (not this repo) + +- Persisting cards and comments. +- Spawning the assigned profile into the chosen workspace. +- Keeping a living dispatcher so board cards do not sit unstarted forever. +- Treating missing complete/block as a violation (010). +- Leaving unknown assignees unstarted. +- In-process helpers that die with the parent turn. +- CLI, gateway, and profile files. + +Operator recovery (reclaim, reassign, unblock) is Hermes operations. This +plan does not catalog those commands. + +## 7. What this repo is responsible for + +- Keep this SDD folder as the dispatch-model source of truth in git. +- When fleet workers touch **this** tree, they obey 009, 010, and this spec: + durable directory for `specs/` and `docs/`, worktree for `src/` / tests, + accepted spec before non-trivial code, no secrets on the card, human merge + for code. +- Do **not** encode dispatch in pytest or GitHub Actions. Spec 005 CI is for + this package’s Python, not for Hermes process health. +- Spoken escalation remains 011’s problem. Do not add a ticket tool under 012. + +## 8. Language and packets + +| Concern | Where | +|---------|--------| +| Which layer / which role | This spec | +| How architect talks to coder (STE, five artifacts, human accept) | 009 | +| How a board worker stops (complete/block, workspace kinds) | 010 | +| Local vs hosted slot | 007 | +| Speakable replies / escalate instead of dumping a patch | 011 | + +This SDD folder is human prose. Agent-consumed procedures stay STE per 009. + +## 9. Risks + +| Risk | Mitigation | +|------|------------| +| Chat replaces the board for multi-step work | Durability rule: inspect / pause / resume → board | +| Orchestrator implements or “reviews” in character | Role rule: decompose, do not impersonate | +| Invented assignee looks running | Closed roster; idle-unstarted is the failure mode | +| Spec/code on scratch | 010 workspace table | +| Coder complete treated as ship | FR-6: human merge | +| In-process helper used for a spec | Dies with the turn; promote to board | +| Building a dispatcher in `thelab-langchain` | Explicit non-goal | +| How-to copied into git | Distill here; CLI stays in Hermes | +| Two local LLMs / 120B loops | 007 slot policy | + +## 10. Success (qualitative) + +No invented metrics. The model is working when: + +- The human uses one front door for small work. +- Multi-step work is on a board a human can open later. +- Architect / researcher / designer cards have no product patches. +- Reviewer cards never contain the fix. +- Merged PRs, not completed coder cards, are what landed in `main`. +- Unknown names sit idle instead of spawning ghosts. +- This package still has no dispatcher. + +## 11. What this plan is not + +It is not a Hermes CLI cheat sheet. It is not a gateway runbook. It is not +messenger delivery. It is not a rewrite of 007, 009, or 010. It is not a +promise that CI will catch a mis-routed card. diff --git a/specs/012-fleet-dispatch-model/spec.md b/specs/012-fleet-dispatch-model/spec.md new file mode 100644 index 0000000..1affb81 --- /dev/null +++ b/specs/012-fleet-dispatch-model/spec.md @@ -0,0 +1,317 @@ +# Feature Spec: Workstation fleet dispatch model + +**Feature ID**: 012-fleet-dispatch-model +**Status**: Living practice (documented here; **not** a product feature of `thelab-langchain`) +**Created**: 2026-08-20 +**Owner**: Derek Clair +**Related**: [009-architect-coder-handoff](../009-architect-coder-handoff/spec.md), +[010-worker-completion-protocol](../010-worker-completion-protocol/spec.md), +[007-dgx-hardware-optimization](../007-dgx-hardware-optimization/spec.md) + +## Record-keeping note + +This spec records **how work is routed** on the workstation: one front door, a +closed specialist roster, and three durability layers. The practice is already +in use. This folder is the SDD trail in the brain repo so later specs (and +humans) can see the model without opening a local operating manual. + +It is **not** a dispatcher, CLI, gateway, LangGraph node, or CI gate in this +package. Runtime process control stays in Hermes. Do **not** copy Hermes +profile files, `SOUL.md`, or the fleet how-to into git. This document +**distills** the design. + +Spec **009** is how architect and coder talk. Spec **010** is how a durable +worker is allowed to stop. Spec **007** is who may occupy the local inference +slot. This spec is **which layer** and **which role**. + +## Overview + +A human talks to **one orchestrator**. That agent answers small work itself, +spawns short-lived helpers that die with the turn, or decomposes larger work +onto a durable board for named specialists. The orchestrator does not pretend +to be every specialist. + +Three layers: + +1. **Conversation** — one orchestrator (front door). Questions, decisions, + small one-shots. +2. **In-process subagent** — short parallel reasoning. Dies with the parent + turn. Not inspectable later. +3. **Durable board** — specs, multi-lane work, review, anything a human may + inspect, pause, or resume. + +Rule of thumb: if a human might want to inspect, pause, or resume it later, +it belongs on the board. If it is a two-minute sub-question, answer it in +conversation or spawn an in-process subagent. + +``` +Human + └─ orchestrator (conversation front door) + ├─ answers / small one-shots + ├─ in-process subagent (dies with the parent turn) + └─ durable board + ├─ architect / researcher / designer + ├─ coder + └─ reviewer +Human accept (specs) and human merge (code) sit outside the roster. +``` + +## Goals + +- Keep a single front door so the human is not picking specialists by filename. +- Route by durability: conversation vs in-process vs board. +- Dispatch only the closed roster. Invented names must not look queued to run. +- Keep the orchestrator as decomposer, not as a fake architect/coder/reviewer. +- Point non-trivial implementation at spec-first (009) and complete-or-block (010). +- Keep human merge as the definition of done for code. +- Stay honest: this package does not implement a dispatcher. + +## Non-goals + +- A dispatcher, board, worker runner, or gateway inside `thelab-langchain`. +- CI in this repo that asserts routing, roster names, or Hermes process health. +- Vendoring the Hermes operating manual, CLI recipes, gateway runbooks, or + profile `SOUL.md`. +- Treating Hermes profile *filenames* as the product vocabulary. Roles are + the product; filenames are a runtime mapping (see [plan.md](./plan.md)). +- Replacing 009 (handoff language), 010 (terminal action / workspace kinds), + or 007 (inference slot / local vs hosted). +- Chat/notification plumbing, messenger delivery, or ticket-tracker IDs as + required reading. +- Voice I/O, speakability, or a spoken ticket-filing tool (see 011 / 008). +- Inventing assignee names, latency numbers, or fleet health metrics. + +## Domain terms (define once) + +| Term | Meaning | +|------|---------| +| **Front door** | The single conversation the human uses day to day. The orchestrator. | +| **Orchestrator** | Role that answers small work, decomposes the rest, and assigns the roster. | +| **Layer** | One of: conversation, in-process subagent, durable board. | +| **Conversation** | The live turn with the orchestrator. Questions, decisions, small one-shots. | +| **In-process subagent** | A helper spawned inside the parent turn. Short parallel reasoning. Dies with that turn. Not a board worker. | +| **Durable board** | Cards that outlive a chat turn. Specs, multi-lane work, review, pause/resume. | +| **Roster** | The closed set of roles that may be assigned: orchestrator, architect, researcher, coder, designer, reviewer. | +| **Dispatch** | Choosing a layer and, for the board, a roster role plus a workspace kind (010). | +| **Task graph** | Named work with dependencies. Proposed before flooding the board. | +| **Assignee** | A roster role on a card. Not an invented string. Not a person’s family name. | +| **Spec-first** | Non-trivial implementation waits for a human-accepted spec (009). | +| **Complete-or-block** | Legal finishes for a board worker (010). Silent exit is a violation. | +| **Human merge** | Definition of done for product code. Worker complete is not merge. | + +Do not reuse these words for other meanings in the same packet (for example, +do not call an in-process subagent a “board worker”). + +## Roster (roles, not runtime filenames) + +Board dispatch and specialist ownership use **only** these roles. Hermes +profile filenames are today’s wiring; they are not this spec’s names. + +| Role | Owns | Must not | +|------|------|----------| +| **orchestrator** | Front door. Small one-shots. Decompose multi-step work. Assign the roster. Keep the board moving. | Pretend to be architect, coder, or reviewer. Invent assignees. Flood the board without a task graph. | +| **architect** | Specs, architecture, plans. | Ship product code. Open an implementation PR as the architect. | +| **researcher** | Sources, findings, comparisons. | Ship product code. Treat a research note as an accepted spec. | +| **coder** | Implementation from an **accepted** spec; branch, tests, PR. | Merge. Implement from chat only. Widen the spec. | +| **designer** | Visual / UI deliverables. | Backend ownership. Product implementation. | +| **reviewer** | Review only. Approve or block with comments. | Implement the fix. | + +Unknown role names are **not** dispatched. They remain unstarted. That idle +state is a routing bug, not a running worker. + +Other chat profiles may exist on the workstation (for example a spoken path). +They are not board assignees unless they are added to this table in a spec +revision. + +The **human** is not a roster role and is not optional: + +- Accepts or rejects the architect packet before code (009). +- Merges product code. That merge is done. +- Inspects, pauses, resumes, and unblocks board work. + +## Layers + +### 1. Conversation (orchestrator) + +Use for questions, decisions, and small one-shots the orchestrator can finish +in the live turn. The human should not have to name a specialist filename to +get a straight answer. + +The orchestrator may **propose** a task graph. Proposing is not the same as +doing the specialist work in chat. Long interactive sessions with a specialist +are the exception; auditable board handoffs are the default for multi-step work. + +### 2. In-process subagent + +Use for short parallel reasoning that only the parent turn needs. The helper +dies with that turn. It does not survive restart. It is not 010’s complete-or-block +protocol. It is not a place for specs, PRs, or review gates. + +If a human might inspect, pause, or resume the work later, do **not** put it +here. Promote it to the board. + +### 3. Durable board + +Use for specs, multi-lane work, review, and anything that must outlive the +turn. Workers on this layer obey 010 (complete or block, workspace kinds, +secrets off the card). Architect → coder packets obey 009. + +Standard shapes (not a command sheet): + +**Spec → implement → review** + +``` +architect (durable directory, spec) + → human accepts + → coder (git worktree, PR + tests) + → reviewer (approve or block) + → human merges +``` + +**Research-heavy** + +``` +researcher (lane A) ─┐ +researcher (lane B) ─┼─► architect (synthesize) ─► human accept ─► coder ─► reviewer ─► human merge +``` + +**Design + build** + +``` +designer ─┐ +architect ─┴─► coder ─► reviewer ─► human merge +``` + +Research and design lanes still complete or block. They still do not land +product code. + +## Decision table + +| Request | Layer | Owner | +|---------|-------|--------| +| Quick question / small one-shot | Conversation | Orchestrator | +| Short parallel sub-question inside a turn | In-process subagent | Orchestrator-spawned helper; dies with the turn | +| What should we build / how? | Board | Architect → human accept | +| Look up / compare options (more than a glance) | Board | Researcher (architect synthesizes if it becomes a spec) | +| Implement the accepted spec | Board | Coder | +| Visual / UI deliverable | Board | Designer | +| Is this safe/correct to merge? | Board | Reviewer | +| Mix of the above | Conversation proposes a task graph → board | Orchestrator decomposes; specialists execute | + +When in doubt, prefer the board over a long specialist chat. + +## Functional requirements + +### FR-1 Layer by durability + +- Conversation: live, small, discarded with the turn except as ordinary chat history. +- In-process subagent: parallel, short, **dies with the parent turn**. +- Durable board: inspect / pause / resume / review / multi-lane. + +### FR-2 Single front door + +- Day-to-day human traffic hits the orchestrator. +- The orchestrator decomposes. It does not impersonate the rest of the roster. + +### FR-3 Closed roster + +- Dispatch uses only the six roles in the roster table. +- Invented assignee names are not dispatched. Idle-unstarted is the failure mode. + +### FR-4 Spec-first for non-trivial code + +- Non-trivial implementation is not assigned to coder until a human has accepted + the spec (009). +- Architect, researcher, and designer do not ship product code. +- Reviewer never implements. + +### FR-5 Board workers finish per 010 + +- Every board run ends **complete** or **block**. +- Clean exit with neither is a protocol violation, not success. +- Workspace kinds and secrets-off-the-card stay in 010. This spec does not + restate the CLI. + +### FR-6 Definition of done (code) + +- Coder complete with a PR is *ready for review / merge*, not done. +- Reviewer complete is not merge. +- **A human merges.** That merge is done. + +### FR-7 Model class follows 007 + +- Quality-critical roles (orchestrator, architect, reviewer, designer) use a + **hosted** model so they do not occupy the Spark’s single local LLM slot. +- Coder and researcher **may** use the one local ~30B-class slot, with hosted + fallback if the slot is busy or the local endpoint is down. +- Do not co-schedule a second serious local generative LLM. Do not run 120B+ + agent loops on one Spark. Numbers and harness work stay in 007. + +### FR-8 This package does not dispatch + +- `thelab-langchain` does not spawn specialists, persist cards, or own a + gateway. +- Recording the model in `specs/012-fleet-dispatch-model/` is not + implementing it. + +## Non-functional requirements + +- Protocol is role-first and runtime-agnostic. If the board product changes, + keep the roles and layers; rewrite the runtime mapping in [plan.md](./plan.md). +- Honest SDD: practiced on Hermes; do not write tasks as if this package will + grow a dispatcher. +- No metrics theater: do not invent pass rates, spawn latency, or fleet health + numbers in this spec. +- No secrets, tracker IDs, or home-directory paths as the layout of record. + +## User stories + +1. As a human, I talk to one orchestrator for questions and small work. +2. As a human, I see multi-step work on a board I can inspect, pause, or resume. +3. As orchestrator, I decompose and assign the roster; I do not fake a code review. +4. As architect / researcher / designer, I deliver artifacts, not product patches. +5. As coder, I implement only an accepted spec and I do not merge. +6. As reviewer, I approve or block; I never implement the fix. +7. As a dispatcher, I never treat a typo as a worker. + +## Acceptance criteria (for this SDD record) + +- [x] This folder names the three layers and the closed roster as **roles**. +- [x] Status is “living practice,” not a `thelab-langchain` feature. +- [x] Spec-first points at 009; complete-or-block points at 010; local vs hosted + points at 007. +- [x] Human merge is definition of done for code. +- [x] No Hermes how-to, CLI cheat sheet, `SOUL.md`, or profile file is copied here. +- [ ] A dispatcher, board, or gateway in this package — **not done** (out of scope). + +## Relationship to other specs + +- **009** — architect ↔ coder packet (spec, acceptance criteria, kanban body, + blockers, residual risks, STE for procedures). 012 decides *that the work + is a board architect card*, not how the packet is worded. 009’s non-goal of + “the full fleet manual” is this spec. +- **010** — durable-board terminal action, workspace kinds, closed roster, + human merge. 012 adds the conversation and in-process layers that 010 + explicitly excluded. Do not duplicate 010’s complete payload rules here. +- **007** — one local LLM slot; hosted models for quality-critical roles; + coder/researcher may use the slot with hosted fallback. 012 does not pick + weights or publish GB figures. +- **011** — voice-facing reply contract. Spoken multi-step coding/research + escalates to this fleet (acknowledge; do not dump a patch through TTS). + 012 does not add a ticket tool to `get_agent()`. +- **008 / 001** — voice I/O spike and long-term voice-agent goal. Dispatch + does not change their stacks. + +## What this spec is not + +It is not a Hermes CLI manual. It is not a request to build +`thelab_langchain.dispatch`. It is not permission to treat chat-profile +filenames as board roles. It is not a copy of `SOUL.md`. + +## Open questions + +- Whether a later spec should give the spoken path a **board escalation + seam** in this package (011 Phase 3). Default until then: no ticket-filing + tool on `get_agent()`. +- Whether conversation-layer “small one-shot” needs a hard size limit. Default: + human judgment at the front door; when in doubt, board. diff --git a/specs/012-fleet-dispatch-model/tasks.md b/specs/012-fleet-dispatch-model/tasks.md new file mode 100644 index 0000000..9eeac1d --- /dev/null +++ b/specs/012-fleet-dispatch-model/tasks.md @@ -0,0 +1,60 @@ +# Tasks: Workstation fleet dispatch model (012) + +**Feature**: 012-fleet-dispatch-model +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +This model is **practiced on Hermes**. This Python package does not implement +a dispatcher. Runtime CLI and gateway stay in Hermes. There is **no CI** in +thelab for routing or roster names. Checkboxes below are a record of that +split, not a backlog to build dispatch here. + +## Phase 0 — Model (living practice, not this package) + +Practiced on the workstation fleet; not code in `src/thelab_langchain/`. + +- [x] Three layers: conversation, in-process subagent, durable board +- [x] Single orchestrator as front door; decomposes; does not impersonate specialists +- [x] Closed roster only (orchestrator, architect, researcher, coder, designer, reviewer) +- [x] Unknown role names are not dispatched (idle-unstarted, not a ghost worker) +- [x] Architect / researcher / designer do not ship product code +- [x] Reviewer never implements +- [x] Coder implements accepted specs only +- [x] Spec-first for non-trivial work (009) +- [x] Board workers complete or block (010) +- [x] Human merge is definition of done for code +- [x] Quality-critical roles hosted; coder/researcher may use the one local slot (007) + +## Phase 1 — SDD record (this repo) + +- [x] `specs/012-fleet-dispatch-model/spec.md` +- [x] `specs/012-fleet-dispatch-model/plan.md` (Hermes as current runtime; roles not filenames) +- [x] `specs/012-fleet-dispatch-model/tasks.md` (this file) + +Do not copy profile files, `SOUL.md`, or the Hermes operating manual into this +tree to “complete” a checkbox. + +## Explicitly not tasks in thelab + +Do not open work in this package for: + +- A dispatcher, board, worker runner, or gateway under `thelab_langchain` +- Pytest or GitHub Actions that assert Hermes routing or process health +- Copying the local Hermes how-to, CLI recipes, or `SOUL.md` into git +- Chat/notification integration as part of 012 +- A spoken ticket-filing tool on `get_agent()` (see 011 Phase 3) + +If the board runtime is replaced, update [plan.md](./plan.md) mapping — do not +add a dispatcher here to “finish” 012. + +## Traceability + +| Concern | Record | +|---------|--------| +| Layers and roster | This folder | +| Handoff wording | [009](../009-architect-coder-handoff/tasks.md) | +| Complete-or-block / workspaces | [010](../010-worker-completion-protocol/tasks.md) | +| Local vs hosted slot | [007](../007-dgx-hardware-optimization/spec.md) | +| Runtime CLI / gateway | Hermes (not vendored) | + +This tasks file is only the checklist view. It does not claim CI or package +enforcement. diff --git a/specs/013-reviewer-quality-gate/plan.md b/specs/013-reviewer-quality-gate/plan.md new file mode 100644 index 0000000..d5ba27b --- /dev/null +++ b/specs/013-reviewer-quality-gate/plan.md @@ -0,0 +1,164 @@ +# Plan: Reviewer quality gate (013) + +**Feature**: 013-reviewer-quality-gate +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-20 + +## 1. What this plan is + +A map of **who reviews what, how findings are written, and how the gate +finishes**. It is not a plan to add a reviewer bot to `thelab-langchain`. + +Success is a practiced gate: structured findings, no reviewer patches, complete +or block, human merge. This repo only records the contract. + +Living practice sits in the Hermes reviewer specialist. GitHub pull-request +review is the public analog for this repository. + +## 2. Review flow + +``` +accepted packet (009) + coder deliverable (PR or spec files) + │ + ▼ + reviewer applies lenses + correctness · security · privacy + SDD completeness · tests · acceptance criteria + │ + ├── no blocker/major → complete (approve) + │ residual risks stay documented + │ minors/notes optional follow-up for coder + │ + └── blocker or major, failed lens, or missing evidence + → block (findings + fix guidance) + → follow-up card for coder (or architect if packet is wrong) + → reviewer does not patch + │ + ▼ + human merges code (010). Reviewer complete ≠ merge. +``` + +### Step 1 — Orient + +The reviewer reads the accepted spec, acceptance criteria, residual risks, and +the actual artifact (diff or spec files). The implementer’s summary is a claim, +not proof. + +### Step 2 — Apply lenses + +All six lenses. Security-sensitive and user-facing changes cannot skip +security, privacy, or user-facing correctness. A pass with those lenses +unexamined is a rubber-stamp. + +### Step 3 — Write findings + +Each finding: severity, location, fix guidance. STE for pass/fail and fix +lines. Short prose for the verdict narrative. No secrets in the text. + +### Step 4 — Finish + +Complete or block (010). If a fix is required, the reviewer creates or returns +work to the **coder**. The reviewer does not implement. + +### Step 5 — Human merge (code) + +A human merges. That is done. + +## 3. Hermes practice vs GitHub analog + +Abstract spec terms map as follows. Profile models, CLI flags, and gateway +startup stay in the local Hermes manual (not vendored). + +| Spec term | Hermes fleet (current runtime) | GitHub public analog | +|-----------|--------------------------------|----------------------| +| Reviewer | Reviewer specialist; never implements | PR reviewer (human). This package has **no** bot. | +| Workspace | Same tree as the change (010) | The PR branch | +| Finding | Card comment: severity, location, fix guidance | Inline or summary PR comment, same shape | +| **Complete** | Board complete + short evidence summary | Approve (or comment if only minors/notes) | +| **Block** | Board block + findings | Request changes | +| Follow-up | Card for **coder** | Same PR; coder addresses review | +| Done (code) | Human merge | Human merge | + +If the board product changes, keep the spec terms and rewrite the Hermes +column. Do not fork a second severity scale. + +GitHub review without a board is still a valid analog for public PRs on this +repo. When the work *is* on the board, 010 still requires complete or block; +a GitHub comment does not replace the terminal action. + +## 4. What is STE vs human prose + +| Text | Language | +|------|----------| +| This SDD folder | Human prose | +| Finding issue + fix guidance | STE | +| Severity and location labels | The four severity words; path or section | +| Verdict narrative (“what was inspected”) | Human prose, short | +| Complete/block summary evidence (paths, PR URL, observed test counts) | Facts; no vibes (010) | + +Skill `asd-ste100` remains the procedure reference (009). Skills `code-review` +and `security-scan` may be named at dispatch. Do not paste skill bodies into +the packet or this repo. + +## 5. Follow-up routing + +| Highest finding | Reviewer does | Coder does | +|-----------------|---------------|------------| +| blocker or major | **Block.** Do not patch. | Fix **this** change; request review again. | +| minor only | **Complete** allowed. Optional follow-up card. | Optional later fix. | +| note only | **Complete.** | Nothing required. | +| Packet incomplete / design wrong | **Block.** Follow-up is **architect**, not a silent coder redesign. | Do not implement from a blocked packet. | + +The reviewer never takes the follow-up card. + +## 6. What this repo does and does not run + +| Mechanism | Status | +|-----------|--------| +| Hermes reviewer specialist | Practiced on the workstation (outside this repo) | +| SDD record in `specs/013-reviewer-quality-gate/` | This folder | +| GitHub PR review by a human | Public analog; already how this repo merges | +| Reviewer bot / GitHub Action that posts a verdict | **Not done** | +| CI that lints finding shape or blocks merges without a reviewer assignee | **Not done** | +| Runtime enforcement in `thelab-langchain` | Out of scope | + +Do not add a review agent, webhook, or “auto-approve” job under this spec. + +## 7. Relationship to 009, 010, 012 + +- **009** supplies the packet the reviewer ticks. This plan does not change + STE rules of thumb or the five artifacts. +- **010** supplies complete/block, roster, workspace kinds, and human merge. + Reviewer workspace stays the same tree as the change. +- **012** routes the reviewer onto the durable board. This plan is the gate + that specialist runs; it does not restate dispatch layers. + +Work toward 001/008 still uses this gate. 013 does not pick ASR/TTS stacks. + +## 8. Risks + +| Risk | Mitigation | +|------|------------| +| Reviewer implements “a small fix” | FR-1: review only; follow-up is coder | +| Rubber-stamp on security or user-facing work | FR-2: lenses required; complete without them is a failed review | +| Vague “needs work” | Finding shape: severity, location, fix guidance | +| Chat LGTM replaces the gate | 010: silent exit is a violation; human merge still required | +| Secrets copied into findings | Privacy lens; redact; never paste values | +| This spec treated as a bot to build | Status: living practice; no reviewer module here | +| Reviewer complete treated as ship | 010 FR-5: human merge | + +## 9. Success (qualitative) + +No invented metrics. The gate is working when: + +- Findings a coder can open (path, severity, what to change). +- Blocked reviews name the human or coder action required. +- Approved reviews name what was actually inspected. +- Reviewer diffs contain no product patches from the reviewer role. +- Merged PRs, not completed review cards, are what landed in `main`. + +## 10. What this plan is not + +It is not a Hermes CLI cheat sheet. It is not a request to vendor a reviewer +profile into git. It is not spec 005’s pytest job. It is not permission to +skip human merge because a specialist approved. diff --git a/specs/013-reviewer-quality-gate/spec.md b/specs/013-reviewer-quality-gate/spec.md new file mode 100644 index 0000000..8683ce1 --- /dev/null +++ b/specs/013-reviewer-quality-gate/spec.md @@ -0,0 +1,288 @@ +# Feature Spec: Reviewer quality gate + +**Feature ID**: 013-reviewer-quality-gate +**Status**: Living practice (documented here; **not** a product feature of `thelab-langchain`) +**Created**: 2026-08-20 +**Owner**: Derek Clair +**Related**: [009-architect-coder-handoff](../009-architect-coder-handoff/spec.md), +[010-worker-completion-protocol](../010-worker-completion-protocol/spec.md), +[012-fleet-dispatch-model](../012-fleet-dispatch-model/spec.md) + +## Record-keeping note + +This spec records the **reviewer quality gate**: how a review specialist judges +specs and pull requests, how findings are written, and how the gate may finish. + +The practice is already used on the workstation fleet (Hermes reviewer +profile). This folder is the SDD trail in the brain repo so the rule is not +only a local profile note. + +This package has **no reviewer bot**, no review LangGraph node, and no CI job +that posts a verdict. GitHub pull-request review on this repo is the **public +analog**: a human (or a forge reviewer) leaves structured comments; a human +still merges. + +Do **not** copy a Hermes profile `SOUL.md`, skill body, or operating manual +into this tree. This document **distills** the gate. + +## Overview + +A **reviewer** reads the deliverable and the accepted packet. The reviewer does +not ship the fix. The job is a quality gate, not a second coder. + +The gate applies to: + +- **Specs / SDD** (durable directory): completeness, honesty, acceptance + criteria, residual risks. +- **Pull requests / code** (git worktree): correctness, security, privacy, + tests, and match to accepted acceptance criteria. + +Every finding has a **severity**, a **location**, and **fix guidance**. The +reviewer always **completes** or **blocks** (spec 010). A quiet exit is a +protocol violation. Reviewer complete is not merge. A human merges. + +## Goals + +- Keep review and implementation in different roles. +- Make the gate’s lenses explicit: correctness, security, privacy, SDD + completeness, tests, acceptance criteria. +- Make findings actionable: severity, location, what to change. +- Stop rubber-stamps on security-sensitive or user-facing work. +- Finish every review run with complete or block; keep human merge as done + for code. + +## Non-goals + +- A reviewer service, bot, or module inside `thelab-langchain`. +- CI in this repo that assigns a reviewer profile or posts a verdict. +- Vendoring Hermes profile files, skill bodies, or the fleet operating manual. +- The architect ↔ coder packet language (that is spec 009). +- The complete/block terminal itself (that is spec 010). This spec says how + the **reviewer** uses that terminal. +- Coverage percentages, latency SLOs, or invented pass rates. +- Permission for the reviewer to “just quickly” patch the branch. + +## Roles + +| Role | Does | Does not | +|------|------|----------| +| **Reviewer** | Reviews specs and PRs. Writes structured findings. Completes (approve) or blocks. Opens a follow-up card for the **coder** when a fix is required. | Implement the fix. Merge. Accept the design in place of the human. Rubber-stamp security-sensitive or user-facing changes. | +| **Coder** | Implements the accepted spec, including fixes the reviewer required. | Review their own PR as the quality gate. Merge. | +| **Architect** | Designs; names residual risks the reviewer will read. | Implement. Wear the reviewer hat on the same packet they just wrote without an independent pass. | +| **Human** | Accepts the design packet (009). Merges code (010). May also review. | Treat reviewer complete as ship. Skip the gate on security-sensitive or user-facing work because the coder was confident. | + +The reviewer never implements. A required fix is a **coder** follow-up (same +PR / returned card for blocker and major; optional later card for minor). + +## Domain terms (define once) + +| Term | Meaning | +|------|---------| +| **Quality gate** | The review pass on a spec or PR before the next legal step (human accept of a spec is 009; human merge of code is 010). | +| **Finding** | One issue. It has severity, location, and fix guidance. | +| **Location** | Path and line, or spec section heading. Enough for a coder to open the artifact. | +| **Fix guidance** | What must change. Not a pasted patch from the reviewer. | +| **Blocker** | Must be fixed before merge (code) or before human accept / coder start (spec). | +| **Major** | Must be fixed in **this** PR (or this spec revision). Not a later card. | +| **Minor** | Nit. Follow-up card is allowed. Does not by itself block complete. | +| **Note** | Non-blocking observation. No fix required. | +| **Rubber-stamp** | Approve without applying the required lenses, especially on security-sensitive or user-facing changes. | +| **User-facing** | Anything a person at the desk sees or hears: CLI text, docs, spoken replies (011), UI. | +| **Security-sensitive** | Auth, secrets, privacy, untrusted input, tool/exec boundaries, network exposure. | +| **Public analog** | GitHub PR review: comments and approve / request-changes. Same gate, different surface. Not a bot in this package. | +| **Follow-up card** | Work for the **coder** that exists because review found a defect. The reviewer does not take that card. | + +Do not call a chat “LGTM” a finding. Do not call reviewer complete “merged.” + +## Severity scale + +| Severity | Meaning | Effect on the gate | +|----------|---------|-------------------| +| **blocker** | Must fix before merge (code) or before the packet is ready (spec). | **Block.** Do not approve. | +| **major** | Must fix in this PR / this spec revision. | **Block.** Do not approve. Follow-up is the same change, not a later PR. | +| **minor** | Nit. Follow-up OK. | Complete is allowed. Optional coder follow-up card. | +| **note** | Non-blocking observation. | Complete is allowed. No fix required. | + +If both a blocker and a note exist, the gate is **block**. The highest +severity present wins. + +Residual risks named by the architect (009) are **not** automatic blockers. +Unstated scope, failed acceptance criteria, and new security/privacy defects +**are**. + +## Finding shape + +Each finding is structured. Missing any field is an incomplete finding. + +1. **Severity** — blocker, major, minor, or note. +2. **Location** — `path:line` or spec section. No private board/chat + identifiers required. No secret values. +3. **Issue** — one sentence. What is wrong. STE for this line. +4. **Fix guidance** — one or more sentences. What the coder (or architect, + if the packet is wrong) must change. STE. Not the implementation. + +Pass/fail lines are STE. The short verdict narrative may be ordinary prose +(009 default, now the rule here). + +Do not paste keys, tokens, serials, or other secrets into a finding. Name the +class of leak and the location. Redact values. + +## Check lenses + +The reviewer applies all of these. Skipping a lens on security-sensitive or +user-facing work is a rubber-stamp. + +### Correctness + +- The change does what the accepted spec claims. +- Edge cases and error paths that the spec named are handled. +- Unstated work is fail (scope creep) or a new spec, not a silent extra. + +### Security + +- No secrets in the diff, specs, tests, comments, or board fields (010 FR-3). +- Untrusted input is not passed to shell, SQL, or path joins without a check. +- Authz and tool boundaries match the spec. Do not invent a threat model that + the spec did not ask for; do report an obvious hole. + +### Privacy + +- No keys, serials, private chat/issue identifiers, family data, or host home + paths as required layout in the change or in the review text. +- Findings themselves obey this rule. + +### SDD completeness + +- Specs under review have honest status, goals, non-goals, and binary + acceptance criteria. +- Architect → coder packets still have the five artifacts (009). +- Related specs are cited; this package is not claimed to own out-of-tree + work (008 honesty). + +### Tests + +- New behavior has tests, or the spec explicitly waived them. +- Tests assert behavior, not a snapshot of source text. +- Observed test counts only (010). Do not invent coverage. + +### Acceptance criteria + +- Each accepted criterion maps to evidence the reviewer actually inspected. +- A criterion with no evidence is not a pass. +- “Looks good” is not evidence. + +## Verdicts + +Exactly one terminal action (010): + +| Verdict | When | Board | GitHub public analog | +|---------|------|-------|----------------------| +| **Complete** (approve) | No blocker, no major. Lenses applied. Evidence named. | Complete with a short summary and artifact list. | Approve, or comment if only minors/notes. | +| **Block** | Any blocker or major, failed lens on security-sensitive / user-facing work, missing evidence, or a human decision is required. | Block. Findings on the card. Follow-up for **coder** (or architect if the packet is wrong). | Request changes. Same findings as inline or summary comments. | + +Reviewer complete **is not merge**. Human merge remains the definition of done +for code (010 FR-5). + +A GitHub comment-only review with no board complete/block is still a +**protocol violation** when the work is on the durable board. The analog is +the comment shape, not permission to go silent. + +## Functional requirements + +### FR-1 Review only + +- Reviewer output is findings and a verdict. +- Reviewer MUST NOT edit product code, specs under review, or tests to “help.” +- Required fixes go to a **coder** follow-up card (blocker/major: this change; + minor: optional later card). + +### FR-2 Lenses + +- Every review of a spec or PR MUST consider correctness, security, privacy, + SDD completeness, tests, and acceptance criteria. +- Security-sensitive or user-facing changes MUST NOT be completed unless those + lenses were actually applied. Rubber-stamp is a failed review. + +### FR-3 Structured findings + +- Each finding MUST include severity, location, and fix guidance. +- Severity MUST be one of: blocker, major, minor, note. +- Findings MUST NOT contain secrets or private coordination identifiers. + +### FR-4 Complete or block + +- The reviewer run MUST end with **complete** or **block** (010 FR-1). +- Clean exit with neither is a protocol violation, not an implicit approve. + +### FR-5 Human merge + +- Reviewer approve does not land the PR. +- A human merges. That merge is done for code. + +### FR-6 Spec vs PR + +- Spec review: packet completeness and honesty; block if the five artifacts + or binary acceptance criteria are missing when 009 applies. +- PR review: diff against the **accepted** spec; block if acceptance criteria + fail or the coder redesigned in the PR. + +## Non-functional requirements + +- Practiced as the Hermes reviewer specialist on the workstation. This + repository does not run that profile. +- Specs in `specs/` remain human-readable SDD. Finding *lines* on a card are + STE; this folder is prose. +- No reviewer metrics theater (approval rate, mean time to review). +- Skill names `code-review` and `security-scan` may be used at dispatch. + Do not vendor those skill bodies here. + +## User stories + +1. As reviewer, I only judge the artifact; I never become the coder on the + same card. +2. As coder, I get findings I can open (path, severity, what to change), not + “needs work.” +3. As human, I still merge; a green review card is not ship. +4. As human, security-sensitive and user-facing changes are not rubber-stamped. +5. As architect, residual risks I named are read, not silently “fixed” by the + reviewer. + +## Acceptance criteria (for this SDD record) + +- [x] This folder contains `spec.md`, `plan.md`, and `tasks.md` that name + review-only, the four severities, structured findings, the six lenses, + complete-or-block, and human merge. +- [x] Status is “living practice,” not a `thelab-langchain` feature. +- [x] Honest that this package has no reviewer bot; GitHub PR review is the + public analog. +- [x] No Hermes profile file or skill body is copied into this repo. +- [ ] CI or a bot in this package that runs the reviewer — **not done** + (out of scope until a later spec). + +## Relationship to other specs + +- **009** — handoff language and packet. Reviewer ticks acceptance criteria + and reads residual risks. Fail/pass lines in STE; narrative in prose + (closes 009’s open question for reviewer output). +- **010** — terminal action, roster, workspace, human merge. Reviewer is on + the closed roster and MUST complete or block. This spec is *how* that + role reviews. +- **012** — who is dispatched and on which layer. Reviewer is a **durable + board** specialist, not an in-process subagent and not the orchestrator. + This spec is *how* that role reviews; 012 does not define severity or + finding shape. +- **005** — tests and CI for this package’s Python. Reviewer checks test + *gaps* against the accepted spec; 013 does not add a coverage gate. +- **007** — quality-critical roles. Model routing stays there / in Hermes, + not in this spec. +- **011** — spoken replies are user-facing. A voice-facing change is not + rubber-stamped. +- **008** — out-of-tree I/O spike. Review of that work still does not + implement, and still does not pretend this package owns playback. + +## Open questions + +- Whether a later spec should add a GitHub Action that only *checks* “PR has + a human review,” still not a reviewer bot. Default: **not in 013**. +- Whether minor-only GitHub reviews should be Approve or Comment. Default: + **Approve or Comment is allowed; board complete is still required.** diff --git a/specs/013-reviewer-quality-gate/tasks.md b/specs/013-reviewer-quality-gate/tasks.md new file mode 100644 index 0000000..0e6337b --- /dev/null +++ b/specs/013-reviewer-quality-gate/tasks.md @@ -0,0 +1,71 @@ +# Tasks: Reviewer quality gate (013) + +**Feature**: 013-reviewer-quality-gate +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) + +Checkboxes are honest. The gate is **practiced in Hermes**. This package has +**no reviewer bot**. GitHub PR review is the public analog. This folder is the +SDD record, not a `thelab-langchain` feature. + +## Phase 0 — Record the practice (this repo) + +- [x] Write `spec.md` with review-only role, four severities, structured + findings, six lenses, complete-or-block, human merge +- [x] Write `plan.md` with Hermes vs GitHub analog, follow-up routing, and + “no bot in this package” +- [x] Write `tasks.md` (this file) +- [x] State status as living practice, not a product feature +- [x] Point at related specs 009, 010, and 012 (fleet dispatch) without vendoring profile files + +## Phase 1 — Workstation practice (Hermes, outside this repo) + +Practiced on the fleet; not code in `src/thelab_langchain/`. + +- [x] Reviewer reviews specs and PRs; does not implement fixes +- [x] Fixes go to a coder follow-up (blocker/major on this change) +- [x] Lenses: correctness, security, privacy, SDD completeness, tests, + acceptance criteria +- [x] Findings use severity, location, and fix guidance +- [x] Severity words: blocker, major, minor, note +- [x] No rubber-stamp of security-sensitive or user-facing changes +- [x] Reviewer always completes or blocks (010) +- [x] Human merge remains definition of done for code + +Do not copy profile files or skill bodies into this tree to “complete” a +checkbox. + +## Phase 2 — Public analog (this GitHub repo, not a bot) + +- [x] Humans review pull requests on this repo (comments / approve / + request-changes) as the public analog of the gate +- [ ] GitHub Action or app that posts a reviewer-profile verdict — **not done** + +Phase 2’s empty box is out of scope for 013. Leave it empty. + +## Phase 3 — Enforcement in this package (**not done**) + +- [ ] Reviewer module or LangGraph node in `thelab-langchain` +- [ ] CI job that requires a reviewer assignee or lints finding shape +- [ ] Auto-approve or auto-merge on specialist complete + +Phase 3 is **out of scope** for 013. Do not implement them under this spec. + +## Explicitly not tasks in thelab + +Do not open work in this package for: + +- A reviewer bot, webhook, or forge app +- Pytest that asserts Hermes complete/block or finding markdown +- Copying the reviewer profile or `code-review` / `security-scan` skill + bodies into git +- Changing 009 packet shape or 010 terminals + +## Traceability + +- Practice: Hermes reviewer specialist (outside this repo). +- Terminals and roster: spec 010. +- Packet language: spec 009. +- Dispatch layer: spec 012 (reviewer is a durable-board specialist). +- Public analog: GitHub pull-request review; human merge. +- This tasks file is only the checklist view. It does not claim a bot or + package enforcement. diff --git a/specs/014-memory-injection-graph/plan.md b/specs/014-memory-injection-graph/plan.md new file mode 100644 index 0000000..7bba045 --- /dev/null +++ b/specs/014-memory-injection-graph/plan.md @@ -0,0 +1,138 @@ +# Plan: LangGraph memory-injection graph (014) + +**Feature**: 014-memory-injection-graph +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-20 +**Status**: Implemented (graph) + +This plan records the architecture that **already shipped**. It is not a +proposal to rewrite `graph.py`. Follow-on work is only checkpointer (004), +an eval harness, and a **measured** injection-vs-summarization comparison. + +## 1. Architecture (as shipped) + +This repo is the brain. Live voice I/O imports `get_agent()` and does not +fork the graph (spec 008). + +``` +Caller (voice I/O or tests) + session messages + user_id + │ + ▼ + get_agent(user_id) graph.compile() # no checkpointer (004) + │ + ▼ + memory_injection entry + last HumanMessage + create_memory_tools(state.user_id) + get_user_profile fail-open invoke → "" + recall_memories(limit=3) fail-open invoke → "" + prepend SystemMessage raw text; no summarization LLM + │ + ▼ + call_llm tools bound at graph-build user_id + │ + ├── tool_calls → execute_tools (ToolNode) → call_llm + └── else END +``` + +| Piece | Owner | +|-------|--------| +| Graph, injection, tool loop | **this repo** (`src/thelab_langchain/agent/graph.py`) | +| Tool factory | `create_memory_tools(user_id)` (spec 006) | +| `user_id` → `container_tag` | spec 002 | +| Short-term turns | Caller list. No checkpointer (spec 004) | +| Live speakerphone | [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) calls `get_agent()` | +| Text CLI | `MemoryChat` — **not** this graph | + +Two `create_memory_tools` call sites: + +- `build_agent_graph(user_id)` binds tools for `call_llm` / `execute_tools`. +- `_memory_injection` rebuilds tools from `state.user_id`. + +Callers must pass the same id both places. The graph does not reconcile them. + +## 2. Tech choices (locked for what shipped) + +| Concern | Choice | Why | +|---------|--------|-----| +| Graph type | Custom `StateGraph(AgentState)`, not `create_react_agent` | Control the injection node | +| Entry | `memory_injection` before `call_llm` | Context without a first tool-call | +| Recall depth on inject | `limit=3` | Small raw block; main LLM still has reactive `recall_memories` | +| Injection payload | Raw tool strings as `SystemMessage` | Skip a second model call per turn | +| “500-1000ms saved” | **Unmeasured** comment in `graph.py` | Design rationale only; not a result | +| Live n=3 timings | Sibling I/O README | Do not copy into this package | +| Fail-open | `except Exception: ""` on **invoke** | Empty context, not a dead turn | +| Factory errors | Not swallowed | Missing memory key still raises | +| Reactive tools | Same three names, `ToolNode`, loop to `call_llm` | Store / deeper recall when the model asks | +| Checkpointer | None | Spec 004 | +| Summarization node | **Forbidden** until measured and chosen | Extra round-trip is the thing this design skipped | + +001’s older note about “optional LLM summarization of retrieved memories” +did **not** land. 014 supersedes that micro-iteration: raw injection is the +shipped path. + +## 3. Phases + +### Phase 0 — SDD record (this folder) + +- Write spec / plan / tasks that match `graph.py` and + `tests/test_agent_graph.py`. +- Label the in-code latency range unmeasured. +- Point at the I/O README for live spoken timings; do not paste that table. + +### Phase 1 — Graph (shipped) + +Already in the tree. Do not re-check these as future work: + +- `memory_injection` → `call_llm` → optional `execute_tools` loop. +- Last `HumanMessage` as recall query; profile + `limit=3`. +- Fail-open invoke; empty combined → `{}`. +- Raw `SystemMessage`; unit test that `get_chat_model` is not used here. +- `get_agent(user_id)` compiled with no checkpointer. +- Public export: `thelab_langchain.get_agent`. + +### Phase 2 — Not this graph (unchecked) + +Do not mark 014 “complete including follow-ons” until these exist: + +1. **Checkpointer** — spec 004. Not a 014 rewrite of injection. +2. **Eval harness** — injection quality with mocked (or gated live) memory. + None in `tests/` today. +3. **Measured latency** — injection vs an extra summarization LLM, recorded + as a measurement. Until then, keep the `graph.py` range labeled + unmeasured and off the résumé. + +## 4. Risks + +| Risk | Mitigation | +|------|------------| +| Treating “500-1000ms” as a result | Spec + tasks: unmeasured; live table stays in I/O README | +| Extra summarization “to improve quality” | Forbidden without a measured comparison | +| Injection `user_id` ≠ bound-tool `user_id` | Same id on `get_agent` and state (spec 002) | +| Factory raise vs invoke fail-open | Document: missing key is not empty context | +| Double-storing history if 004 lands | 004 plan: pick one owner of short-term turns | +| Copying transcripts / identifiers into eval fixtures | Privacy: no keys, serials, transcripts, household ids | +| I/O reimplementing injection | Spec 008: one import, `get_agent()` | + +## 5. Success metrics + +Shipped (graph): + +- A compiled `get_agent()` turn always hits `memory_injection` before the + LLM. +- Tests prove injection does not call `get_chat_model`. +- Empty / failed invoke does not require a summarization model to recover. + +Not shipped (do not claim): + +- Process restart keeps short-term turns (004). +- Eval scores for recall/profile usefulness. +- A measured delta for injection vs summarization. + +## 6. What this plan is not + +It is not a checkpointer. It is not a second memory backend. It is not +`MemoryChat`. It is not the I/O latency table. It is not permission to print +unmeasured milliseconds as a benchmark. It is not spec 011 speakability. +It does not replace spec 001’s broader desktop-voice goal. diff --git a/specs/014-memory-injection-graph/spec.md b/specs/014-memory-injection-graph/spec.md new file mode 100644 index 0000000..3271154 --- /dev/null +++ b/specs/014-memory-injection-graph/spec.md @@ -0,0 +1,235 @@ +# Feature Spec: LangGraph memory-injection graph + +**Feature ID**: 014-memory-injection-graph +**Status**: Implemented (graph) +**Created**: 2026-08-20 +**Owner**: Derek Clair +**Related**: [002-multi-user-support](../002-multi-user-support/spec.md) (`user_id`), +[004-persistence-checkpointers](../004-persistence-checkpointers/spec.md) (not wired), +[006-alternative-memory-systems](../006-alternative-memory-systems/spec.md) (narrow tool interface), +[008-local-tts-lenovo-go-spike](../008-local-tts-lenovo-go-spike/spec.md) (voice uses `get_agent()`) + +This folder documents **what shipped** in this repo. It is not a redesign. + +## Honest current state + +This package’s distinctive code is the LangGraph in +`src/thelab_langchain/agent/graph.py`. + +The compiled agent is `get_agent(user_id)`. Entry node is `memory_injection`. +It runs **before** `call_llm`. It pulls the last `HumanMessage`, then calls +`get_user_profile` and `recall_memories(limit=3)` from +`create_memory_tools(user_id)`. Tool **invoke** failures become empty strings +(fail-open). Non-empty raw profile + recall is prepended as a `SystemMessage`. +There is **no** extra LLM summarization pass. + +The in-code comment that this avoids “500-1000ms of latency per voice turn” +is a **rationale, unmeasured**. Do not put that range on a résumé, in a +benchmark claim, or in this spec as a result. Live spoken-turn timings (n=3 +table) live in the +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) +README, not here. This spec does not copy that table and does not invent +numbers. + +Then `call_llm` runs with the same three memory tools bound. If the model +emits `tool_calls`, `ToolNode` executes and the graph loops back to +`call_llm`. Otherwise it ends. + +`get_agent()` is `graph.compile()` with **no checkpointer** (spec 004). +Callers that need a session keep a `HumanMessage` / `AIMessage` list +themselves. Text CLI (`MemoryChat`) is a **different** path: it does not use +this graph. + +Unit tests in `tests/test_agent_graph.py` cover routing and injection with +mocked tools (including “no summarization LLM”). There is no eval harness and +no measured injection-vs-summarization latency in this tree. + +## Overview + +On every `get_agent()` turn, give the model user context **without waiting for +it to tool-call**, then still let it store or recall reactively. + +Two memory modes, one factory: + +1. **Proactive** — `memory_injection` fetches profile + a few recalled + memories from the last user utterance and prepends them as raw context. +2. **Reactive** — `call_llm` has `get_user_profile`, `recall_memories`, and + `store_memory` bound so the model can deepen recall or write a fact. + +Long-term store is Supermemory, scoped by `user_id` as `container_tag` +(spec 002 / 006). Short-term turns are the caller’s message list (spec 004). +Live speakerphone I/O imports this graph and does not reimplement it +(spec 008). + +## Goals + +- Document the shipped graph: entry injection, LLM + tools, tool loop, no + checkpointer. +- Keep injection raw (no extra summarization round-trip). +- Fail-open on memory **invoke** so a recall/profile error does not kill the + turn. +- Keep the tool interface the three names in spec 006. +- Stay honest about latency: unmeasured comment in `graph.py`; live n=3 table + is in the sibling I/O README, not this package. + +## Non-goals + +- Wiring a LangGraph checkpointer (spec 004). +- A second memory backend or `MemoryBackend` ABC (spec 006). +- Speaker ID / tenant product (spec 002). +- Speakability filter on `AIMessage.content` (spec 011). +- Copying or paraphrasing Hermes `SOUL.md`. +- Copying the sibling-repo live latency table into this tree. +- Inventing or reprinting millisecond numbers as if measured here. +- Changing `MemoryChat` / `thelab-chat` to use this graph. +- ALSA, Piper, Parakeet, LED, or button interrupt (spec 008 / I/O repo). + +## User stories + +1. As the person at the desk, I speak a turn and the model already has my + profile and a few relevant memories, without a tool-call round-trip first. +2. As that person, I can still have the model store a new fact or search + memory more deeply in the same turn. +3. As a developer, I know `get_agent(user_id)` is the brain seam the I/O + process calls. +4. As a developer, I know injection is fail-open on tool invoke, and that + missing context is a no-op (`{}`), not a crash of the node. +5. As a developer, I do not treat the “500-1000ms” comment as a benchmark. + +## Functional requirements + +### FR-1 Graph shape (shipped) + +``` +memory_injection → call_llm → execute_tools → call_llm (loop) + │ + └── END (no tool_calls) +``` + +- Entry point is `memory_injection`. +- Unconditional edge: `memory_injection` → `call_llm`. +- Conditional: last message is `AIMessage` with `tool_calls` → + `execute_tools`; otherwise `END`. +- `execute_tools` always returns to `call_llm`. +- Public factory: `get_agent(user_id="default-user")` compiles that graph. + +### FR-2 Proactive injection (shipped) + +`_memory_injection(state)`: + +1. `user_id` from `state.user_id`, default `"default-user"`. +2. Last `HumanMessage` content, scanning `state.messages` from the end. +3. `create_memory_tools(user_id)` (same factory as the bound tools). +4. `get_user_profile.invoke({"query": last_user_msg or None})` if that tool + exists. +5. `recall_memories.invoke({"query": last_user_msg, "limit": 3})` if that + tool exists **and** there is a last user utterance. +6. Combine: profile text, then `## Relevant Long-term Memories` + recall. +7. If combined is empty, return `{}` (no injection). +8. Else prepend + `SystemMessage("## User Context (from long-term memory)\n" + combined)` + and return `{"messages": [injection] + list(state.messages)}`. + +Do **not** call `get_chat_model()` in this node. Raw tool strings go to the +main LLM. + +`AgentState.long_term_context` exists on the state model and is **not** +written by this node. Injection is the `SystemMessage` on `messages`. + +### FR-3 Fail-open on invoke (shipped) + +- Exceptions from `profile_tool.invoke` / `recall_tool.invoke` become `""`. +- Missing tools are skipped. +- Empty combined context → `{}`. +- **Not** fail-open: `create_memory_tools` itself (missing + `SUPERMEMORY_API_KEY` still raises in the factory). Documented so operators + do not confuse “empty context” with “missing key”. + +### FR-4 LLM + reactive tools (shipped) + +- `call_llm` binds `create_memory_tools(user_id)` from **graph build** + (`get_agent(user_id)` / `build_agent_graph(user_id)`). +- Tools: `get_user_profile`, `recall_memories`, `store_memory` (spec 006). +- `execute_tools` is LangGraph `ToolNode(tools)`. +- Injection recreates tools from `state.user_id`. Callers should pass the + same `user_id` to `get_agent` and in `AgentState` so the two scopes match + (spec 002). + +### FR-5 No checkpointer (shipped absence) + +- `get_agent()` is `return graph.compile()` — no `checkpointer` argument. +- Spec 004 remains not wired. This spec does not pretend persistence exists. + +### FR-6 Honesty about latency + +- Design intent: skip a summarization LLM so a voice turn does not pay an + extra model round-trip. That intent is in `graph.py` and in unit tests + (`get_chat_model` must not be called from `_memory_injection`). +- The “500-1000ms” figure is **unmeasured in-code rationale**. Not a result. +- Measured spoken-path tables, if any, stay in the I/O repo README. Do not + duplicate them here. + +## Non-functional requirements + +- No secrets, serials, transcripts, or household identifiers in this spec or + in example utterances. +- No invented latency numbers. +- Same `get_agent(user_id)` seam as spec 008. +- Long-term access only through the three tools (spec 006). +- Privacy: `container_tag=user_id`; do not search across users. + +## Acceptance criteria + +- [x] Entry node is `memory_injection`; it runs before `call_llm`. +- [x] Injection uses last `HumanMessage` + `get_user_profile` + + `recall_memories(limit=3)` via `create_memory_tools(user_id)`. +- [x] Invoke failures become empty strings; empty combined context returns + `{}`. +- [x] Injection is a prepended `SystemMessage` of raw context (no extra LLM). +- [x] `call_llm` binds tools; `ToolNode` loops back when `tool_calls` exist. +- [x] `get_agent()` compiles with no checkpointer. +- [x] Unit tests: routing + injection without a summarization LLM + (`tests/test_agent_graph.py`). +- [ ] Checkpointer wired (spec 004 — out of this graph’s shipped scope). +- [ ] Eval harness for injection quality. +- [ ] Measured latency of injection vs an extra summarization round-trip + (do not treat the in-code range as that measurement). + +## Seams this package must keep stable + +| Seam | Contract | +|------|----------| +| `get_agent(user_id)` | Compiled graph. Voice I/O (spec 008) invokes this. | +| `graph.invoke({"messages": ...})` | Caller supplies the turn (and any session history). | +| `create_memory_tools(user_id)` | Only long-term memory factory (spec 006). | +| `user_id` | Opaque string; Supermemory `container_tag` (spec 002). | +| Injection message | `SystemMessage` titled `## User Context (from long-term memory)`. | +| Checkpointer | None. Caller owns short-term turns (spec 004). | + +## Relationship to other specs + +- **002** — `user_id` on `get_agent` / `AgentState` / tools. Injection does + not identify speakers. +- **004** — checkpointer not wired. This graph does not add one. +- **006** — three-tool interface; no second backend. +- **008** — live voice consumes `get_agent()`. I/O does not copy `graph.py`. +- **011** — reply speakability is a wanted contract, not this node. +- **001 / 007** — broader desktop-voice / Spark budget. This folder is the + brain graph that actually shipped. + +## Open questions + +- Should `_memory_injection` close over the graph-build `user_id` instead of + reading `state.user_id`, so the two tool sets cannot diverge? +- Should the node return only the new `SystemMessage` (rely on `add_messages`) + instead of `[injection] + list(state.messages)`? +- Should `AgentState.long_term_context` be removed or actually used? +- How (and where) to measure injection vs summarization without putting + unmeasured ranges in this package. +- Eval harness: offline fixtures with mocked tools, or a gated live-memory + job? Neither exists. + +--- + +**Status**: Graph implemented in this tree. Checkpointer, eval harness, and a +measured injection-vs-summarization comparison are **not** done. diff --git a/specs/014-memory-injection-graph/tasks.md b/specs/014-memory-injection-graph/tasks.md new file mode 100644 index 0000000..ff840a4 --- /dev/null +++ b/specs/014-memory-injection-graph/tasks.md @@ -0,0 +1,84 @@ +# Tasks: LangGraph memory-injection graph (014) + +**Feature**: 014-memory-injection-graph +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) +**Status**: Implemented (graph) + +Checkboxes record what is in this tree. Graph work is done. Checkpointer, +eval harness, and a **measured** injection-vs-summarization comparison are +not. + +## Phase 0 — SDD record (this folder) + +- [x] Write `spec.md` (shipped graph, fail-open, raw injection, no + checkpointer, unmeasured latency comment) +- [x] Write `plan.md` (architecture as shipped; 001 summarization note + superseded) +- [x] Write `tasks.md` (this file) +- [x] Point live n=3 spoken timings at the I/O README; do not copy the table +- [x] In this SDD, label the `graph.py` “500-1000ms” comment as unmeasured rationale, not a result (code comment unchanged) + +## Phase 1 — Graph (shipped) + +- [x] `StateGraph(AgentState)` with nodes `memory_injection`, `call_llm`, + `execute_tools` +- [x] Entry point `memory_injection`; edge to `call_llm` +- [x] Pull last `HumanMessage` as the recall query +- [x] `create_memory_tools(user_id)` for injection (`state.user_id`) +- [x] `get_user_profile` + `recall_memories(limit=3)` on the injection path +- [x] Fail-open: invoke exceptions → `""`; empty combined → `{}` +- [x] Prepend raw context as `SystemMessage` (`## User Context (from + long-term memory)`) +- [x] No extra summarization LLM in `_memory_injection` +- [x] `call_llm` binds the three memory tools from graph-build `user_id` +- [x] `ToolNode` when last `AIMessage` has `tool_calls`; else `END` +- [x] Loop `execute_tools` → `call_llm` +- [x] `get_agent(user_id)` → `graph.compile()` with **no** checkpointer +- [x] Export `get_agent` from `thelab_langchain` +- [x] Unit tests: `_should_continue` routing +- [x] Unit tests: injection prepends `SystemMessage`; `get_chat_model` not + called (`tests/test_agent_graph.py`) +- [x] Unit test: empty profile+recall returns `{}` + +## Phase 2 — Follow-ons (not done) + +These are **not** required to call the graph implemented. Leave unchecked +until code or measurements exist. + +- [ ] LangGraph checkpointer on `get_agent()` (spec 004 — factory, namespaced + `thread_id`, tests that two threads do not share state) +- [ ] Eval harness for injection quality (fixtures; no transcripts or + household identifiers in git) +- [ ] Measured latency of raw injection vs an extra summarization round-trip + (record the method and the result; do **not** promote the in-code + 500-1000ms comment) + +## Out of scope (stay unchecked here) + +- [ ] Second memory backend / `MemoryBackend` ABC (spec 006) +- [ ] Speaker ID / tenant product (spec 002) +- [ ] Voice speakability filter (spec 011) +- [ ] Switching `MemoryChat` onto this graph +- [ ] Copying sibling-repo n=3 latency tables into this package +- [ ] Fail-open around `create_memory_tools` / missing API key (factory still + raises; not a 014 bug-fix unless a later spec asks) + +## Traceability + +| Want | Code today | +|------|------------| +| Proactive memory before LLM | `_memory_injection` entry node | +| Raw context, no extra LLM | `SystemMessage` prepend; test asserts `get_chat_model` unused | +| Fail-open invoke | `except Exception: ""` on profile/recall invoke | +| Reactive store/recall | tools bound + `ToolNode` loop | +| Per-user scope | `create_memory_tools(user_id)` (spec 002 / 006) | +| Short-term persistence | **None** — spec 004 | +| Eval / measured injection latency | **Not in this tree** | + +Code: `src/thelab_langchain/agent/graph.py`, +`src/thelab_langchain/agent/tools/memory.py`, +`src/thelab_langchain/agent/state.py`, +`tests/test_agent_graph.py`. + +Live consume path: [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) +calls `thelab_langchain.agent.graph.get_agent`. diff --git a/specs/015-content-free-telemetry/plan.md b/specs/015-content-free-telemetry/plan.md new file mode 100644 index 0000000..fcbee81 --- /dev/null +++ b/specs/015-content-free-telemetry/plan.md @@ -0,0 +1,140 @@ +# Plan: Content-free / privacy-tiered telemetry (015) + +**Feature**: 015-content-free-telemetry +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-20 +**Status**: Specified. Turns implemented in the voice sibling; graph OTEL not +in this package. + +## 1. Two planes (do not mix them) + +``` +desk loop (008) + utterance → STT → get_agent() → TTS + │ + ├─ on-host JSONL / transcripts (gitignored; may contain text) + │ + └─ opt-in OTLP metrics ──► lan-agent-otel + allow-list: asr/agent/tts/total/eou/ttfa ms + + turn/error/cancel counts + + service.name, generic host id + never: transcripts, prompts, tool bodies +``` + +`thelab_langchain` today sits only on the `get_agent()` box. It does not sit +on either telemetry arrow. + +The LAN hub (`collector → Prometheus / Loki / Tempo / Grafana`) is a **different +git repo**. Point at [`derekclair/lan-agent-otel`](https://github.com/derekclair/lan-agent-otel). +Do not copy compose, collector YAML, IPs, or hostnames into this tree. + +Example bases an operator may set in **local env** (not in git): + +- `http://127.0.0.1:4318` +- `http://collector-host:4318` + +## 2. What already exists (voice sibling) + +Executed in [`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent), +not here. + +| Piece | Behavior | +|-------|----------| +| Local JSONL | Structured events, including `turn_complete` with stage durations | +| Duration keys | `asr_ms`, `agent_ms`, `tts_ms`, `total_ms`, `eou_ms`, `tts_ttfa_ms` | +| Opt-in OTLP | No-op unless an endpoint env is set; histograms of those keys + counters | +| Content rule | OTLP module reads **numeric keys only**; text keys are ignored | +| Fail-open | Missing SDK / down collector does not stop the loop | + +008’s tasks already mark “content-free opt-in OTEL” done **in that repo**. +This plan does not re-implement it in `thelab`. + +On-host JSONL may still hold speech for the operator. That is the on-host +plane. Remote export of that same payload is out of policy (spec FR-2 / FR-3). + +## 3. What does not exist (this package) + +No OpenTelemetry dependency, bootstrap, or span around the LangGraph graph. +`graph.py` injects memory and calls the LLM. It does not record histograms. + +Honest consequence: Grafana cannot show “thelab graph stages” until someone +implements a later phase. Voice-turn `agent_ms` is wall time around `invoke` +in the I/O process. That is enough for 008 latency tables. It is not a graph +trace. + +## 4. Privacy policy (restate; enforce in review) + +**Forbidden in git** (tracked files, commit messages, PR bodies, CI logs, +example JSONL, screenshots): + +- Transcripts and session dumps +- Prompts, system / memory-injection text, completions +- Tool args/results that carry user content (including 011 unspeakable dumps) +- API keys, tokens, `.env` contents, bearer headers +- Lab IPs (RFC1918 and any site-identifying address) +- Hardware serials, MACs, GPU/CPU/disk identifiers + +**Allowed in git and on the wire (tier A):** + +- Durations +- Counts +- Low-cardinality labels: `service.name`, generic host id + +Do not use user id, session id, or free-text as an OTLP label. + +## 5. Suggested sequence (if we implement graph metrics later) + +Not a commitment. Default order if someone picks this up: + +1. **Keep the sibling path as the live turn exporter.** Do not duplicate + `agent_ms` inside `get_agent()` just to look busy. +2. **If graph-internal stages are needed**, add a tiny, fail-open recorder in + *this* package that emits only numeric stage times (memory injection, LLM, + tools). Same allow-list as FR-1. Opt-in via `OTEL_EXPORTER_OTLP_ENDPOINT` + (or a package-specific alias). Example base still `http://127.0.0.1:4318` + or `http://collector-host:4318`. +3. **Allow-list in code**, not redaction. Tests: a payload that includes + `user_text` / `prompt` / tool args must not appear in the exported metric + attributes. +4. **Do not** enable OpenInference prompt capture, LangSmith SaaS dual-export, + or OTEL log-user-prompt flags. +5. **Do not** vendor the hub stack. Operators already have `lan-agent-otel`. + +Tier B (token counts, model id, tool *names*) is optional later and still +must not include bodies. Tier C stays off. + +## 6. 011 and logs + +A markdown table or fenced dump that slips through Piper is already a 011 +failure. Shipping that string to Loki “so we can see what Piper said” is a +015 failure on top. Debug on-host JSONL if needed; do not open a content +back-channel. + +## 7. What we will not do in this plan + +- Copy `lan-agent-otel` compose, IPs, or hostnames. +- Claim `thelab_langchain` exports OTEL. +- Put measured TTS timings in this tree (008 / sibling README). +- Add an extra LLM call to “summarize the turn for metrics.” +- Route alerts to Slack or any chat product from this spec. +- Check off graph implementation because the sibling already times `invoke`. + +## 8. Risks + +| Risk | Mitigation | +|------|------------| +| Docs imply the brain already has Grafana series | Status line on every file in this folder | +| Hub compose copied “for convenience” | Point at the repo; no YAML here | +| JSONL text POSTed to a custom ingest | FR-2 applies to every remote path, not only OTLP | +| High-cardinality labels (`user_id`, session) | Forbidden on the wire | +| Prompt-capture instrumentation added with a tutorial | Explicit non-goal; default flags off | +| 011 dumps in Loki | Same allow-list; no message bodies on spans/logs | + +## 9. Success + +- A developer reading this folder can say: sibling does turns; this package + does not; hub is the other repo; content never goes on the wire. +- Git history of `thelab` still has no transcripts, keys, lab IPs, or serials + introduced by this work. +- If graph export ships later: unit tests prove text keys are dropped, no + hardware and no live collector required. diff --git a/specs/015-content-free-telemetry/spec.md b/specs/015-content-free-telemetry/spec.md new file mode 100644 index 0000000..cd11b3b --- /dev/null +++ b/specs/015-content-free-telemetry/spec.md @@ -0,0 +1,184 @@ +# Feature Spec: Content-free / privacy-tiered telemetry + +**Feature ID**: 015-content-free-telemetry +**Status**: Specified. Turn telemetry is implemented in the voice sibling; +not implemented in this package’s graph. +**Created**: 2026-08-20 +**Owner**: Derek Clair +**Related**: [008-local-tts-lenovo-go-spike](../008-local-tts-lenovo-go-spike/spec.md), +[011-voice-reply-contract](../011-voice-reply-contract/spec.md), +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent), +[`lan-agent-otel`](https://github.com/derekclair/lan-agent-otel) + +## Honest current state + +This is a **contract**, not a new exporter in `thelab_langchain`. + +| Surface | Today | +|---------|--------| +| Voice I/O sibling | Emits per-turn durations locally as JSONL. Opt-in OTLP ships **durations and counts only**. Transcripts must not go on that wire. | +| `thelab_langchain` graph | **Does not** export OpenTelemetry. No meter, no tracer, no OTLP bootstrap in this package. | +| LAN hub | Lives in [`derekclair/lan-agent-otel`](https://github.com/derekclair/lan-agent-otel) (collector → Prometheus / Loki / Tempo / Grafana). This repo **points at** that hub. It does **not** vendor the compose stack, lab IPs, or hostnames. | + +008 already named content-free opt-in OTEL as an I/O-repo concern. 011 forbids unspeakable reply dumps through Piper. This spec is the **workstation telemetry policy** those two sit under: same content must not leave the host as “observability.” + +Do not read this folder as “the LangGraph brain now reports to Grafana.” It does not. + +## Overview + +The desk has two telemetry planes: + +1. **On-host** — structured JSONL (and session transcripts, if kept). Operator debugging. Gitignored. May contain speech and replies because it never leaves the machine. +2. **Off-host / LAN hub** — opt-in OTLP to the collector in `lan-agent-otel`. **Content-free by construction**: durations, counts, and low-cardinality labels only. + +Privacy is **tiered**. The default for this workstation’s voice path and for any future graph metrics is **tier A** (ops, no user content). Higher tiers exist so a future operator can name them; they are not the default and they are not implemented here. + +The hub is a **push sink**. Agents fail open if the collector is down. This package does not run the hub. + +## Goals + +- State what may leave the host, and what must not, in one place. +- Record that voice-turn latency export is real in the sibling, and that this package’s graph has **no** OTEL today. +- Point at `lan-agent-otel` as the LAN hub without copying its compose, addressing, or inventory. +- Align with 008 (I/O owns ears/mouth/hands/telemetry) and 011 (do not dump unspeakable content — including into logs and OTLP). +- Restate git policy: no transcripts, prompts, tool bodies, keys, lab IPs, or hardware serials in this tree. + +## Non-goals + +- Implementing OTEL inside `thelab_langchain` in this spec’s delivery. Future graph metrics are **specified as a contract**, not shipped. +- Copying `lan-agent-otel` compose, collector YAML, Grafana dashboards, or scrape targets into this repo. +- Publishing lab IPs, RFC1918, Tailscale names, or hub hostnames. Example endpoints in this folder are loopback or a placeholder only. +- A SaaS default (LangSmith, cloud APM) that ships prompts off-LAN. +- Using telemetry as a transcript archive, a speakability dump (011), or a secrets store. +- Alert routing, Slack, PagerDuty, or household identifiers in this spec. +- Changing 008’s hardware loop or 011’s reply-shape rules except to bind them to this privacy policy. + +## User stories + +1. As the person at the desk, I can look at per-turn latency without anyone else seeing what I said. +2. As an operator, I point opt-in OTLP at the LAN collector and get durations/counts, never transcripts. +3. As a developer of this package, I know `get_agent()` does not emit OTEL today and must not grow content-bearing spans later without a spec change. +4. As a developer of the I/O sibling, I keep JSONL local and treat OTLP as a numeric allow-list. +5. As a reviewer of git, I reject commits that contain transcripts, prompts, tool args/results with user content, API keys, lab IPs, or hardware serials. + +## Privacy tiers + +Names match the hub’s baseline. This package’s **required default** is tier A. + +| Tier | What | Default for voice + this graph | May leave the host? | +|------|------|--------------------------------|---------------------| +| **A — content-free ops** | Durations, counts, error/cancel counters, `service.name`, generic `host.id` | **Yes (only this)** | Yes, via opt-in OTLP | +| **B — usage** | Token counts, model ids, tool *names* (not args), cost counters | Not implemented here | Hub-only, explicit review, still no bodies | +| **C — content** | Transcripts, prompts, completions, tool args/results, raw API JSON | **Forbidden as default** | **No.** On-host JSONL/transcript files only, gitignored | + +Tier C is not “OTLP with a flag.” If content is retained at all, it stays on the host. Shipping it to the collector is a policy violation unless a later spec explicitly opens a lab-only, default-off path — and even then it must not land in git or in a public dashboard. + +## Functional requirements + +### FR-1 Allowed off-host fields (tier A) + +OTLP and any other remote export **MAY** include: + +- Numeric durations (milliseconds). +- Counts (turns, errors, cancellations). +- Low-cardinality resource/labels: `service.name`, a **generic** host id (not a serial, not an IP). + +Voice-turn duration keys already used by the sibling (the contract for turns): + +`asr_ms`, `agent_ms`, `tts_ms`, `total_ms`, `eou_ms`, `tts_ttfa_ms` + +Future graph metrics, **if** this package ever exports, are the same shape: stage durations (for example memory-injection / LLM / tools), turn counts, error counts. Not message text. + +### FR-2 Forbidden off-host fields + +Remote export **MUST NOT** include: + +- Transcripts (user speech, STT text) +- Prompts, system messages, memory-injection text +- Completions / `AIMessage.content` +- Tool arguments or tool results that contain user content (including 011 unspeakable dumps: tables, fenced code, diffs, JSON/YAML, path/URL soup) +- API keys, tokens, `.env` values, OTLP bearer material +- Lab IPs (including RFC1918), Tailscale/CGNAT addresses, hub hostnames that identify the site +- Hardware serials, MAC addresses, GPU/CPU/disk identifiers + +A content-free exporter is an **allow-list of numeric keys**, not a redaction regex. Unknown payload keys are ignored. + +### FR-3 On-host JSONL and transcripts + +- Local JSONL **MAY** hold richer events for debugging, including text. +- Session transcripts, if kept, stay on the host and are **gitignored** (008). +- On-host files are not a license to POST the same payload to a collector, custom ingest URL, or SaaS. +- Do not commit JSONL, transcript dumps, or Grafana screenshots that contain speech. + +### FR-4 Opt-in and fail-open + +- OTLP is **opt-in**. No collector endpoint in env → no export (no-op). +- Missing SDK or unreachable collector **MUST NOT** break the voice loop or `get_agent()`. +- Do not bake a hub URL into this package’s code or SDD. Documented examples: + + - `http://127.0.0.1:4318` + - `http://collector-host:4318` + + Prefer OTLP/HTTP. `/v1/metrics` (and traces/logs if a future exporter adds them) are appended by the client; SDD shows the **base** URL only. + +### FR-5 Hub ownership + +- The LAN hub is [`derekclair/lan-agent-otel`](https://github.com/derekclair/lan-agent-otel). +- This repo **does not** copy that compose file, collector config, IPs, or hostnames. +- Operators configure the sibling (and any future graph exporter) with a local env var. They do not learn the hub topology from `thelab` git. + +### FR-6 Honesty in this package + +- Until a meter/tracer exists under `thelab_langchain`, docs **MUST NOT** claim graph OTEL. +- Do not add OpenInference / LangSmith / “capture prompts” flags as a default. +- `src/thelab_langchain/voice/` (Riva helpers) is not the live path (008) and is not an observability implementation. + +### FR-7 Relation to 011 (unspeakable dumps) + +011 says the spoken reply must not dump tables, fences, or runbooks unless the user asked. Telemetry is not a second mouth: + +- Do not log those dumps to OTLP “for debugging.” +- Do not attach `AIMessage.content` to spans so Grafana can “show the turn.” +- On-host JSONL may retain the string; the wire must not. + +## Non-functional requirements + +- No secrets, serials, household identifiers, Slack, or real IPs in this spec, plan, or tasks. +- Example OTLP bases only: `http://127.0.0.1:4318` or `http://collector-host:4318`. +- Low cardinality: do not use user id, session id, or free-text as a metric label on the wire. +- Fail-open, non-blocking export (background; never stall STT/TTS or the graph). +- Same `get_agent(user_id)` seam as 008. Telemetry must not require a forked graph. + +## Acceptance criteria + +- [x] Spec states allowed vs forbidden fields and privacy tiers, with tier A as default. +- [x] Voice-turn duration keys are named; OTLP is opt-in and content-free. +- [x] Honest status: sibling implements turns; `thelab_langchain` does not export OTEL. +- [x] Hub is referenced as `lan-agent-otel` without copying compose, IPs, or hostnames. +- [x] Git policy restated (transcripts, prompts, tool content, keys, lab IPs, serials). +- [x] 011 dumps are explicitly out of remote logs. +- [ ] Graph-side metrics in this package — **not done** (see [tasks.md](./tasks.md)). + +## Seams this package must keep stable + +| Seam | Contract | +|------|----------| +| `get_agent(user_id)` | Unchanged. No telemetry side-effect required for 008. | +| Voice sibling JSONL | Local `turn_complete` (and related) events; gitignored; may include text on host. | +| Voice sibling OTLP | Opt-in; allow-list of `*_ms` + counters; no transcripts. | +| Hub OTLP HTTP | Base `http://127.0.0.1:4318` or `http://collector-host:4318` (operator env). | +| Future graph exporter | If added: same FR-1/FR-2 allow-list. Not present today. | + +## Relationship to other specs + +- **008** — I/O spike owns telemetry emission for the desk loop. FR-6 there is the ancestor of this contract. 008’s open question (“LAN hub vs JSONL-only”) is answered here: JSONL always on-host; OTLP opt-in to `lan-agent-otel`. +- **011** — reply content contract. Unspeakable formatting is also unspeakable as a log line on the collector. +- **007** — Spark budget. Any future graph exporter stays fail-open and must not add an extra LLM call (same rule as memory injection). +- **004** — checkpointers. Session memory is not a telemetry sink. +- **010** — keep secrets off the board; same classes stay out of git and OTLP. + +## Open questions + +- If graph metrics are built later: wrap `get_agent()` internally, or let the I/O process time `invoke` only (sibling already has `agent_ms`)? Default bias: do not duplicate `agent_ms`; only add stages the I/O process cannot see (memory injection, tool node). +- Tier B token/model counts for this package: useful, still content-free, but **not** required to call this spec done. +- Custom HTTP ingest besides OTLP: bound by FR-2. Prefer the hub’s OTLP path; do not grow a second content-bearing shipper. diff --git a/specs/015-content-free-telemetry/tasks.md b/specs/015-content-free-telemetry/tasks.md new file mode 100644 index 0000000..9d97433 --- /dev/null +++ b/specs/015-content-free-telemetry/tasks.md @@ -0,0 +1,84 @@ +# Tasks: Content-free / privacy-tiered telemetry (015) + +**Feature**: 015-content-free-telemetry +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) +**Status**: Specified. Turns implemented in the voice sibling; graph OTEL not +in this package. + +Checkboxes are honest. Spec-only work can be marked done. Do not mark graph +export done because the I/O process already times `invoke`. + +## Phase 0 — Specify the contract (this folder) + +- [x] Write `spec.md` (tiers, allow-list, forbidden fields, hub pointer, + honesty about this package) +- [x] Write `plan.md` (two planes, sibling vs graph, no compose copy) +- [x] Write `tasks.md` (this file) +- [x] Restate git policy: no transcripts, prompts, tool content, keys, lab + IPs, hardware serials +- [x] Example OTLP bases only: `http://127.0.0.1:4318`, + `http://collector-host:4318` +- [x] Bind 008 (I/O telemetry) and 011 (do not log unspeakable dumps) + +## Phase 1 — Voice-turn export (sibling; already executed) + +Out of tree. Listed so this package does not re-build it. + +Implementation: +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) + +- [x] Local JSONL `turn_complete` with `asr_ms`, `agent_ms`, `tts_ms`, + `total_ms`, `eou_ms`, `tts_ttfa_ms` +- [x] Opt-in OTLP/HTTP metrics of those durations + turn/error/cancel counts +- [x] OTLP path ignores text keys (allow-list, not redaction) +- [x] Fail-open if SDK missing or collector down +- [x] On-host transcripts gitignored (008) + +Do **not** copy that exporter into `thelab_langchain` as a “port.” + +## Phase 2 — Graph metrics in this package (not built) + +Not built. Leave unchecked until a fail-open, content-free exporter exists +**in this tree** and tests prove text cannot leak. + +- [ ] Decide whether graph-internal stages are worth emitting (memory + injection / LLM / tools) vs keeping I/O-timed `agent_ms` only +- [ ] Opt-in via env; no baked hub URL; examples remain + `http://127.0.0.1:4318` or `http://collector-host:4318` +- [ ] Allow-list numeric fields + `service.name` + generic host id +- [ ] Unit tests: payload with transcript/prompt/tool args does not appear on + exported attributes (no collector required) +- [ ] Fail-open: missing SDK / down collector does not break `get_agent()` +- [ ] No OpenInference / LangSmith prompt capture; no extra LLM round-trip +- [ ] Do not treat sibling `agent_ms` as this checkbox + +## Phase 3 — Hub (other repo; do not vendor) + +- [x] Point operators at [`derekclair/lan-agent-otel`](https://github.com/derekclair/lan-agent-otel) + (collector → Prometheus / Loki / Tempo / Grafana) +- [ ] **Out of this repo forever:** copy compose, collector YAML, lab IPs, + hostnames, scrape targets, Grafana JSON + +## Out of scope (stay unchecked here) + +- [ ] SaaS default that ships prompts off-LAN +- [ ] Tier C content on the collector +- [ ] Alert routing to Slack or any chat product +- [ ] Speakability filter (011) — different contract; this spec only forbids + logging those dumps remotely +- [ ] Hardware latency tables copied from the sibling README + +## Traceability + +| Want | Where it lives today | +|------|----------------------| +| Turn durations JSONL | Voice sibling, on-host | +| Opt-in content-free OTLP for turns | Voice sibling | +| LAN hub stack | `lan-agent-otel` (do not copy) | +| Graph OTEL | **Not in `thelab_langchain`** | +| Transcripts | On-host, gitignored; never OTLP | +| Unspeakable reply dumps | 011 content rule; 015: not on the wire | + +Live consume path for turns: +[`conversational-voice-agent`](https://github.com/derekclair/conversational-voice-agent) +calls `thelab_langchain.agent.graph.get_agent` and times the turn itself. diff --git a/specs/README.md b/specs/README.md index 052d333..9a9179b 100644 --- a/specs/README.md +++ b/specs/README.md @@ -17,7 +17,11 @@ or partial. | [009](009-architect-coder-handoff/spec.md) | Architect ↔ coder handoff (STE) | Living practice. Not a library feature; not CI-enforced. | | [010](010-worker-completion-protocol/spec.md) | Worker complete-or-block | Living practice. This package does not implement a board. | | [011](011-voice-reply-contract/spec.md) | Voice-facing reply contract | Wanted speakability rules. No filter in code yet. | +| [012](012-fleet-dispatch-model/spec.md) | Fleet dispatch model | Conversation vs subagent vs durable board; closed roster. Not a dispatcher in this package. | +| [013](013-reviewer-quality-gate/spec.md) | Reviewer quality gate | Never implements; severity scale. No reviewer bot here. | +| [014](014-memory-injection-graph/spec.md) | Memory-injection graph | **Shipped** in `get_agent()`. Raw context; fail-open; no extra summarizer. | +| [015](015-content-free-telemetry/spec.md) | Content-free telemetry | Voice sibling implements turns; graph does not export OTEL. Hub is `lan-agent-otel`. | Hermes **operating manual** (CLI, gateway, profile files) stays at -`~/.hermes/docs/agentic-workflow.md`. Specs 009–010 record the *protocol*, -not that file. +`~/.hermes/docs/agentic-workflow.md`. Specs 009–010 and 012–013 record +*protocol*, not that file. From 67b9922198c7c1db4cf70d767ddbec8ee1fb6189 Mon Sep 17 00:00:00 2001 From: Derek Clair Date: Thu, 20 Aug 2026 01:50:18 -0600 Subject: [PATCH 3/3] =?UTF-8?q?Add=20SDD=20016=E2=80=93017=20from=20workst?= =?UTF-8?q?ation=20research=20tracks.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 016 records Mac+Spark hybrid inference: MCDMA is watch-only until OSS; ds4 Spark-then-identity-gated KV is a plan, not executed; 007 one-slot still owns the agent path. 017 records MCP/tool runtime as a host trust boundary; the Linux POC stays out of tree; no CVE catalogs. Explicitly not imported: church, client sites, SITE-Bench clones, skillopt bootstrap, literature surveys, KEV dumps. --- specs/016-mac-spark-hybrid-inference/plan.md | 199 +++++++++++ specs/016-mac-spark-hybrid-inference/spec.md | 332 ++++++++++++++++++ specs/016-mac-spark-hybrid-inference/tasks.md | 116 ++++++ specs/017-mcp-runtime-trust-boundary/plan.md | 133 +++++++ specs/017-mcp-runtime-trust-boundary/spec.md | 299 ++++++++++++++++ specs/017-mcp-runtime-trust-boundary/tasks.md | 80 +++++ specs/README.md | 15 + 7 files changed, 1174 insertions(+) create mode 100644 specs/016-mac-spark-hybrid-inference/plan.md create mode 100644 specs/016-mac-spark-hybrid-inference/spec.md create mode 100644 specs/016-mac-spark-hybrid-inference/tasks.md create mode 100644 specs/017-mcp-runtime-trust-boundary/plan.md create mode 100644 specs/017-mcp-runtime-trust-boundary/spec.md create mode 100644 specs/017-mcp-runtime-trust-boundary/tasks.md diff --git a/specs/016-mac-spark-hybrid-inference/plan.md b/specs/016-mac-spark-hybrid-inference/plan.md new file mode 100644 index 0000000..311af93 --- /dev/null +++ b/specs/016-mac-spark-hybrid-inference/plan.md @@ -0,0 +1,199 @@ +# Plan: Hybrid Apple Silicon + DGX Spark inference research (016) + +**Feature**: 016-mac-spark-hybrid-inference +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-20 +**Status**: Specified / not executed. Human go required before Phase A. + +## 1. Two planes (do not mix them) + +``` +desk today + Mac ── Ethernet / Tailscale ── Spark agent plane (already real) + get_agent() / Hermes + Grok default; optional ~30B slot + +research (not green) + Spark CUDA ds4 ── disk .kv file ──► Mac Metal ds4 Phase B, gated + Mac Metal memory ⇄ USB-C RDMA ⇄ Spark CUDA memory MCDMA, watch only +``` + +The agent plane does not wait on this research. KV / tensor work, if it ever +ships, is a **second** plane. Do not route Hermes over MCDMA. Do not treat +file copy as RDMA. + +## 2. What already exists + +- Spec 007 living slot: one local generative LLM on the Spark; CPU STT/TTS; + hosted Grok for quality-critical roles; optional ~30B-class worker. +- Spec 008 spoken loop: I/O sibling + `get_agent()`. Sentence-chunked TTS and + `tts_ttfa_ms` already exist **there**. +- Mac ↔ Spark agent use over Ethernet / Tailscale. + +This plan does not re-tune those paths. + +## 3. MCDMA — watch, do not implement + +Public 2026-08 posts describe Metal↔CUDA RDMA over USB-C. Closed source until +the author publishes source and license. + +**Do:** + +- Watch the public account for OSS + license. +- Keep author-reported BW / RTT labeled **author-reported** (spec table). +- If source + license land: clone **their** tree, run **their** tests on + one Spark + Apple Silicon Mac in the lab, write a decision memo. Still not + a Hermes cutover. + +**Do not:** + +- Start an implementation repo, bindings layer, or “thin wrapper” while the + code is closed. +- Build two-Spark CX7 fabric. +- Treat 939 MB/s / 24 µs (or any other author figure) as a lab result. +- Block ds4 Phase A on MCDMA. The tracks are independent. + +On OSS drop the first honest work is a single-link bench, then a toy tensor +or KV shuttle — **after** a human go, **after** license review. Prefer +upstream hooks over a lab fork. + +## 4. ds4 spike — plan only until go + +Execution is **out of tree**. Do not vendor [antirez/ds4](https://github.com/antirez/ds4) +into `thelab`. This repo keeps SDD only. + +### Human gates + +| Gate | Who | Unlocks | +|------|-----|---------| +| Specify (this folder) | Done | Nothing executable | +| **Go Phase A** | Human | Spark-only CUDA + disk KV + loopback server | +| **Go Phase B** | Human, after A green | File-based Spark-prefill → Mac-decode + identity | +| Hermes default change | Out of scope for the spike | — | + +An agent must not clone, download weights, or start `ds4-server` from this +card without the Phase A go. + +### Phase A — Spark-only CUDA (must pass) + +Goal: prove ds4 is usable on this Spark for Flash q2 with disk KV and a +localhost server. No Mac. No RDMA. + +Suggested sequence after go: + +1. **Quiesce the 007 slot.** Unload / stop the local Nemotron (or any other + serious generative LLM). Sequential, not stacked. Do not delete models. +2. **Clone + pin.** `git clone https://github.com/antirez/ds4.git` outside + this repo. Record `git rev-parse HEAD`. `make cuda-spark`. If the Spark + target fails, capture the log; do not silently switch to a generic CUDA + target without checking GB10 flags. +3. **Weights.** `./download_model.sh ds4f-q2` only. Skip PRO, MXFP4, and + DSpark until CLI + disk KV are green. Engine loads **ds4 GGUFs only**. +4. **CLI smoke.** Load Flash q2 at a modest context (start around 8k). + Greedy (`--temp 0`) short prompt. Pass = completes without OOM or driver + crash; host stays interactive. +5. **Disk KV + server.** Dedicated on-host directory with an explicit size + cap (`--kv-disk-dir`, `--kv-disk-space-mb`). + `./ds4-server ... --host 127.0.0.1 --port 8090` + Cold chat completion → restart process → same prefix should hit disk KV. + Do not bind `0.0.0.0`. +6. **Notes + teardown.** Record commit, quant, pass/fail, blockers. Stop + `ds4-server`. Restore the 007 agent path (Ollama / ~30B or hosted Grok). + +Spark is single-GPU: no `--cuda-tensor-parallel`. + +Phase A exit: all of the above, or an explicit fail with a blocker. Phase A +fail → no Phase B. + +### Phase B — optional file-based handoff + +Proceed only if Phase A is green, identical GGUF can live on both boxes, and +a human still wants the experiment. + +1. **Same commit, same GGUF.** Metal `make` on the Apple Silicon Mac in the + lab. Checksum the GGUF against Spark. +2. **Same-machine baselines** before handoff: greedy tokens for prompt P on + Mac-only and Spark-only. Document backend delta so handoff noise is + separable. +3. **Handoff.** Spark prefills P and writes disk KV. Copy the KV artifact + over the existing network (Wi-Fi first). Mac loads KV and decodes + **without** prefilling P. +4. **Identity gate.** ≥99% greedy token match vs Mac-local full prefill + (temperature 0, continuation length N recorded in notes, e.g. 64 or 128). + Fail closed on miss — that is a valuable negative result. +5. **Timing only after the gate.** Small contexts first (8k, 32k — not a + 500k safari). Compare time-to-first-decode-token: + Spark prefill + ship + Mac load vs Mac-local prefill. Then Mac decode. + Schedule 10GbE only if identity holds and ship time dominates. + +Pacary shipping projections and tweet tok/s figures stay **author-reported**. +Upstream README GB10 / Metal tables stay **upstream-reported**. Our numbers +are whatever Phase A/B notes record after go. + +### Decision tree after a real spike + +``` +Phase A fail → document blocker; no Mac work +Phase A pass, no Mac → optional loopback ds4-server for DeepSeek-shaped + research/coding only; still not Hermes default +Phase B identity fail → keep Spark-only; negative result is the finding +Phase B identity pass + + ship < Mac prefill → interesting hybrid; write a follow-up spec + + ship > Mac prefill → interesting science; not a daily driver without + a faster second plane (still not MCDMA-by-hope) +``` + +## 5. Slot, voice, and defaults + +| Rule | Plan consequence | +|------|------------------| +| 007 one local LLM | Flash q2 **is** the occupied slot while the spike runs | +| No 120B+ | Unchanged | +| No Hermes profile cutover | Do not point architect/coder/reviewer at ds4-server | +| 008 TTFA | Hybrid decode is not the spoken path in this plan. If a later spec + proposes it, first-audio must not regress; measure `tts_ttfa_ms` before + calling it a win | +| Second plane | Tailscale/Ethernet agents keep working if ds4 is down | + +## 6. What we will not do in this plan + +- Execute Phase A or B from this folder without a human go. +- Copy research-note trees, home-directory layouts, or cache paths into git. +- Implement RDMA, wrap a closed MCDMA binary, or start a lab MCDMA repo. +- Two-Spark CX7 fabric. +- Multi-tenant `ds4-server`. +- Stack ds4 + full local Nemotron. +- Change Hermes profile defaults. +- Vendor `ds4` as a submodule of `thelab`. +- Claim author or upstream benches as ours. +- Open `ds4-server` on a non-loopback bind as part of the spike. +- Use hybrid decode on the 008 loop “to try it” without a TTFA comparison. + +## 7. Risks + +| Risk | Mitigation | +|------|------------| +| Unified-memory fight with the 007 slot | Sequential load; teardown restores agents | +| Closed MCDMA copied or wrapped | Watch-only until source + license | +| False hybrid speed claims | Identity ≥99% before any timing narrative | +| KV not portable CUDA → Metal | That is Phase B; fail closed | +| Beta `main` churn | Pin commit after green smoke | +| Policy leak into Hermes | Explicit non-goal; review rejects default edits | +| Voice first-audio regression | Hybrid is not the live I/O path; 008 gate if it ever is | +| Docs imply it already runs | Status line on every file in this folder | +| Disk fill from KV | Size-capped dedicated directory | +| Author numbers become “our” numbers | Citations section; labels on every borrowed figure | + +## 8. Success + +A developer reading this folder can say: + +- Hybrid Metal/MLX + CUDA is the north star for **KV/tensor research**. +- MCDMA is watch-only; numbers in the spec are author-reported. +- ds4 is a planned spike, not executed, not the agent backend. +- Phase A is Spark-only; Phase B is file KV with an identity gate. +- 007 slot and 008 TTFA still constrain any future green light. +- Nothing here was committed as running code. + +Phase A/B success criteria live in [spec.md](./spec.md) and +[tasks.md](./tasks.md). They stay unchecked until a human go and real notes. diff --git a/specs/016-mac-spark-hybrid-inference/spec.md b/specs/016-mac-spark-hybrid-inference/spec.md new file mode 100644 index 0000000..ce04d80 --- /dev/null +++ b/specs/016-mac-spark-hybrid-inference/spec.md @@ -0,0 +1,332 @@ +# Feature Spec: Hybrid Apple Silicon + DGX Spark inference research + +**Feature ID**: 016-mac-spark-hybrid-inference +**Status**: Specified / not executed. Human go required before Phase A. +**Created**: 2026-08-20 +**Owner**: Derek Clair +**Related**: [007-dgx-hardware-optimization](../007-dgx-hardware-optimization/spec.md) +(one local generative slot), +[008-local-tts-lenovo-go-spike](../008-local-tts-lenovo-go-spike/spec.md) +(TTFA must not regress if hybrid decode is ever used) + +Public sources (not our benches): +[antirez/ds4](https://github.com/antirez/ds4), +[danpacary](https://x.com/danpacary/status/2086851964261003615), +[ashxhart](https://x.com/ashxhart/status/2089749434087227672) + +## Honest current state + +This folder is a **research contract**, not a feature in `thelab_langchain`. + +Nothing in this spec has been executed. There is no ds4 build, no disk-KV +handoff, and no Metal↔CUDA RDMA path in this lab. Do not read the folder as +“hybrid inference is running.” + +| Surface | Today | +|---------|--------| +| Agent path | Unchanged: this repo `get_agent()`; hosted Grok default; optional local ~30B-class on the Spark (spec 007). | +| Voice I/O | Spec 008 sibling. STT/TTS on CPU. Spoken loop does **not** use ds4 or MCDMA. | +| Mac ↔ Spark agents | Ethernet / Tailscale already exists. That is the **agent** plane. | +| ds4 (DwarfStar) | Spike **plan** only. Not cloned, not built, not served. | +| MCDMA | **Watch** only. Closed source until the author publishes source and license. | + +Ethernet/Tailscale stays the agent plane. MCDMA or a ds4 KV ship would be a +**second plane** (tensor / KV), if either ever goes green. They do not replace +the agent network. + +## Overview + +**North star:** evaluate hybrid local inference — Apple Silicon Mac in the lab +(Metal / MLX) plus DGX Spark (CUDA) — especially KV and tensor paths, not +chat-over-LAN. + +Two independent tracks: + +1. **MCDMA** (Metal CUDA Direct Memory Access) — public posts, 2026-08. + Claimed USB-C RDMA between Metal unified memory and CUDA memory. **Watch.** + Do not start an implementation repo until source and license are public. +2. **DwarfStar `ds4`** — public engine at [antirez/ds4](https://github.com/antirez/ds4). + Optional file-based Spark-prefill → Mac-decode, in the shape of the Pacary + experiment. **Plan a spike; do not execute until a human says go.** + +`ds4` does **not** replace the Hermes / thelab agent path (Nemotron ~30B / +Grok). It is a parallel research engine for ds4-specific DeepSeek V4 (and +related) GGUFs. + +## Goals + +- Write down the north star so later work does not silently become “new default + agent backend.” +- Keep MCDMA as watch-only until OSS + license. +- Specify a ds4 spike that is Spark-only first (Phase A), then optional + file-based heterogeneous KV (Phase B). +- Require an identity gate (≥99% greedy token match) **before** any hybrid + speed claim. +- Bind any `ds4-server` to loopback (`127.0.0.1`). +- Obey spec 007: one local generative LLM on the Spark; do not stack ds4 with + a full local Nemotron. +- Protect spec 008: if hybrid decode is ever used on a spoken path, time to + first audio must not regress. + +## Non-goals + +- 120B+ agent loops (spec 007). +- Multi-tenant serving. +- Implementing RDMA / MCDMA ourselves, or wrapping a closed binary. +- Two-Spark ConnectX-7 fabric (this lab is one Spark). +- Changing Hermes profile defaults in the spike. +- Vendoring `ds4` into this repo. +- Replacing Ethernet / Tailscale agent traffic with a KV plane. +- Treating author-reported or upstream README numbers as our benches. +- Wiring hybrid decode into the live 008 voice loop in this spec’s delivery. + +## User stories + +1. As the operator, I can tell a researcher: hybrid Metal/CUDA is a **watch + + planned spike**, not production, and the agent still uses Grok / ~30B. +2. As the person who would run Phase A, I know Spark-only CUDA + disk KV + + localhost server is the whole first gate, and I must not start without a + human go. +3. As the person who might run Phase B, I know file copy of KV is the + experiment, identity comes before speed, and Wi-Fi is first. +4. As a reviewer of git, I reject MCDMA implementation work, 007 slot stacking, + Hermes default cutover, and pasted author benches labeled as ours. +5. As the 008 voice owner, I know this track must not worsen `tts_ttfa_ms` if + it ever touches the spoken path. + +## Two tracks (do not merge them) + +``` +existing agent plane + Mac ── Ethernet / Tailscale ── Spark (Hermes / get_agent(); already real) + +research second plane (not green) + (A) ds4 file KV: Spark CUDA prefill → disk .kv → copy → Mac Metal decode + (B) MCDMA: Metal ↔ CUDA RDMA over USB-C [watch; closed] +``` + +MCDMA is not “faster ds4.” File-based KV is not a substitute for RDMA. Prove +or reject each on its own evidence. + +### Track 1 — MCDMA (watch) + +Public posts describe: + +- Registered memory and rkeys +- One-sided READ/WRITE +- Two-sided SEND/RECV with credit-based flow control +- Symmetric verbs (no master/slave) +- Transport: USB-C (author: USB3-class rates today; USB4 if a locked + controller can train) + +**Our topology if it ever opens:** one Spark + Apple Silicon Mac in the lab +over USB-C. The author’s two-Spark CX7 + dual USB-C Studio diagram is +**reference only**. We do not build that fabric. + +Author-reported figures from the 2026-08 public post +([ashxhart](https://x.com/ashxhart/status/2089749434087227672)). **Unverified +here. Label as author-reported. Do not treat as our benches.** + +| Metric (author-reported) | Value | +|--------------------------|-------| +| Single USB-C link | 939 MB/s | +| Mac → both Sparks, concurrent | 1.80 GB/s | +| Both Sparks → Mac, concurrent | 1.25 GB/s | +| Round-trip | 24 µs | +| Small-message rate | 41k msg/s | + +“Every byte delivery verified” is the author’s claim. Reproduce only after +source + license are public. + +**Standing rule:** do not start an implementation repo until source and +license are public. Prefer upstream integration (when it exists) over a +closed-source fork. + +### Track 2 — ds4 spike (plan, not executed) + +[antirez/ds4](https://github.com/antirez/ds4) is a narrow native engine +(Metal / CUDA / ROCm) for ds4-specific GGUFs — not a general llama.cpp zoo, +not a Nemotron loader. Spark target: `make cuda-spark`. Mac target: Metal +`make`. Surfaces: CLI, `ds4-server` (OpenAI- and Anthropic-style HTTP), +optional agent binary. First-class **disk KV** (content-addressed prefix +files) is why a file-based handoff is even thinkable. + +Pacary’s public experiment +([danpacary](https://x.com/danpacary/status/2086851964261003615)): Spark +prefill, Mac decode, same byte-identical GGUF, ship disk KV, Wi-Fi then +10GbE. Shipping-time projections and tweet prefill rates in that post are +**author-reported, not our benches.** We adopt the **correctness gate**, not +the speed narrative: + +> A handed-off cache must produce ≥99% token-identical greedy output vs +> prefilling locally. Correctness first, then speed. + +Upstream `ds4` README GB10 vs Metal tables are **upstream-reported**, not +lab results. They motivate why Spark-prefill / Mac-decode is interesting +(CUDA prefill vs Metal decode asymmetry). They are not a substitute for +Phase A notes. + +#### Phase A — Spark-only (must pass before any Mac work) + +- CUDA build (`make cuda-spark`); record commit SHA. +- Flash q2 weights only for the first spike (skip PRO / MXFP4 / tensor-parallel). +- CLI greedy short prompt succeeds. +- Disk KV: cold prefill → process restart → warm prefix hit. +- `ds4-server` chat completion on **localhost**. +- Teardown: stop the server; restore the 007 agent slot (do not leave Flash + resident next to Nemotron). + +Spark is a **single-GPU** target. Do not pass CUDA multi-GPU tensor-parallel +flags on this box. + +#### Phase B — optional file-based handoff (separate human go) + +Only if Phase A is green **and** a human wants Mac time. + +- Same commit and **byte-identical** GGUF on Mac and Spark (checksum). +- Mac Metal build. +- Spark writes KV for prefix P; file lands on the Mac; Mac decodes + continuation **without** prefilling P. +- Identity: ≥99% greedy token match vs Mac-local full prefill of P + (same prompt, temperature 0, fixed continuation length recorded in notes). +- Fail closed: if the gate fails, stop speed work. A negative result is + still a result. +- Transfer timing on the existing network first; 10GbE tuning is deferred + until identity passes. +- No “faster E2E” claim until identity **and** a timed comparison: + `(Spark prefill + ship + Mac load)` vs `Mac-local prefill`, plus Mac decode. + +Phase B is file copy. It is not MCDMA. + +## Functional requirements + +### FR-1 Status honesty + +Docs, PRs, and commit messages **MUST** say specified / not executed until +Phase A notes exist. Do not imply a live hybrid path. + +### FR-2 MCDMA is watch-only + +- No implementation repo, bindings, or vendored blob while source or license + is unpublished. +- Author-reported BW / RTT **MUST** stay labeled author-reported. +- Two-Spark CX7 + dual Mac links are out of scope. + +### FR-3 ds4 does not replace the agent path + +- Hermes / thelab defaults stay Grok (quality-critical) and ~30B-class local + (optional worker). +- `get_agent()` is unchanged by this spec. +- A localhost `ds4-server`, if it ever stands, is a **named research/coding + endpoint**, not a silent profile cutover. + +### FR-4 One-slot rule (007) + +- Do not load ds4 Flash and a full local Nemotron (or any second serious + generative LLM) at the same time on the Spark. +- Sequential use: quiesce the occupied slot, run the spike, teardown, restore. +- No 120B+ loops. + +### FR-5 Loopback bind + +- `ds4-server` **MUST** bind `127.0.0.1` unless a later spec explicitly opens + a firewalled bind (still not a public bind by default). +- Proposed research port if executed: `8090` on loopback. Not baked into this + package. + +### FR-6 Identity before speed + +- Phase B **MUST NOT** publish speed comparisons until ≥99% greedy token + identity vs the Mac-local prefill baseline. +- Tweet / README prefill and decode rates are citations, not results. + +### FR-7 Voice TTFA (008) + +- Live spoken path stays 008 until a later spec says otherwise. +- If hybrid decode is ever used on that path, `tts_ttfa_ms` / time-to-first-audio + **MUST NOT** regress vs the then-current all-on-Spark (or hosted) loop. +- Fail the hybrid voice idea rather than ship a slower first chunk. + +### FR-8 Execution is out of tree + +- Clone and build `ds4` outside this repo. Do not submodule it here. +- Spike notes (commit, quant, pass/fail, identity %) stay out of git if they + include prompts, transcripts, host identifiers, or hardware serials. +- This folder remains the SDD; it is not the run log. + +### FR-9 Second plane + +- Agent RPC stays on the existing Ethernet / Tailscale path. +- KV / tensor research, if green, is a second plane. Do not collapse the two. + +## Non-functional requirements + +- No secrets, serials, household identifiers, chat-product routing, lab IPs + (including RFC1918), or required hostnames in this spec, plan, or tasks. +- Hardware in prose: “DGX Spark” and “Apple Silicon Mac in the lab.” Do not + inventory a named personal Mac generation or a return date. +- Disk KV lives in a **dedicated on-host directory with a size cap**, not a + path committed here. +- Beta engine: pin a commit after a green smoke; do not chase `main` mid-spike. +- Weights: only `download_model.sh` targets. First spike = Flash q2. +- Privacy: prompts used for identity tests stay on-host; do not commit them. + +## Acceptance criteria + +- [x] Spec states north star, two tracks, and honest “not executed” status. +- [x] MCDMA is watch-only; author-reported numbers labeled; no implementation + repo until source + license. +- [x] ds4 Phase A (Spark-only) and Phase B (optional file KV) are specified + with the ≥99% identity gate. +- [x] `ds4-server` loopback bind and 007 one-slot rule are written down. +- [x] ds4 does not replace Hermes / thelab (Nemotron ~30B / Grok). +- [x] 008 TTFA non-regression is named if hybrid decode is ever used. +- [x] Non-goals include 120B+, multi-tenant, implementing RDMA, two-Spark + CX7, and Hermes default changes. +- [ ] Phase A executed — **not done** (human go required). +- [ ] Phase B executed — **not done**. +- [ ] MCDMA OSS evaluation — **not done** (blocked on public source + license). + +## Seams this package must keep stable + +| Seam | Contract | +|------|----------| +| `get_agent(user_id)` | Unchanged. No ds4 or MCDMA side-effect. | +| Spec 007 slot | One local generative LLM; ds4 occupies it if loaded. | +| Spec 008 I/O | Still the spoken path; TTFA protected. | +| Agent network | Existing Ethernet / Tailscale. | +| `ds4-server` | Loopback only if/when executed; not this package. | + +## Relationship to other specs + +- **007** — one-slot policy. This spike **is** occupying the slot while Flash + is loaded. Stacking with Nemotron is a 007 violation. 120B+ remains + forbidden as a daily loop. +- **008** — live voice. Hybrid decode is not the desk loop. If it ever is, + first-audio latency is a hard gate. +- **001** — long-term desktop voice. This research does not revive Riva/NIM + compose as production. +- **012 / 009** — fleet roles and architect/coder handoff stay on Grok / + existing local workers. Do not retarget profiles at ds4 in the spike. +- **015** — if any hybrid timings are exported later, they are content-free + durations only. This spec does not add OTEL. + +## Open questions (do not block specifying; do block speed claims) + +- Exact `ds4` flags to **export** a KV file another backend will accept + (confirm in `--help` / source during Phase A). +- Whether CLI session KV and server KV files are the same format for handoff. +- Whether CUDA-written KV is portable to Metal at all (that **is** Phase B). +- Standing localhost server after a green Phase A: optional, still not Hermes + default, still loopback, still sequential with the 007 slot. +- MCDMA OSS date and license: unknown. Watch; do not schedule implementation. + +## Citations + +Use as **sources**. Do not claim this lab reproduced them. + +- Engine: https://github.com/antirez/ds4 +- Heterogeneous file-KV experiment (author-reported): + https://x.com/danpacary/status/2086851964261003615 +- MCDMA (author-reported): + https://x.com/ashxhart/status/2089749434087227672 diff --git a/specs/016-mac-spark-hybrid-inference/tasks.md b/specs/016-mac-spark-hybrid-inference/tasks.md new file mode 100644 index 0000000..fa6c857 --- /dev/null +++ b/specs/016-mac-spark-hybrid-inference/tasks.md @@ -0,0 +1,116 @@ +# Tasks: Hybrid Apple Silicon + DGX Spark inference research (016) + +**Feature**: 016-mac-spark-hybrid-inference +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) +**Status**: Specified / not executed. Human go required before Phase A. + +Checkboxes are honest. Spec-only work can be marked done. Do **not** mark +Phase A, Phase B, or MCDMA evaluation done because public posts or the ds4 +README exist. Do not clone, build, or serve from this list without a human go. + +## Phase 0 — Specify the contract (this folder) + +- [x] Write `spec.md` (north star, two tracks, non-goals, 007/008 binds) +- [x] Write `plan.md` (watch vs spike, planes, decision tree, no execute) +- [x] Write `tasks.md` (this file) +- [x] Label MCDMA BW/RTT as author-reported; cite public posts without + claiming we ran them +- [x] Cite [antirez/ds4](https://github.com/antirez/ds4) as the engine; + Pacary post as the file-KV experiment shape +- [x] Bind `ds4-server` to `127.0.0.1`; 007 one-slot; no Hermes default change +- [x] Privacy: no lab IPs, RFC1918, serials, required hostnames, or cache + paths in this folder + +## Phase 1 — MCDMA watch (not an implementation) + +Policy (specified): + +- [x] Track is **watch only** until source **and** license are public +- [x] Do **not** start an implementation repo while closed +- [x] Do **not** implement RDMA ourselves +- [x] Two-Spark CX7 fabric is out of scope +- [x] Author-reported figures stay labeled; not our benches + +Blocked on OSS (leave unchecked): + +- [ ] Public source + license reviewed +- [ ] Human go to run **upstream** tests on one Spark + Apple Silicon Mac + in the lab (USB-C) +- [ ] Independent single-link BW / RTT capture, labeled as *our* run +- [ ] Decision memo: keep watching vs toy tensor/KV shuttle vs drop +- [ ] Any Hermes or 008 wiring — **forbidden until** a later spec + +Do not treat an OSS rumor as a checkbox. + +## Phase 2 — ds4 Phase A (Spark-only; not started) + +**Human go required.** Out of tree. Do not vendor into `thelab`. + +- [ ] Explicit human go for Phase A +- [ ] Quiesce the 007 slot (no ds4 + full local Nemotron) +- [ ] Clone [antirez/ds4](https://github.com/antirez/ds4) outside this repo; + record commit SHA +- [ ] `make cuda-spark` succeeds (Spark is single-GPU; no CUDA TP flags) +- [ ] `ds4f-q2` only (skip PRO / MXFP4 / DSpark on the first spike) +- [ ] CLI greedy short prompt OK (modest context first) +- [ ] Disk KV: cold prefill → restart → warm prefix hit, in a dedicated + size-capped on-host directory (path not committed here) +- [ ] `ds4-server` chat completion on `127.0.0.1` (proposed port 8090) +- [ ] Notes: commit, quant, pass/fail, blockers (no secrets, no serials, + no prompts in this git tree) +- [ ] Teardown: stop server; restore 007 agent path +- [ ] Decision: stop | optional loopback-only standing server (still not + Hermes default) | ask for Phase B go + +Phase A fail → do not start Phase B. + +## Phase 3 — ds4 Phase B (optional file KV; gated) + +**Separate human go** after Phase A green. + +- [ ] Explicit human go for Phase B +- [ ] Same commit + byte-identical GGUF on Mac and Spark (checksum) +- [ ] Mac Metal build OK +- [ ] Same-machine greedy baselines recorded (Mac-only vs Spark-only) +- [ ] Spark writes KV; file copied over existing network (Wi-Fi first); + Mac decodes without local prefill of that prefix +- [ ] Identity gate: ≥99% greedy token match vs Mac-local prefill + (temp 0; continuation length N in notes) +- [ ] If gate fails: stop speed work; keep Spark-only; record negative result +- [ ] If gate passes: time Spark prefill + ship + Mac load vs Mac-local + prefill (8k / 32k first). 10GbE only if identity holds and ship dominates +- [ ] No “faster E2E” claim until identity **and** that comparison exist +- [ ] Identity notes stay out of this git tree if they include prompts + +Phase B is **file copy**. It is not MCDMA. + +## Out of scope (stay unchecked here) + +- [ ] Implementing MCDMA / RDMA / USB-C verbs +- [ ] Two-Spark ConnectX-7 fabric +- [ ] 120B+ agents +- [ ] Multi-tenant ds4 serve +- [ ] Hermes profile default cutover to ds4 +- [ ] Stacking ds4 with a full local Nemotron +- [ ] Binding `ds4-server` off loopback as part of the spike +- [ ] Vendoring `ds4` into this repo +- [ ] Hybrid decode on the 008 spoken path +- [ ] Claiming author or upstream benches as lab results +- [ ] Changing 007 slot policy or 008 TTFA contract except to obey them + +## Traceability + +| Want | Where it lives | +|------|----------------| +| North star (Metal/MLX + CUDA KV/tensor) | This folder | +| Agent plane Mac ↔ Spark | Existing Ethernet / Tailscale; unchanged | +| One local LLM | Spec 007 | +| Spoken loop + TTFA | Spec 008 sibling | +| ds4 engine | Public `antirez/ds4` (not executed here) | +| File-KV experiment shape | Pacary public post (author-reported) | +| MCDMA | ashxhart public post (watch; author-reported) | +| Hybrid running in this lab | **Does not** | + +Live consume path for agents remains +`get_agent()` (this package) plus the 008 I/O sibling. ds4 and MCDMA are not +on that path. diff --git a/specs/017-mcp-runtime-trust-boundary/plan.md b/specs/017-mcp-runtime-trust-boundary/plan.md new file mode 100644 index 0000000..3942222 --- /dev/null +++ b/specs/017-mcp-runtime-trust-boundary/plan.md @@ -0,0 +1,133 @@ +# Plan: MCP / tool-runtime trust boundary (017) + +**Feature**: 017-mcp-runtime-trust-boundary +**Spec**: [spec.md](./spec.md) +**Date**: 2026-08-20 +**Status**: Specified. Linux POC out of tree. Not wired into `get_agent()`. + +## 1. What this plan is + +A map of **where the trust boundary sits** and **what the five detection +classes mean**. It is not a plan to add sensors, an MCP client, or a +security node to `thelab-langchain`. + +Success is an honest SDD: the desk treats MCP stdio / tool exec as a +process boundary; defense in depth is runtime watching; the POC stays out +of tree; this package does not pretend to run it. + +## 2. Boundary (do not collapse it) + +``` +desk agent (Hermes, IDE, or future graph tools) + │ + │ exec / MCP stdio command + ▼ +┌───────────────────────────────────────────┐ +│ tool process ← TRUST BOUNDARY │ +│ children · config · env · network │ +└───────────────────────────────────────────┘ + │ + │ host observations (POC out of tree) + ▼ + five detection classes → content-free alert (015) +``` + +`get_agent()` today sits **above** that picture. It compiles a graph with +memory tools (014). It does not launch MCP servers. The voice sibling +calls `get_agent()` (008) and also does not run these sensors. + +Collapsing the boundary into “MCP JSON looks fine” or “a future protocol +release will patch it” is a failed design. + +## 3. Five classes (distill only) + +Implementations stay out of tree. This table is the contract. + +| Class | Signal (idea) | Not the signal | +|-------|----------------|----------------| +| Shell metacharacters in child cmdline | Child cmdline of a tool/MCP parent contains shell operators | Full argv dump in git or on the wire | +| Unexpected subprocess vs allow-list | Child binary not on that server’s expected list | A global “malware” catalog in this repo | +| Config file integrity | Watched MCP/tool-host config diverges from a known-good hash | Pasting the new file (secrets) into an alert | +| New-server network watch | Shortly after register/start, outbound peer not on allow-list | Lab IPs, household names, or a full netflow archive | +| Dangerous env var changes | Loader/interpreter-related env changed after snapshot | Dump of the whole environment | + +Host techniques the POC *may* use (process table, file watch, connection +table, environ snapshot) are **examples of where to look**. This plan does +not copy scripts, regexes, or audit rule files. + +## 4. What already exists vs what does not + +| Mechanism | Status | +|-----------|--------| +| This SDD folder | This plan | +| Linux POC (five classes) | Out of tree. Prototype. Detection, not prevention. | +| Workstation MCP processes | Real, outside this package | +| Sensors in `thelab_langchain` | **Not done** | +| MCP client in `get_agent()` | **Not done** (and not this spec’s delivery) | +| Content-free alert path for these classes | Specified via 015; **not** implemented here | +| CVE database / KEV ingest in this repo | **Never** under 017 | + +Do not check off “workstation is monitored” because the POC directory +exists on a research machine. + +## 5. Sequence (if anyone implements later) + +Not a commitment. Default order: + +1. **Keep sensors off this package.** Prefer a host-side watcher next to + the actual MCP/tool processes (workstation runtime), not a LangGraph + node. +2. **Do not import the POC.** Re-implement against the five classes, or + keep the POC where it is. Do not vendor it into `src/thelab_langchain/`. +3. **Alerts obey 015.** Rule id + counts/flags + generic names. No + cmdline-with-args, no config diffs, no transcripts, no keys, no lab IPs. +4. **Fail open.** A dead sensor does not break `get_agent()` or the voice + loop. +5. **013 before widening the graph.** Adding MCP or shell tools to + `ToolNode` is a new spec plus reviewer security lens. 017 is not + permission to add them. + +Prevention (kill, freeze config, netns) is a **different** spec with an +operator model. This plan stays detect-and-record. + +## 6. Binding 013 and 015 + +| Spec | Binding | +|------|---------| +| **013** | Tool/exec boundary is security-sensitive. Rubber-stamp is a failed review. Findings: severity, location, fix guidance; no secret dumps. | +| **015** | Sensor alerts are remote/ops data. Tier A only unless a later spec says otherwise. Allow-list fields; ignore unknown keys. No chat-product routing. | + +Do not “debug” a class-1 hit by shipping the child cmdline to a collector. +On-host logs, if kept, follow 015’s on-host vs off-host split. + +## 7. What we will not do in this plan + +- Copy POC Python, tests, or audit rules into this repo. +- Copy research-cycle SUMMARY files, NVD/KEV JSON, or CVE tables. +- Repeat unverifiable stats or flaw chains the research STATUS already + flagged. +- Claim `thelab_langchain` implements the five classes. +- Wire a monitor into `get_agent()` under 017. +- Put keys, IPs, Slack, or household identifiers in this folder. +- Invent detection rates or coverage percentages. + +## 8. Risks + +| Risk | Mitigation | +|------|------------| +| Docs imply the brain already watches MCP | Status line on every file in this folder | +| POC copied “for convenience” | FR-7: idea only; no vendor | +| Protocol patch treated as the control | FR-1 / FR-2: process effects + runtime watch | +| Alert contains secrets or transcripts | FR-5 + spec 015 | +| Reviewer skips the boundary on a tool PR | FR-6 + spec 013 | +| Short-lived children missed by polling | Residual risk named; do not claim prevention | +| Unverifiable research stats leak into SDD | Explicit refuse list in the spec | + +## 9. Success + +- A developer reading this folder can say: tool/MCP exec is a process + boundary; five classes exist as a POC elsewhere; this package does not + run them; `get_agent()` is unchanged. +- Git history of `thelab` is not loaded with CVE catalogs, POC source, or + secret-bearing alert examples. +- Review of later tool-runtime work has a named boundary to tick (013). diff --git a/specs/017-mcp-runtime-trust-boundary/spec.md b/specs/017-mcp-runtime-trust-boundary/spec.md new file mode 100644 index 0000000..90b25f4 --- /dev/null +++ b/specs/017-mcp-runtime-trust-boundary/spec.md @@ -0,0 +1,299 @@ +# Feature Spec: MCP / tool-runtime trust boundary + +**Feature ID**: 017-mcp-runtime-trust-boundary +**Status**: Specified. Linux POC exists out of tree. **Not** implemented in +`thelab-langchain`. **Not** wired into `get_agent()`. +**Created**: 2026-08-20 +**Owner**: Derek Clair +**Related**: [013-reviewer-quality-gate](../013-reviewer-quality-gate/spec.md), +[015-content-free-telemetry](../015-content-free-telemetry/spec.md), +[014-memory-injection-graph](../014-memory-injection-graph/spec.md) + +## Record-keeping note + +This spec records a **workstation design**: MCP stdio and other tool +runtimes are a **trust boundary**. A tool process can spawn children, touch +config, set environment variables, and make network calls. Defense in depth +is **runtime monitoring of what that process does**, not a bet that “the +protocol will be patched.” + +A Linux proof-of-concept with five detection classes exists **outside this +repository**. This folder distills the boundary and those classes. It does +**not** vendor the POC, copy research-cycle dumps, or import CVE catalogs. + +This package has **no** MCP client, **no** host sensors, and **no** +LangGraph node that watches tool processes. `get_agent()` still binds +memory tools only (spec 014). Do not read this folder as “the brain now +monitors MCP.” + +Do **not** copy POC source, audit rule files, NVD/KEV JSON, or research-cycle +SUMMARY notes into this tree. + +## What this spec refuses to record + +The research tree that produced the POC also accumulated CVE lists, CVSS +scores, and cycle stats. Some of those notes are flagged **in that tree** as +unverifiable or hallucinated. This SDD **does not** restate them. + +Out of this folder (and out of this git repo) forever: + +- CVE identifiers, CVSS numbers, or catalog tables +- Unverifiable counts or “instances affected” figures +- Flaw-chain narratives that the research STATUS already marked unverifiable +- Alert routing to chat products +- Keys, tokens, lab IPs, household identifiers, or private host paths as + required layout + +The design below stands without those claims. + +## Overview + +On the agent workstation, **tool execution is not a library call with a +pretty schema**. MCP stdio starts a command. That child is a real OS +process. So is any other tool runner that shells out or execs a server. + +Once running, that process can: + +- spawn further children +- read or rewrite client/server config +- change its environment +- open network connections + +Those abilities are the **trust boundary**. Protocol-level review of MCP +messages is useful and is **not** sufficient. A later protocol revision does +not replace host-side observation. + +Defense in depth for this desk: + +1. **Treat the tool process as untrusted relative to the operator session.** +2. **Watch runtime effects** (children, config, env, new-server network). +3. **Keep alerts content-free** (spec 015). +4. **Do not rubber-stamp** changes that widen this boundary (spec 013). + +The five detection classes below are the recorded sensor *ideas*. They are +implemented in an out-of-tree Linux POC. They are **not** shipped here. + +``` +operator / orchestrator + │ + ▼ + tool runtime (MCP stdio, or any exec/shell tool) + │ + ├── children (cmdline, unexpected binaries) + ├── config files + ├── environment + └── network (especially a newly registered server) + │ + ▼ + host sensors (POC out of tree; not in get_agent()) + │ + ▼ + content-free alerts (015) — no secrets, no transcripts +``` + +## Goals + +- Name MCP stdio / tool execution as a trust boundary on this workstation. +- Record that defense in depth is runtime monitoring, not protocol hope. +- Distill five detection classes without vendoring POC code. +- Stay honest: specified; POC out of tree; this package does not implement + sensors; `get_agent()` is not wired to them. +- Bind 013 (security-sensitive review) and 015 (alerts must not carry + secrets or transcripts). + +## Non-goals + +- Implementing sensors, auditd loaders, eBPF, or an MCP client in + `thelab-langchain`. +- Wiring a monitor into `get_agent()`, `ToolNode`, or the voice sibling. +- Copying POC source, test suites, or generated audit rules into git. +- A CVE program, KEV tracker, or vulnerability database in this repo. +- Claiming the workstation is “protected” because a POC exists. +- Prevention (kill/quarantine) as a shipped control — the recorded POC is + **detection**. +- Slack, PagerDuty, or any chat product as an alert sink. +- Changing memory-tool behavior (014) except to note that tool invoke is + already a boundary, currently HTTP memory rather than MCP stdio. + +## Domain terms (define once) + +| Term | Meaning | +|------|---------| +| **Trust boundary** | The line where the agent (or MCP client) starts a process whose OS effects are no longer “just a function return.” Children, config, env, and network on the other side are in scope. | +| **Tool runtime** | Anything that execs or shells a tool: MCP stdio servers, LangGraph `ToolNode`, CLI tool hosts. This spec is about that runtime, not about a particular vendor product. | +| **MCP stdio** | MCP transport that launches a server as a child command and talks over stdio. The child is a host process. | +| **Detection class** | One kind of runtime check. Five are recorded here. Not a CVE. | +| **Allow-list** | Expected child executables (or expected endpoints) for a given server. Unknown is alert-worthy. | +| **Sensor** | Host-side watcher that implements a detection class. Lives out of tree today. | +| **Content-free alert** | A signal that a class fired, with low-cardinality labels (rule id, generic process name, path *class*). No secrets, transcripts, tool argument bodies, or config diffs that may contain tokens (015). | +| **Out of tree** | Not in this git repository. The Linux POC is research/workspace code, not a `thelab` module. | + +Do not call a protocol changelog “the fix.” Do not call the POC “production +monitoring.” Do not call `get_agent()` an MCP host — it is not. + +## Detection classes (ideas, not source) + +Five classes. Names are the design. Do not paste POC implementations. + +| # | Class | What to watch | +|---|--------|----------------| +| 1 | **Shell metacharacters in child cmdline** | A child of a tool/MCP process whose command line contains shell operators that look like injection (for example `;`, `\|`, `&&`, backticks, `$(…)`). | +| 2 | **Unexpected subprocess vs allow-list** | A child executable that is not on the server’s expected list. | +| 3 | **Config file integrity** | MCP (or tool-host) config files changing away from a known-good baseline, especially from a process that is not a known editor. | +| 4 | **New-server network watch** | Shortly after a server is registered or first started, outbound connections that are not on an allow-list (unknown peers, not loopback). | +| 5 | **Dangerous env var changes** | After a baseline snapshot, changes to process environment that alter loader or interpreter behavior (class examples: `LD_PRELOAD`, `LD_LIBRARY_PATH`, `PYTHONPATH`, `NODE_PATH`). | + +These are **host observations**. They do not parse MCP JSON-RPC as the +primary control. They do not require a CVE id to fire. + +Residual limits (name them; do not hide them): + +- Detection is not prevention. A short-lived child can exit before a poll. +- Allow-lists can be wrong or incomplete. +- Config watch without a baseline is noise. +- Class 4 is a **new-server window**, not a full network IDS. + +## Honest current state + +| Surface | Today | +|---------|--------| +| This spec folder | Contract and distillation | +| Linux POC | Out of tree. Five classes as a prototype. Not a product. | +| `thelab_langchain` | **No** sensors. **No** MCP stdio client. | +| `get_agent()` | Memory tools + `ToolNode` only (014). **Not** wired to the POC. | +| Workstation MCP (Hermes and similar) | Real tool processes on the desk. Outside this package. The boundary still applies there. | + +`get_agent()` tool invoke is already a **smaller** boundary: memory tools +talk to a network store and fail open (014). That is not MCP stdio, and this +spec does not add monitoring around it. + +## Functional requirements + +### FR-1 Boundary is process effects + +- MCP stdio and any exec/shell tool runtime **MUST** be treated as a trust + boundary: children, config, environment, and network are in scope. +- Reviewers and architects **MUST NOT** treat “the protocol will be patched” + as the only control. + +### FR-2 Runtime monitoring is the defense-in-depth layer + +- The recorded control is **watching runtime effects**, not a protocol + patch, not a prompt filter, and not a CVE chase. +- Sensors, when they exist, run **beside** the tool process (host), not as + a required step inside `get_agent()`. + +### FR-3 Five classes + +- The distilled class list is the five rows above. +- A later implementation **MAY** refine signals. It **MUST NOT** drop a + class silently without a spec change. +- This package **MUST NOT** claim those sensors are present until they live + in a named, accepted follow-up and actually run. + +### FR-4 Honesty of this package + +- Docs **MUST NOT** claim `thelab-langchain` monitors MCP or tool processes. +- `get_agent()` **MUST NOT** grow an MCP client or sensor loop under this + spec. +- Do not vendor POC source into `src/` “as a port.” + +### FR-5 Alerts follow 015 + +If sensors emit (out of tree or in a later spec): + +- **MUST NOT** include transcripts, prompts, completions, tool argument or + result bodies, API keys, tokens, `.env` values, or config file contents + that may hold secrets. +- **MUST NOT** include lab IPs or household identifiers (015 FR-2). +- **MAY** include: rule class id, generic process name, boolean/count, path + *class* (for example “mcp-config”), not a dump of the file. +- Unknown payload keys are ignored. Allow-list, not redaction. +- Do not route those alerts to a chat product from this spec. + +### FR-6 Review (013) + +- Tool/exec boundaries are **security-sensitive**. +- A spec or PR that adds MCP, shell tools, or a wider `ToolNode` **MUST NOT** + be rubber-stamped. Reviewer applies the security and privacy lenses (013). +- Findings **MUST NOT** paste secrets or cmdline dumps that contain secrets. + +### FR-7 Out-of-tree POC is not this repo + +- Point at the **idea** of the five classes. +- Do not copy POC Python, audit rule files, or demo transcripts into git. +- Do not make a private research path required layout for this package. + +## Non-functional requirements + +- No keys, IPs, Slack, household identifiers, or CVE dump tables in this + spec, plan, or tasks. +- No invented detection rates, false-positive percentages, or “instances + protected.” +- Sensors, if later built, must fail open relative to the voice loop and + `get_agent()`: a down monitor does not break the turn (same fail-open + spirit as 015 / 014). +- SDD in this folder stays human prose. Do not vendor skill bodies. + +## User stories + +1. As the person at the desk, I know a tool/MCP child is a real process, not + a sandboxed RPC. +2. As an operator, I know defense in depth is watching children, config, + env, and new-server network — not waiting for a protocol patch. +3. As a developer of this package, I know `get_agent()` does not host MCP + and does not run those sensors. +4. As a reviewer (013), I do not approve a wider tool/exec boundary without + the security lens. +5. As an operator of telemetry (015), an alert that a class fired does not + ship my conversation or my keys. + +## Acceptance criteria (for this SDD record) + +- [x] This folder contains `spec.md`, `plan.md`, and `tasks.md` that name + the trust boundary, runtime monitoring as defense in depth, and the + five detection classes. +- [x] Status is specified; POC out of tree; not a `thelab-langchain` + feature; not wired into `get_agent()`. +- [x] Related specs 013 and 015 are cited (no rubber-stamp; content-free + alerts). +- [x] No CVE tables, CVSS scores, unverifiable stats, POC source, keys, + IPs, or chat-product routing in this folder. +- [ ] Sensors in this package — **not done**. +- [ ] Wiring into `get_agent()` — **not done** (out of scope for 017). + +## Seams this package must keep stable + +| Seam | Contract | +|------|----------| +| `get_agent(user_id)` | Unchanged. No MCP client. No sensor side-effect. | +| `ToolNode` / memory tools | Still 014: profile, recall, store; fail-open invoke. Not an MCP host. | +| Workstation MCP | Outside this package. Boundary still applies. | +| Sensor process | Out of tree. Not imported by this package. | +| Alert payload | 015 allow-list if anything is exported. | + +## Relationship to other specs + +- **013** — Tool/exec boundaries are security-sensitive. Reviewer does not + rubber-stamp them. This spec is *what* the boundary is; 013 is *how* + review treats it. +- **015** — Telemetry and alerts are content-free. Sensor output is not a + transcript archive and not a secrets store. +- **014** — Shipped graph. Memory `ToolNode` is the only tool runtime in + this package today. 017 does not add tools. +- **010** — Secrets stay off the board; same classes stay out of findings + and alerts. +- **012** — Fleet dispatch. This spec does not add a “security scanner” + roster slot or a dispatcher in this package. +- **008** — Voice I/O calls `get_agent()`. I/O does not become the MCP + monitor. + +## Open questions + +- Whether a later spec should run host sensors next to Hermes (workstation) + rather than inside this Python package. Default: **workstation / out of + tree, not `get_agent()`.** +- Whether a later spec should add MCP tools to the graph at all. Default: + **not in 017.** If it happens, 013 + this boundary apply first. +- Prevention (kill child, freeze config) vs detect-and-alert. Default: + **detection only**, until a later spec with an explicit operator model. diff --git a/specs/017-mcp-runtime-trust-boundary/tasks.md b/specs/017-mcp-runtime-trust-boundary/tasks.md new file mode 100644 index 0000000..1aac7bf --- /dev/null +++ b/specs/017-mcp-runtime-trust-boundary/tasks.md @@ -0,0 +1,80 @@ +# Tasks: MCP / tool-runtime trust boundary (017) + +**Feature**: 017-mcp-runtime-trust-boundary +**Spec**: [spec.md](./spec.md) · **Plan**: [plan.md](./plan.md) +**Status**: Specified. Linux POC out of tree. Not wired into `get_agent()`. + +Checkboxes are honest. Spec-only work can be marked done. Do not mark +sensors done because a POC exists on another machine. Do not mark +`get_agent()` monitoring done because memory `ToolNode` already runs. + +## Phase 0 — Specify the boundary (this folder) + +- [x] Write `spec.md` with trust boundary, runtime monitoring as defense + in depth, five detection classes, honesty about this package +- [x] Write `plan.md` (boundary diagram, class table as ideas, 013/015 + binding, no CVE/POC copy) +- [x] Write `tasks.md` (this file) +- [x] Status: specified; POC out of tree; not a `thelab-langchain` feature +- [x] Cite 013 (no rubber-stamp of tool/exec) and 015 (content-free alerts) +- [x] Refuse CVE tables, unverifiable stats, keys, IPs, chat-product + routing, and POC source in this tree + +## Phase 1 — Linux POC (out of tree; already exists) + +Out of tree. Listed so this package does not re-build or vendor it. + +- [x] Prototype of five detection classes on Linux: + shell metacharacters in child cmdline; unexpected subprocess vs + allow-list; config file integrity; new-server network watch; + dangerous env var changes +- [x] POC treated as detection, not as production prevention + +Do **not** copy POC Python, audit rule files, or test suites into this +repo to “complete” a checkbox. + +## Phase 2 — Workstation wiring (not done) + +Host-side watcher next to real MCP/tool processes. Not this package. + +- [ ] Run sensors beside the workstation tool runtime (outside + `thelab-langchain`) +- [ ] Content-free alerts (015): rule class, counts/flags, generic names; + no secrets, transcripts, tool bodies, config dumps, or lab IPs +- [ ] Fail-open: dead sensor does not break the desk loop + +Phase 2 is **out of scope for 017 delivery**. Leave unchecked. + +## Phase 3 — This package / `get_agent()` (**not done**) + +- [ ] MCP client in `thelab_langchain` +- [ ] Sensors or a monitor node in the graph +- [ ] Wiring the out-of-tree POC into `get_agent()` +- [ ] Prevention (kill child, freeze config) as a shipped control + +Phase 3 is **out of scope** for 017. Do not implement them under this spec. + +## Explicitly not tasks in thelab + +Do not open work in this package for: + +- Vendoring POC source or research-cycle SUMMARY/JSON dumps +- A CVE / KEV tracker, CVSS tables, or “instances affected” claims +- Slack or any chat product as an alert sink +- Pytest that asserts Hermes MCP process trees +- Changing 014 memory tools in order to look like a monitor +- Adding MCP or shell tools to `ToolNode` without a new spec and 013 review + +## Traceability + +| Want | Where it lives today | +|------|----------------------| +| Boundary + five classes | This folder | +| Linux POC | Out of tree (not this git repo) | +| Graph / `get_agent()` | Spec 014; memory tools only; **no** sensors | +| Review of tool/exec changes | Spec 013 | +| Alert/telemetry privacy | Spec 015 | +| Sensors in this package | **Not present** | + +This tasks file is only the checklist view. It does not claim the brain +monitors MCP. diff --git a/specs/README.md b/specs/README.md index 9a9179b..ca102e4 100644 --- a/specs/README.md +++ b/specs/README.md @@ -21,7 +21,22 @@ or partial. | [013](013-reviewer-quality-gate/spec.md) | Reviewer quality gate | Never implements; severity scale. No reviewer bot here. | | [014](014-memory-injection-graph/spec.md) | Memory-injection graph | **Shipped** in `get_agent()`. Raw context; fail-open; no extra summarizer. | | [015](015-content-free-telemetry/spec.md) | Content-free telemetry | Voice sibling implements turns; graph does not export OTEL. Hub is `lan-agent-otel`. | +| [016](016-mac-spark-hybrid-inference/spec.md) | Mac + Spark hybrid inference | Research contract. MCDMA watch-only; ds4 spike not executed. Not the agent path. | +| [017](017-mcp-runtime-trust-boundary/spec.md) | MCP runtime trust boundary | Tool processes are a host trust boundary. POC out of tree; not in `get_agent()`. | Hermes **operating manual** (CLI, gateway, profile files) stays at `~/.hermes/docs/agentic-workflow.md`. Specs 009–010 and 012–013 record *protocol*, not that file. + +### Research tracks not imported + +Workspace research that is **not** SDD in this package (wrong product, client, +or a dump we will not vendor): + +- Church captioning / ProPresenter pipelines +- Client marketing sites +- World-models / SITE-Bench eval clones (upstream academic bench) +- Skill-optimization bootstrap (no design locked) +- Multi-agent *literature* surveys (012 is our chosen dispatch model) +- CVE/KEV catalogs and cycle summaries (017 records the trust boundary only) +- Alternate observability compose experiments (015 points at `lan-agent-otel`)