Plan: Persist QA artefacts linked to canonical episodes (2.2.7)

[leynos]

Summary

This branch carries the pre-implementation execution plan for roadmap item
2.2.7, "Persist QA artefacts linked to canonical episodes". No production code
changes; the single deliverable is the plan itself.

Today the service can run two quality-assurance evaluators — Pedante and
Chrono — but their results live only in an in-memory LangGraph state object and
are discarded when the graph finishes. Nothing reaches the database, so no
operator can ask what an evaluator said about an episode. The plan authorizes
building a durable QA artefact linked to a canonical episode, plus retrieval
over both HTTP and a new first-party command-line interface, filtered by
evaluator and compliance status.

Roadmap task: (2.2.7)

Execplan: docs/execplans/2-2-7-persist-qa-artefacts-linked-to-canonical-episodes.md

What the plan authorizes

Seven milestones, each ending in a coherent repository state: import-purity and
architecture grouping (EP-M0); the domain model and ports (EP-M1); the
PostgreSQL adapter, migration, and store observability (EP-M2); the recording
and query services with the Pedante and Chrono projections (EP-M3); two REST
endpoints (EP-M4); one CLI command (EP-M5); and two architecture decision
records plus the documentation and roadmap updates (EP-M6).

Wiring evaluators into the generation-run execution path is explicitly out of
scope; that remains roadmap item 4.4.1.

Review walkthrough

Reviewers short of time should read the decision log and the three structural
decisions it records; everything else follows from those.

• Start with
  Purpose / big picture
  for the observable outcome, then
  Constraints
  for what the plan promises not to disturb — in particular that QaStatus and
  QualityMode stay as ADR 017 left them.
• Then read the
  Decision log.
  Three decisions deserve scrutiny because each deviates from an upstream
  artefact or fixes a semantic that will be hard to change later: replacing the
  design document's brand_compliance_results table with a
  compliance_status column on a unified evaluation record; recording a
  compliance_policy_version so a stored verdict states the rule that produced
  it; and treating an errored artefact as provisional so a transient evaluator
  failure is not permanent.
• Surprises & discoveries
  records what reconnaissance and review turned up, all verified against the
  tree. Two findings change the work rather than describing it: Hecate
  classifies a module by the first matching group, so an ancestor prefix
  silently voids a later group's rules while make check-architecture still
  exits 0; and importing any evaluator contract currently loads LangGraph and
  httpx transitively, because episodic/qa/__init__.py imports the graph
  builders.
• Verification plan
  is the longest section and the one worth reviewing hardest. Each obligation
  names its method, artefact, command, and a negative control that must fail
  for the intended reason. Two obligations exist purely to stop other
  obligations being vacuous: INV-BLOCKING-SET pins the blocking support
  levels against an independently written literal, and INV-COUNT-AGREES
  cross-checks the listed finding count against the detail representation.
• Interfaces and dependencies
  gives the prescriptive shapes: the domain types, the port protocol, the
  versioning semantics, the SQL schema with its two indexes, the adapter's
  narrowed IntegrityError handling, the HTTP contract, and the CLI contract.
• Milestones and plateaus
  and
  Module budget
  close the loop on scope. The budget is honest about the size of this item:
  roughly 46 files against a 50-file tolerance, driven by the CLI bootstrap.

Validation

Documentation gates on the branch head:

$ make fmt
Summary: 0 error(s)

$ make markdownlint
markdownlint-cli2 v0.23.2 (markdownlint v0.41.1)
Linting: 116 files
Summary: 0 issues in 0 files

$ make nixie
🧜‍♀️✨ All diagrams validated successfully!

Claims in the plan that were checked against the tree rather than asserted:

$ uv run python -c "import sys, ast, episodic.qa.pedante.types; \
    print('langgraph', 'langgraph' in sys.modules, '/ httpx', 'httpx' in sys.modules)"
langgraph True / httpx True

$ uv run hecate check          # with `episodic.llm` added to domain_ports
hecate: architecture check passed          # <- false green; see policy.py::first_matching_group

$ uv run python -c "import ast; ast.parse(open('episodic/api/errors.py').read())"
parse OK                                    # PEP 758 syntax, valid on 3.14 only

Code gates were not run because this branch changes no code.

Notes

The plan was produced with reconnaissance across the QA, persistence, API, and
orchestration subsystems, then revised after a six-lens design review covering
structural integrity, contracts, failure modes, verification non-vacuity,
alternatives, and long-term viability. The
Revision note
records what that review changed and why.

Two design questions were put to the reviewer and have since been ruled on;
both acceptances are recorded in the plan's decision log, so neither is an open
proposal any more:

1. The brand_compliance_results deviation — accepted. The plan updates
   the design document's Data Model bullet rather than building the table it
   names, because that table's only producer, Anthem, is unimplemented, and a
   two-table split would turn the roadmap's own filtered-retrieval requirement
   into a union query. The acceptance names the upstream edits it obliges:
   EP-M6 cannot be marked complete while the design document still names a
   table that was never built.
2. The CLI stays in scope — approved as written. Roadmap 2.2.7 requires
   retrieval "via API and CLI", but no CLI exists and roadmap 4.6.1
   ("Extend CLI client") assumes one does. Splitting the bootstrap out up
   front was offered and declined. The budget-triggered escalation to a roadmap
   addendum item 2.2.8 remains available to the implementer and needs no
   further approval when taken.

The plan's status remains DRAFT: these two decisions are settled, but the
plan as a whole has not yet been approved for implementation.

Two divergences from docs/episodic-tui-api-design.md are recorded rather than
fixed here: an unauthenticated request receives 404 rather than 401, following
the existing episode_tei.py precedent, and the plan raises 401 handling as a
separate item.

References

• Lody session: https://lody.ai/leynos/sessions/c456469d-cdcd-4f92-bc74-871616264ba0

🤖 Generated with Claude Code

Summary by Sourcery

Define the implementation plan for durably storing and retrieving QA evaluator artefacts associated with canonical episodes without changing production code.

Enhancements:

• Add a detailed execution plan for persisting evaluator results as versioned QA artefacts linked to canonical episodes, with PostgreSQL storage, filtered HTTP retrieval, and a first-party CLI.
• Define the planned domain model, persistence boundaries, compliance and idempotency semantics, architecture constraints, implementation milestones, and verification obligations for roadmap item 2.2.7.

Documentation:

• Document the proposed QA artefact schema, API and CLI contracts, architectural decisions, risks, milestones, and validation strategy in a new execution plan.

Tests:

• Specify comprehensive unit, property, integration, behavioural, architecture, import-purity, serialization, and CLI verification coverage for the planned implementation.

[coderabbitai]

Warning

Your free Security trial is over. An organization admin can activate billing to continue.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 39abac74-256a-491a-aecd-d830daa8d423

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

This PR adds a single, very large execution-plan document describing how to implement persistent QA artefacts linked to canonical episodes, including milestones, domain and storage design, HTTP and CLI interfaces, verification strategy, and architectural constraints; there are no production code changes yet, only the plan.

Sequence diagram for recording and retrieving a QA evaluation

sequenceDiagram
    participant Evaluator
    participant Recorder as QA_recording_service
    participant Store as PostgreSQL_store
    participant API as HTTP_API
    participant CLI

    Evaluator->>Recorder: evaluation_from_pedante(result)
    Recorder->>Store: record_evaluation(evaluation)
    Store-->>Recorder: stored evaluation with findings
    Recorder-->>Evaluator: recorded QA artefact
    API->>Store: list_evaluations(request)
    Store-->>API: summaries and total
    API-->>CLI: HTTP response
    CLI->>CLI: render evaluations

Loading

Entity relationship diagram for persisted QA artefacts

erDiagram
    CANONICAL_EPISODE ||--o{ QA_EVALUATION : has
    GENERATION_RUN o|--o{ QA_EVALUATION : correlates
    QA_EVALUATION ||--o{ QA_FINDING : contains

    CANONICAL_EPISODE {
        uuid id PK
    }
    GENERATION_RUN {
        uuid id PK
    }
    QA_EVALUATION {
        uuid id PK
        uuid episode_id FK
        uuid generation_run_id FK
        enum evaluator
        enum compliance_status
        int finding_count
        int compliance_policy_version
    }
    QA_FINDING {
        uuid id PK
        uuid evaluation_id FK
        int ordinal
        boolean is_blocking
    }

Loading

File-Level Changes

Change	Details	Files
Introduce a detailed execution plan document for persisting QA artefacts linked to canonical episodes, covering milestones, architecture, data model, interfaces, and verification.	Add execplan markdown document under docs/execplans describing seven milestones (EP-M0–EP-M6) for QA artefact persistence and retrieval Define planned domain model for QA artefacts, including evaluators, compliance status, findings, and versioning semantics Describe planned PostgreSQL schema (qa_evaluations and qa_findings), indexes, and adapter behaviour including idempotent recording and errored supersession Specify planned HTTP API endpoints and contracts for listing and retrieving QA evaluations, including auth, filtering, pagination, and error semantics Specify planned CLI interface (episodic qa evaluations list) behaviour, options, exit codes, and its relationship to the REST API Lay out verification plan with invariants, property tests, behavioural scenarios, and constraints on tools like Hecate, Skylos, Alembic, and py-pglite Record risks, constraints, tolerances, and architectural decisions (e.g., unified evaluation table, compliance_policy_version, CLI as REST client) in a decision log and ADR pointers	docs/execplans/2-2-7-persist-qa-artefacts-linked-to-canonical-episodes.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[codescene-access]

No quality gates enabled for this code.