feat(insights): add bounded Hermes integration health rollup#3230
Merged
Conversation
Ref polylogue-fs1.15 Composes existing evidence into one read-only Hermes integration health view: per-source freshness/cursor position, a bounded dry-run parser/fidelity pass, convergence debt, lifecycle-event pairing debt, and context-delivery correlation state. Co-Authored-By: Claude <noreply@anthropic.com>
Ref polylogue-fs1.15 devtools render topology-projection/topology-status/cli-reference after adding polylogue/insights/hermes_integration_health.py and the `polylogue ops insights hermes-health` command. Co-Authored-By: Claude <noreply@anthropic.com>
Ref polylogue-fs1.15 Problem: polylogue/insights may not import polylogue/daemon directly (docs/plans/layering.yaml). The initial composer imported convergence_debt_status/convergence_debt_alert straight from polylogue.daemon, which devtools verify layering rejects. Solution: move the daemon-touching convergence-debt bucketing into the surface-adapter layer (api/archive.py._archive_hermes_integration_health, which is unrestricted), and have build_hermes_integration_health accept pre-computed convergence_debt_failed_count/retry_due_count instead. Also: add a log call to the one broad except-handler flagged by devtools verify degrade-loudly, and document the new CLI command in docs/daemon.md (verbatim "polylogue ops insights hermes-health") so devtools verify docs-coverage passes without baseline growth. Verification: devtools verify layering -> no violations; devtools verify degrade-loudly -> 0 unallowlisted; devtools verify docs-coverage -> clean (pre-existing unrelated stale baseline entry left untouched). Co-Authored-By: Claude <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 25 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (25)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
## Summary Adds the missing automated actor for the MCP `judge` dispatcher's candidate queue: a policy engine plus a daemon periodic-loop trigger surface that decides which `CANDIDATE`-status assertions are auto-judgeable and calls the same storage chokepoint the MCP `judge` dispatcher uses, escalating the residue to an explicit, queryable `handoff` assertion. ## Problem Per the polylogue-800m design note (PR #3202), the MCP `judge` dispatcher lets an operator judge assertion candidates in bulk, but nothing calls it automatically. Every agent-authored candidate (pathology findings, ontology/ transform candidates, findings, ...) defaults to waiting on an explicit human decision, which the operator has said does not scale — most agent-authored annotations will never be seen by a human. ## Solution - **`polylogue/daemon/judgment_automation.py`** (new): a policy engine (`evaluate_candidate`) applies per-kind confidence thresholds (`auto_accept_min_confidence`/`auto_reject_max_confidence`) decoded from a new `[judgment_automation.policies]` config table (`parse_judgment_automation_policy`). A candidate is judged (`run_judgment_automation_sweep_once`) via the real `judge_assertion_candidates` storage chokepoint — the same one the MCP `judge` dispatcher calls — when the policy is unambiguous; every other candidate (no policy for its kind, no `confidence` value, or a confidence inside the undecided band) is escalated: a deterministic `handoff`-kind assertion pointing at the candidate is upserted so the residue is a queryable review queue, not silent limbo. - **`polylogue/daemon/cli.py`**: wires `periodic_judgment_automation_sweep` into the existing `periodic_loops` list next to `_periodic_db_optimize`/ `periodic_embedding_backlog_check`, following the established periodic- maintenance-loop pattern rather than a `DaemonConverger` stage — judgment candidates are not file/session-scoped, and `docs/retro/2026-05-24-1498-cascade.md` flags `convergence_stages.py` as due for a refactor before a 4th stage, so this keeps the bead's footprint out of that hot file entirely. - **`polylogue/config.py`**: four new layered config keys — `judgment_automation_enabled` (fail-closed bool, off by default, following the `mcp_judge_enabled` pattern), `judgment_automation_interval_s`, `judgment_automation_batch_limit`, and the raw `judgment_automation_policy` table (decoded by the owning module, same shape as `health_convergence_debt`). - **Dual capability gate**: the sweep only judges when *both* `judgment_automation_enabled` AND `mcp_judge_enabled` (polylogue-800m's independent capability boundary) are true — it exercises the same judge write authority as the MCP dispatcher, so its own opt-in must not, by itself, grant that authority. Both flags are re-read every tick (`reload_behavior="daemon-loop"`), so flipping either off in `polylogue.toml` takes effect on the next tick without a daemon restart. - Docs: `docs/configuration.md` (config key table) and `docs/daemon.md` (new "Judgment Automation" section with a worked `polylogue.toml` example). ## Verification - `devtools test tests/unit/daemon/test_judgment_automation.py tests/unit/core/test_config_inventory.py tests/unit/daemon/test_daemon_cli.py` → `183 passed` - `mypy --strict` on `polylogue/config.py polylogue/daemon/cli.py polylogue/daemon/judgment_automation.py tests/unit/daemon/test_judgment_automation.py` → `Success: no issues found in 4 source files` - `devtools verify --quick` (ruff format/check, mypy, render-all --check, topology/layering/closure-matrix/manifests/ci-workflows/doc-commands/docs-coverage/test-infra-currency/test-clock-hygiene/pytest-timeout-overrides/degrade-loudly) → `exit_code: 0` - Not run: full `devtools verify` (testmon-affected) / `--seed-testmon` — per-repo policy this is the default pre-PR baseline, but it was explicitly deferred for this lane in favor of the focused set above plus `--quick`. ## Residual risk - No real-world confidence-producing detector is wired to a `[judgment_automation.policies]` entry yet in this PR — the policy table starts empty by default, so the sweep is a bounded no-op until an operator configures at least one kind. This is intentional (config-gated OFF, fail-closed on missing policy) but means the automation has no live exercise beyond the test fixtures until an operator opts a real detector kind in. - The escalation `handoff` assertion is discoverable only via an explicit `kind=handoff` filter today (not in `ASSERTION_CLAIM_KINDS`'s default set); wiring a dedicated MCP/CLI "review queue" surface for it is left as follow-up rather than expanding this PR's scope. Ref polylogue-6qjc --------- Co-authored-by: Claude <noreply@anthropic.com>
## Summary Fixes the Hermes ATIF+ATOF session-identity collision named in polylogue-fs1.14 and adds a typed, provenance-preserving read-side topology projection that composes every retained Hermes artifact class (conversational state.db session, ATIF trajectory, ATOF event stream, verification ledger) for one raw Hermes session id without ever physically merging message/event bodies. ## Problem `hermes_spans.py`'s `parse_atif_document` and `parse_atof_stream` both minted the identical `observer:<hermes_session_id>` `provider_session_id` for a given raw Hermes session id. Since `sessions.session_id` is computed from `origin` + this native id, importing an ATIF trajectory export and an ATOF event stream for the *same* Hermes session physically collided on one archive row: the archive's content-hash full-replace rule made the second ingest silently discard the first artifact's `session_events`. This was directly, empirically documented (and left unfixed) in `docs/hermes-operators.md`'s "What Polylogue cannot claim" section — item 1 stated "ATIF and ATOF do not compose when they share a session id," verified by ingesting the checked-in real ATIF and ATOF fixtures into one archive and observing only one survived. The two real-fixture tests in `test_hermes_spans.py` even asserted the identical `provider_session_id` for both artifact families, which is the collision made visible in test form. Beyond the identity collision, fs1.14's design asked for a typed composition across every Hermes artifact class stating field-level provenance (source artifact, fidelity) and rendering an unpaired trace or a conflicting producer-reported subagent identity as explicit, visible evidence rather than silently resolved. ## Solution **Identity fix** (`polylogue/sources/parsers/hermes_spans.py`): ATIF and ATOF now mint artifact-qualified, non-colliding identities — `atif_session_provider_id` → `observer:atif:<id>`, `atof_session_provider_id` → `observer:atof:<id>` — replacing the shared `observer_session_provider_id` / `hermes_observer_session_id_for` helpers with `hermes_atif_session_id_for` / `hermes_atof_session_id_for`. Every test asserting the old shared id was updated (`test_hermes_spans.py`, `test_dispatch_payloads.py`, `test_live_watcher.py`), and a new regression test (`test_atif_and_atof_no_longer_collide_on_one_session_identity_fs1_14`) proves the two artifact families no longer collide for an identical raw Hermes session id through the real dispatch/parse route both artifacts actually ingest through. **Topology projection** (`polylogue/insights/hermes_topology_projection.py`, new module): `project_hermes_topology` is a pure aggregator (no I/O, mirrors the existing `hermes_verification_coverage.py` / `hermes_delivery_correlation.py` design precedent) over caller-fetched `SessionEventRecord` rows. It composes one `HermesArtifactObservation` per artifact class (availability, event count, a fidelity status derived from structural evidence already present in the event stream — e.g. `hermes_atof_unpaired_scope` events degrade ATOF, unrecognized-shape `hermes_observer_span` events degrade ATIF), extracts producer-reported subagent evidence from both ATIF's `subagent_trajectories` and ATOF's `hermes.subagent.*` marks as `HermesSubagentEvidenceRef` entries tagged by source artifact, flags an ATIF-only or ATOF-only session as `unpaired_artifacts` (visible debt), and renders a self-referential subagent id or a disjoint ATIF/ATOF subagent-identity disagreement as an explicit `HermesTopologyConflict` — never silently trusting one source over the other. All output collections are sorted, so composing from the same retained raw evidence twice reproduces byte-identical output. **Not done here** (explicitly deferred, not silently assumed): physical `session_links`/`topology_edges` materialization from subagent evidence remains out of scope — the projection surfaces evidence refs read-side, the same discipline `hermes_spans.py` already documents for the ATIF/ATOF ↔ state-db relationship itself. Wiring this projection into any CLI/MCP surface is also deferred, following the same standalone-first precedent `hermes_verification_coverage.py` (fs1.4) already set — it shipped unit-tested before any surface consumed it. **Docs**: `docs/hermes-operators.md`'s "What Polylogue cannot claim" item 1 is updated from "unfixed, verified empirically" to "[Fixed, fs1.14]", the session-identity table and CLI examples now show the split `observer:atif:`/`observer:atof:` prefixes, and item 5 now mentions the new projection's conflict detection. `docs/design/hermes-archival-export-contract.md` and stale docstring cross-references in `hermes_verification.py` were also updated. ## AC matrix (polylogue-fs1.14) | AC | Status | | --- | --- | | Fixture set with parent, ATIF, ATOF, and snapshot evidence produces one logical-session topology with stable links and no duplicate messages/actions | **Satisfied** — `project_hermes_topology` composes all four artifact classes read-side with per-artifact isolated event counts (`test_per_artifact_event_counts_never_cross_contaminate` proves no cross-artifact leakage); the identity-collision fix independently guarantees no duplicate/clobbered messages when both ATIF and ATOF exist for one session | | Replay and rebuild are idempotent | **Satisfied** — `test_projection_is_idempotent_and_deterministic_across_repeated_calls`; deterministic sorted output; existing `test_atif_parse_is_idempotent_and_deterministic` / `test_atof_stream_is_idempotent_across_append_replay_and_keeps_scope_edges` cover the parser side | | An unpaired trace becomes visible debt | **Satisfied** — `unpaired_artifacts` + explicit caveat when ATIF exists without ATOF or vice versa (`test_atif_present_without_atof_sibling_becomes_visible_unpaired_debt` and its ATOF-mirror) | | A conflicting parent identifier fails closed or renders an explicit conflict | **Satisfied** — self-referential subagent and disjoint ATIF/ATOF subagent-identity sets both render an explicit `HermesTopologyConflict` (`test_self_referential_subagent_fails_closed_as_explicit_conflict`, `test_disjoint_atif_atof_subagent_identities_fail_closed_as_explicit_conflict`) | | Query/topology surfaces show evidence refs and per-link fidelity | **Partially satisfied / deferred** — the typed projection *is* the evidence-refs + per-artifact-fidelity surface, following the same standalone-first precedent as `hermes_verification_coverage.py`; it is not yet wired into a CLI/MCP surface (that wiring is separate follow-up work, matching how fs1.4's verification-coverage primitive shipped) | | Focused storage/topology/parser tests pass | **Satisfied** — see Verification below | | "Session links and subagent topology edges from producer-positive IDs only" | **Deferred, not misframed** — subagent evidence is surfaced read-side (producer-positive ids only, never inferred from proximity); *physical* `session_links`/`topology_edges` row materialization from that evidence remains out of scope for this pass, consistent with `hermes_spans.py`'s existing documented deferral of the ATIF/ATOF ↔ state-db physical merge | ## Verification - `devtools test tests/unit/insights/test_hermes_topology_projection.py tests/unit/sources/parsers/test_hermes_spans.py tests/unit/sources/parsers/test_hermes_verification.py tests/unit/sources/test_dispatch_payloads.py tests/unit/sources/test_live_watcher.py tests/unit/insights/test_hermes_verification_coverage.py` → `140 passed` - `/realm/project/polylogue/.venv/bin/python -m mypy --strict` on every touched/added file → `Success: no issues found` - `devtools render topology-projection && devtools render topology-status` (required — new module under `polylogue/`) - `devtools render all --check` → exit 0, no `out of sync` - `devtools verify --quick` (ruff format/check, mypy, render all, topology/layering/closure-matrix/manifests/ci-workflows/doc-commands/docs-coverage/test-infra-currency/test-clock-hygiene/pytest-timeout-overrides/degrade-loudly) → exit 0, run twice (pre-push hook + manual) - Not run: full `devtools verify` (testmon not seeded in this fresh worktree checkout; per repo policy this is a one-time harness-level seed, not a per-PR requirement) and the broader `devtools test tests/unit` sweep (anti-pattern per repo policy — testmon/focused selection is the intended inner loop) ## Residual risk - The topology projection has no CLI/MCP consumer yet — it is a typed, tested read model waiting for a surface, matching the existing `hermes_verification_coverage.py` precedent. A follow-up bead should wire both into a named surface together. - Subagent evidence is still not materialized into `session_links`/ `topology_edges` — a real physical child-session write for ATIF `subagent_trajectories` was considered and deliberately not attempted this pass: no real `subagent_trajectories` evidence has been observed in a live export yet (per `hermes_spans.py`'s existing fidelity notes), so committing to a session-linking schema for an unverified real shape was judged premature. This is the same honesty boundary `hermes_spans.py` already drew for the ATIF/ATOF ↔ state-db merge itself. - I found another worktree already had a branch named `feature/ingest/hermes-atif-atof-topology-projection` checked out and locked at the same base commit with zero divergence, suggesting a possible parallel dispatch of this same bead. I used a different branch name (`feature/ingest/hermes-atif-atof-provenance-projection`) rather than touch that branch/worktree. The coordinator should check for a duplicate PR before merging this one. Ref polylogue-fs1.14 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added unified Hermes topology views across conversational, ATIF, ATOF, and verification evidence. * Added provenance for subagent evidence, artifact availability, fidelity, and unpaired artifacts. * Added explicit conflict reporting for inconsistent or self-referential topology evidence. * **Bug Fixes** * Prevented collisions between ATIF and ATOF sessions and across profiles. * Improved profile-aware session correlation and artifact-specific archive identities. * **Documentation** * Updated Hermes identity, correlation, topology, and CLI usage documentation. * **Tests** * Added comprehensive coverage for projections, conflicts, fidelity, determinism, and session isolation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude <noreply@anthropic.com>
…h-rollup' into tmp/pr3230-merge # Conflicts: # docs/plans/topology-target.yaml # docs/topology-status.md
Owner
Author
|
Coordinator review stand-in (CodeRabbit rate-limited — no findings produced). Read the diff. Assessment: composes five EXISTING primitives (explain_import_path dry-run parse, project_named_source_freshness, convergence-debt summary, fs1.7 lifecycle reconciliation, fs1.11 delivery correlation) into one bounded read-only rollup — no new tables/writes, exactly the bead's no-new-monitoring-database constraint. Layering respected: insights/ never imports daemon/ (the API adapter computes Hermes-scoped debt and passes it in). Surface choice (readiness-report pattern: facade method + CLI subcommand, not INSIGHT_REGISTRY) is right for a heterogeneous singleton snapshot; MCP tool deferred with fs1.11 precedent. Degraded-state coverage is fixture-backed for all five failure classes; payload carries filenames/ids only, no raw paths/transcripts/credentials. Coordinator resolved the generated-topology merge conflict vs #3225/#3229 by regenerating both surfaces on the merged tree (focused tests + full quick baseline green via pre-push hook). 322 focused tests, mypy --strict, quick-gate green. Merging. |
Sinity
added a commit
that referenced
this pull request
Jul 20, 2026
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QUsH3Rhq6oAZpYPWcsZqnZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds one bounded, read-only Hermes-to-Polylogue integration health
rollup composing existing evidence, and a
polylogue ops insights hermes-healthCLI surface (plusPolylogue.hermes_integration_health()on the async facade) to read it.
Problem
Bead polylogue-fs1.15 asks for continuous observability of the declared
Hermes integration: enabled/disabled, freshness of each Hermes source
class, parser/materializer failures, unpaired/debt counts, latest
imported evidence refs, and delivery-correlation state, with explicit
degraded states rather than silent zeros. Per the bead's own 2026-07-18
note, the per-source freshness primitive
(
project_named_source_freshness, polylogue-1xc.13) already provesHermes coverage for state.db/ATOF; the remaining scope was only the
rollup/composition view on top.
Solution
New module
polylogue/insights/hermes_integration_health.py(
build_hermes_integration_health) composes, with zero new schema orpersistent write path:
explain_import_path— a bounded, non-mutating dry-run parse of everyfile under the Hermes runtime root; already returns per-file fidelity
declarations and parser-failure/decode-failure reasons.
project_named_source_freshness— per-file freshness/cursor/parse/index/FTS evidence (polylogue-1xc.13).
convergence_debt_summary_info, bucketed to the Hermes source familyby the caller (see layering note below).
reconcile_hermes_session_lifecycle— per-session lifecycle-eventpairing debt (fs1.7).
correlate_hermes_context_deliveries— per-session context-deliverycorrelation state (fs1.11).
It renders an explicit
disabled/unavailable/degraded/healthyverdict, and never carries raw transcript text, credentials,or absolute filesystem paths — source references are filenames only,
evidence refs are ids, not rendered bytes.
Surface choice: followed the existing
insights/readiness.py+insight_readiness_reportpattern (a standalone composed "one-shotreport" wired onto the
Operations/Polyloguefacade and a CLIsubcommand), not the
INSIGHT_REGISTRYdescriptor machinery — thatmachinery assumes a queryable list of homogeneous per-session items
with pagination, not one composed rollup snapshot. No MCP tool was
added in this pass (consistent with the fs1.11 precedent of deferring
MCP-surface growth when the API/CLI path is already fully queryable) —
left as an explicit, named option rather than silently omitted.
Layering fix:
polylogue/insightsmay not importpolylogue/daemondirectly (docs/plans/layering.yaml). Thedaemon-touching convergence-debt bucketing lives in the surface-adapter
layer (
api/archive.py::_archive_hermes_integration_health, which isunrestricted);
build_hermes_integration_healthaccepts pre-computedconvergence_debt_failed_count/convergence_debt_retry_due_countinstead of importing daemon modules.
AC matrix (polylogue-fs1.15)
enabled_reasonHermesSourceStatusviaproject_named_source_freshnesssession_refper source (session id, never a raw path)parser_failuresfrom the dry-run explain pass, root-level "no files yet" excluded from failure countingfidelity_capabilities(e.g.unpaired_scope_debt),convergence_debt_failed_count/retry_due_count,lifecycle_debtdelivery_correlationvia fs1.11'scorrelate_hermes_context_deliveriesstr(hermes_root) not in ...)Verification
devtools test tests/unit/insights/test_hermes_integration_health.py tests/unit/api/test_facade_contracts.py tests/unit/cli/test_insights.py→322 passed in 10.93smypy --strict polylogue/insights/hermes_integration_health.py polylogue/api/archive.py polylogue/cli/commands/insights.py→Success: no issues found in 3 source filesdevtools verify --quick→"exit_code": 0(ruff format/check, mypy --strict, render all, topology, layering, closure-matrix, manifests, ci-workflows, doc-commands, docs-coverage, test-infra-currency, test-clock-hygiene, pytest-timeout-overrides, degrade-loudly all green)devtools render topology-projection && devtools render topology-status && devtools render cli-reference— regenerated for the new module + CLI command;devtools render all --checkclean.Not run: the heavy post-merge
testsuite (per-PR CI convention) anddevtools verify --all.Residual risk
access is needed, adding one requires
EXPECTED_TOOL_NAMES+_ToolRow+TOOL_CONTRACT+render openapi/render cli-output-schemas; left as a follow-up rather than addedspeculatively.
observed Hermes session ids (
session_limit); a very high-volumeinstall could have debt in sessions outside that window. This
mirrors the bounded-evidence discipline the rest of the archive uses
(row/query budgets) rather than an unbounded scan.
entry (
cli: ops maintenance rebuild-index) — left untouched perscope discipline; not caused by this change.
Ref polylogue-fs1.15