Conversation
…edevops-rag + agent-harness)
Reusable AgentConsole every agentic-os app mounts as its chat surface. Runs
entirely on our own planes, no raw provider SDK:
- LLM = OpenAICompatibleModel (new stdlib ModelPlugin, cost-tiered Tier
routing, ModelRequest->ModelResult) so slim images without the
[litellm] extra still route through the Context Runtime model plane;
degrades to StubModel offline.
- grounding = PrimerIndex (redevops-rag hybrid_search contract; [rag] extra
swaps in the reranked path) for how-do-I / explain answers.
- actions = agent-harness ToolRegistry + ApprovalPolicy — side-effecting
tools are gated and every decision audited.
respond() returns a transparent show-your-work payload; panel_html() the panel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OpenAICompatibleModel.from_env now falls back to the self-hosted OpenAI-compatible endpoint the apps already point at (REDEVOPS_LLM_BASE_URL / REDEVOPS_LLM_MODEL, keyless DeepSeek) when no provider key is set — the agent runs on our own model plane without a key in every container. Branch the token param (max_completion_tokens for OpenAI, max_tokens for vLLM/DeepSeek). AgentConsole degrades to the grounded passage / raw tool summary if the model call fails, so a model outage never 500s. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ools in apps) - context_runtime/tools/mcp_servers/web_search.py: a keyless, dependency-free MCP stdio server exposing a read-only web_search tool (Wikipedia + Hacker News/Algolia). - AgentConsole.mount_mcp(client, prefix=, allow=): mounts an MCP server's tools into the agent-harness registry AND the classify/dispatch catalog, so the assistant can pick + run them like native tools. readOnlyHint tools run ungated; side-effecting ones stay gated. Proven end-to-end: console routes 'search the web…' to web_search, runs it over real stdio, returns live results. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ightwell pre-deploy half SupplyChainScanner wraps Trivy (CVEs in OS+language deps, IaC misconfig, secrets, SBOM) and Syft (SBOM) as subprocesses, normalizes findings (id, pkg, installed, fixed pin, severity), and degrades gracefully when the binaries aren't installed. The pre-deploy half of the Security & Compliance block; Edge Sentinel's AgentConsole triages the output. Hermetic tests parse canned Trivy JSON + cover the degrade path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…se-image CVEs a lockfile scan misses Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…SV consumer) VulnDB reads the vulns table vuln_feeds populates — lookup(package/cve/min_cvss) + enrich(findings) for apps like Edge Sentinel to annotate scan results against our NVD/OSV store. Applies the same row-scope + column-mask permissioning as the enterprise DorisStore (Principal: roles + owns_rows_of; non-privileged callers see only owned sources, refs masked). pymysql deferred/optional; degrades to available()=False. 5 hermetic tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…recommendation Finding.cls (os|lang) from Trivy's Result.Class; advise() turns the split into an actionable recommendation — when OS/base-image CVEs dominate (the common case), it recommends HARDENING THE BASE (rebase to a patched python:3.12-slim digest / apt-get upgrade the flagged system packages), fleet-wide, rather than chasing individual CVEs; otherwise it points at the app's own pins. Powers Edge Sentinel's scan recommendation.
Two optional Protocols (plugins/base.py) let the enterprise open-core layer compose INTO the cost optimizer without the engine importing any enterprise types: - PolicyProvider.feasible(candidate, goal, score) narrows the feasible space BEFORE cost ranking; a policy-rejected candidate is infeasible regardless of score, its reason recorded in Plan.rejected (observable in EXPLAIN). - TrustProvider.score(candidate, goal) breaks ties between equally cost-ranked feasible plans. KnapsackOptimizer gains optional policy/trust params; both default None → identical pre-seam behavior (verified: full suite green). Realizes the Whitepaper v3 pipeline seam Goal -> Policy Engine -> Feasible Plans -> Cost Model -> Planner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…prompt Whitepaper v3's scaling property: an assistant's abilities (retrieval methods, model tiers, …) are registered OUT OF BAND with cost/quality priors + the intent buckets they serve; the planner selects among them by intent/policy/cost, so capability count scales in a cheap registry rather than in the model's context window. - registry/capabilities.py: Capability + CapabilityRegistry (register/get/list/for_intent). CapabilityRegistry.from_rules() lifts the existing planner rule tables into registry form, so the default registry is behavior-equivalent (single source of truth). - planner/candidates_registry.py: RegistryCandidateGenerator — a drop-in CandidateGenerator that draws methods/tiers from the registry instead of hard-coded tables (injectable via ContextRuntime(candidates=...)). - tests: bucket-by-bucket parity with RuleCandidateGenerator + registering a new capability widens the candidate set with no rules/prompt edit. example: examples/capability_registry.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…y evaluation Plan selection becomes a contextual bandit keyed by intent bucket (Whitepaper v3, Gen 4): - BanditOptimizer (optimizer/online.py) implements CostOptimizer. score() is unchanged (estimate + hard/policy feasibility). select() runs a subset-aware epsilon-greedy over the FEASIBLE candidates, seeded with the cost-model total as an optimistic prior and refined by measured reward; records selection propensity + mode in Plan.extra for off-policy eval. learn()/learn_from_plan() fold reward into the shared EpsilonGreedyBandit. - Layered exploration: Tier 3 = live epsilon branch; Tier 2 = shadow=True (off-path pick); Tier 1 = offpolicy_values() — self-normalized IPS over logged probabilities ranks arms the planner never served, at zero live latency. - Composes with the Phase-0 seam: a PolicyProvider filters the feasible space BEFORE any exploration (never explore an infeasible plan); a TrustProvider breaks ties. Wiring: CostOptimizer.select gains an optional context="" (intent bucket); KnapsackOptimizer ignores it, the runtime passes intent.bucket. Backward-compatible — full suite green. 6 tests (exploit/explore/learn/policy-filter/off-policy) + examples/online_learning.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…est abstention Two Gen-5 behaviors (Whitepaper v3) on the online planner: - trust as an OBJECTIVE: BanditOptimizer gains trust_weight; when > 0 the trust score is folded into the ranking value (base = (1-w)*base + w*trust), not merely a tie-breaker, so a relied-upon plan can win over a nominally cheaper one. - honest abstention: abstention.py = AbstentionGate (serve / escalate / abstain) on a plan's calibrated confidence (expected_accuracy through an optional calibration map). Injected via abstain_gate; the verdict is recorded in Plan.extra['abstention']. Abstaining beats a confident-wrong answer — it protects operator trust (and the TrustLedger credits it). Also fixes a latent bug: _arm_value referenced goal without receiving it (masked because Phase-2 tests set no TrustProvider on the bandit); it now takes goal, so trust works on the online planner. 8 tests + example; full suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-> snapshots The policy loop that makes the planner scale (Whitepaper v3): the planner never learns on the hot path. Executions emit OutcomeEvents to a bus; one aggregator folds them into the shared bandit off the serving path and republishes a versioned snapshot; stateless replicas select against a local snapshot and reconcile. Neither stream touches the model's context window. - learning/events.py: OutcomeEvent (context, arm, reward, propensity, abstention + operator signals); from_plan() reads the optimizer's Plan.extra. Serializable. - learning/bus.py: EventBus Protocol + thread-safe InMemoryBus (publish/poll). A Kafka binding drops into the same seam for a fleet. - learning/aggregator.py: LearningAggregator (one writer) drains events, updates the bandit (skips the arm on an abstention but still forwards trust signal via on_trust sink), tracks a version, and emits a LearnedStateSnapshot; apply_to() reconciles a replica. Idempotent by seq. 5 tests (event-from-plan, off-hot-path fold, snapshot distribution to a replica optimizer, idempotent replay, trust sink) + examples/learning_loop.py. Full suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…the loop Satisfies the same publish/poll EventBus seam as InMemoryBus, backed by Kafka, so the async policy loop and the context-freshness/CDC stream run unchanged across stateless replicas. kafka-python is imported lazily (engine stays dependency-free); JSON codec by default, pluggable codec_out/codec_in for a typed event (OutcomeEvent) — plain dicts round-trip for the vuln/CDC stream. 3 tests via an injected fake client (no broker) incl. the aggregator draining from the Kafka seam. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nd when?' The forthcoming retrieval method (Whitepaper v3): a dependency-free bi-temporal store where every fact carries valid time (valid_from/valid_to — when it's true in the world) and transaction time (recorded_at — when we learned it), so the planner can answer point-in-time questions the other methods can't. TemporalStore satisfies the RetrieverPlugin seam (method='temporal', added to the Retrieval literal): search() = current state; as_of(at, known_at) = world-time + transaction-time travel (a late correction doesn't silently rewrite what was known then); changes(since, until) = what began/ended validity. Graphiti can back it as an optional binding. 5 tests + example. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…property + a benchmark
Completes the Gen-4 non-stationarity property the whitepaper names ('discounted updates that fade
stale evidence'): EpsilonGreedyBandit gains an optional discount (constant step size); 0 (default) =
sample-average 1/n, unchanged. BanditOptimizer passes it through.
examples/online_vs_static_bench.py benchmarks a planner under drift (best plan A->C at t=200):
static 0.20 · online plain 0.33 · online discounted 0.67 (post-drift, oracle 0.80)
i.e. a static v2 planner is pinned to the stale plan; plain sample-average online adapts poorly (200
stale samples drown the signal); discounting tracks the drift. 3 unit tests lock the estimator in.
Also surfaced (and the benchmark now avoids) a footgun: learn() must key on the plan's arm
(method:tier via learn_from_plan), not the bare method string.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every tool call for every AgentConsole app funnels through ToolRegistry.run(). It now takes an optional authorizer (principal, ToolSpec, args)->reason|None run BEFORE the side-effect gate; deny short-circuits with an audited [blocked] result. set_default_authorizer(fn) installs one process-wide, so a single enterprise install() call gates the whole console fleet. AgentConsole.__init__ accepts authorizer=, respond(message, principal=) threads the caller identity through. Default None ⇒ unchanged (verified: full suite green). This is the ToolRegistry analog of the PolicyProvider seam — engine imports no enterprise code. 3 tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ket Radar)
A company hiring an AI Data Engineer/Developer in-house signals real internal AI-adoption pain. This
module sources such listings, drops consultancies/forward-deployed/agency shops (they build AI FOR
clients — no internal pain), writes a listing-tailored pitch from an EDITABLE, shared template
(placeholders {title}{company}{match}; {match} is LLM-tailored to the listing when a model is given),
dedupes via an append-only OutreachLedger so a listing that reappears months later is never pitched
twice, and drafts distribution to an outbox (real sending is an approval-gated deployment step).
Pluggable sourcing (StaticJobSource/CallableJobSource over web_search/a job API); resume attached when
the listing asks for a CV. Dependency-light + fully testable offline. 7 tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ly) with graceful degrade + test
…e to the caller - tools/base.py: current_principal() — the in-flight tool call's principal (set by ToolRegistry.run around the call, reset after), so a tool can scope to the user without changing the ToolPlugin API. - job_leads: LeadProfile + ProfileStore (per-user, persisted <dir>/<user>.json). The owner (LEAD_OWNER_USER) is seeded with AI-infrastructure targets + the Context Runtime template; everyone else starts blank and configures their own include/exclude + template. classify_for()/find_leads_for() score against ONE user's targets; consultancies excluded by default. Templates are per-user now. 6 tests (owner-vs-generic, isolation, profile-scoped classify/find). Backward-compatible (default None principal ⇒ owner fallback); full suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…isions+audit, AgentConsole wiring Implements docs/policy-runtime.md phases 1-5 (OSS, open-core): - policy/store.py: Rule + RuleStore — persisted, scoped long-term memory (global | <app> | <app>:<user>), JSONL, atomic; content-hash ids; scopes_for(principal, app). - policy/plane.py: Decision (allow|deny|redact|flag|require_approval), PolicyDecision + DecisionSink (the mandatory audit event), GuardrailProvider (input/output, pattern) + ApprovalProvider (tool → require_approval), composed Policy.check() that emits a PolicyDecision for EVERY decision; set_default_policy/current_policy. - policy/commands.py: Command + CommandRegistry (dual-path /dispatch, permission-gated via injected can) + parse_args (flags + comma lists). - AgentConsole: /command dispatch (no LLM); input+output guardrails; tool-phase approval pauses irreversible actions; returns compact policy[] (progressive disclosure §10.1). All optional + fall back to the process default → one install governs the fleet; no policy ⇒ byte-for-byte unchanged (full suite green). 12 tests (RuleStore CRUD/scopes, guardrail deny/redact/phase, approval, per-user scope, parse_args, command permission/help/alias, console dispatch/block/approval/fold/backcompat). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…role_gate
Ready-made command sets over a RuleStore: /showpolicy /addpolicy /removepolicy /modifypolicy (aliases
/showguardrails /addguardrail /removeguardrail), read open + write gated by 'policy-admin'. role_gate()
is a simple role-based can(principal, requires) for deployments without the enterprise plane
(''=any · self=authenticated · policy-admin=admin role · role:X). 1 test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the consolidated v1-vs-v2 benchmark with a v3 section (Generation 4): the best plan drifts mid-run; a static v1/v2 planner is pinned to the stale plan (post-drift reward 0.20) while the v3 online planner re-explores and recency-weighted (discounted) learning tracks the shift (0.67; without discounting 0.34; oracle 0.80). 24-seed average. A distinct axis from — and preserving — the v2 calibration gains. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… only Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…imes) Generated by examples/consolidated_benchmark.py --html. Final post-fix numbers (bandit persist-context fix 1c2e135 already in v2). Hand-off artifact for the redevops.io/benchmarks page; BENCHMARKS.md is the canonical markdown twin. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…c writer Re-adds v3_results() (online optimization under drift, Gen-4) to consolidated_benchmark.py behind a --v3-doc PATH flag, and generates docs/BENCHMARKS-v3-preliminary.md. Kept off v2's shipped BENCHMARKS.md and the default v1/v2 table — this is the v3 line's forward-looking axis (post-drift served-plan reward 0.20 static → 0.67 online, 24-seed avg, oracle 0.80). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…matched tool
The deterministic fallback (_keyword_route, used when the LLM classifier times out) scored a tool on a
single incidental word — 'How do I track just part of a page?' matched add_watch's 'monitor a page',
and 'What is dunning?' fired a real CHASE_OVERDUE. Interrogative/how-to leads now short-circuit to help,
mirroring the LLM classify rule; explicit action queries ('what changed this week?') still keyword-match.
Regression test added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…are one-liner The help prompt said 'be concise — give the concrete steps', which is a how-to framing; for 'what is X?' questions the model collapsed the answer to a single definition clause, dropping the surrounding domain context that was in the retrieved passage (e.g. that http-bruteforce is a *scenario* that drives an *alert*/*ban* on the source *IP*). Reworked the prompt to, for explain questions, define the term AND explain how it fits the system from the passages. Validated A/B against the live model. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…m (Kimi fallback) Point CR_UPSTREAM at the host model shim (host.docker.internal:8000 → shared Qwen3.6-35B-A3B on GPU0, Kimi fallback in the shim) instead of Kimi directly; add host-gateway extra_hosts. Rebuilt on v3 context-runtime so redevops.io/planner (→ chat.redevops.io/librechat/explain) runs the v3 engine. CR_MODEL_* overrides let you A/B another model without editing the compose. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pre-existing working-tree changes, committed as part of the deploy sync: - OpenAICompatibleModel default_model gpt-5.5 -> grok-4.5 (matches the dedicated CR_JUDGE_MODEL=grok-4.5 the live self-learning RAG now uses). - examples/fleet_tenants.py: add an outreach tenant probe. - docs/BENCHMARK-REPORT.md: the 30/30 v3 benchmark run + judge-swap note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restore the three-model chat.redevops.io demo (wiped to FinanceBench-only by the v3 deploy): one control plane serves finance/medical/mixed tenants, each its own retriever + learned policy, routed by model id in /v1 and /librechat/*. - CR_TENANTS "id=/corpus | id=shards(a:/p1,b:/p2)"; Mixed = coverage-routed finance x medical (cross-domain noise -> 0, the whitepaper win). - Per-tenant cross-language (CR_TENANT_LANGS, opt-in) for e.g. Spanish speakers. - deploy/medical: PubMedQA MedicalBench downloader + corpus builder. - proxmox-demo: 3-model librechat.yaml + compose (judge + medical mount) + docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…presentation (#3) Whitepaper v4 groundwork: the planner routes across knowledge *representations*, not just retrieval algorithms. This adds temporal (bi-temporal) as a first-class routable representation and a representation taxonomy, without a big refactor and fully backward compatible. - types: add `temporal` IntentBucket + a `KnowledgeRepresentation` taxonomy - planner/representations.py (new): retrieval method -> representation map (document / graph / temporal / analytical / community / code / multimodal) - planner/rules: temporal intent cues (as-of / what-changed / superseded / point-in-time), placed before multi_hop; BUCKET_DEFAULTS + BUCKET_TIERS (temporal candidate + hybrid fallback) - store_router: optional `temporal` slot with empty-fallback — routes to the bi-temporal store only when one is bound and matches; otherwise falls back to single-hop, so a new representation never regresses default behavior - costmodel: temporal recall + a temporal-bucket edge so the planner prefers the bi-temporal store for time-dependent questions (mirrors how multi_hop prefers graph) - tests: 8 tests (routing, cost edge, router dispatch + fallback, bi-temporal point-in-time and what-changed semantics, representation taxonomy, end-to-end run) The reference temporal backend is the in-memory TemporalStore already in the tree; a real Graphiti engine binds via the same RetrieverPlugin seam at construction time. HippoRAG (graph) already routes end-to-end; this closes the temporal gap. Full suite green. Co-authored-by: redevops <redevops@redevops.io> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
fleet_tenants.py claims to cover "every agentic-os-stack module", but guide and growth-assistant (both live in modules.yaml / on demo.redevops.io) were missing — so redevops.io had no measured learned-vs-baseline number for their cards. Add both to CATALOG + PROBES so their numbers are reproducible from the same harness as the rest of the fleet: guide 0.946 vs 0.800 (docs) growth_assistant 0.928 vs 0.800 (playbooks) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Whitepaper-v4's representation-first planning was taxonomy-only (representations.py mapped
method→representation but nothing called it; candidates routed off the flat BUCKET_DEFAULTS).
Make it real, in three stages that compose with — not replace — the v2/v3 loop below them:
1. Classify — RuleIntentAnalyzer now emits a KnowledgeRepresentation (Intent.representation).
representations.classify() maps bucket→representation (multi_hop→graph, temporal→temporal,
code→code) with content-hint overrides that reach representations no bucket produces on its
own: "how many … per week" → analytical (OLAP), "screenshot" → multimodal.
2. Constrain — RuleCandidateGenerator restricts methods to methods_for(representation) instead
of the flat bucket table, keeping a document (hybrid) fallback so a missing/infeasible
representation engine degrades gracefully, and widening under low confidence so uncertain
intents explore across representations.
3. Record & learn — representation is recorded on the plan, the librechat EXPLAIN output and the
OutcomeEvent. A routed plan keeps its document fallback, so the bandit's arms span two
representations — choosing between them IS representation selection, learned per context.
tests/test_representation_planning.py (7): classify head incl. analytical/multimodal hints,
analyzer sets representation, candidates constrained + document fallback, OLAP candidates now
reachable, bandit arms span representations, representation on the outcome event. Full suite
316 passed, 4 skipped (no regressions).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ion) (#1) The HippoRAG indexing LLM defaulted to gpt-4o-mini, which OpenAI is deprecating. Move to gpt-5-mini (the GPT-5 small tier). Still overridable via llm_model_name. Co-authored-by: redevops <redevops@redevops.io> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Superseded — main's squashed AgentConsole history conflicts with v4's granular version, so a direct v4→main squash can't auto-create. Replaced by a reconciled single-commit PR based on main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Point
mainat thev4generation branch.Brings the v2 development line (Generations 4–6) onto main as one squash:
Preserves the main-only hipporag
gpt-5-minidefault (cherry-picked onto v4 so this squash doesn't revert it). Generation branchesv3/v4and tagsv3.0/v4.0are pushed. The Graphiti/HippoRAG engine bindings + structured-routing land as a follow-up PR (post-benchmark).Recommend Squash and merge to match the existing main PR history.