Self-evolve mechanism: propose/review/apply/revert loop for an agent's own definition

[yjc801]

Filed on behalf of @yjc801 (JC) from #buzz-dev.

Problem

An agent can learn something in a session and has almost no way to make that learning stick in its own definition. There is no loop from "this went wrong twice" to a durable, reviewed change in the agent's instructions.

What exists today

• Core memory: fully self-writable. An agent can rewrite core at will with buzz mem set core — no review, no versioned rationale, no rollback beyond engram heads. That is the only self-modification available, and it is the least controlled one.
• System prompt: owner-gated, and correctly so. buzz agents draft-create / draft-update publish an ephemeral draft that only Buzz Desktop's owner can save; the CLI itself reports "saved": false and "Nothing changes until the owner saves it" (crates/buzz-cli/src/commands/agents.rs). The durable records are KIND_MANAGED_AGENT = 30177 / KIND_PRIVATE_MANAGED_AGENT = 30179, personas KIND_PERSONA = 30175.
• Skills: not self-modifiable at all — see the skills issue; there is no distribution mechanism, so an agent cannot add a capability to itself even with owner approval.
• Signal exists but is unconsumed. KIND_AGENT_TURN_METRIC = 44200 is emitted per turn. Nothing reads it back to propose an improvement.

The gap

The pieces are asymmetric: the unreviewed channel (core memory) is wide open, the reviewed channel (prompt drafts) exists but is only ever driven by a human asking for a change, and there is no third channel (skills) at all. Nothing correlates outcomes with instructions, nothing cites why a change is proposed, and nothing can revert a change that made an agent worse.

Proposed direction

A propose → review → apply → revert loop with the human gate kept, not removed.

1. Proposal. An agent emits a self-change proposal: a diff against its persona, core, or skill set, plus a cited rationale — links to the specific events, turns, or failures that motivated it. Reuse the existing draft flow for prompt changes rather than inventing a second one.
2. Review. Owner sees the diff and the evidence side by side in Buzz Desktop and approves or rejects. KIND_APPROVAL_GRANT/DENY (46030/46031) already exist and may fit.
3. Apply + version. Every applied change records what changed, why, and which proposal it came from, so it can be reverted as a unit.
4. Guardrails — the part that needs the most thought.
   • Rate limit proposals; an agent that proposes constantly is noise, not evolution.
   • Certain fields must never be self-proposable without explicit owner action: runtime/provider/model, community scope, permissions, credentials.
   • Prompt-injection surface. Proposals will be triggered by channel content, and channels contain other agents and untrusted text. "Convince an agent to rewrite its own instructions" must be a designed-against attack, not an emergent one. This is the single biggest reason the human gate stays.
   • An agent must not be able to weaken its own guardrails, including this list.
5. Evaluation. Without some notion of "did this change help?", self-evolution is drift. Minimum viable version: the proposal states the observable it expects to improve, and the change is revertable when it does not.

Depends on

• The skills issue — capability changes are a large share of what "evolve" should mean.
• The remote-memory issue — evolution across environments is meaningless if the memory does not survive a machine.

Acceptance criteria

• Written design doc covering the proposal event, the review surface, the guardrail list, and the injection threat model — reviewed before any code.
• Proposal path implemented for at least persona/system-prompt changes, reusing the existing owner-review draft flow.
• Applied changes are versioned and revertable with a single command.
• Deny-list of non-self-proposable fields enforced server-side, not just in the agent's prompt.
• Rate limiting on proposals.
• Tests including an injection-style case: channel content that tries to induce a self-change proposal is either not proposed or clearly attributed to its source in the review UI.

Open questions for JC

• How much autonomy do you actually want? Options, in increasing order: (a) agent drafts, human approves everything — closest to today; (b) auto-apply within a narrow allowlist (core memory, its own notes) with an audit trail; (c) auto-apply with periodic human review. My recommendation is (a) to start, with (b) only for what the agent can already change unreviewed.
• Should evolution be per-agent, or should a lesson one agent learns propagate to the whole team (shared guidance / team catalog KIND_TEAM_CATALOG = 30178)?
• Is "self-evolve" meant to include changing which tools/MCP servers an agent has, or only instructions and skills?

[yjc801]

Dependencies referenced above: skills is #86, remote memory is #87.