Skip to content

product: define golden workflows and decision-outcome metrics #1721

Description

@jonathanhaaswriter

Summary

Define the product workflows and measurement contract that Cerebro will optimize before adding more surface area.

The primary outcome should be evidence-backed decisions completed, not connector count, route count, graph size, or finding volume.

Golden workflows

Define these three workflows end to end:

  1. Change decision — before a pull request or deployment, determine applicable risk, controls, owners, affected assets, evidence freshness, coverage gaps, and whether to proceed.
  2. Finding to verified fix — explain why a finding matters, identify its owner and attack path, propose an action, record approval, execute, and verify closure.
  3. Continuous evidence — maintain current control evidence, identify missing or stale proof, track exceptions, and produce an audit packet.

Each workflow must name its actor, trigger, inputs, decision states, completion state, failure states, required evidence, and measurable value.

Measurement contract

Define a versioned event and metric contract for:

  • decision requested
  • evidence packet built
  • decision accepted, rejected, or deferred
  • recommendation dismissed, including a bounded reason
  • owner assigned
  • action proposed, approved, started, completed, and verified
  • finding reopened
  • audit packet delivered

The north-star metric is weekly evidence-backed decisions completed.

Supporting metrics:

  • time to first trusted decision
  • percentage of decisions with fresh evidence
  • percentage with explicit coverage gaps
  • top-ranked finding acceptance and dismissal rate
  • time from finding to owner, action, and verified closure
  • evidence reuse across controls
  • estimated audit preparation time avoided
  • runtime cost per completed decision

Acceptance criteria

  • A checked-in workflow specification defines all three workflows and their real states.
  • A versioned telemetry schema defines the decision and outcome events.
  • Metrics can be segmented by workflow, tenant, source cohort, and outcome without exposing sensitive values.
  • One end-to-end reference flow emits the complete event sequence.
  • A baseline report can distinguish activity from completed value.
  • New roadmap work can state which workflow metric it is expected to improve.

Non-goals

  • Do not build a web UI in this issue.
  • Do not require every source or product surface to support all three workflows.
  • Do not use connector count or finding count as a value proxy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions