Skip to content

Feat: Human-in-the-loop effect for APL and elicitation plugin#115

Draft
terylt wants to merge 3 commits into
devfrom
feat/hil_apl
Draft

Feat: Human-in-the-loop effect for APL and elicitation plugin#115
terylt wants to merge 3 commits into
devfrom
feat/hil_apl

Conversation

@terylt

@terylt terylt commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds human-in-the-loop (HIL) elicitation to APL: a policy can pause an operation to ask a human — manager approval, a confirm, a step-up re-auth, an attestation — and resume once the human responds, without blocking the request path. The human's decision is bound to the actual request args and recorded from CPEX-owned state, so it is the source of truth for "what was approved," never an LLM summary. Includes a working Keycloak CIBA channel plugin.

A policy expresses it with a sugar verb that parses to a single Step::Elicit:

plugins:
  - name: manager-approver
    kind: elicitation/ciba
    hooks: [elicit]

routes:
  - tool: approve_raise
    pre_invocation:
      - "require_approval(manager-approver, from: claim.manager, channel: \"ciba\", scope: \"args.amount <= 25000\", purpose: \"Approve raise\")"

from is who to ask (approver, resolved from the bag — may differ from the subject); scope is the args-binding the runtime checks against the live request; purpose is the audited, human-readable description.

The model: async, retry-based resume

An elicitation has three short, synchronous touch-points — the hours-long human gap lives in the channel (Keycloak CIBA), never in a blocking call:

  1. Dispatch — first arrival: register the intent, open the channel backchannel, return a correlation id.
  2. Check — on each agent retry: read status (Pending / Resolved{approved|denied} / Expired) without blocking.
  3. Validate — once resolved: verify the response is genuine (signed token, intent binding, responder identity). The runtime then layers the scope-over-args sufficiency check before honoring the approval.

While pending, the phase suspends rather than denies. Decision stays binary — a suspended phase reports Allow + Some(PendingElicitation), and the host maps that to JSON-RPC -32120 ("not complete, retry echoing this id"). The forwarding rule stays one clause: forward iff Allow AND pending.is_none(). Expiry / channel error fails closed (on_error: deny default).

What's included

Area Change
DSL verbs (apl-core) require_approval / confirm / step_up / attestation / info / review — sugar over one Step::Elicit(ElicitStep) with an ElicitKind. Fields: from, purpose, scope, timeout, channel, config_override, on_error. Parser + evaluator (dispatch/check/validate flow, pending short-circuit, scope-over-args) + route.rs resume wiring.
Elicitation hook family (cpex-core) New ElicitationHook (single hook name "elicit") + ElicitationPayload with a Dispatch/Check/Validate operation discriminator — mirrors the delegation hook layout; dispatch is free via invoke_entries::<ElicitationHook>. cpex-core keeps its own ElicitationOp/ElicitationStatusKind/ElicitationOutcomeKind (no apl-core dep); the bridge maps.
Bridge / invoker (apl-cpex) ElicitationInvoker (dispatch/check/validate) + elicitation_invoker.rs; dispatch-plan resolves an elicit entry by name; route_handler drives the pending/resume flow and surfaces elicitation.* bag attrs (id, approver, intent_id, channel) for audit.
CIBA channel plugin (builtins/plugins/elicitation-ciba, new crate) kind: elicitation/ciba — a HookHandler<ElicitationHook> implementing the three operations against Keycloak CIBA: fromlogin_hint, purposebinding_message, timeoutrequested_expiry. Intent store, config, factory. Registered via cpex-builtins.
Args binding via scope The approval is bound to the request with an APL boolean expression (args.amount <= 25000) the runtime evaluates at validate — kept in APL because Keycloak has no RFC 9396 RAR.

Notable design decisions

  • Approver ≠ subject. from resolves the party to ask from the request bag (e.g. claim.manager); the resolved identity is cross-checked against the actual responder at validate.
  • Sufficiency vs genuineness split. The channel plugin proves the response is genuine (signature / intent binding / responder); the sufficiency check (scope over live args) stays in the runtime because it is an APL expression the plugin can't evaluate.
  • One hook, three operations. A single elicit hook + payload discriminator (not three hook names) keeps registration and the dispatch plan trivial — one plugin, one entry, resolved by name like token.delegate.
  • Tri-state without widening Decision. PendingElicitation rides alongside an Allow so the deny path stays untouched; the host's one-clause forward rule and -32120 retry contract carry the suspend/resume.
  • Fail-closed. Expiry, channel error, or a failed validation deny (subject to on_error).
  • cpex-core decoupled from apl-core. The hook payload defines its own op/status/outcome enums; the apl-cpex bridge translates to/from apl-core's ElicitKind / ElicitationStatus / ElicitationOutcome.

Public API surface

  • cpex-core::elicitation: ElicitationHook, HOOK_ELICIT ("elicit"), ElicitationPayload, ElicitationOp, ElicitationStatusKind, ElicitationOutcomeKind.
  • apl-core::step: Step::Elicit, ElicitStep, ElicitKind, ElicitationInvoker (dispatch/check/validate), ElicitationDispatch, ElicitationStatus, ElicitationOutcome, ElicitationValidation, PendingElicitation, ElicitationError.
  • apl-cpex: ElicitationInvoker bridge, dispatch-plan elicitation_entries, route-handler pending/resume wiring.
  • builtins/plugins/elicitation-ciba: kind = "elicitation/ciba" (CibaApproverFactory), registered through cpex-builtins.

Testing

  • CIBA pluginbuiltins/plugins/elicitation-ciba/tests/ciba_e2e.rs (7 tests) + live_keycloak.rs (1 live integration test, ignored by default).
  • apl-cpextests/elicit_step_e2e.rs: end-to-end through a real PluginManager (dispatch → pending → resolve → validate; approve / deny / expire; args-binding). Existing suites (delegate_step_e2e, end_to_end_route, cmf_invoker_dispatch) updated for the new step/plan shape.
  • apl-core — parser / evaluator / step.rs / route.rs unit tests; tests/yaml_end_to_end.rs updated for the elicit verbs; apl-cmf/tests/end_to_end.rs touched.

Note: I have not re-run the full workspace suite on this branch — worth a cargo test --workspace / make test befor

Follow-ups

  • Site docs page for elicitation under docs/content/docs/apl/ (none in this PR).
  • Additional channel plugins beyond CIBA (Slack, in-band); capability-gating for the elicit hook; richer per-kind validation contracts.

terylt added 3 commits July 8, 2026 16:01
Signed-off-by: Teryl Taylor <terylt@ibm.com>
Signed-off-by: Teryl Taylor <terylt@ibm.com>
Signed-off-by: Teryl Taylor <terylt@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant