Plan: integrate request correlation across HTTP, tasks, and outbound provider calls (4.1.3)

[leynos]

Summary

Adds the execution plan for roadmap item 4.1.3, integrating request
correlation across the Falcon HTTP boundary, Celery tasks, and outbound
OpenAI-compatible provider calls.

• Plan: docs/execplans/4-1-3-integrate-request-correlation.md
• Roadmap: docs/roadmap.md lines 606-626
• Design: docs/episodic-podcast-generation-system-design.md:2033-2044

This PR contains no production code — only the plan and its index entry.

What the plan covers

Ten milestones: pin falcon-correlate; build an episodic-owned correlation
seam with sanitization and runtime configuration; wire
CorrelationIDMiddlewareASGI ahead of the authorization middleware; split
episodic/observability.py below the 400-line limit; bootstrap logging and
carry the identifier into log lines; configure Celery propagation; correlate
outbound provider calls; two behavioural-test milestones; and documentation
with a new ADR 018.

Design review

The first draft was stress-tested by a six-lens review panel — structure,
alternatives, scaling and observability, contracts, failure modes, and
long-term viability. It falsified four load-bearing premises. Each was verified
by probing the runtime rather than by reading documentation, and each changed
the plan.

make check-architecture cannot enforce the boundary the draft relied on.
Hecate's include_external_packages defaults to false and [tool.hecate]
does not set it, so the gate sees zero third-party imports and would pass a
domain module importing httpx directly. Constraint C2 is reworded as a review
obligation and a new import-graph test replaces the imagined gate.

Not every log call funnels through the logging helpers. 29 of 48 call sites
bypass them — including _log_authorization_denial, the roadmap's own headline
example, which logs at DEBUG through a direct logger.log call. The draft's
marquee transcript did not correspond to any message the codebase emits; it is
replaced with the real one, and the plan now schedules the edits instead of
claiming coverage it did not have.

import falcon_correlate registers the Celery signal handlers globally, at
import time, and pulls Celery in at ~87 ms. configure_celery_correlation(app)
ignores its argument entirely, so the draft's negative control — "delete the
call and the test must fail" — was unfalsifiable. It is now signal
disconnection.

episodic/observability.py is 399 lines against a blocking 400-line Pylint
limit, so the planned edit would have failed make lint on the first run. A
new milestone splits it first, including the ~20 Skylos entry-point rules that
must be renamed in lockstep.

Two fixes that looked right were tested and rejected: importing a
falcon_correlate submodule does not avoid the Celery side effect, because
Python executes the package __init__ first; and hypothesis.event() cannot
enforce class coverage, because it only feeds the statistics report and a
module-level counter is unsafe under pytest-xdist.

Security findings folded in

Verified against Granian: a 60,000-byte correlation header is accepted,
stored, and echoed in full, and the payload abc method=GET path=/admin principal_id=root round-trips intact — audit-trail forgery in a last-wins
logfmt parser, since the identifier is appended last. Sanitization is now
unconditional at the seam rather than dependent on an optional upstream
validator.

Risk R2 is rewritten. Granian does populate scope["client"], so the
draft's stated hazard was largely theoretical. The real one is that Traefik on
DOKS gives operators no stable range to trust — pod IPs change on every
reschedule — while relaying the header verbatim, so the path of least
resistance trusts every pod in the cluster. The plan now ships a Traefik
middleware that overwrites the header at the edge.

Notes for the reviewer

• One deviation needs sign-off. Decision D10 adds a correlate_client
  helper beyond the roadmap's literal "falcon-correlate transport support",
  because every existing adapter fixture injects a client and the transport
  approach structurally cannot reach those. Flagged as a reviewer gate before
  EP-M5.
• falcon-correlate is pinned at v0.1.0, which upstream cut in response
  to the draft. It resolves to caea7a6a — the exact commit every finding here
  was probed against — so nothing needed re-verifying. Risk R1 drops from
  medium/certain to low/low; what survives is that the package is still absent
  from PyPI, single-author, and alpha-status. Because a Git tag is mutable where
  a commit is not, EP-M0 asks the implementer to confirm the locked revision in
  uv.lock, so a moved tag surfaces as a reviewable lockfile change.
• Two upstream issues are identified for filing rather than local shimming.
• make markdownlint (which runs the spelling gate) passes with 0 issues.

References

• Lody session: https://lody.ai/leynos/sessions/d18130de-405b-4adf-a88f-ccf6940c7394

🤖 Generated with Claude Code

Summary by Sourcery

Define the implementation plan for end-to-end request correlation across Episodic’s HTTP, worker, logging, and provider boundaries.

Enhancements:

• Add a comprehensive execution plan for propagating sanitized request correlation identifiers across HTTP requests, Celery tasks, logs, and outbound provider calls.
• Document design decisions, security risks, verification obligations, implementation milestones, and operational limitations for the correlation rollout.

Documentation:

• Index the new request-correlation execution plan in the documentation contents.

[coderabbitai]

Warning

Your free Security trial is over. An organization admin can activate billing to continue.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Warning

Your free Security trial is over. An organization admin can activate billing to continue.

---

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

Adds a detailed execution plan document for roadmap item 4.1.3 (request correlation across HTTP, Celery, and outbound provider calls) and indexes it in the docs contents, with no production code changes.

Sequence diagram for planned HTTP request correlation

sequenceDiagram
    participant Client
    participant Traefik
    participant Falcon
    participant Authorization
    participant Celery
    participant Provider

    Client->>Traefik: HTTP request
    Traefik->>Traefik: Overwrite X-Correlation-ID
    Traefik->>Falcon: Forward request
    Falcon->>Falcon: CorrelationIDMiddlewareASGI.process_request
    Falcon->>Authorization: AuthorizationMiddleware.process_request
    Authorization-->>Falcon: Allow or deny
    Falcon->>Celery: Publish task with correlation_id
    Falcon->>Provider: HTTP request with X-Correlation-ID
    Falcon-->>Client: Response with X-Correlation-ID

Loading

Sequence diagram for planned Celery correlation propagation

sequenceDiagram
    participant HTTP as HTTP Service
    participant Celery as Celery Publish Signal
    participant Worker
    participant Task

    HTTP->>Celery: before_task_publish
    Celery->>Celery: propagate_correlation_id_to_celery
    Celery-->>Worker: Message with correlation_id
    Worker->>Worker: task_prerun restores correlation_id
    Worker->>Task: Execute task
    Task-->>Worker: task_postrun clears correlation_id

Loading

Sequence diagram for planned outbound provider correlation

sequenceDiagram
    participant Request as Request Context
    participant Adapter as OpenAICompatibleLLMAdapter
    participant Client as httpx AsyncClient
    participant Provider as OpenAI-Compatible Provider

    Request->>Adapter: LLM operation
    Adapter->>Client: Build correlated client
    Client->>Provider: HTTP request with X-Correlation-ID
    Provider-->>Client: Provider response
    Client-->>Adapter: Response

Loading

File-Level Changes

Change	Details	Files
Index the new request-correlation execution plan in the documentation contents so it appears alongside other exec plans.	Add a bullet linking to execplans/4-1-3-integrate-request-correlation.md under the execution plans section Provide a short description for the plan entry (request correlation propagation plan)	docs/contents.md
Introduce a comprehensive execution plan document for integrating request correlation across Falcon HTTP, Celery workers, and outbound OpenAI-compatible provider calls, including constraints, risks, milestones, and verification strategy.	Define purpose, scope, and desired behaviour for request correlation identifiers across HTTP, tasks, and provider calls Capture design-review outcomes and surprises that changed the plan’s assumptions List constraints, tolerances, risks, and axioms governing correlation behaviour and security Lay out a multi-milestone implementation plan (EP-M0–EP-M7) covering dependency pinning, middleware seam, logging, Celery propagation, provider correlation, behavioural tests, and documentation Specify verification plan with concrete test obligations and artefacts (INV-1 through INV-10) and residual gaps Document decision log (D1–D12) explaining key design choices, security posture, and coupling to falcon-correlate Provide progress checklist, conformance basis to roadmap/design docs/ADRs, and operational notes for implementation and recovery	docs/execplans/4-1-3-integrate-request-correlation.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[codescene-access]

No quality gates enabled for this code.