Skip to content

Feat: stream live apply logs to audit API - #231

Merged
brandonmontijo merged 5 commits into
mainfrom
bmontijo/log-chunk-streaming
Sep 3, 2026
Merged

Feat: stream live apply logs to audit API#231
brandonmontijo merged 5 commits into
mainfrom
bmontijo/log-chunk-streaming

Conversation

@brandonmontijo

@brandonmontijo brandonmontijo commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Buffers apply stdout line-by-line and flushes to POST /log_chunk every 10 lines or 5s
  • No-op when audit_api_url is not set — existing behavior unchanged
  • Version bump to 0.11.6 (rebased twice onto main since this PR opened: chunk posting now loops over every URL in audit_api_url to match main's multi-URL support, and buffering sits on top of main's incremental UTF-8 decoder in _execute_command so a final partial multi-byte char still flushes to a chunk)

Test plan

  • Point audit_api_url at devops-testing, run tf apply, confirm chunks POST successfully
  • Confirm apply still succeeds if chunk POST fails (errors logged, not raised)
  • Unit tests added for _post_log_chunk and the on_chunk buffering/failure-handling in execute_command

Testing before merge

This feature spans 3 repos — terraform-audit-api#193 and launchpad#130 — and can't be verified in isolation:

  1. Deploy terraform-audit-api#193 to devops-testing (creates the log_chunks table + new endpoints).
  2. Install this branch locally and run a real tf apply/destroy against a devops-testing directory with audit_api_url pointed at that deploy. Confirm POST /log_chunk succeeds and local console output isn't stalling.
  3. Run launchpad#130 locally against the devops-testing API, trigger the same apply from the tfaudit UI, confirm the Output page streams live and swaps to the full output on completion.
  4. Repeat step 3 for a failing apply and for two concurrent applies in different directories.

@brandonmontijo
brandonmontijo requested a review from a team as a code owner June 30, 2026 15:09
@brandonmontijo
brandonmontijo marked this pull request as draft June 30, 2026 15:11
@brandonmontijo
brandonmontijo marked this pull request as ready for review July 6, 2026 21:35
- Bump version to 0.11.5 (was based on stale 0.11.4/0.10.26)
- _chunk_callback now posts to every URL in audit_api_url, matching
  the multi-URL support added to _post_audit_info on main
- Bump version to 0.11.6 (main independently landed its own 0.11.5)
- Layer chunk buffering on top of main's incremental UTF-8 decoder in
  _execute_command, keeping the decoder's is_eof-after-processing
  ordering so a final partial multi-byte char still gets chunked
Bump version to 0.11.7 (main independently landed its own 0.11.6).
cli.py auto-merged clean this time; only the version bump collided.
@brandonmontijo brandonmontijo changed the title [EXPERIMENTAL] feat: stream live apply logs to audit API Feat: stream live apply logs to audit API Sep 3, 2026
@brandonmontijo
brandonmontijo merged commit 203dd1c into main Sep 3, 2026
11 checks passed
@brandonmontijo
brandonmontijo deleted the bmontijo/log-chunk-streaming branch September 3, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants