Skip to content

fix(ci): don't red-fail Olympix Static Analysis on analyzer timeout / missing SARIF#2096

Open
0xDEnYO wants to merge 3 commits into
mainfrom
feature/exsc-olympix-static-analysis-timeout-sarif-guard
Open

fix(ci): don't red-fail Olympix Static Analysis on analyzer timeout / missing SARIF#2096
0xDEnYO wants to merge 3 commits into
mainfrom
feature/exsc-olympix-static-analysis-timeout-sarif-guard

Conversation

@0xDEnYO

@0xDEnYO 0xDEnYO commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Which Linear task belongs to this PR?

https://linear.app/lifi-linear/issue/EXSC-665/ci-olympix-static-analysis-red-fails-on-analyzer-timeout-missing-sarif

Why did I implement it this way?

Problem

The Static Analysis Security Check job (.github/workflows/olympixStaticAnalysis.yml) fails deterministically on large-diff PRs (e.g. #2055, ~14 changed .sol files):

  1. The Run Olympix Integrated Security step reaches the Olympix analyzer's fixed internal ~3-minute analysis timeout, prints Code analysis finished unsuccessfully because of timeout, and — critically — exits 0 (green) while writing no olympix.sarif.
  2. The next step, Upload Result to GitHub Code Scanning, then hard-fails with ##[error]Path does not exist: olympix.sarif.

Net: an uncontrollable infra timeout surfaces as a cryptic file-not-found red on every large-diff PR. It passed on an earlier, smaller-diff commit of the same PR and reproduced across two re-runs.

The olympix/integrated-security action exposes no timeout input (only -w/-p/-f/-o/--no-<vuln> per its README/action.yaml), so the timeout cannot simply be raised.

Fix

  • Only upload the SARIF when it was actually produced — the upload step is now gated on a sarif_exists check.
  • A new Verify Olympix produced a SARIF report step turns a missing report into a loud ::warning annotation + a job-summary note (naming the likely timeout and the analyzer step outcome) instead of an opaque file-not-found error.
  • The analyzer step is continue-on-error so an incomplete scan can't hard-fail the gate on an infra limit we don't control. When a SARIF is produced, real findings still upload and gate exactly as before.

Why not the alternatives

  • Raise the timeout (a): no action input exists for it.
  • Batch the -p files (c): rejected — the run logs show Scan Type: full regardless of -p, so batching's benefit is uncertain; it also adds SARIF-merge complexity and up to N×3-min sequential Docker runs.
  • Hard-fail loudly on timeout (b, fail variant): would permanently red-block legitimate large PRs on an infra limit. This check is not a required status check on main, so keeping the job green on an incomplete scan does not weaken merge protection; the warning + job summary keep it honest, and other gates (Aikido, check-security-alerts, CodeRabbit, human audit) provide defense in depth.

Tradeoff called out

With continue-on-error, a genuine analyzer misconfiguration (e.g. a bad API token) that produces no SARIF now surfaces as the same warning rather than a red — acceptable given the check is non-required and the warning names the analyzer step outcome. A follow-up option, if the team wants the scan to actually complete on large diffs, is upstream Olympix perf / a configurable timeout.

Checklist before requesting a review

Checklist for reviewer (DO NOT DEPLOY and contracts BEFORE CHECKING THIS!!!)

  • I have checked that any arbitrary calls to external contracts are validated and or restricted
  • I have checked that any privileged calls (i.e. storage modifications) are validated and or restricted
  • I have ensured that any new contracts have had AT A MINIMUM 1 preliminary audit conducted on by <company/auditor>

… missing SARIF

The Olympix analyzer has a fixed internal ~3-minute analysis timeout with no
action input to raise it. On large changed-file sets it hits that timeout,
prints "Code analysis finished unsuccessfully because of timeout", exits 0
(green), and writes NO olympix.sarif. The downstream upload-sarif step then
hard-failed with a cryptic "Path does not exist: olympix.sarif", turning an
uncontrollable infra timeout into a red gate on every large-diff PR (e.g. #2055).

Fix: only upload the SARIF when it was actually produced, and surface an
incomplete scan as a loud warning + job-summary note instead of a file-not-found
error. The analyzer step is continue-on-error so an incomplete scan can't hard
-fail the gate; real findings, when produced, still upload and gate normally.

This check is not a required status check, so keeping the job green on an
incomplete scan does not weaken merge protection; the warning keeps it honest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 07678135-4c3c-4dba-95b7-8c9421b42a12

📥 Commits

Reviewing files that changed from the base of the PR and between 52703bb and 710f40a.

📒 Files selected for processing (1)
  • .github/workflows/olympixStaticAnalysis.yml
💤 Files with no reviewable changes (1)
  • .github/workflows/olympixStaticAnalysis.yml

Walkthrough

The Olympix workflow documents analyzer timeouts, verifies whether olympix.sarif was produced, and conditionally uploads the report to GitHub Code Scanning.

Changes

Olympix SARIF handling

Layer / File(s) Summary
Timeout-tolerant scan and conditional upload
.github/workflows/olympixStaticAnalysis.yml
Workflow comments describe missing SARIF after timeout, a verification step records whether the report exists, and SARIF upload requires changed Solidity files and the generated report.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the workflow fix for analyzer timeouts and missing SARIF.
Description check ✅ Passed The description matches the template and includes the Linear task, problem, fix, alternatives, tradeoff, and checklists.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/exsc-olympix-static-analysis-timeout-sarif-guard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lifi-action-bot
lifi-action-bot marked this pull request as draft July 21, 2026 03:05
@0xDEnYO
0xDEnYO marked this pull request as ready for review July 21, 2026 03:10
@0xDEnYO
0xDEnYO enabled auto-merge (squash) July 21, 2026 03:11
gvladika
gvladika previously approved these changes Jul 21, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/olympixStaticAnalysis.yml:
- Around line 54-57: Remove the continue-on-error: true setting from the Olympix
analyzer step in the workflow. Preserve the existing timeout behavior and the
subsequent “Verify Olympix produced a SARIF report” step so successful timeout
exits remain handled while genuine analyzer failures stop the job.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 8af4e120-96b7-472b-97bb-1b9f643d9da2

📥 Commits

Reviewing files that changed from the base of the PR and between e142f07 and 52703bb.

📒 Files selected for processing (1)
  • .github/workflows/olympixStaticAnalysis.yml

Comment thread .github/workflows/olympixStaticAnalysis.yml Outdated
maxklenk
maxklenk previously approved these changes Jul 21, 2026
Timeouts already exit 0; continue-on-error only masked real analyzer
failures. Missing SARIF is still handled by the verify/warn step.

Co-authored-by: Cursor <cursoragent@cursor.com>
@0xDEnYO
0xDEnYO dismissed stale reviews from maxklenk and gvladika via 710f40a July 22, 2026 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants