Add clone block latency monitoring and epoch soak - #3027
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
| env: | ||
| RUST_BACKTRACE: full | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[HIGH] Pin actions used by the privileged soak workflow
This new workflow uses mutable @v4 tags for checkout, artifact transfer, Node setup, and artifact upload while running builds on self-hosted infrastructure. A moved or compromised tag can therefore replace reviewed workflow code. Pin every action in this workflow to a full commit SHA; the repository already contains vetted pins for these actions.
| - uses: actions/checkout@v4 | |
| - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 |
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: |
There was a problem hiding this comment.
[HIGH] Pin the newly added setup-node action
This adds a mutable action tag to a PR-triggered job. Because the referenced code can change independently of this repository, compromise or retargeting of the tag would execute unreviewed code during CI. Use the vetted full SHA already used elsewhere in the repository.
| with: | |
| - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 |
| done | ||
|
|
||
| - name: Upload clone block performance report | ||
| if: always() |
There was a problem hiding this comment.
[HIGH] Pin the newly added artifact action
The new artifact-upload step executes a mutable @v4 action in the clone job. Pinning prevents a tag update from silently changing executable CI code. The repository already uses the following vetted v4 commit.
| if: always() | |
| uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 |
🛡️ AI Review — Skeptic (security review)VERDICT: SAFE VERY HIGH scrutiny (young account and no public repositories), mitigated by repository admin permission and substantial merged history; no Gittensor association found. Branch: clone-block-time-epoch-soak → release-v438. Static analysis found no malicious behavior or actionable security vulnerability. The self-hosted soak remains limited to explicitly labeled, same-repository PR heads, uses read-only permissions, pins its added actions, and binds downloaded release artifacts to the selected PR merge SHA and digest. FindingsNo findings. ConclusionThe current head introduces no actionable security issue. 🔍 AI Review — Auditor (domain review)VERDICT: 👎 Gittensor association: UNKNOWN. The account is young, but the author is a repository admin with substantial recent contributions. The implementation matches the substantive PR description. The overlapping PRs share only one file each and are unrelated. No auto-fixes were needed. Findings
Prior-comment reconciliation
ConclusionThe epoch soak remains unreachable as a pre-merge gate because its workflow does not exist on a trusted base/default branch. Bootstrap it through an existing workflow before relying on its results. 📜 Previous run (superseded)
|
|
🔄 AI review updated — Skeptic: VULNERABLE |
| - uses: ./.github/actions/rust-setup | ||
| with: | ||
| cache-key: clone-epoch-soak-node | ||
| sccache-credential-mode: auto | ||
| sccache-writer-access-key-id: ${{ secrets.SCCACHE_R2_WRITE_ACCESS_KEY_ID }} | ||
| sccache-writer-secret-access-key: ${{ secrets.SCCACHE_R2_WRITE_SECRET_ACCESS_KEY }} |
There was a problem hiding this comment.
[CRITICAL] Do not expose secrets to PR-controlled execution
This pull_request job checks out the PR head, runs its local rust-setup action, and subsequently builds its Cargo workspace on a self-hosted runner while supplying cache credentials. Any same-repository PR receiving this label can modify the local action, build scripts, or dependencies to extract those credentials or affect the runner. The maintainer-applied label gates execution but does not make the checked-out code trusted. Remove secrets from the PR path and run secret-bearing setup only from trusted base-ref code, or restrict this workflow to trusted manual dispatch against an immutable reviewed SHA.
| env: | ||
| RUST_BACKTRACE: full | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[HIGH] Pin actions used by the privileged soak workflow
This newly introduced self-hosted workflow uses mutable major-version action tags. A compromised or retargeted tag can change what executes without a reviewed repository commit. Pin every third-party action in this workflow to a full commit SHA.
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: |
There was a problem hiding this comment.
[HIGH] Pin the newly added setup-node action
The new action invocation uses a mutable major-version tag, allowing its executed code to change independently of this repository. Pin it to a reviewed full commit SHA.
| done | ||
|
|
||
| - name: Upload clone block performance report | ||
| if: always() |
There was a problem hiding this comment.
[HIGH] Pin the newly added artifact action
The newly added artifact step executes an action through a mutable major-version tag. Pin it to a reviewed full commit SHA so future tag movement cannot alter CI execution.
|
🔄 AI review updated — Skeptic: VULNERABLE |
| cache-key: clone-epoch-soak-node | ||
| sccache-credential-mode: auto | ||
| sccache-writer-access-key-id: ${{ secrets.SCCACHE_R2_WRITE_ACCESS_KEY_ID }} | ||
| sccache-writer-secret-access-key: ${{ secrets.SCCACHE_R2_WRITE_SECRET_ACCESS_KEY }} |
There was a problem hiding this comment.
[CRITICAL] Do not expose secrets to PR-controlled execution
The labeled pull_request path checks out the PR merge ref, invokes a PR-local action, and builds PR-controlled code on a self-hosted runner while supplying cache writer credentials. A same-repository PR can therefore execute arbitrary code with those credentials. Remove secrets from PR-triggered execution and isolate the untrusted build from credentialed jobs.
| env: | ||
| RUST_BACKTRACE: full | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[HIGH] Pin actions used by the privileged soak workflow
This privileged workflow uses mutable major-version references for checkout, artifact upload/download, and Node setup. Pin every third-party action to a reviewed full commit SHA so upstream tag movement cannot alter code executed on the self-hosted runner.
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: |
There was a problem hiding this comment.
[HIGH] Pin the newly added setup-node action
actions/setup-node@v4 is a mutable tag. Pin it to a reviewed full commit SHA to prevent the executed action from changing without a repository diff.
| done | ||
|
|
||
| - name: Upload clone block performance report | ||
| if: always() |
There was a problem hiding this comment.
[HIGH] Pin the newly added artifact action
actions/upload-artifact@v4 is a mutable tag. Pin it to a reviewed full commit SHA to keep artifact-handling code immutable.
|
🔄 AI review updated — Skeptic: VULNERABLE |
| size: ${{ steps.release.outputs.size }} | ||
| run_id: ${{ steps.release.outputs.run_id }} | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[HIGH] Pin actions used by the privileged soak workflow
This labeled-PR workflow executes checked-out PR code on a self-hosted runner, but its checkout, Node setup, and artifact-upload actions use mutable @v4 tags. Pin every action in this workflow to a reviewed full commit SHA so an upstream tag move cannot inject code into this privileged execution path.
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added setup-node action
The PR newly makes this mutable @v4 action part of Runtime Checks. Pin it to a reviewed full commit SHA so an upstream tag move cannot alter code executed by this workflow.
|
|
||
| - name: Upload clone block performance report | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added artifact action
This newly introduced action reference is mutable. Pin actions/upload-artifact to a reviewed full commit SHA to prevent an upstream tag move from changing code executed in the clone job.
|
🔄 AI review updated — Skeptic: VULNERABLE |
| size: ${{ steps.release.outputs.size }} | ||
| run_id: ${{ steps.release.outputs.run_id }} | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[HIGH] Pin actions used by the privileged soak workflow
This workflow executes PR-controlled code on a self-hosted runner, but its checkout, setup-node, and upload-artifact actions use mutable major-version tags. A compromised or retargeted upstream tag would gain execution in this job. Pin every action to a reviewed full commit SHA.
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added setup-node action
The newly reachable setup-node invocation uses the mutable @v4 tag. Pin it to a reviewed full commit SHA so upstream tag movement cannot change code executed by Runtime Checks.
|
|
||
| - name: Upload clone block performance report | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added artifact action
The new artifact-upload step executes actions/upload-artifact through a mutable major-version tag. Pin the action to a reviewed full commit SHA to prevent upstream tag movement from changing this CI execution path.
|
🔄 AI review updated — Skeptic: VULNERABLE |
36bb744 to
424a54f
Compare
| id: deadline | ||
| run: echo "deadline_epoch_ms=$(( $(date -u +%s) * 1000 + 140 * 60 * 1000 ))" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[HIGH] Pin actions used by the privileged soak workflow
This new workflow still references actions/checkout, actions/setup-node, and actions/upload-artifact through mutable major-version tags. Pin every action in this workflow to a reviewed full commit SHA so an upstream tag move cannot change code executed on the self-hosted soak runner.
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added setup-node action
The newly reachable actions/setup-node@v4 reference is mutable. Pin it to a reviewed full commit SHA to prevent an upstream tag move from changing code executed by Runtime Checks.
|
|
||
| - name: Upload clone block performance report | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added artifact action
The newly added actions/upload-artifact@v4 reference is mutable. Pin it to a reviewed full commit SHA to prevent an upstream tag move from changing code executed by the clone job.
|
🔄 AI review updated — Skeptic: VULNERABLE |
| id: deadline | ||
| run: echo "deadline_epoch_ms=$(( $(date -u +%s) * 1000 + 140 * 60 * 1000 ))" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
[HIGH] Pin actions used by the privileged soak workflow
This new workflow executes actions/checkout, actions/setup-node, and actions/upload-artifact through mutable @v4 tags, including on a self-hosted runner. Pin every action to a reviewed full commit SHA so an upstream tag move cannot replace code executed by the job.
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added setup-node action
The newly reachable actions/setup-node@v4 reference is mutable. Pin it to a reviewed full commit SHA so upstream tag movement cannot change code executed during Runtime Checks.
|
|
||
| - name: Upload clone block performance report | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 |
There was a problem hiding this comment.
[HIGH] Pin the newly added artifact action
The newly added actions/upload-artifact@v4 reference is mutable and runs in the clone job. Pin it to a reviewed full commit SHA so upstream tag movement cannot replace the action implementation.
|
|
||
| on: | ||
| pull_request: | ||
| types: [labeled, synchronize] |
There was a problem hiding this comment.
[HIGH] The new workflow still cannot run before merge
This remains unresolved. GitHub only discovers pull_request and workflow_dispatch workflows that already exist on the applicable trusted/default branch; adding labeled and synchronize triggers inside this new PR-only workflow does not bootstrap it. Consequently, the existing run-clone-epoch-soak label cannot execute the advertised acceptance soak for this PR. Land the workflow trigger separately on the base/default branch, or invoke this soak from an existing base-branch workflow that explicitly checks out and validates this PR's immutable head SHA.
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👎 |
26919cf to
82e0222
Compare
| pull_request: | ||
| types: [labeled, synchronize] |
There was a problem hiding this comment.
[HIGH] The new workflow still cannot run before merge
Neither trigger bootstraps this workflow for the current PR: pull_request workflows are selected from the base branch, while workflow_dispatch requires the workflow to exist on the default branch. Since this file exists only on the PR branch, applying the label or manually dispatching cannot execute the advertised pre-merge soak. Add the trigger to an existing trusted base/default-branch workflow (for example, dispatching a reusable implementation at the immutable PR SHA), then demonstrate the soak result before merge.
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👎 |
82e0222 to
b5f3129
Compare
|
|
||
| on: | ||
| pull_request: | ||
| types: [labeled, synchronize] |
There was a problem hiding this comment.
[HIGH] The new workflow still cannot run before merge
This workflow is absent from both release-v438 and the default branch. GitHub evaluates pull_request workflows from the base branch and exposes workflow_dispatch only for workflows present on the default branch, so neither the label nor synchronization event can start this soak for PR #3027. Bootstrap the dispatch from an existing trusted base/default-branch workflow (or merge that trigger separately first); otherwise the advertised full epoch validation cannot execute before this PR merges.
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👎 |
b5f3129 to
348980c
Compare
| pull_request: | ||
| types: [labeled, synchronize] |
There was a problem hiding this comment.
[HIGH] [BOOTSTRAP] The new workflow still cannot run before merge
This workflow is absent from both the PR base and the repository’s default branch. GitHub evaluates pull_request triggers from the base branch, while workflow_dispatch is available only for workflows already present on the default branch. Consequently, neither trigger can start this soak for the PR that introduces it, despite the run label being applied. Bootstrap the dispatch from an existing trusted workflow/branch, passing the immutable PR head SHA, before treating this as pre-merge validation. After this workflow merges, the bootstrap limitation becomes structurally unreachable for subsequent PRs; reintroducing a PR-only trigger later would be a strong red flag.
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👎 |
|
|
||
| on: | ||
| pull_request: | ||
| types: [labeled, synchronize] |
There was a problem hiding this comment.
[HIGH] [BOOTSTRAP] The new workflow still cannot run before merge
GitHub cannot trigger either pull_request or workflow_dispatch for a workflow that exists only on this PR branch; the workflow is absent from the base/default branch. Consequently, the labeled soak advertised as pre-merge validation cannot run at this head.
Bootstrap the soak through an existing trusted workflow such as runtime-checks.yml, or land an entry-point workflow on the trusted branch first. The PR-controlled workflow requests read-only contents, actions, and pull-requests permissions and reaches a self-hosted runner; after the entry point lands on the default branch, this bootstrap limitation becomes structurally irrelevant for subsequent PRs.
|
🔄 AI review updated — Skeptic: SAFE Auditor: 👎 |
Summary
This is stacked on #3019.
Behavior
Validation
npm run typechecknpm run test:clone-performance(10 tests).github/scripts/test-clone-regression-phase.sh.github/scripts/test-clone-epoch-soak-workflow.sh.github/scripts/test-runtime-change-filter.shgit diff --checkThe full PR #3019 clone acceptance run is intentionally left to CI because it requires the release node and trusted mainnet snapshot.