[wip] publish_test_results: annotate test failures on GitHub Actions#3484
Open
jstarks wants to merge 2 commits into
Open
[wip] publish_test_results: annotate test failures on GitHub Actions#3484jstarks wants to merge 2 commits into
jstarks wants to merge 2 commits into
Conversation
On the GitHub Actions CI backend, JUnit XML results are uploaded as raw artifacts but never parsed — making it hard to find which tests failed without downloading and reading XML. The ADO backend doesn't have this problem because PublishTestResults@2 natively integrates results into the UI. Add a Rust step on the GitHub backend that parses the JUnit XML with roxmltree (already a dependency) and emits ::error:: workflow commands for each failure. These show as annotations on the PR checks tab. The step also writes a Markdown summary table to $GITHUB_STEP_SUMMARY for a quick visual overview on the job summary page.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds GitHub Actions-native visibility for failed Rust test runs by parsing published JUnit XML and emitting annotations plus job-summary output.
Changes:
- Adds a GitHub-only Flowey Rust step to parse JUnit XML and annotate failed/error test cases.
- Writes a Markdown failure summary to
$GITHUB_STEP_SUMMARY. - Regenerates OpenVMM GitHub workflow YAMLs to include the new annotation steps.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
flowey/flowey_lib_common/src/publish_test_results.rs |
Adds JUnit failure parsing and GitHub annotation/summary emission. |
.github/workflows/openvmm-pr.yaml |
Regenerated PR workflow with annotation steps. |
.github/workflows/openvmm-pr-release.yaml |
Regenerated PR release workflow with annotation steps. |
.github/workflows/openvmm-ci.yaml |
Regenerated CI workflow with annotation steps. |
|
|
||
| // GitHub Actions workflow command — shows as an annotation on | ||
| // the PR checks tab. | ||
| eprintln!("::error title=Test failure: {full_name}::{short_msg}"); |
Replace :: with / in the annotation title to avoid breaking GitHub Actions workflow command parsing (:: is the command delimiter). Remove the step name prefix so the annotation shows the test path directly. Drop the source file mapping code—it was unreliable and added too much complexity for uncertain benefit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On the GitHub Actions CI backend, JUnit XML results are uploaded as raw artifacts but never parsed — making it hard to find which tests failed without downloading and reading XML. The ADO backend doesn't have this problem because PublishTestResults@2 natively integrates results into the UI.
Add a Rust step on the GitHub backend that parses the JUnit XML with roxmltree (already a dependency) and emits ::error:: workflow commands for each failure. These show as annotations on the PR checks tab. The step also writes a Markdown summary table to $GITHUB_STEP_SUMMARY for a quick visual overview on the job summary page.