Skip to content

Add incremental system test progress to job summary#821

Open
PranjalManhgaye wants to merge 3 commits into
precice:developfrom
PranjalManhgaye:issue-791-summary-progress
Open

Add incremental system test progress to job summary#821
PranjalManhgaye wants to merge 3 commits into
precice:developfrom
PranjalManhgaye:issue-791-summary-progress

Conversation

@PranjalManhgaye
Copy link
Copy Markdown
Collaborator

Write progress entries to GITHUB_STEP_SUMMARY after each finished system test while keeping the final results table.

Description

Closes #791.

This PR adds incremental progress reporting to the system tests job summary, instead of only writing the summary table after the full suite completes, the runner now:

  • initializes a System tests progress section in GITHUB_STEP_SUMMARY
  • appends one markdown bullet after each finished system test
  • keeps the final system test results table at the end
    Locally verified incremental GITHUB_STEP_SUMMARY updates (progress bullets after each test + final table).
    This makes long-running system test workflows easier to follow from the GitHub Actions summary page.

Validation

verified locally by writing to a temporary GITHUB_STEP_SUMMARY file and by running release_test locally to confirm that progress entries are appended during execution and that the final results table is still written.

Checklist

  • I added a summary of any user-facing changes (compared to the last release) in the changelog-entries/<PRnumber>.md.
  • New tutorial case (e.g., new fluid-openfoam folder for existing tutorial)? Add it to the respective README.md.
  • New tutorial? Update the website.

Resources

Write progress entries to GITHUB_STEP_SUMMARY after each finished system test while keeping the final results table.

Fixes precice#791.
@PranjalManhgaye PranjalManhgaye requested a review from MakisH May 31, 2026 09:44
Copy link
Copy Markdown
Member

@MakisH MakisH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for looking into this. I tested it in this run: https://github.com/precice/tutorials/actions/runs/26725640841

However, it looks like the summary is only posted after the step finishes. Is there a way to tell GitHub Actions to display the summary while the job is running?

Comment thread changelog-entries/821.md Outdated
Link the pull request instead of the issue and keep the entry short.
@PranjalManhgaye
Copy link
Copy Markdown
Collaborator Author

thanks for checking ,
but ig on the live summary question: with one long “Run tests” step, i don’t think GitHub will refresh the Summary tab after each test. i do append to GITHUB_STEP_SUMMARY when each test finishes, but the UI usually only shows it when the step ends , so your run looks normal to me, not like the writes failed ,,
For progress while the job is still going, the log is what actually updates (and #801 helps there). and a summary that updates in real time would probably mean changing the workflow , e.g. more steps — which feels out of scope for this PR.
Happy to either ig go with #821 as “summary fills in at step end” or park the live-update part for a follow-up if you’d rather go that way. let me know what you prefer ,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System tests job summary is only written at the end

2 participants