Skip to content

Add Process.ReadAllTextAsync and ReadAllBytesAsync#126942

Open
Copilot wants to merge 34 commits intomainfrom
copilot/add-readalltextbytes-async-methods
Open

Add Process.ReadAllTextAsync and ReadAllBytesAsync#126942
Copilot wants to merge 34 commits intomainfrom
copilot/add-readalltextbytes-async-methods

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 15, 2026

Description

Adds asynchronous ReadAllTextAsync(CancellationToken) and ReadAllBytesAsync(CancellationToken) APIs to System.Diagnostics.Process, enabling fully async multiplexed reads of both stdout and stderr.

Implementation

  • Async per-stream helper: ReadPipeToBufferAsync(Stream, CancellationToken) reads a single stream into a pooled buffer, owns the buffer lifecycle (returns to pool on failure via catch, transfers ownership via ArraySegment on success).
  • Shared helper: ReadAllBytesIntoRentedArraysAsync validates state, launches two parallel ReadPipeToBufferAsync tasks via Task.WhenAll, handles buffer cleanup for the successful task when the other fails, and surfaces both exceptions via AggregateException when both fail.
  • Sync path improvements: Refactored GetSafeFileHandleFromStreamReader into OS-specific GetSafeHandleFromStreamReader helpers (Process.Multiplexing.Windows.cs, Process.Multiplexing.Unix.cs), using SafePipeHandle on Unix and SafeFileHandle on Windows. Simplified sync ReadAllText/ReadAllBytes to use AsSpan() directly.
  • Tests: Extended ProcessMultiplexingTests with useAsync parameter across all existing test methods. Added cancellation test (ReadAllAsync_ThrowsOperationCanceledOnCancellation) and multi-exception test (ReadAllAsync_ThrowsAllAvailableExceptions).

Testing

  • ✅ Build verified (0 errors, 0 warnings)
  • ✅ Tests verified (467 total, 0 failed, 5 skipped)
  • ✅ Multi-model code review: no blocking issues found

Copilot AI and others added 22 commits April 12, 2026 07:31
…le fixes needed)

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/45ece5ba-166f-4418-887f-3ecaba54615e

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…ead of GetAwaiter().GetResult()

Replace the async Task-based Windows implementation with proper Win32 overlapped IO:
- Allocate NativeOverlapped* with ManualResetEvent for each pipe
- Use Interop.Kernel32.ReadFile with overlapped structs
- Use WaitHandle.WaitAny for synchronous multiplexing
- Use GetOverlappedResult to retrieve bytes read
- Use CancelIoEx for timeout cancellation
- Add System.Threading.Overlapped project reference
- Add ReadFile, GetOverlappedResult, CancelIoEx interop references
- Fix nullable warning in Unix implementation

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/8af96aef-e52b-459d-a945-e1159e8be19b

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…in32Exception

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/8af96aef-e52b-459d-a945-e1159e8be19b

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…elper method, DangerousAddRef/Release, remove partial

Changes:
1. Require both stdout and stderr to be redirected (throw if either is missing)
2. Create ReadPipesToBuffers helper shared between ReadAllText and ReadAllBytes
   (ReadAllText avoids extra buffer copy by decoding from rented buffers directly)
3. GetSafeFileHandleFromStreamReader now returns an owner SafeHandle; caller uses
   DangerousAddRef/DangerousRelease to prevent handle disposal during reads
4. Remove partial method declaration; ReadPipes is a regular static method in each
   platform file (non-nullable parameters since both handles always provided)
5. Update tests: all tests redirect both stdout and stderr; add two new tests for
   missing-stream validation

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/cd917b13-f344-47c5-ac1e-cfb2c2da34f6

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…indows ReadPipes fully unsafe with proper cleanup, use is not null

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/451cf437-2de0-409a-937a-0031c3fa69d4

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
- refactor the code
- restore deleted comments
- fix AV bug by setting the lower bit for EventHandle
- fix test bug: process argument can't be that long
- refactor the tests to use Theories rather than very similar Fact for bytes and text
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…RentLargerBuffer overflow

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/34214748-db27-499b-88ef-23a1af15515f

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…mode test

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/f6ca8588-63c1-4cd8-aeaf-48c716a4ea11

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…rlapped cleanup, new tests, StreamBody->Dummy

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/dac42b05-c591-4e1c-b751-9eadeb376728

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…al in interleaved test

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/560e33ea-a2eb-4f65-a9c9-a2bc3fc32986

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…provements, simplified poll check

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/8b364ac1-c21d-4c74-88ca-44d6a5ea4064

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…ly official docs references

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/ef4fa489-ab10-4574-9069-ca32ccaac99b

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…Read with EAGAIN handling

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/2646b5c6-b0ae-4ae3-99ca-064187661e73

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/9d7b276f-4c66-4ff0-88ee-087b732ad8ce

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/9d7b276f-4c66-4ff0-88ee-087b732ad8ce

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Base automatically changed from copilot/implement-readalltext-and-readallbytes to main April 15, 2026 10:13
@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process
See info in area-owners.md if you want to be subscribed.

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 15, 2026 16:05
@adamsitnik
Copy link
Copy Markdown
Member

@copilot please check latest commits, do NOT touch the code, just update the PR description and then invoke the code-review skill and post your analysis/results as a comment on this PR

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Copilot AI requested review from Copilot and removed request for Copilot April 15, 2026 16:19
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 15, 2026

@copilot please check latest commits, do NOT touch the code, just update the PR description and then invoke the code-review skill and post your analysis/results as a comment on this PR

PR description updated. Here is the code review:

🤖 Copilot Code Review — PR #126942

Holistic Assessment

Motivation: The changes since the merge base (8875214c) simplify the async implementation by replacing ReadPipesToBuffersAsync (a complex WhenAny-based multiplexing loop) with a simple per-stream ReadPipeToBufferAsync helper called twice in parallel. @adamsitnik's follow-up commit (442fb0f2) further improves the buffer return pattern.

Approach: The two-task Task.WhenAll pattern is simpler, has fewer allocations (eliminates per-read WhenAny task), and uses the modern ReadAsync(Memory<byte>) overload. Buffer ownership is cleanly split between the helper (owns on failure) and the caller (owns on success).

Summary: ✅ LGTM. The implementation is correct, simpler than what it replaces, and handles all buffer lifecycle edge cases properly. Verified by multi-model review (Gemini 2.5 Pro, GPT 5.3 Codex) — zero issues found across both models.


Detailed Findings

✅ Buffer lifecycle — Correct ownership transfer

ReadPipeToBufferAsync rents a buffer, returns it in the catch block on failure, and transfers ownership via ArraySegment on success. The callers' finally blocks use IsCompletedSuccessfully guards to return only successfully-completed buffers — preventing double-return when a task faults/cancels (since the helper's catch already returned that buffer). Flagged as correct by 2/2 models.

✅ Exception handling — Exactly-once buffer return

When one task faults and the other succeeds: the faulted task's buffer was returned in ReadPipeToBufferAsync.catch, and the successful task's buffer is returned in the caller's finally. When both fault: both helpers return their own buffers. When both succeed: caller's finally returns both. All paths result in exactly one return per buffer. Verified by 2/2 models.

✅ Cancellation — Clean propagation

CancellationToken flows through to stream.ReadAsync. On cancellation, OperationCanceledException triggers the helper's catch (buffer returned), then propagates through Task.WhenAll to the caller. Verified by 2/2 models.

✅ Performance — Reduced allocations

Eliminates per-read Task.WhenAny allocation from the old multiplexing loop. Each stream reads independently without coordination overhead. Uses ReadAsync(Memory<byte>) instead of legacy ReadAsync(byte[], int, int). Confirmed improvement by 2/2 models.

✅ No new public API surface in this diff

The ref assembly was not modified in these commits — new public APIs were added in earlier commits that are already part of the merge base.

Copilot AI review requested due to automatic review settings April 15, 2026 18:14
@adamsitnik adamsitnik force-pushed the copilot/add-readalltextbytes-async-methods branch from d6c1a3f to c0c1d8b Compare April 15, 2026 18:14
@adamsitnik
Copy link
Copy Markdown
Member

@copilot please invoke the code-review skill and post your analysis/results as a comment on this PR

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 15, 2026

@copilot please invoke the code-review skill and post your analysis/results as a comment on this PR

PR description updated and code review completed. Posting results below.

@adamsitnik adamsitnik enabled auto-merge (squash) April 15, 2026 18:37
@github-actions
Copy link
Copy Markdown
Contributor

🤖 Copilot Code Review — PR #126942

Note

This review was AI/Copilot-generated. Multi-model review: primary analysis by Claude Opus 4.6, cross-checked by GPT-5.3-Codex and Claude Sonnet 4.5.

Holistic Assessment

Motivation: Justified. This implements @MihaZupan's suggestion to replace the Task.WhenAny multiplexing loop with two independent parallel ReadPipeToBufferAsync tasks. The old approach allocated a Task per WhenAny call on every read iteration and was unnecessarily complex for reading two independent pipes. The parallel approach is the natural design.

Approach: Correct and simpler. Each stream gets its own read loop (ReadPipeToBufferAsync), eliminating the interleaved state tracking. Buffer ownership is clean: the helper owns on failure (returns to pool in catch), transfers on success (caller returns in finally). The shared ReadAllBytesIntoRentedArraysAsync helper properly handles the "one succeeds, one fails" case by returning the successful buffer to the pool before rethrowing. The rename from GetSafeFileHandleFromStreamReader to GetSafeHandleFromStreamReader is appropriate since Unix returns SafePipeHandle, not SafeFileHandle.

Summary: ✅ LGTM. The refactoring is correct, simpler than what it replaces, and handles all buffer lifecycle edge cases properly. No blocking issues found. Two minor suggestions below.


Detailed Findings

✅ Buffer ownership — Correct across all code paths

Verified all paths through ReadPipeToBufferAsync and ReadAllBytesIntoRentedArraysAsync:

  • Success: ReadPipeToBufferAsync returns ArraySegment → caller (ReadAllTextAsync/ReadAllBytesAsync) returns buffer in finally. ✓
  • Per-task failure: ReadPipeToBufferAsync returns its buffer in catch, rethrows. ✓
  • Mixed success+failure: ReadAllBytesIntoRentedArraysAsync catches, returns the successful task's buffer via IsCompletedSuccessfully check, rethrows the failure. ✓
  • Both tasks fail: Neither task is IsCompletedSuccessfully, so no pool return at this level (each task already returned its own buffer in its catch). No double-return. ✓
  • Cancellation: Both tasks cancel via shared token → both ReadPipeToBufferAsync catch blocks return buffers → whenAll is Canceled (not Faulted) → whenAll.Exception is null → throw; rethrows OperationCanceledException. ✓

✅ Exception handling — Correct for all scenarios

  • Both tasks fault (non-cancellation): whenAll.Exception.InnerExceptions.Count > 1 → throws AggregateException with both exceptions. Tested by ReadAllAsync_ThrowsAllAvailableExceptions(multiple: true). ✓
  • One task faults: Single inner exception → throw; rethrows the original unwrapped exception. Tested by ReadAllAsync_ThrowsAllAvailableExceptions(multiple: false). ✓
  • Both tasks cancel: Tasks transition to Canceled state (not Faulted), so whenAll.Exception is null → throw; rethrows TaskCanceledException. Tested by ReadAllAsync_ThrowsOperationCanceledOnCancellation. ✓

Note: Claude Sonnet 4.5 flagged a potential cancellation bug where both tasks being cancelled would throw AggregateException instead of OperationCanceledException. This is a false positive: when an async method throws OperationCanceledException, the AsyncTaskMethodBuilder calls TrySetCanceled, putting the Task in Canceled state (not Faulted). Task.WhenAll on all-Canceled tasks produces a Canceled task whose Exception property is null, so whenAll.Exception?.InnerExceptions.Count > 1 evaluates to false and throw; correctly rethrows the TaskCanceledException.

✅ Modern API usage — Good improvement

ReadPipeToBufferAsync uses stream.ReadAsync(buffer.AsMemory(bytesRead), cancellationToken) (the ValueTask<int>-returning overload) instead of the old stream.ReadAsync(byte[], int, int, CancellationToken) (the Task<int>-returning overload). This avoids a Task<int> allocation when the read completes synchronously.

✅ Test coverage — Adequate

The new ReadAllAsync_ThrowsAllAvailableExceptions test covers both the "both fail → AggregateException" and "one fails → single exception" scenarios with all four combinations of (multiple, bytes). The use of reflection to access _standardOutput/_standardError is pragmatic — the comment explains why (accessing the properties would set StreamReadMode, preventing ReadAllBytesAsync from being called). The existing cancellation test is preserved unchanged.

💡 throw whenAll.Exception stack trace — Non-issue, just noting

Process.Multiplexing.cs line 219: throw whenAll.Exception; creates a new stack trace on the AggregateException wrapper. This is fine — the important stack traces (in the inner exceptions) are fully preserved. Since Task.WhenAll constructs the AggregateException internally without ever throwing it, there's no prior stack trace to preserve. ExceptionDispatchInfo.Throw would not help here.

Flagged by GPT-5.3-Codex as ⚠️; assessed as non-issue after deeper analysis.

💡 Test suggestion: mixed-outcome with stderr (follow-up, not blocking)

The current ReadAllAsync_ThrowsAllAvailableExceptions(multiple: false) covers the "stdout disposed, stderr succeeds" case. A variant with stderr disposed (and stdout succeeds) would exercise the mirror path through the IsCompletedSuccessfully checks. Low priority — the code is symmetric and already tested.

Generated by Code Review for issue #126942 ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants