ref(download): Use download service for objectstore requests - #1928
Draft
jjbayer wants to merge 9 commits into
Draft
ref(download): Use download service for objectstore requests#1928jjbayer wants to merge 9 commits into
jjbayer wants to merge 9 commits into
Conversation
This reverts commit 3de2a6a.
Dav1dde
added a commit
that referenced
this pull request
May 6, 2026
Ideally we'd be using the download service directly (as noted in the comment), there is an attempt in #1928 but requires a bit more thought and work. Right now we do see some download failures which can be mitigated with retries. So while it's not great and makes the `GenericErrorHandler` public with a helper fn, it's what we got.
Member
Author
|
Closed the wrong PR. |
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
Comment on lines
+31
to
+36
| fetch_file( | ||
| download_svc, | ||
| RemoteFile::Http(http_remote_file), | ||
| &mut temp_file, | ||
| ) | ||
| .await?; |
There was a problem hiding this comment.
Remote minidump attachment decompression has no effective CPU bound despite 15GiB output cap
An attacker-controlled storage URL can serve a tiny zstd/gzip/zlib/zip/CAB payload that fetch_file synchronously inflates into a temporary file after the download timeout has ended. Although output is capped at the default 15GiB, the decompression and subsequent minidump processing have no decompression-specific CPU budget; the outer request timeout cannot preempt this synchronous block, allowing substantial CPU and disk exhaustion on the shared worker pool.
Evidence
/symbolicate-anyaccepts attacker-controlledstorage_urlvalues and constructsAttachmentFile::Remote;process_minidumppasses them todownload_attachment(crates/symbolicator/src/endpoints/symbolicate_any.rs:42-72,crates/symbolicator-native/src/symbolication/process_minidump.rs:602-634).download_attachmentcallsfetch_file(crates/symbolicator-native/src/symbolication/attachments.rs:23-36), which invokesDownloadService::downloadand then synchronously callsmaybe_decompress_fileafter the download timeout (crates/symbolicator-service/src/download/fetch_file.rs:18-37).maybe_decompress_fileusestake(max_uncompressed_size + 1)but still copies up to the configured limit into a temporary file before checking its size; the shipped default is 15GiB and no CPU or decompression-time budget exists (crates/symbolicator-service/src/download/compression.rs:31-145,crates/symbolicator-service/src/config.rs:553-556,670-674).- The resulting file is mapped and parsed before minidump stackwalking (
crates/symbolicator-native/src/symbolication/process_minidump.rs:628-634,491); processing runs on the CPU pool and the default request admission limit allows 200 concurrent requests (crates/symbolicator/src/service.rs:242-259,428-518).
Identified by Warden · wrdn-dos-review · MZN-DLX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow up to #1924: Using the existing service would have prevented the bug in the first place.