Add more file reading benchmarks#8063
Open
threecgreen wants to merge 1 commit intotokio-rs:masterfrom
Open
Conversation
c21254a to
f087d1b
Compare
Contributor
Author
|
FreeBSD failure looks flaky: |
Member
|
Probably we would need to run a cargo clean, I’ll open a PR. Can you trigger the CI again with an empty commit? |
Add file reading benchmarks that read from actual files of different sizes instead of /dev/zero. Also test concurrent reading with small blocking thread pool to try and test the effect of contention System: AMD Ryzen 9 5950X, 64 GB RAM, Linux 6.19.11 Storage: Samsung SSD 980 NVMe 1TB (ext4) Tokio: 4 worker threads, 2 blocking threads (limited_blocking) benchmark sync blocking io-uring vs sync vs blk ------------------------------------------------ ----- -------- -------- ------- ------ Sequential fs::read() 64KiB 7 us 15 us 38 us +461% +147% 1MiB 43 us 55 us 79 us +84% +43% 16MiB 1.5 ms 1.9 ms 1.4 ms -8% -23% 256MiB 172 ms 177 ms 186 ms +8% +5% Sequential File stream (64K buf, multi_thread) 64KiB 6 us 42 us 47 us +624% +10% 1MiB 48 us 235 us 323 us +571% +38% 16MiB 947 us 5.0 ms 6.0 ms +534% +19% 256MiB 27.8 ms 73.5 ms 96.8 ms +248% +32% Sequential File stream (64K buf, current_thread) 64KiB 6 us 41 us 29 us +352% -28% 1MiB 48 us 253 us 100 us +108% -60% 16MiB 947 us 4.5 ms 1.8 ms +94% -59% 256MiB 27.8 ms 81.8 ms 44.0 ms +58% -46% Concurrent fs::read() (256 MiB files) multi_thread/4 - 216 ms 770 ms - +257% multi_thread/16 - 790 ms 3.2 s - +308% multi_thread/64 - 6.4 s 12.4 s - +95% current_thread/4 - 219 ms 744 ms - +239% current_thread/16 - 785 ms 3.0 s - +277% current_thread/64 - 5.6 s 11.8 s - +109% Concurrent File stream (256 MiB, multi_thread) multi_thread/4 - 151 ms 229 ms - +52% multi_thread/16 - 493 ms 823 ms - +67% multi_thread/64 - 1.5 s 2.8 s - +86% Concurrent File stream (256 MiB, limited_blocking) limited_blocking/4 - 249 ms 229 ms - -8% limited_blocking/16 - 509 ms 822 ms - +61% limited_blocking/64 - 1.2 s 2.9 s - +143%
f087d1b to
bc8e566
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
I wanted to add more representative benchmarks to compare blocking reads against io_uring for the current
AsyncReadandfs::readimplementations.Solution
This PR adds file reading benchmarks that read from actual files of different sizes. The existing fs benchmarks read from
/dev/zero. I also added benchmarks for concurrent reading with a small blocking thread pool to try and test the effect of contention.Results
io_uringperforms well on thecurrent_threadruntime, about 30-60% faster thanspawn_blocking. I was surprisedio_uringperforms worse onmulti_thread, even on concurrent reads.Median results (lower is better).
System: AMD Ryzen 9 5950X, 64 GB RAM, Linux 6.19.11
Storage: Samsung SSD 980 NVMe 1TB (ext4)
Sequential
fs::read()spawn_blockingio-uringspawn_blockingSequential
Filestream (64K buf,multi_thread)spawn_blockingio-uringspawn_blockingSequential
Filestream (64K buf,current_thread)spawn_blockingio-uringspawn_blockingConcurrent
fs::read()(256 MiB files)spawn_blockingio-uringspawn_blockingmulti_thread/4multi_thread/16multi_thread/64current_thread/4current_thread/16current_thread/64Concurrent
Filestream (256 MiB files)spawn_blockingio-uringspawn_blockingmulti_thread/4multi_thread/16multi_thread/64limited_blocking/4limited_blocking/16limited_blocking/64