This report compares performance of the latests versions of Geth and evmone EVMs.
[toc]
In total we have compared 3 EVM implementations: Geth and 2 evmone interpreters (Advanced & Baseline).
The original evmone/Advanced has relatively sophisticated design: during code analysis it also collects information about gas cost and stack requirements of groups of instructions (basic blocks). This makes the later execution more efficient.
In contrast, the evmone/Baseline has "classic" EVM design with the code analysis limited only to the mandatory collecting of valid JUMPDEST locations.
The latest stable releases of the projects have been used:
- geth 1.10.12 released 2021-11-08, built with go 1.16.2
- evmone 0.8.2 released 2021-08-26, built with clang 13.0.1
The performance of EVM implementations is compared with the evmone's benchmark suite. The suite is a small set of computation heavy EVM bytecodes (e.g. cryptographic hash functions) and some micro-benchmarks (e.g. stress testing single MSTORE instruction). The suite does not necessarily represent Mainnet workloads.
To benchmark evmone interpreters, the internal tool evmone-bench based on the benchmark library has been used. It outputs results in a JSON format. To benchmark geth, the evm run --bench tool has been used. The loading of the benchmark suite and execution runs have been conducted via the evm-bench.py Python script. Comparison and statistical analysis have been in the end processed with benchstat.
Each benchmark sample contains code analysis and execution times combined. These timings may be separated with some additional tuning of the tooling in the future.
Hardware: Intel Haswell CPU i7-4790K with turbo frequency 4.4 GHz, Linux x86-64 (Ubuntu 20.04)
The evmone/Baseline has been used as the baseline performance.
name Baseline time Geth time delta
main/blake2b_huff/empty 22.1µs ± 0% 111.2µs ± 0% +402.22% (p=0.000 n=9+10)
main/blake2b_huff/2805nulls 360µs ± 0% 2145µs ± 1% +495.70% (p=0.000 n=9+10)
main/blake2b_huff/5610nulls 697µs ± 0% 4163µs ± 0% +496.90% (p=0.000 n=9+10)
main/blake2b_huff/8415nulls 1.02ms ± 1% 6.07ms ± 0% +495.14% (p=0.000 n=9+10)
main/blake2b_shifts/2805nulls 3.23ms ± 0% 16.77ms ± 0% +419.30% (p=0.000 n=10+8)
main/blake2b_shifts/5610nulls 6.63ms ± 0% 33.43ms ± 1% +403.85% (p=0.000 n=9+9)
main/blake2b_shifts/8415nulls 10.0ms ± 0% 49.7ms ± 0% +399.47% (p=0.000 n=9+8)
main/sha1_divs/empty 55.1µs ± 0% 222.1µs ± 0% +302.91% (p=0.000 n=10+9)
main/sha1_divs/1351 1.12ms ± 0% 4.49ms ± 0% +301.83% (p=0.000 n=9+9)
main/sha1_divs/2737 2.19ms ± 1% 8.74ms ± 0% +298.66% (p=0.000 n=10+9)
main/sha1_divs/5311 4.25ms ± 0% 17.10ms ± 0% +302.58% (p=0.000 n=10+9)
main/sha1_shifts/empty 33.4µs ± 0% 176.7µs ± 1% +429.77% (p=0.000 n=9+10)
main/sha1_shifts/1351 691µs ± 3% 3640µs ± 1% +427.11% (p=0.000 n=10+10)
main/sha1_shifts/2737 1.34ms ± 0% 7.07ms ± 1% +429.74% (p=0.000 n=9+10)
main/sha1_shifts/5311 2.61ms ± 1% 13.85ms ± 1% +430.30% (p=0.000 n=10+10)
main/weierstrudel/0 171µs ± 0% 494µs ± 0% +188.37% (p=0.000 n=10+10)
main/weierstrudel/1 364µs ± 1% 1053µs ± 0% +189.20% (p=0.000 n=10+10)
main/weierstrudel/3 565µs ± 0% 1653µs ± 0% +192.78% (p=0.000 n=8+10)
main/weierstrudel/9 1.16ms ± 0% 3.47ms ± 0% +197.77% (p=0.000 n=9+10)
main/weierstrudel/14 1.67ms ± 0% 4.95ms ± 0% +196.94% (p=0.000 n=10+10)
micro/memory_grow_mload/nogrow 71.9µs ± 0% 295.6µs ± 1% +310.99% (p=0.000 n=9+10)
micro/memory_grow_mload/by1 73.7µs ± 0% 302.8µs ± 0% +310.73% (p=0.000 n=10+10)
micro/memory_grow_mload/by16 83.7µs ± 1% 369.1µs ± 0% +341.01% (p=0.000 n=10+10)
micro/memory_grow_mload/by32 90.7µs ± 0% 449.3µs ± 1% +395.55% (p=0.000 n=8+10)
micro/memory_grow_mstore/nogrow 75.6µs ± 0% 530.1µs ± 0% +601.47% (p=0.000 n=9+10)
micro/memory_grow_mstore/by1 77.4µs ± 0% 542.2µs ± 0% +600.57% (p=0.000 n=10+10)
micro/memory_grow_mstore/by16 87.1µs ± 0% 627.0µs ± 1% +619.89% (p=0.000 n=8+10)
micro/memory_grow_mstore/by32 95.1µs ± 0% 695.9µs ± 1% +631.54% (p=0.000 n=8+10)
[Geo mean] 447µs 1874µs +319.26%
name Baseline gas/s Geth gas/s delta
main/blake2b_huff/empty 632M ± 0% 126M ± 0% -80.09% (p=0.000 n=9+10)
main/blake2b_huff/2805nulls 853M ± 0% 143M ± 1% -83.21% (p=0.000 n=9+10)
main/blake2b_huff/5610nulls 860M ± 0% 144M ± 0% -83.25% (p=0.000 n=9+10)
main/blake2b_huff/8415nulls 862M ± 1% 145M ± 1% -83.20% (p=0.000 n=9+10)
main/blake2b_shifts/2805nulls 1.06G ± 0% 0.20G ± 0% -80.74% (p=0.000 n=10+8)
main/blake2b_shifts/5610nulls 1.12G ± 0% 0.22G ± 1% -80.15% (p=0.000 n=9+9)
main/blake2b_shifts/8415nulls 1.21G ± 0% 0.24G ± 0% -79.98% (p=0.000 n=9+8)
main/sha1_divs/empty 661M ± 0% 164M ± 0% -75.18% (p=0.000 n=10+9)
main/sha1_divs/1351 704M ± 0% 175M ± 0% -75.11% (p=0.000 n=9+9)
main/sha1_divs/2737 701M ± 1% 176M ± 0% -74.92% (p=0.000 n=10+9)
main/sha1_divs/5311 706M ± 0% 175M ± 0% -75.16% (p=0.000 n=10+9)
main/sha1_shifts/empty 1.06G ± 0% 0.20G ± 1% -81.12% (p=0.000 n=9+10)
main/sha1_shifts/1351 1.10G ± 3% 0.21G ± 1% -81.03% (p=0.000 n=10+10)
main/sha1_shifts/2737 1.11G ± 0% 0.21G ± 1% -81.12% (p=0.000 n=9+10)
main/sha1_shifts/5311 1.11G ± 1% 0.21G ± 1% -81.14% (p=0.000 n=10+10)
main/weierstrudel/0 277M ± 0% 96M ± 0% -65.32% (p=0.000 n=10+10)
main/weierstrudel/1 358M ± 1% 124M ± 0% -65.42% (p=0.000 n=10+10)
main/weierstrudel/3 358M ± 0% 122M ± 0% -65.84% (p=0.000 n=8+10)
main/weierstrudel/9 359M ± 0% 120M ± 0% -66.42% (p=0.000 n=9+10)
main/weierstrudel/14 359M ± 0% 121M ± 0% -66.32% (p=0.000 n=10+10)
micro/memory_grow_mload/nogrow 797M ± 0% 194M ± 1% -75.67% (p=0.000 n=9+10)
micro/memory_grow_mload/by1 784M ± 0% 191M ± 0% -75.65% (p=0.000 n=10+10)
micro/memory_grow_mload/by16 857M ± 1% 194M ± 0% -77.32% (p=0.000 n=10+10)
micro/memory_grow_mload/by32 1.13G ± 0% 0.23G ± 1% -79.82% (p=0.000 n=8+10)
micro/memory_grow_mstore/nogrow 813M ± 0% 116M ± 0% -85.74% (p=0.000 n=9+10)
micro/memory_grow_mstore/by1 799M ± 0% 114M ± 0% -85.73% (p=0.000 n=10+10)
micro/memory_grow_mstore/by16 870M ± 0% 121M ± 1% -86.11% (p=0.000 n=8+10)
micro/memory_grow_mstore/by32 1.12G ± 0% 0.15G ± 1% -86.33% (p=0.000 n=8+10)
[Geo mean] 784M 161M -79.53%
name Baseline time Advanced time delta
main/blake2b_huff/empty 22.1µs ± 0% 67.0µs ± 0% +202.72% (p=0.000 n=9+9)
main/blake2b_huff/2805nulls 360µs ± 0% 349µs ± 0% -3.19% (p=0.000 n=9+10)
main/blake2b_huff/5610nulls 697µs ± 0% 629µs ± 0% -9.76% (p=0.000 n=9+9)
main/blake2b_huff/8415nulls 1.02ms ± 1% 0.90ms ± 0% -12.00% (p=0.000 n=9+10)
main/blake2b_shifts/2805nulls 3.23ms ± 0% 3.54ms ± 0% +9.47% (p=0.000 n=10+10)
main/blake2b_shifts/5610nulls 6.63ms ± 0% 7.05ms ± 0% +6.23% (p=0.000 n=9+9)
main/blake2b_shifts/8415nulls 10.0ms ± 0% 10.6ms ± 0% +5.93% (p=0.000 n=9+9)
main/sha1_divs/empty 55.1µs ± 0% 57.6µs ± 0% +4.40% (p=0.000 n=10+9)
main/sha1_divs/1351 1.12ms ± 0% 1.06ms ± 0% -5.23% (p=0.000 n=9+9)
main/sha1_divs/2737 2.19ms ± 1% 2.06ms ± 0% -6.08% (p=0.000 n=10+7)
main/sha1_divs/5311 4.25ms ± 0% 4.01ms ± 0% -5.65% (p=0.000 n=10+10)
main/sha1_shifts/empty 33.4µs ± 0% 35.5µs ± 0% +6.43% (p=0.000 n=9+9)
main/sha1_shifts/1351 691µs ± 3% 616µs ± 0% -10.78% (p=0.000 n=10+9)
main/sha1_shifts/2737 1.34ms ± 0% 1.20ms ± 0% -10.42% (p=0.000 n=9+9)
main/sha1_shifts/5311 2.61ms ± 1% 2.33ms ± 0% -10.93% (p=0.000 n=10+9)
main/weierstrudel/0 171µs ± 0% 231µs ± 0% +34.69% (p=0.000 n=10+9)
main/weierstrudel/1 364µs ± 1% 429µs ± 0% +17.71% (p=0.000 n=10+10)
main/weierstrudel/3 565µs ± 0% 631µs ± 0% +11.82% (p=0.000 n=8+9)
main/weierstrudel/9 1.16ms ± 0% 1.24ms ± 0% +6.11% (p=0.000 n=9+7)
main/weierstrudel/14 1.67ms ± 0% 1.74ms ± 0% +4.39% (p=0.000 n=10+9)
micro/memory_grow_mload/nogrow 71.9µs ± 0% 148.2µs ± 1% +106.10% (p=0.000 n=9+10)
micro/memory_grow_mload/by1 73.7µs ± 0% 149.6µs ± 0% +102.95% (p=0.000 n=10+9)
micro/memory_grow_mload/by16 83.7µs ± 1% 162.4µs ± 0% +94.07% (p=0.000 n=10+9)
micro/memory_grow_mload/by32 90.7µs ± 0% 172.6µs ± 1% +90.33% (p=0.000 n=8+10)
micro/memory_grow_mstore/nogrow 75.6µs ± 0% 156.0µs ± 0% +106.37% (p=0.000 n=9+9)
micro/memory_grow_mstore/by1 77.4µs ± 0% 157.9µs ± 0% +103.98% (p=0.000 n=10+9)
micro/memory_grow_mstore/by16 87.1µs ± 0% 170.9µs ± 0% +96.21% (p=0.000 n=8+9)
micro/memory_grow_mstore/by32 95.1µs ± 0% 181.5µs ± 1% +90.75% (p=0.000 n=8+10)
[Geo mean] 447µs 511µs +14.27%
name Baseline gas/s Advanced gas/s delta
main/blake2b_huff/empty 632M ± 0% 209M ± 0% -66.97% (p=0.000 n=9+9)
main/blake2b_huff/2805nulls 853M ± 0% 881M ± 0% +3.29% (p=0.000 n=9+10)
main/blake2b_huff/5610nulls 860M ± 0% 954M ± 0% +10.82% (p=0.000 n=9+9)
main/blake2b_huff/8415nulls 862M ± 1% 980M ± 0% +13.63% (p=0.000 n=9+10)
main/blake2b_shifts/2805nulls 1.06G ± 0% 0.97G ± 0% -8.65% (p=0.000 n=10+10)
main/blake2b_shifts/5610nulls 1.12G ± 0% 1.05G ± 0% -5.87% (p=0.000 n=9+9)
main/blake2b_shifts/8415nulls 1.21G ± 0% 1.14G ± 0% -5.60% (p=0.000 n=9+9)
main/sha1_divs/empty 661M ± 0% 633M ± 0% -4.22% (p=0.000 n=10+9)
main/sha1_divs/1351 704M ± 0% 743M ± 0% +5.52% (p=0.000 n=9+9)
main/sha1_divs/2737 701M ± 1% 746M ± 0% +6.47% (p=0.000 n=10+7)
main/sha1_divs/5311 706M ± 0% 749M ± 0% +5.99% (p=0.000 n=10+10)
main/sha1_shifts/empty 1.06G ± 0% 0.99G ± 0% -6.04% (p=0.000 n=9+9)
main/sha1_shifts/1351 1.10G ± 3% 1.23G ± 0% +12.05% (p=0.000 n=10+9)
main/sha1_shifts/2737 1.11G ± 0% 1.24G ± 0% +11.64% (p=0.000 n=9+9)
main/sha1_shifts/5311 1.11G ± 1% 1.25G ± 0% +12.27% (p=0.000 n=10+9)
main/weierstrudel/0 277M ± 0% 206M ± 0% -25.76% (p=0.000 n=10+9)
main/weierstrudel/1 358M ± 1% 305M ± 0% -15.05% (p=0.000 n=10+10)
main/weierstrudel/3 358M ± 0% 320M ± 0% -10.57% (p=0.000 n=8+9)
main/weierstrudel/9 359M ± 0% 338M ± 0% -5.75% (p=0.000 n=9+7)
main/weierstrudel/14 359M ± 0% 344M ± 0% -4.20% (p=0.000 n=10+9)
micro/memory_grow_mload/nogrow 797M ± 0% 387M ± 1% -51.48% (p=0.000 n=9+10)
micro/memory_grow_mload/by1 784M ± 0% 386M ± 0% -50.73% (p=0.000 n=10+9)
micro/memory_grow_mload/by16 857M ± 1% 441M ± 0% -48.47% (p=0.000 n=10+9)
micro/memory_grow_mload/by32 1.13G ± 0% 0.59G ± 1% -47.46% (p=0.000 n=8+10)
micro/memory_grow_mstore/nogrow 813M ± 0% 394M ± 0% -51.54% (p=0.000 n=9+9)
micro/memory_grow_mstore/by1 799M ± 0% 392M ± 0% -50.98% (p=0.000 n=10+9)
micro/memory_grow_mstore/by16 870M ± 0% 444M ± 0% -49.03% (p=0.000 n=8+9)
micro/memory_grow_mstore/by32 1.12G ± 0% 0.59G ± 1% -47.57% (p=0.000 n=8+10)
[Geo mean] 784M 589M -24.89%