Repository navigation
perf: exact custom %16.8f float formatter for the writer (~2.4x, byte-identical) - #40
Merged
Merged
Conversation
jameskermode
force-pushed
the
perf/write-float-fmt
branch
from
June 10, 2026 06:53
bc2d961 to
73ea590
Compare
Profiling (#39) showed the write path is bound by formatting the per-atom floats, not I/O: buffering alone was 1.14x while a custom formatter promised ~2x. This adds that formatter for the default "%16.8f", proven byte-identical to printf. A finite double is |v| = m*2^e exactly, and 10^8 = 2^8*5^8, so v*10^8 = m*390625*2^(e+8) is an exact rational. We round it to nearest (ties to even, matching printf's default) with integer-only arithmetic -- a 128-bit shift + half-bit test, no floating-point error -- then format the digits manually (no snprintf). It falls back to snprintf for non-finite / |v|>=1e15, for any custom format_dict (#22), and on compilers without __int128 (MSVC), so output is byte-identical everywhere. new libextxyz/fast_format.{c,h}; wired into the #39 buffered per-atom loop only when the float format is the default (fmt_f == NULL). Int/bool/string unchanged. Result: 200k-atom write 210 -> 84 ms (~2.4x over #39, ~3x over the original fprintf writer); now ~6x faster than ASE's writer and ~3x faster than extxyz-ng. Validation: libextxyz/test_fmt_float.c checks fmt_default_f16_8 == snprintf over 60M+ doubles incl. dense sweeps across round-half-even ties, subnormals, +/-0.0, and the fallback boundary (wired as a `meson test`); the naive round-half-up formatter fails it ~1 in 5M, the exact one passes 0/66M. tests/test_write_float_ format.py validates byte-identity end-to-end through the writer (default fast path == explicit %16.8f snprintf path) over 300k random + crafted-edge values. #39 golden/round-trip + #22 precision stay green; leaks 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the read benchmark for writing: bench_write.py now emits write_results.csv (same column shape as results.csv) and plot_bench.py auto-detects read vs write from the columns and renders benchmarks/write_speedup.png with the same two-panel structure (time vs size, speedup vs ASE built-in). README gains the matching write table + figure. The C writer is ~5-6x faster than ASE's writer across sizes.
jameskermode
force-pushed
the
perf/write-float-fmt
branch
from
June 10, 2026 08:35
73ea590 to
a092267
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #39 (base
perf/write-buffer). Review/merge #39 first; this retargets tomasterafter.What
#39's profiling showed the write path is bound by formatting the per-atom floats, not I/O (buffering alone was 1.14×; a custom formatter promised ~2×). This adds that formatter for the default
"%16.8f", proven byte-identical to printf.Exact, zero floating-point error
A finite double is
|v| = m·2^eexactly, and10^8 = 2^8·5^8, sois an exact rational. We round it to nearest (ties to even, matching printf's default mode) with integer-only arithmetic — a 128-bit shift + half-bit test, no FP error — then emit the digits manually (no
snprintf). Falls back tosnprintffor non-finite /|v|≥1e15, for any customformat_dict(#22), and on compilers without__int128(MSVC) — so output is byte-identical on every platform. Newlibextxyz/fast_format.{c,h}, wired into #39's buffered loop only when the float format is the default (fmt_f == NULL); int/bool/string unchanged.Result (200k-atom frame)
fprintfsnprintfAlso ~3× faster than extxyz-ng (7.1 vs 24 ms on the 20k frame).
pyPython/ASE ≈ 510 ms.Validation (the point of the task)
libextxyz/test_fmt_float.c, run bymeson test):fmt_default_f16_8(v) == snprintf("%16.8f", v)over 66M+ doubles — uniform random across magnitudes, dense sweeps straddling 8th-decimal round-half-to-even ties (k + 0.5e-8 ± ulps), subnormals,±0.0, the fallback boundary, and the known12.031073044999999case. TDD: the naive round-half-up formatter fails this ~1-in-5M (demonstrated); the exact one is 0/66M.tests/test_write_float_format.py): default fast path vs explicitformat_dict={'R':'%16.8f'}(snprintf) byte streams must be identical, over 300k random + crafted-edge values through the real writer.test_write_precision.py(Precision specification when writing to extxyz #22) stay green; core (52) + ase-extxyz (38) green;leaks --atExit= 0.🤖 Generated with Claude Code