Skip to content

perf: exact custom %16.8f float formatter for the writer (~2.4x, byte-identical) - #40

Merged
jameskermode merged 2 commits into
masterfrom
perf/write-float-fmt
Jun 10, 2026
Merged

jameskermode merged 2 commits into
masterfrom
perf/write-float-fmt

Conversation

@jameskermode

Copy link
Copy Markdown
Member

Stacked on #39 (base perf/write-buffer). Review/merge #39 first; this retargets to master after.

What

#39's profiling showed the write path is bound by formatting the per-atom floats, not I/O (buffering alone was 1.14×; a custom formatter promised ~2×). This adds that formatter for the default "%16.8f", proven byte-identical to printf.

Exact, zero floating-point error

A finite double is |v| = m·2^e exactly, and 10^8 = 2^8·5^8, so

v·10^8 = m·390625·2^(e+8)

is an exact rational. We round it to nearest (ties to even, matching printf's default mode) with integer-only arithmetic — a 128-bit shift + half-bit test, no FP error — then emit the digits manually (no snprintf). Falls back to snprintf for non-finite / |v|≥1e15, for any custom format_dict (#22), and on compilers without __int128 (MSVC) — so output is byte-identical on every platform. New libextxyz/fast_format.{c,h}, wired into #39's buffered loop only when the float format is the default (fmt_f == NULL); int/bool/string unchanged.

Result (200k-atom frame)

writer time vs ASE
original fprintf 266 ms 2.0×
#39 buffered snprintf 210 ms 2.5×
+ this formatter 84 ms 6.0×

Also ~3× faster than extxyz-ng (7.1 vs 24 ms on the 20k frame). pyPython/ASE ≈ 510 ms.

Validation (the point of the task)

  • C exhaustive (libextxyz/test_fmt_float.c, run by meson test): fmt_default_f16_8(v) == snprintf("%16.8f", v) over 66M+ doubles — uniform random across magnitudes, dense sweeps straddling 8th-decimal round-half-to-even ties (k + 0.5e-8 ± ulps), subnormals, ±0.0, the fallback boundary, and the known 12.031073044999999 case. TDD: the naive round-half-up formatter fails this ~1-in-5M (demonstrated); the exact one is 0/66M.
  • Python end-to-end (tests/test_write_float_format.py): default fast path vs explicit format_dict={'R':'%16.8f'} (snprintf) byte streams must be identical, over 300k random + crafted-edge values through the real writer.
  • perf: profile the write path; buffer per-atom output (byte-identical) #39 golden/round-trip + test_write_precision.py (Precision specification when writing to extxyz #22) stay green; core (52) + ase-extxyz (38) green; leaks --atExit = 0.

🤖 Generated with Claude Code

@jameskermode
jameskermode force-pushed the perf/write-float-fmt branch from bc2d961 to 73ea590 Compare June 10, 2026 06:53
jameskermode and others added 2 commits June 10, 2026 09:30
Profiling (#39) showed the write path is bound by formatting the per-atom floats,
not I/O: buffering alone was 1.14x while a custom formatter promised ~2x. This
adds that formatter for the default "%16.8f", proven byte-identical to printf.

A finite double is |v| = m*2^e exactly, and 10^8 = 2^8*5^8, so
  v*10^8 = m*390625*2^(e+8)
is an exact rational. We round it to nearest (ties to even, matching printf's
default) with integer-only arithmetic -- a 128-bit shift + half-bit test, no
floating-point error -- then format the digits manually (no snprintf). It falls
back to snprintf for non-finite / |v|>=1e15, for any custom format_dict (#22),
and on compilers without __int128 (MSVC), so output is byte-identical everywhere.

new libextxyz/fast_format.{c,h}; wired into the #39 buffered per-atom loop only
when the float format is the default (fmt_f == NULL). Int/bool/string unchanged.

Result: 200k-atom write 210 -> 84 ms (~2.4x over #39, ~3x over the original
fprintf writer); now ~6x faster than ASE's writer and ~3x faster than extxyz-ng.

Validation: libextxyz/test_fmt_float.c checks fmt_default_f16_8 == snprintf over
60M+ doubles incl. dense sweeps across round-half-even ties, subnormals, +/-0.0,
and the fallback boundary (wired as a `meson test`); the naive round-half-up
formatter fails it ~1 in 5M, the exact one passes 0/66M. tests/test_write_float_
format.py validates byte-identity end-to-end through the writer (default fast path
== explicit %16.8f snprintf path) over 300k random + crafted-edge values. #39
golden/round-trip + #22 precision stay green; leaks 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the read benchmark for writing: bench_write.py now emits write_results.csv
(same column shape as results.csv) and plot_bench.py auto-detects read vs write
from the columns and renders benchmarks/write_speedup.png with the same two-panel
structure (time vs size, speedup vs ASE built-in). README gains the matching write
table + figure. The C writer is ~5-6x faster than ASE's writer across sizes.
@jameskermode
jameskermode force-pushed the perf/write-float-fmt branch from 73ea590 to a092267 Compare June 10, 2026 08:35
@jameskermode
jameskermode changed the base branch from perf/write-buffer to master June 10, 2026 08:35
@jameskermode
jameskermode merged commit 7e3c010 into master Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant