Skip to content

Preserve extended precision in negative-binomial probabilities - #156

Merged
jmh530 merged 5 commits into
libmir:masterfrom
jmh530:codex/fp-negative-binomial-review
Oct 13, 2026
Merged

jmh530 merged 5 commits into
libmir:masterfrom
jmh530:codex/fp-negative-binomial-review

Conversation

@jmh530

@jmh530 jmh530 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Fix precision loss and count handling in the extended-precision negative-binomial PMF, and make it available through negativeBinomialPMF alongside the built-in floating-point implementation.

Previously, forming 1-p through double could turn an Fp probability just below one into exactly one, incorrectly producing zero mass. Coefficient construction also narrowed success counts to uint, could overflow when combining counts, and failed to compile for Fp!256 because intermediate arithmetic mixed precision widths.

Changes

  • Validate probabilities and calculate their complements directly from the Fp representation.
  • Construct coefficients without narrowing or overflowing counts. Retain exact ulong arithmetic for small coefficients; use same-width products and integer significand division for larger coefficients, avoiding Fp division's conversion through hardware real.
  • Replace linear repeated multiplication in fp_powi with exponentiation by squaring. Preserve the exponent-one identity directly to avoid premature underflow near the minimum exponent.
  • Add an Fp overload of negativeBinomialPMF, retaining fp_negativeBinomialPMF as a compatibility wrapper without deprecation.
  • Document rounding and computational limits, and add regressions across Fp!64, Fp!128, and Fp!256.

The built-in float, double, and real PMF implementation is unchanged. Extended-precision results still involve rounding, and coefficient construction remains linear in the smaller count.

Validation

  • Full Windows LDC suite: 58 modules passed.
  • Optimized focused tests passed with DMD and LDC on Windows and Linux.
  • Optimized downstream binomial and Poisson tests passed with both Windows compilers.
  • DMD documentation build passed.
  • Compared 180 probability cases against exact rational references.
  • Investigated 2,983 exact power-of-two boundary cases to distinguish the exponent-one regression from existing Fp multiplication limitations.

Paired local benchmarks showed comparable or improved LDC performance in the tested cases. The medium DMD case was approximately 6% slower; large-exponent cases improved substantially on both compilers.

The existing exponent-boundary limitation in Mir's general Fp multiplication is recorded separately for follow-up.

AI-Assisted-By: Codex (GPT-6 Astra Medium)

Validate Fp probabilities from their represented value and form 1-p with
integer significand arithmetic. Avoid rounding through double, which could
turn a probability just below one into one and discard a positive mass.

Build coefficients at the requested Fp width without narrowing success
counts to uint or overflowing k+r-1. Use exact ulong coefficients for small
counts and symmetric numerator/denominator products for larger counts.
Divide their integer significands with guard bits instead of using Fp
division, which currently converts through hardware real.

Replace linear fp_powi multiplication with exponentiation by squaring so
large exponents remain practical. Avoid an unused final square and retain
the zero-exponent identity. Document rounded arithmetic and the remaining
linear cost in the smaller coefficient count.

Add Fp64, Fp128, and Fp256 regressions for cancellation near one, complement
rounding, endpoint probabilities, full-width counts, and coefficients that
require more precision than hardware floating point. Keep the existing
public function names and built-in floating-point implementation unchanged.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Expose the corrected Fp implementation through negativeBinomialPMF, using
disjoint template constraints to select it from the probability type.
Retain fp_negativeBinomialPMF as a forwarding compatibility wrapper so
existing callers remain supported without a deprecation warning.

Add a documented example that checks the Fp return type and probability,
and verifies that a double argument still selects the built-in overload.
Document the compatibility name and preserve safe, pure, nothrow, and nogc
attributes on both entry points.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Return x directly when fp_powi receives exponent one. Multiplying by
one can underflow in Mir's Fp implementation because it adds raw
exponents before normalization, discarding a representable result.

Add representation-level regressions for positive and negative values
near the minimum exponent across Fp64, Fp128, and Fp256. Check signs,
coefficients, and exponents without conversion to hardware floats.

Cover negativeBinomialPMF(0, 1, p) at the exponent floor, where the mass
must equal p. Retain exponentiation by squaring for other powers;
the broader Fp multiplication limitation remains a separate issue.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Import Fp inside the documented unittest. CircleCI extracts documentation
examples into standalone modules, where the source module's private
imports are unavailable, causing the example to fail compilation.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Replace the precision regression's conditional absolute-error calculation
with lower and upper bounds on the computed significand. Preserve the
existing 16-unit tolerance and exponent check.

Avoid an uncovered test branch whose execution depends on rounding
direction. Production arithmetic is unchanged.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@jmh530
jmh530 merged commit e8b063c into libmir:master Oct 13, 2026
24 checks passed
@jmh530
jmh530 deleted the codex/fp-negative-binomial-review branch October 13, 2026 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant