Skip to content

Preserve binomial probabilities, tails, and quantiles at numerical boundaries - #154

Merged
jmh530 merged 8 commits into
libmir:masterfrom
jmh530:codex/binomial-numerical-review
Oct 12, 2026
Merged

jmh530 merged 8 commits into
libmir:masterfrom
jmh530:codex/binomial-numerical-review

Conversation

@jmh530

@jmh530 jmh530 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Binomial calculations could lose valid probabilities through coefficient overflow or intermediate underflow, erase small tails through subtraction, and select the wrong quantile near a CDF boundary. Approximation algorithms also mishandled degenerate distributions and could return quantiles outside the binomial support.

This PR addresses those cases while retaining inexpensive paths for ordinary inputs.

  • Keep the coefficient-and-powers PMF calculation when its intermediates retain precision; otherwise use a stable logarithmic calculation. Preserve full-width counts and reduce cancellation in large-count log masses.
  • Calculate the appropriate tail directly, without reflecting the probability through a rounded 1-p. Use adjacent-mass recurrence for small distributions and stop larger sums when the remaining contribution is negligible.
  • Handle inverse-CDF endpoints explicitly and bound the search. Compare upper-half queries using the CCDF, with bounded exact integer arithmetic for close comparisons—including subnormal probability boundaries.
  • Give approximation algorithms exact endpoint behavior and constrain their quantiles to [0, n].

Regression tests cover overflow, underflow, tiny tails, exact CDF ties, adjacent floating-point queries, and approximation boundaries. Expected results use independent high-precision calculations or exact integer binomial sums.

Validation

  • Full Windows LDC suite: 58 modules passed.
  • Focused optimized tests passed with DMD and LDC on Windows and Linux.
  • DMD documentation check passed.
  • Expanded exact-reference quantile grids and paired performance comparisons completed during development.

Scope

Exact quantile comparisons use a fixed arithmetic budget; unsupported cases retain floating-point tail accuracy. Broader large-count CDF/CCDF algorithms and improvements to interior approximations remain separate follow-ups.

…flow

Fall back to logarithmic PMF evaluation when the binomial coefficient
overflows or intermediate powers lose precision. Limit the direct
calculation to modest trial counts to avoid amplifying rounding in 1-p.

Use a centered log-mass calculation for larger counts, with short
logarithmic products near the endpoints. Preserve full size_t counts
instead of narrowing the success count to uint.

Keep the ordinary direct path and calculate its coefficient before
the powers to reduce overhead without weakening the safety checks.

Add regressions for central probabilities, subnormal intermediates,
degenerate endpoints, full-width counts, and helper transitions using
independent high-precision references.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Share direct CDF and CCDF evaluation, selecting the tail by the mean
without reflecting the input probability. Use log1p and expm1 for
zero-count tails, handle degenerate probabilities explicitly, and
avoid overflowing n + 1.

For n <= 64, derive adjacent masses from one PMF value and accumulate
float inputs in double. For larger counts, retain PMF summation and
bound the remaining tail before stopping.

Add regressions for tiny probabilities, maximum trial counts,
recurrence boundaries, and independent high-precision references.
Leave the general huge-count fallback for separate work.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Handle degenerate distributions before choosing an approximation and
bound the direct inverse search to [0, n]. Compare upper probabilities
through the CCDF and re-evaluate tails after cancellation.

Use adjacent-mass recurrence for short distributions and double
working precision for float inputs, including the initial guess.

Resolve close comparisons with exact integer probability numerators
when their binary denominators fit a fixed arithmetic budget. Retain
floating-point tail evaluation outside that budget and document its
accuracy limits.

Add regressions for exact ties, neighboring probabilities, tiny upper
tails, underflowed recurrence seeds, and arithmetic-budget boundaries.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Handle zero trials and degenerate probabilities before evaluating
normal or Poisson approximations. Return exact CDF and CCDF values
at the upper support boundary and align inverse endpoint conventions
with the direct algorithm.

Cap Poisson quantiles at the trial count. Bound normal quantiles
before unsigned conversion so negative rounding near the lower
cutoff cannot wrap into a large count.

Retain the existing interior approximation formulas and compile
the new endpoint guards out of the direct algorithm wrappers.

Document the endpoint behavior and add regressions across floating
types, approximation choices, Poisson sub-algorithms, and normal
lower-cutoff rounding and underflow.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Add a minimum scale to the close-comparison screen so subnormal
rounding does not bypass the existing exact quantile comparison.
Preserve the quantile inequality and fixed arithmetic budget.

Explain the absolute allowance and add regressions for adjacent
double probabilities around a lower-tail CDF step, using exact
integer binomial sums to establish the expected results.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Add lower-tail boundary tests for queries beyond the exact integer
arithmetic budget, using independent high-precision references.

Exercise downward searches from the upper support endpoint with
both CDF and CCDF comparisons, using exact rational reference
quantiles. Check positive queries below the normal approximation's
lower cutoff.

Retain defensive guards without changing production behavior or
excluding them from coverage.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Check the support endpoint before evaluating the remaining-tail
bound. Preserve explicit unsigned-index overflow protection and
avoid endpoint branches already bypassed by the zero-ratio check.

Remove the probability-sum clamp from the tail calculation, which
sums away from the mean rather than accumulating nearly all mass.
Retain the existing probability-bounds regression tests.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Remove the initial-guess clamp because every source of the guess
already returns a value within the binomial support.

Retain the normal approximation's upper guard before integer
conversion. Add float, double, and real regressions for queries
just below the upper cutoff, where the continuous quantile can
round back to the endpoint.

Allow either the floored interior count or the rounded endpoint
to account for differences in floating-point evaluation.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@jmh530
jmh530 merged commit 65b07e6 into libmir:master Oct 12, 2026
24 checks passed
@jmh530
jmh530 deleted the codex/binomial-numerical-review branch October 12, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant