Skip to content

Fix weighted batch accumulation and stabilize the normality test - #147

Merged
jmh530 merged 3 commits into
libmir:masterfrom
jmh530:codex/weighted-batch-products
Oct 7, 2026
Merged

jmh530 merged 3 commits into
libmir:masterfrom
jmh530:codex/weighted-batch-products

Conversation

@jmh530

@jmh530 jmh530 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Fix three correctness issues in weighted accumulation and the D’Agostino–Pearson normality test, with a separate commit for each.

Weighted batch multiplication

Convert observations and weights to their accumulator operand types before multiplying, matching scalar insertion. Previously, products could overflow in the input type even when the accumulator could represent them—for example, multiplying two 50_000 integers before accumulating into double.

Keep the existing multiplication mapping when input and accumulator types already match. Mixed-type calculations can change rounding because conversion now happens before multiplication.

Shared-cursor input ranges

Capture the observation count before summation consumes a shaped input range. Previously, ranges whose copies share a cursor could report zero remaining observations after summation, producing an incorrect total weight.

This changes operation ordering without adding a traversal or allocation.

Normality-test stability

Replace log(x + sqrt(x*x + 1)) with std.math.asinh(x) in the skewness transformation. This avoids cancellation for negative and near-zero inputs and restores consistent results for reflected samples.

Validation

  • Added regression tests for integer and floating-point product overflow, batch/scalar agreement, shared-cursor insertion, and skewness symmetry.
  • All 55 test modules passed with DMD and LDC debug builds, DMD DIP1000, and optimized LDC.
  • Matching-type weighted paths retain identical LDC assembly in the tested cases; mixed-type performance varies.
  • Full normality-test benchmarks showed roughly 2% overhead with LDC for 16 observations and less than 1% difference for 256 and 4,096 observations. DMD timings were essentially unchanged.
  • Older compiler and GDC validation remains for CI.

jmh530 added 3 commits October 7, 2026 05:08
Convert mixed-type observations and weights before multiplication, matching scalar insertion and avoiding overflow in input types.

Preserve the existing mapping when operand and accumulator types match. Add regression tests for integer and floating-point overflow and batch/scalar consistency.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Capture the element count before summation consumes the input range. Add regression tests for initial, exhausted, and repeated insertion with naive and pairwise summation.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Use std.math.asinh to avoid cancellation for negative and near-zero skewness. Add regression tests for transformation symmetry and reflected samples across float, double, and real.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@codecov

codecov Bot commented Oct 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@jmh530
jmh530 merged commit 377f163 into libmir:master Oct 7, 2026
24 checks passed
@jmh530
jmh530 deleted the codex/weighted-batch-products branch October 7, 2026 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant