Run from the repository root:
dub run mir-stat:benchmarks --build=release --compiler=ldc2 -- --function=correlation --transform=standardize --algorithms=all --iterations=10000 --rounds=10 --warmup=10 --size=1000 --seed=5489Use --function=all (the default) to run skewness, kurtosis, covariance, and
correlation. Use --help to list options. DMD is also supported with
--compiler=dmd. Use a release build for performance measurements.
Each statistics comparison uses double and
Summation.fast. Iterations and size default to 10000 and 1000, respectively.
--rounds defaults to 10 measured rounds; --warmup defaults to 10 untimed
iterations once per function before its first measured round. --iterations
is per algorithm per measured round, so the defaults execute 100000 measured
calls and 10 warm-up calls per selected algorithm. Set --rounds=1 --warmup=0
for a single round without warm-up. Rounds and iterations must be positive;
zero warm-up is allowed.
Skewness requires at least three observations, kurtosis four, and covariance
and correlation two. Each result records the round's mean computed value and
total elapsed time, excluding input generation, preparation, reference checks,
and warm-up. Each algorithm accumulates native clock ticks across calls before
converting the round total to Duration, avoiding per-call rounding to 100 ns.
Warm-up still runs correctness checks.
Inputs are generated once per iteration and shared through read-only views
across all algorithms. --seed selects a local MT19937-64 engine's seed
(default 5489); it does not use or modify the thread-local random engine.
Running a function alone or as part of all uses the same seed. The generator
runs continuously through warm-up and all measured rounds; it is not reseeded
at round boundaries. Reproduction therefore requires the same seed, warm-up,
iteration count, compiler, build and dependency versions. Measured values
are reproducible under those conditions; elapsed times are not. Bitwise
agreement across platforms is not promised.
Select the input transformation independently of the algorithms:
--transform=none(default): leave generated inputs unchanged.--transform=center: subtract each input's mean, without scaling variance.--transform=standardize: calculate sample z-scores for each input.
Select --algorithms=all (default) or a comma-separated list, for example:
dub run mir-stat:benchmarks --build=release --compiler=ldc2 -- --function=correlation --transform=standardize --algorithms=twoPass,assumeStandardizedall runs every algorithm valid for the transformation. assumeZeroMean
requires center or standardize; assumeStandardized requires
standardize and is only a correlation algorithm. Explicitly requesting an
incompatible or unknown algorithm fails before any measurements start.
With --function=all, an explicit algorithm list must be valid for every
function. Duplicates, empty list items, and combining all with names are
rejected. The first round uses enumeration order, regardless of list order.
Each subsequent round cyclically shifts the first selected algorithm by one;
disabled algorithms take no positions. The order stays fixed within a round.
Every algorithm takes each position once per complete cycle of rounds, which
reduces position bias but does not eliminate all order or carryover effects.
Results list round and execution position alongside function and transformation.
The general inputs use x = 2 + 2*z1 and y = -1 + z1 + 3*z2, where z1
and z2 are independent standard normal draws. Preparation costs are excluded
when centering or standardizing: these comparisons model data already transformed,
not an end-to-end speedup from choosing a shortcut on arbitrary data.
Preparation and reference calculations run outside timing. Every result is
checked against the three-pass (skewness/kurtosis) or two-pass
(covariance/correlation) implementation using precise summation. Checks reject
nonfinite values and errors exceeding 1e-10 + 1e-10 * abs(reference). This
mixed tolerance accommodates rounding and near-zero statistics for these
normal inputs; it is not a general accuracy guarantee. A failed check stops
the run with a nonzero exit status.
This replaces the old unittest-perf configuration/build. The shared-input
loop, centering, and reference checks change cache state and the timing
baseline, so old timings are not directly comparable. Round timings are retained
individually; the runner does not perform significance tests or claim speedups.
Use --format=csv for one row per algorithm per measured round. For example:
dub run --quiet mir-stat:benchmarks --build=release --compiler=ldc2 -- --function=correlation --transform=standardize --algorithms=twoPass,assumeStandardized --rounds=10 --warmup=10 --iterations=10000 --format=csv > results.csv--quiet suppresses DUB progress messages. Text is the default format. CSV
includes function, transform, algorithm, round, execution position, iterations,
input size, seed, warm-up count, mean value, and total elapsed nanoseconds.
Round and position are one-based. Nanoseconds are the output unit, not a promise
of clock resolution. Means use 17 significant digits for double round trips.
Rows are emitted in enumeration order within each round; use position to
recover execution order.
Pair algorithms within the same function, transform, seed, warm-up, input size, iteration count, and round. Each such pair used identical inputs. Repeated rounds from one process are not independent process-level replications; retain that distinction in subsequent statistical analysis. Output occurs after a function's measured rounds, keeping printing out of the round sequence.
source/mir/stat/benchmark/cases.ddefines cases and returns results.source/mir/stat/benchmark/runner.dhandles command-line options and output.source/mir/stat/benchmark/report.dformats per-round CSV rows.mir.math.internal.benchmarkremains the shared helper module in the library.
Case interfaces use package(mir.stat) visibility. The executable depends on
the local parent package; benchmark code and mir-random are not added to the
normal library configuration.
dub test mir-stat:benchmarks --compiler=dmd
dub test mir-stat:benchmarks --compiler=ldc2
dub test mir-stat:benchmark-helpers --compiler=dmd
dub test mir-stat:benchmark-helpers --compiler=ldc2The subpackage tests run small cases and check results and input validation,
without asserting timing thresholds. The focused helper package compiles
mir.math.internal.benchmark with both ordinary and random-input helper tests
enabled, without rebuilding the full library test suite. The existing
dub test --config=unittest-benchmark command remains available to run helper
tests alongside the normal library tests.
CI runs these tests with stable DMD and LDC on Linux and Windows. Short release runs exercise all functions and applicable algorithms with untransformed, centered, and standardized inputs, including CSV output. These are correctness checks; CI does not compare timings or enforce performance thresholds.
Run the separate group with smaller defaults (matrix order 32 and 20 iterations):
dub run mir-stat:benchmarks --config=native-d --build=release --compiler=ldc2 -- --function=linearAlgebra --size=64 --iterations=20 --rounds=8 --warmup=3 --format=csv--function=all retains the existing statistics cases. Linear algebra accepts
--transform=none only. Choose --layout=rowMajor or --layout=transposed;
the latter stores the same logical matrices through transposed descriptors,
including output and factor workspaces. CSV function names include the layout.
--size is the square matrix order (1 through 4096), not the element count.
Buffers and precomputed factors are allocated only for the selected algorithms.
Large matrix cases require substantial memory and untimed reference work;
start with the small defaults. --seed, rotation, warm-up, algorithm selection,
and per-round CSV output follow the existing scheduler.
| Function | Algorithms |
|---|---|
triangularLeft |
dense, triangular, packed |
triangularRight |
dense, triangular, packed |
quadratic |
composed, general, symmetric, singleTriangle, generalFast, symmetricFast, packedSymmetric, packedFast |
mahalanobis |
covarianceGC, factorGC, precisionGC, precisionFactorGC, corresponding Workspace variants, precisionFastGC, precisionFastWorkspace |
Triangular comparisons multiply square matrices into preallocated output,
using explicit zeros in the dense baseline and NaNs in ignored triangular
entries. Every output element is checked outside timing against an independent
calculation with real accumulation. Only the first output element
is returned to the timing helper; it is not the sole correctness check.
Quadratic comparisons use a dense SPD matrix. composed computes a matrix-vector
product into reusable storage followed by a dot product. general and
symmetric use the library's default compensated row and outer reductions.
singleTriangle is an experimental benchmark-only flattened compensated
reduction, reading each upper-triangle entry once. It is not an alternative
library API: it can overflow individual terms where row-wise cancellation
keeps the existing implementation finite. A regression example records that
tradeoff. Comparisons therefore include differences in summation algorithms,
not just loop overhead; these are not general accuracy or speed guarantees.
Mahalanobis comparisons return squared distance. GC variants include their
per-call scratch allocations; workspace variants reuse buffers. covariance
includes factorization each call; factor, precision, and precisionFactor
start from precomputed matrices, whose preparation is excluded. No inversion
is timed. RC/custom allocator overhead and distance's final square root are
not measured by these cases. GC collection is neither disabled nor forced;
allocation timings can include collection and carryover effects.
The fixed metric is S = I + u*u', with independently constructed precision
P = I - u*u'/(1 + u'u). Seeded normal vector/right-hand-side inputs change
once per iteration. Reference calculations use these identities with real
accumulation; checks reject nonfinite results and errors above
1e-10 * (1 + abs(reference)). These are well-conditioned benchmark inputs,
not a substitute for the library's adversarial numerical tests. Setup,
precomputed factors, input generation, and correctness checks are untimed.
The native-d configuration selects the native D implementation and requires no BLAS library. To measure the optional
native backend, use --config=openblas with OpenBLAS installed. The parent
package selects mir-lapack/mir-blas through that configuration. Transposed
triangular matrix products and matrix solves require mir-blas 1.1.18 or later,
as enforced by the optional BLAS configurations. This release includes the fixes in
mir-blas PR #12.
Unsupported native layouts/types use the library's native D kernels.
Record backend/dependency versions, compiler flags, and BLAS thread count
alongside CSV files; the CSV schema does not identify the linked backend.
The benchmark unit tests cover both layouts, selections, invalid requests, and the single-triangle numerical counterexample. CI smoke runs verify native D CLI integration without asserting timing thresholds.
The generalFast and symmetricFast quadratic cases use caller workspace and
explicit Summation.fast. packedSymmetric uses compensated packed rows;
packedFast permits SPMV and DOT. The triangular packed case passes stairs
directly to mtimesInto. Packing happens before timing. All use the same seeded
inputs and independent reference checks as the corresponding dense cases.
The layout option changes dense matrix storage; packed rows remain in their
canonical upper-row order. Packed storage reduces memory use but does not
necessarily improve speed, especially versus dense TRMM with multiple columns.
precisionFastGC includes both temporary vector allocations in its timing;
precisionFastWorkspace reuses both vectors. Both explicitly select
Summation.fast and compare against the same independent precision identity
as the existing Mahalanobis cases.
symmetricVector, symmetricLeft, and symmetricRight compare general dense,
symmetric-view, and packed symmetric multiplication with general, symmetric,
and packed. Matrix cases use 32 columns (or 32 rows on the left of a right-side
metric). Their generalMixed, symmetricMixed, and packedMixed variants use
an output storage orientation opposite to the ordinary input. --layout selects
the input orientation. Output is preallocated, so these isolate computation from
allocation. Packing and analytic reference generation are outside timing.
dub run mir-stat:benchmarks --config=openblas --build=release --compiler=ldc2 -- --function=symmetricLeft --size=256 --iterations=20 --rounds=8 --warmup=3 --format=csvThe quadraticBatch algorithms are repeated, batch, repeatedFast, and
batchFast. mahalanobisBatch provides repeatedFactor/batchFactor,
repeatedPrecision/batchPrecision, repeatedPrecisionFast/batchPrecisionFast,
and repeatedPrecisionFactor/batchPrecisionFactor. Each invocation evaluates
32 observation columns against a shared metric with reusable workspace. Times
are per batch, not per observation. Each pair uses the same summation policy.
Prepared-factor construction is untimed; these do not measure covariance
factorization or allocating batch APIs. Every result is checked against the
independent rank-one identity outside timing.
The symmetric matrix families also provide workspaceInput and workspaceOutput.
Both use opposite input/output orientations and include the selected layout copy
inside timing; caller workspace is allocated beforehand. Their workspace strides
select copying the input or copying the output respectively. On the native D
backend, workspace can be unused and these retain the normal fallback.
symmetricColumn and symmetricRow exercise rank-two one-column and one-row
operands with the same algorithm choices, including the real vector specialization.
crossprod and tcrossprod compare general matrix multiplication with
specialized one-triangle self-products, including output mirroring in timing.
--size is the square output order; contraction length is 32. Input and output
buffers are reused. Both storage layouts are supported, and every result entry
is checked against an extended-precision reference outside timing. These cases
measure caller-output operations, not result allocation costs.
dub run mir-stat:benchmarks --config=openblas --build=release --compiler=ldc2 -- --function=crossprod --size=256 --iterations=20 --rounds=8 --warmup=3 --format=csvThe cross-product benchmarks also include triangleOnly, which calls the
explicit-upper-triangle Into overload. specialized includes mirroring and
fully initializes caller storage; triangleOnly exposes the same logical
symmetric matrix without mirroring. All cases reuse square output storage,
exclude allocation, and validate every logical entry against an independent
extended-precision reference outside timing. This isolates the cost of
mirroring rather than a reduction in allocated storage.
The histogram construction benchmark measures short-lived histograms with GC, reference-counted, and custom-allocator storage. It reports construction and final GC collection separately and uses an independently compiled observer library.