Skip to content

DICOM extract: multi-echo plane detection, modality fallback, and 9 smaller fixes - #152

Closed
robert-graf wants to merge 27 commits into
mainfrom
Point-Reg
Closed

robert-graf wants to merge 27 commits into
mainfrom
Point-Reg

Conversation

@robert-graf

Copy link
Copy Markdown
Collaborator

Summary

Cleanup pass on TPTBox/core/dicom/. Splits into 10 focused commits (plus the pre-existing 07aac40 fix multi echo bug). Every commit is reviewable on its own.

Correctness fixes

  • 88ab7e8 — _extract_nii_from_dicom returns False on unknown exceptions, deletes the half-written sidecar, and logs. Previously the bare except Exception fell through to return True, so _add_grid_info_to_json / _split_multi_echo_dixon then ran on a NIfTI that was never written.
  • 391a28e — _classic_get_grouped_dicoms no longer produces NaN directions on multi-echo series. New helper _split_by_echo_numbers runs stack detection on a single-echo subset, then re-attaches remaining echoes by ImagePositionPatient.
  • f0c70f4 — _split_multi_echo_dixon triggers on any 4-D output (not just part in ("magnitude", "phase")). Also switched frame/TE iteration to zip_strict.
  • 0894e46 — _find_all_files yields Path instances one at a time instead of yielding the raw list[Path] first, which downstream code cast to str and broke on.
  • 7f9e38b — _inc_key caps its retry loop at 10 000 iterations and raises RuntimeError with the offending key instead of spinning forever on a broken path_exists callback.
  • a09de6a — get_plane_dicom's real call now uses 0.8 (matches the docstring default); the bare-except now logs a warning so downstream acq-None files are traceable.

Perf / usability

  • 4c63431 — _read_dicom_files gates on a _looks_like_dicom check (extension blocklist + DICM magic at offset 128) before pydicom.dcmread. Skips reports, thumbnails, JSON sidecars, etc. on mixed source trees.
  • 7a7ade2 — Docs: n_cpu behaviour spelled out (1 sequential / >1 N threads / None → ThreadPoolExecutor-Default ≈ min(32, cpu_count + 4)).

Features

  • e1cddbc — Expanded map_series_description_to_file_format_default:
    • Quantitative maps (MP2RAGE, T2*, R2*, T1map, T2map, T2starmap, SWI) placed before the greedy .*t2.*/.*t1.* catch-alls that were misclassifying them.
    • Multi-echo VIBE variants (ME_vibe_fatquant_*, mevibe, fatquant, fatfrac, pdff, GE ideal).
    • Philips localizers (survey, refscan, smartexam).
  • 5280eb8 — Modality fallback + BIDS format whitelist + view/laterality/bodypart:
    • bids_constants.formats grows by xray, us, nm, sc, photo, endoscopy, rtimage, rtstruct, rtdose, rtplan, ot.
    • extract_keys_from_json no longer raises NotImplementedError for CR/DX/RG/PX/IO/MG/US/NM/SC/OP/XC/ES/RT*/OT — sensible defaults per DICOM modality group, with a warning + tag-based fallback for unknowns.
    • Two helpers _apply_view_keys and _apply_bodypart_key lift ViewPosition, Laterality/ImageLaterality, and BodyPartExamined into acq / part / desc, so MG four-view screenings, bilateral radiographs, ophthalmic photos, and multi-region NM/US sessions no longer collapse onto identical filenames.

Test plan

  • pytest unit_tests/ — 595 passed, 5 skipped (unchanged from base).
  • End-to-end run through the GLP1/RESIST DICOM tree, confirming the 6-echo Philips series is now labelled acq-ax (not acq-iso) and the classic stack detector no longer over-splits it.

🤖 Generated with Claude Code

robert-graf and others added 27 commits August 26, 2026 14:42
Roll the internal-scripts changes together with the new
precompute_grid_info_parallel helper in _load_nako_wh, which fans
per-file _add_grid_info_to_json calls out to a ProcessPoolExecutor so
the sidecar grid cache is populated up front.
The bare `except Exception: print(nii_path)` at the end of _extract_nii_from_dicom
still fell through to `return True`. Any unknown conversion error (network, out
of disk, dicom2nifti internal bug) was silently treated as success — downstream
`_add_grid_info_to_json` and `_split_multi_echo_dixon` then ran on a NIfTI that
was never actually written.

Now the branch logs a warning, deletes the half-written sidecar JSON, and
returns False so the caller stops the pipeline for that series.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Multi-echo series (e.g. Philips mDIX quant) put M echoes at each of N slice
positions inside a single ImageType sub-group. Sorted by InstanceNumber,
consecutive DICOMs then share ImagePositionPatient, so the direction test
computed `0 / 0 = NaN` and `np.allclose(NaN, prev)` false-triggered a stack
split — one physical acquisition would be shattered into arbitrary chunks.

New helper `_split_by_echo_numbers` picks the smallest `EchoNumbers` bucket
for stack detection so each position appears exactly once, then re-attaches
the remaining echoes to the detected stacks by `ImagePositionPatient`. Also
skips the direction update whenever `norm(delta) < 1e-6`, guarding against
residual duplicates and genuinely repeated slices.

No-op for single-echo series (0 or 1 distinct EchoNumbers value).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`_from_dicom_to_nii` only tried to split multi-echo output when the BIDS
`part` entity resolved to "magnitude" or "phase". Any 4-D volume whose part
was mapped to something else via `dixon_mapping` / `parts_mapping` (or none
at all) escaped the split and sat on disk as an opaque 4-D file. Since
`_split_multi_echo_dixon` is already a no-op on 3-D input, drop the gate
and always try to split.

Also switch the frame/TE iteration in `_split_multi_echo_dixon` to
`zip_strict`. `tes` is padded to `n_echo` upstream, so `zip` is currently
safe — but `zip_strict` makes the invariant explicit and traps future
regressions instead of silently truncating.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The first `yield dcm_dirs` handed back the raw argument, which is a
`list[Path]` when the caller passed a list. Downstream code (in
`extract_dicom_folder`) then did `str(dicom_path).endswith(".pkl")` on that
list and eventually `_read_dicom_files(list)` — the list-repr string
match ate the guard and `.rglob` blew up with `AttributeError`.

Iterate the normalised `dcm_dirs` list instead, so we always yield `Path`
instances one at a time. Behaviour for the common single-Path case is
unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously `_read_dicom_files` handed every file under a source folder to
`pydicom.dcmread(force=True)` — README.txt, .DS_Store, thumbnail PNGs, JSON
sidecars from earlier extractions, and so on. Each dcmread cost real time
on top-level scanner exports that mix DICOMs with reports and metadata.

New helper `_looks_like_dicom(path)`:
  1. Skips files with obviously non-DICOM extensions (json/txt/md/pdf, image
     formats, csv/tsv/yaml/html/xml, already-extracted NIfTI / nrrd).
  2. Reads the first 132 bytes and accepts the standard "DICM" magic at
     offset 128.
  3. Falls back to accepting extensionless files (and .dcm / .ima / .dicom),
     matching the previous permissive behaviour for Philips/Siemens exports
     that ship as `IM000001` and skip the preamble.

Archive extensions (`.zip` / `.tar` / …) are intentionally NOT in the block
list because top-level zips are unpacked by `extract_dicom_folder` before
`_read_dicom_files` sees them, and any residual archive would fail the
DICM-magic check anyway.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`_inc_key` incremented its key while `path_exists(keys)` remained True with
no upper bound. A mis-configured `path_exists` callback (e.g. one that
never sees the mutation because it captures stale state) would spin
forever. Add a hard cap and raise `RuntimeError` with the offending key
value instead, so the mistake surfaces immediately.

10 000 tries covers `-a` through `-mnw` letter suffixes — well beyond any
realistic filename-collision run.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous one-liner claimed "Defaults to 1 (sequential)" but did not
mention that `n_cpu=None` — a common thing to try when reaching for "use
all cores" — hits the parallel path with `max_workers=None`, meaning
`min(32, os.cpu_count() + 4)` Python threads. Spell all three cases out
(1 / >1 / None) and note that this workload is I/O-heavy.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two related plane-detection tweaks:

1. The real caller inside `extract_keys_from_json` was passing
   `hires_threshold=1` while the docstring / function default advertised
   0.8. Push the actual call to 0.8 so the behaviour matches the docs
   (and so submillimetre isotropic MRI still resolves cleanly after the
   multi-echo pre-filter took the "everything clamps to 1.0" collapse
   out of the equation).

2. The bare `except Exception: return None` at the end of
   `get_plane_dicom` swallowed every failure silently. A follow-up
   `acq-None` filename would then be unattributable. Log a warning with
   the exception type + message before returning None, guarding the log
   import itself so the fallback still returns None cleanly if the
   logger isn't importable.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Several common series descriptions were misclassified because the greedy
``.*t2.*`` / ``.*t1.*`` catch-alls matched first (first-match wins). Add
specific patterns BEFORE them:

* Quantitative maps: MP2RAGE, T2*, R2*, T1map, T2map, T2starmap.
* SWI (susceptibility weighted).
* Multi-echo VIBE variants for both NAKO Siemens (`ME_vibe_fatquant_*`)
  and the generic mevibe / fatquant labels.
* DIXON auxiliary maps: fatfrac, PDFF, GE Ideal.
* Philips localizer aliases the existing pilot/scout entries missed:
  survey, refscan, smartexam.

Also add an explicit ``.*mdixon.*`` alongside the existing ``.*mdix.*``
so both spellings of the Philips protocol name resolve to "dixon".

No behaviour change for series that were already correctly classified.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ypart

Three coupled additions so `extract_keys_from_json` stops raising on
non-CT/PET/MR/XA modalities.

* `bids_constants.formats`: add `xray`, `us`, `nm`, `sc`, `photo`,
  `endoscopy`, `rtimage`, `rtstruct`, `rtdose`, `rtplan`, `ot`. Without
  these entries, `BIDS_FILE.get_changed_bids` refuses the new formats
  under strict mode.

* New modality branches in `extract_keys_from_json`. Instead of
  `raise NotImplementedError`, dispatch:
    - CR/DX/RG/PX/IO/MG → `xray`
    - US → `us`
    - NM → `nm` (Radiopharmaceutical → `ce` when set)
    - SC → `sc`
    - OP/XC → `photo`
    - ES → `endoscopy`
    - RTIMAGE/RTSTRUCT/RTDOSE/RTPLAN → matching lowercase format
    - OT → `ot`
    - unknown → warn and fall back to `modality.lower()`
  so the pipeline can complete instead of aborting mid-tree.

* Two helpers extract the DICOM tags that actually differentiate these
  files in a session:
    - `_apply_view_keys` lifts `ViewPosition` into `acq` (only overwriting
      a None/iso plane) and `ImageLaterality`/`Laterality` into `part`,
      with ophthalmic OS/OD normalised to L/R.
    - `_apply_bodypart_key` lifts `BodyPartExamined` into `desc` when
      empty, matching common values (ABDOMEN, PELVIS, CHEST, HEAD…).
  Called from CT, PET, and every new modality branch so files that only
  differ by view (MG four-view screening), laterality (ophthalmic),
  tracer (NM), or body region no longer collapse onto identical BIDS
  filenames.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Benchmark comparison

Speed (wall time per call)

baseline 68adb2a vs head 68adb2a · python 3.11.16 · 5 repeats + 1 warmup

Showing the 5 most-changed measurements per case; the rest are collapsed. Rows with a baseline below 3 ms are tagged (noise) — runner jitter on those swamps any real change.

ct_3d — shape (73, 47, 73)

Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
poi_calc_poi_from_subreg_vert 10.27 ±0.10 10.63 ±0.40 +3.5% 0.037
poi_calc_centroids_nocrop 5.43 ±0.04 5.58 ±0.02 +2.7% 0.000
nii_load_img 12.72 ±0.09 12.86 ±0.07 +1.1% 0.228
nii_save 10.57 ±0.06 10.68 ±0.08 +1.0% 0.087
nii_dilate_msk 101.18 ±0.12 102.20 ±0.23 +1.0% 0.000
… 31 more measurements
Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
nii_fill_holes 4.48 ±0.03 4.46 ±0.08 -0.4% 0.616
nii_rescale 42.85 ±0.23 42.68 ±0.98 -0.4% 0.393
nii_filter_connected_components 4.16 ±0.03 4.15 ±0.02 -0.2% 0.542
nii_erode_msk 6.03 ±0.03 6.04 ±0.05 +0.1% 0.877
nii_resample_from_to 44.00 ±0.16 44.04 ±0.22 +0.1% 0.118
metric_voxels 250463.00 ±0.00 250463.00 ±0.00 +0.0% —
metric_labels 3.00 ±0.00 3.00 ±0.00 +0.0% —
metric_foreground_pct 14.51 ±0.00 14.51 ±0.00 +0.0% —
nii_set_dtype 0.41 ±0.01 0.37 ±0.02 -9.6% (noise) 0.050
poi_reorient 0.30 ±0.02 0.31 ±0.01 +5.9% (noise) 0.841
poi_rescale 0.18 ±0.01 0.17 ±0.01 -5.5% (noise) 0.467
nii_apply_crop 0.50 ±0.01 0.52 ±0.02 +3.3% (noise) 0.380
nii_compute_crop 0.28 ±0.01 0.28 ±0.00 -2.4% (noise) 0.484
poi_resample_from_to 0.51 ±0.01 0.52 ±0.01 +2.3% (noise) 0.117
nii_center_of_masses 1.54 ±0.02 1.51 ±0.01 -1.9% (noise) 0.054
poi_to_global 0.16 ±0.01 0.16 ±0.01 -1.7% (noise) 0.235
nii_get_connected_components 2.41 ±0.03 2.45 ±0.01 +1.5% (noise) 0.316
nii_pad_to 0.53 ±0.01 0.52 ±0.02 -1.4% (noise) 0.473
nii_rescale_seg 2.72 ±0.06 2.75 ±0.14 +1.1% (noise) 0.337
poi_local_to_global_arr 0.27 ±0.01 0.27 ±0.01 -1.0% (noise) 0.877
nii_reorient 0.64 ±0.03 0.64 ±0.02 -1.0% (noise) 0.496
poi_load 0.77 ±0.01 0.78 ±0.04 +0.9% (noise) 0.299
nii_extract_label 0.29 ±0.02 0.30 ±0.01 +0.9% (noise) 0.524
nii_map_labels 1.01 ±0.01 1.02 ±0.01 +0.8% (noise) 0.330
poi_save 0.30 ±0.02 0.30 ±0.02 -0.7% (noise) 0.952
nii_load_seg 2.36 ±0.03 2.38 ±0.03 +0.6% (noise) 0.453
nii_unique 0.83 ±0.01 0.83 ±0.01 -0.6% (noise) 0.177
nii_get_array 0.04 ±0.00 0.04 ±0.00 +0.6% (noise) 0.783
poi_calc_centroids 2.06 ±0.03 2.07 ±0.01 +0.4% (noise) 0.667
poi_map_labels 0.14 ±0.01 0.14 ±0.01 +0.4% (noise) 0.996
nii_volumes 1.64 ±0.01 1.63 ±0.02 -0.2% (noise) 0.523

ct_2d — shape (73, 47, 1)

Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
nii_rescale 5.11 ±0.06 5.15 ±0.05 +0.8% 0.630
nii_resample_from_to 6.54 ±0.05 6.52 ±0.07 -0.3% 0.857
nii_dilate_msk 11.66 ±0.04 11.67 ±0.24 +0.1% 0.384
metric_voxels 3431.00 ±0.00 3431.00 ±0.00 +0.0% —
metric_labels 3.00 ±0.00 3.00 ±0.00 +0.0% —
… 30 more measurements
Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
metric_foreground_pct 37.60 ±0.00 37.60 ±0.00 +0.0% —
nii_unique 0.08 ±0.00 0.09 ±0.01 +10.8% (noise) 0.062
nii_set_dtype 0.31 ±0.01 0.29 ±0.01 -7.7% (noise) 0.028
nii_volumes 0.20 ±0.01 0.21 ±0.00 +7.4% (noise) 0.105
poi_reorient 0.34 ±0.01 0.32 ±0.01 -5.1% (noise) 0.262
nii_center_of_masses 0.13 ±0.01 0.14 ±0.01 +4.9% (noise) 0.095
poi_map_labels 0.16 ±0.01 0.15 ±0.01 -3.7% (noise) 0.493
nii_apply_crop 0.50 ±0.02 0.48 ±0.02 -3.6% (noise) 0.240
nii_compute_crop 0.10 ±0.01 0.10 ±0.01 +3.5% (noise) 0.301
nii_extract_label 0.29 ±0.02 0.30 ±0.01 +3.3% (noise) 0.961
nii_reorient 0.62 ±0.02 0.60 ±0.02 -3.2% (noise) 0.175
poi_local_to_global_arr 0.30 ±0.01 0.29 ±0.01 -3.0% (noise) 0.339
nii_load_img 1.14 ±0.01 1.17 ±0.02 +2.7% (noise) 0.027
nii_pad_to 0.41 ±0.02 0.40 ±0.01 -2.4% (noise) 0.145
poi_calc_centroids 0.68 ±0.01 0.70 ±0.01 +2.4% (noise) 0.042
poi_resample_from_to 0.55 ±0.01 0.56 ±0.01 +1.6% (noise) 0.594
nii_get_array 0.01 ±0.00 0.01 ±0.00 +1.5% (noise) 0.384
nii_rescale_seg 0.56 ±0.02 0.57 ±0.02 +1.5% (noise) 0.833
nii_load_seg 1.18 ±0.03 1.20 ±0.03 +1.5% (noise) 0.465
poi_rescale 0.19 ±0.01 0.20 ±0.01 +1.4% (noise) 0.671
nii_save 0.99 ±0.02 1.01 ±0.03 +1.4% (noise) 0.762
poi_calc_centroids_nocrop 0.86 ±0.02 0.87 ±0.01 +1.2% (noise) 0.476
nii_fill_holes 0.64 ±0.01 0.64 ±0.02 +1.1% (noise) 0.829
poi_to_global 0.17 ±0.01 0.17 ±0.01 +0.8% (noise) 0.215
nii_map_labels 0.35 ±0.01 0.35 ±0.01 -0.7% (noise) 0.907
nii_get_connected_components 0.48 ±0.01 0.48 ±0.02 +0.7% (noise) 0.363
nii_erode_msk 0.76 ±0.03 0.76 ±0.01 +0.6% (noise) 0.900
poi_save 0.34 ±0.01 0.34 ±0.01 -0.3% (noise) 0.889
poi_load 0.81 ±0.01 0.81 ±0.02 +0.2% (noise) 0.903
nii_filter_connected_components 0.62 ±0.03 0.62 ±0.02 +0.2% (noise) 0.652

mri_3d — shape (68, 52, 67)

Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
nii_filter_connected_components 4.36 ±0.02 4.46 ±0.06 +2.3% 0.012
poi_calc_centroids_nocrop 11.49 ±0.04 11.28 ±0.07 -1.8% 0.004
nii_erode_msk 7.60 ±0.04 7.67 ±0.04 +0.9% 0.040
nii_resample_from_to 41.57 ±0.62 41.85 ±0.98 +0.7% 0.299
nii_rescale 40.09 ±0.72 40.35 ±0.97 +0.6% 0.473
… 31 more measurements
Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
poi_calc_poi_from_subreg_vert 9.52 ±0.06 9.56 ±0.05 +0.5% 0.100
nii_save 8.50 ±0.07 8.54 ±0.07 +0.5% 0.267
nii_fill_holes 5.49 ±0.03 5.51 ±0.02 +0.4% 0.330
nii_load_img 10.15 ±0.04 10.17 ±0.06 +0.2% 0.994
nii_dilate_msk 132.10 ±0.32 132.26 ±0.53 +0.1% 0.671
metric_voxels 236912.00 ±0.00 236912.00 ±0.00 +0.0% —
metric_labels 8.00 ±0.00 8.00 ±0.00 +0.0% —
metric_foreground_pct 18.21 ±0.00 18.21 ±0.00 +0.0% —
nii_get_array 0.04 ±0.00 0.05 ±0.00 +15.4% (noise) 0.000
nii_pad_to 0.55 ±0.01 0.63 ±0.01 +14.2% (noise) 0.000
nii_extract_label 0.35 ±0.01 0.38 ±0.02 +9.4% (noise) 0.017
poi_local_to_global_arr 0.29 ±0.01 0.32 ±0.01 +9.2% (noise) 0.017
nii_volumes 1.84 ±0.02 1.68 ±0.01 -8.5% (noise) 0.000
nii_center_of_masses 1.88 ±0.01 1.73 ±0.04 -8.3% (noise) 0.001
nii_set_dtype 0.44 ±0.01 0.47 ±0.02 +7.4% (noise) 0.027
poi_to_global 0.20 ±0.00 0.21 ±0.01 +4.3% (noise) 0.035
poi_rescale 0.20 ±0.00 0.21 ±0.02 +4.3% (noise) 0.121
poi_resample_from_to 0.58 ±0.04 0.60 ±0.01 +3.8% (noise) 0.927
nii_map_labels 1.04 ±0.01 1.07 ±0.01 +3.7% (noise) 0.003
nii_get_connected_components 2.32 ±0.03 2.38 ±0.04 +2.5% (noise) 0.031
poi_map_labels 0.20 ±0.01 0.20 ±0.01 -2.5% (noise) 0.224
nii_unique 0.85 ±0.02 0.86 ±0.02 +1.9% (noise) 0.038
nii_compute_crop 0.30 ±0.02 0.30 ±0.02 -1.4% (noise) 0.793
poi_calc_centroids 2.28 ±0.01 2.31 ±0.02 +1.2% (noise) 0.041
nii_load_seg 2.32 ±0.02 2.34 ±0.11 +1.0% (noise) 0.240
poi_load 1.78 ±0.01 1.79 ±0.00 +0.4% (noise) 0.029
nii_apply_crop 0.53 ±0.03 0.54 ±0.00 +0.3% (noise) 0.589
poi_reorient 0.36 ±0.01 0.36 ±0.01 +0.2% (noise) 0.256
nii_reorient 0.67 ±0.01 0.67 ±0.01 +0.1% (noise) 0.975
nii_rescale_seg 2.62 ±0.04 2.62 ±0.07 -0.0% (noise) 0.704
poi_save 0.40 ±0.02 0.40 ±0.02 -0.0% (noise) 0.683

mri_2d — shape (68, 52, 1)

Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
nii_resample_from_to 6.53 ±0.06 6.62 ±0.07 +1.3% 0.198
nii_rescale 5.14 ±0.05 5.18 ±0.02 +0.7% 0.235
nii_dilate_msk 15.46 ±0.09 15.53 ±0.07 +0.4% 0.229
metric_voxels 3536.00 ±0.00 3536.00 ±0.00 +0.0% —
metric_labels 7.00 ±0.00 7.00 ±0.00 +0.0% —
… 30 more measurements
Measurement baseline ms (median ±½·range) head ms (median ±½·range) Δ % p
metric_foreground_pct 50.17 ±0.00 50.17 ±0.00 +0.0% —
poi_to_global 0.21 ±0.00 0.23 ±0.01 +9.2% (noise) 0.026
nii_extract_label 0.33 ±0.02 0.30 ±0.01 -9.0% (noise) 0.060
nii_get_array 0.01 ±0.00 0.01 ±0.00 +8.5% (noise) 0.079
poi_rescale 0.22 ±0.01 0.21 ±0.01 -8.1% (noise) 0.011
poi_save 0.38 ±0.00 0.41 ±0.03 +7.6% (noise) 0.093
nii_get_connected_components 0.46 ±0.02 0.49 ±0.01 +7.6% (noise) 0.065
nii_pad_to 0.41 ±0.02 0.39 ±0.01 -4.5% (noise) 0.099
nii_unique 0.08 ±0.00 0.09 ±0.01 +4.5% (noise) 0.181
nii_center_of_masses 0.31 ±0.01 0.30 ±0.00 -4.3% (noise) 0.059
nii_load_seg 1.17 ±0.02 1.22 ±0.03 +4.2% (noise) 0.004
nii_rescale_seg 0.58 ±0.02 0.56 ±0.01 -4.0% (noise) 0.174
poi_local_to_global_arr 0.30 ±0.01 0.31 ±0.02 +3.9% (noise) 0.283
nii_reorient 0.63 ±0.01 0.61 ±0.03 -3.1% (noise) 0.065
nii_load_img 1.11 ±0.01 1.14 ±0.03 +3.1% (noise) 0.030
nii_volumes 0.35 ±0.01 0.36 ±0.01 +1.9% (noise) 0.758
nii_apply_crop 0.48 ±0.01 0.47 ±0.01 -1.4% (noise) 0.411
poi_load 1.58 ±0.02 1.60 ±0.02 +1.4% (noise) 0.308
nii_compute_crop 0.10 ±0.00 0.10 ±0.00 -1.3% (noise) 0.411
nii_set_dtype 0.30 ±0.02 0.29 ±0.01 -1.3% (noise) 0.137
poi_map_labels 0.19 ±0.01 0.19 ±0.01 +1.2% (noise) 0.703
nii_fill_holes 0.80 ±0.02 0.79 ±0.02 -1.2% (noise) 0.913
nii_filter_connected_components 0.65 ±0.06 0.64 ±0.01 -1.1% (noise) 0.398
poi_resample_from_to 0.59 ±0.02 0.60 ±0.02 +0.7% (noise) 0.985
poi_calc_centroids 0.87 ±0.01 0.86 ±0.02 -0.7% (noise) 0.547
nii_save 0.99 ±0.01 0.98 ±0.02 -0.7% (noise) 0.872
nii_map_labels 0.37 ±0.01 0.36 ±0.01 -0.5% (noise) 0.606
nii_erode_msk 0.97 ±0.00 0.97 ±0.01 -0.4% (noise) 0.183
poi_reorient 0.35 ±0.01 0.35 ±0.01 -0.3% (noise) 0.358
poi_calc_centroids_nocrop 1.09 ±0.01 1.09 ±0.02 +0.1% (noise) 0.677

Gate: a measurement fails when the baseline is ≥ 1 ms, the median grows by ≥ 50%, and Welch's t-test gives p < 0.05. metric_* rows are context only.

Memory (peak RSS growth per call)

baseline 68adb2a vs head 68adb2a · python 3.11.16 · 5 repeats + 1 warmup · sampler proc-status, isolation fork · measurement floor ≈ 0.95 MiB

Showing the 5 most-changed measurements per case; the rest are collapsed. Rows with a baseline below 3 MiB are tagged (noise) — runner jitter on those swamps any real change.

ct_3d — shape (73, 47, 73)

Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
poi_resample_from_to 5.59 ±0.00 5.49 ±0.00 -1.7% —
nii_reorient 6.03 ±0.00 5.93 ±0.00 -1.6% —
nii_load_seg 6.13 ±0.00 6.04 ±0.00 -1.5% —
nii_resample_from_to 6.61 ±0.00 6.51 ±0.00 -1.5% —
nii_rescale 6.48 ±0.00 6.39 ±0.00 -1.4% —
… 31 more measurements
Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
nii_load_img 3.62 ±0.00 3.58 ±0.00 -1.2% —
nii_dilate_msk 6.12 ±0.00 6.05 ±0.00 -1.1% —
nii_set_dtype 5.43 ±0.00 5.37 ±0.00 -1.1% —
nii_rescale_seg 6.25 ±0.00 6.19 ±0.00 -1.1% —
nii_pad_to 5.55 ±0.00 5.49 ±0.00 -1.1% —
nii_extract_label 5.55 ±0.00 5.50 ±0.00 -1.1% —
nii_map_labels 5.62 ±0.00 5.56 ±0.00 -1.0% —
nii_apply_crop 5.99 ±0.00 5.93 ±0.00 -1.0% —
poi_calc_centroids_nocrop 6.11 ±0.00 6.05 ±0.00 -1.0% —
nii_erode_msk 6.12 ±0.00 6.06 ±0.00 -1.0% —
poi_calc_centroids 6.24 ±0.00 6.18 ±0.00 -0.9% —
nii_get_connected_components 6.74 ±0.00 6.68 ±0.00 -0.9% —
nii_fill_holes 6.47 ±0.00 6.41 ±0.00 -0.8% —
nii_filter_connected_components 7.05 ±0.00 7.00 ±0.00 -0.8% —
poi_calc_poi_from_subreg_vert 7.36 ±0.00 7.30 ±0.00 -0.8% —
nii_compute_crop 3.02 ±0.00 3.02 ±0.00 -0.3% —
poi_local_to_global_arr 3.08 ±0.00 3.07 ±0.00 -0.3% —
poi_to_global 3.21 ±0.00 3.20 ±0.00 -0.2% —
nii_center_of_masses 3.27 ±0.00 3.27 ±0.00 -0.2% —
nii_volumes 3.59 ±0.00 3.58 ±0.00 -0.2% —
nii_save 3.90 ±0.00 3.89 ±0.00 -0.2% —
poi_reorient 3.96 ±0.00 3.95 ±0.00 -0.2% —
metric_voxels 250463.00 ±0.00 250463.00 ±0.00 +0.0% —
metric_labels 3.00 ±0.00 3.00 ±0.00 +0.0% —
metric_foreground_pct 14.51 ±0.00 14.51 ±0.00 +0.0% —
poi_save 1.53 ±0.00 1.52 ±0.00 -0.5% (noise) —
poi_load 1.68 ±0.00 1.67 ±0.00 -0.5% (noise) —
nii_get_array 1.71 ±0.00 1.70 ±0.00 -0.5% (noise) —
poi_map_labels 1.84 ±0.00 1.83 ±0.00 -0.4% (noise) —
poi_rescale 2.67 ±0.00 2.66 ±0.00 -0.3% (noise) —
nii_unique 2.96 ±0.00 2.95 ±0.00 -0.3% (noise) —

ct_2d — shape (73, 47, 1)

Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
poi_resample_from_to 5.53 ±0.00 5.43 ±0.00 -1.8% —
nii_reorient 5.97 ±0.00 5.87 ±0.00 -1.6% —
nii_load_seg 6.07 ±0.00 5.98 ±0.00 -1.5% —
nii_resample_from_to 6.55 ±0.00 6.45 ±0.00 -1.5% —
nii_rescale 6.42 ±0.00 6.32 ±0.00 -1.5% —
… 30 more measurements
Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
nii_load_img 3.56 ±0.00 3.52 ±0.00 -1.2% —
nii_dilate_msk 6.05 ±0.00 5.98 ±0.00 -1.2% —
nii_set_dtype 5.37 ±0.00 5.31 ±0.00 -1.1% —
nii_rescale_seg 6.19 ±0.00 6.12 ±0.00 -1.1% —
nii_pad_to 5.49 ±0.00 5.43 ±0.00 -1.1% —
nii_extract_label 5.49 ±0.00 5.43 ±0.00 -1.1% —
nii_map_labels 5.55 ±0.00 5.50 ±0.00 -1.1% —
nii_apply_crop 5.93 ±0.00 5.87 ±0.00 -1.0% —
nii_erode_msk 6.06 ±0.00 6.00 ±0.00 -1.0% —
poi_calc_centroids_nocrop 6.11 ±0.00 6.05 ±0.00 -1.0% —
poi_calc_centroids 6.18 ±0.00 6.12 ±0.00 -0.9% —
nii_get_connected_components 6.68 ±0.00 6.62 ±0.00 -0.9% —
nii_fill_holes 6.41 ±0.00 6.35 ±0.00 -0.9% —
nii_filter_connected_components 6.99 ±0.00 6.93 ±0.00 -0.8% —
poi_local_to_global_arr 3.02 ±0.00 3.01 ±0.00 -0.3% —
poi_to_global 3.14 ±0.00 3.14 ±0.00 -0.2% —
nii_center_of_masses 3.21 ±0.00 3.20 ±0.00 -0.2% —
nii_volumes 3.52 ±0.00 3.52 ±0.00 -0.2% —
nii_save 3.84 ±0.00 3.83 ±0.00 -0.2% —
poi_reorient 3.89 ±0.00 3.89 ±0.00 -0.2% —
metric_voxels 3431.00 ±0.00 3431.00 ±0.00 +0.0% —
metric_labels 3.00 ±0.00 3.00 ±0.00 +0.0% —
metric_foreground_pct 37.60 ±0.00 37.60 ±0.00 +0.0% —
poi_save 1.46 ±0.00 1.46 ±0.00 -0.5% (noise) —
poi_load 1.61 ±0.00 1.61 ±0.00 -0.5% (noise) —
nii_get_array 1.65 ±0.00 1.64 ±0.00 -0.5% (noise) —
poi_map_labels 1.77 ±0.00 1.77 ±0.00 -0.4% (noise) —
poi_rescale 2.61 ±0.00 2.60 ±0.00 -0.3% (noise) —
nii_unique 2.90 ±0.00 2.89 ±0.00 -0.3% (noise) —
nii_compute_crop 2.96 ±0.00 2.95 ±0.00 -0.3% (noise) —

mri_3d — shape (68, 52, 67)

Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
poi_resample_from_to 5.53 ±0.00 5.43 ±0.00 -1.8% —
nii_reorient 5.97 ±0.00 5.87 ±0.00 -1.6% —
nii_load_seg 6.07 ±0.00 5.98 ±0.00 -1.5% —
nii_resample_from_to 6.55 ±0.00 6.45 ±0.00 -1.5% —
nii_rescale 6.42 ±0.00 6.32 ±0.00 -1.5% —
… 31 more measurements
Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
nii_load_img 3.62 ±0.00 3.58 ±0.00 -1.2% —
nii_dilate_msk 6.05 ±0.00 5.98 ±0.00 -1.2% —
nii_set_dtype 5.43 ±0.00 5.37 ±0.00 -1.1% —
nii_rescale_seg 6.19 ±0.00 6.12 ±0.00 -1.1% —
nii_pad_to 5.49 ±0.00 5.43 ±0.00 -1.1% —
nii_extract_label 5.49 ±0.00 5.43 ±0.00 -1.1% —
nii_map_labels 5.55 ±0.00 5.50 ±0.00 -1.1% —
nii_apply_crop 5.93 ±0.00 5.87 ±0.00 -1.0% —
poi_calc_centroids_nocrop 6.05 ±0.00 5.99 ±0.00 -1.0% —
nii_erode_msk 6.06 ±0.00 6.00 ±0.00 -1.0% —
poi_calc_centroids 6.18 ±0.00 6.12 ±0.00 -0.9% —
nii_get_connected_components 6.68 ±0.00 6.62 ±0.00 -0.9% —
nii_fill_holes 6.41 ±0.00 6.35 ±0.00 -0.9% —
nii_filter_connected_components 6.99 ±0.00 6.93 ±0.00 -0.8% —
poi_calc_poi_from_subreg_vert 7.36 ±0.00 7.30 ±0.00 -0.8% —
poi_local_to_global_arr 3.02 ±0.00 3.01 ±0.00 -0.3% —
poi_to_global 3.14 ±0.00 3.14 ±0.00 -0.2% —
nii_center_of_masses 3.21 ±0.00 3.20 ±0.00 -0.2% —
nii_volumes 3.52 ±0.00 3.52 ±0.00 -0.2% —
nii_save 3.84 ±0.00 3.83 ±0.00 -0.2% —
poi_reorient 3.89 ±0.00 3.89 ±0.00 -0.2% —
metric_voxels 236912.00 ±0.00 236912.00 ±0.00 +0.0% —
metric_labels 8.00 ±0.00 8.00 ±0.00 +0.0% —
metric_foreground_pct 18.21 ±0.00 18.21 ±0.00 +0.0% —
poi_save 1.53 ±0.00 1.52 ±0.00 -0.5% (noise) —
poi_load 1.61 ±0.00 1.61 ±0.00 -0.5% (noise) —
nii_get_array 1.65 ±0.00 1.64 ±0.00 -0.5% (noise) —
poi_map_labels 1.77 ±0.00 1.77 ±0.00 -0.4% (noise) —
poi_rescale 2.61 ±0.00 2.60 ±0.00 -0.3% (noise) —
nii_unique 2.90 ±0.00 2.89 ±0.00 -0.3% (noise) —
nii_compute_crop 2.96 ±0.00 2.95 ±0.00 -0.3% (noise) —

mri_2d — shape (68, 52, 1)

Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
poi_resample_from_to 5.53 ±0.00 5.43 ±0.00 -1.8% —
nii_reorient 5.97 ±0.00 5.87 ±0.00 -1.6% —
nii_load_seg 6.07 ±0.00 5.98 ±0.00 -1.5% —
nii_resample_from_to 6.55 ±0.00 6.45 ±0.00 -1.5% —
nii_rescale 6.42 ±0.00 6.32 ±0.00 -1.5% —
… 30 more measurements
Measurement baseline MiB (median ±½·range) head MiB (median ±½·range) Δ % p
nii_load_img 3.62 ±0.00 3.58 ±0.00 -1.2% —
nii_dilate_msk 6.05 ±0.00 5.98 ±0.00 -1.2% —
nii_set_dtype 5.43 ±0.00 5.37 ±0.00 -1.1% —
nii_rescale_seg 6.19 ±0.00 6.12 ±0.00 -1.1% —
nii_pad_to 5.49 ±0.00 5.43 ±0.00 -1.1% —
nii_extract_label 5.49 ±0.00 5.43 ±0.00 -1.1% —
nii_map_labels 5.55 ±0.00 5.50 ±0.00 -1.1% —
nii_apply_crop 5.93 ±0.00 5.87 ±0.00 -1.0% —
nii_erode_msk 6.06 ±0.00 6.00 ±0.00 -1.0% —
poi_calc_centroids_nocrop 6.11 ±0.00 6.05 ±0.00 -1.0% —
poi_calc_centroids 6.18 ±0.00 6.12 ±0.00 -0.9% —
nii_get_connected_components 6.68 ±0.00 6.62 ±0.00 -0.9% —
nii_fill_holes 6.41 ±0.00 6.35 ±0.00 -0.9% —
nii_filter_connected_components 6.99 ±0.00 6.93 ±0.00 -0.8% —
poi_local_to_global_arr 3.02 ±0.00 3.01 ±0.00 -0.3% —
poi_to_global 3.14 ±0.00 3.14 ±0.00 -0.2% —
nii_center_of_masses 3.21 ±0.00 3.20 ±0.00 -0.2% —
nii_volumes 3.52 ±0.00 3.52 ±0.00 -0.2% —
nii_save 3.84 ±0.00 3.83 ±0.00 -0.2% —
poi_reorient 3.89 ±0.00 3.89 ±0.00 -0.2% —
metric_voxels 3536.00 ±0.00 3536.00 ±0.00 +0.0% —
metric_labels 7.00 ±0.00 7.00 ±0.00 +0.0% —
metric_foreground_pct 50.17 ±0.00 50.17 ±0.00 +0.0% —
poi_save 1.53 ±0.00 1.52 ±0.00 -0.5% (noise) —
poi_load 1.61 ±0.00 1.61 ±0.00 -0.5% (noise) —
nii_get_array 1.65 ±0.00 1.64 ±0.00 -0.5% (noise) —
poi_map_labels 1.77 ±0.00 1.77 ±0.00 -0.4% (noise) —
poi_rescale 2.61 ±0.00 2.60 ±0.00 -0.3% (noise) —
nii_unique 2.90 ±0.00 2.89 ±0.00 -0.3% (noise) —
nii_compute_crop 2.96 ±0.00 2.95 ±0.00 -0.3% (noise) —

Gate: a measurement fails when the baseline is ≥ 1 MiB, the median grows by ≥ 50%, and Welch's t-test gives p < 0.05. metric_* rows are context only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant