Repository navigation
DICOM extract: multi-echo plane detection, modality fallback, and 9 smaller fixes - #152
robert-graf wants to merge 27 commits into
Conversation
Roll the internal-scripts changes together with the new precompute_grid_info_parallel helper in _load_nako_wh, which fans per-file _add_grid_info_to_json calls out to a ProcessPoolExecutor so the sidecar grid cache is populated up front.
The bare `except Exception: print(nii_path)` at the end of _extract_nii_from_dicom still fell through to `return True`. Any unknown conversion error (network, out of disk, dicom2nifti internal bug) was silently treated as success — downstream `_add_grid_info_to_json` and `_split_multi_echo_dixon` then ran on a NIfTI that was never actually written. Now the branch logs a warning, deletes the half-written sidecar JSON, and returns False so the caller stops the pipeline for that series. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Multi-echo series (e.g. Philips mDIX quant) put M echoes at each of N slice positions inside a single ImageType sub-group. Sorted by InstanceNumber, consecutive DICOMs then share ImagePositionPatient, so the direction test computed `0 / 0 = NaN` and `np.allclose(NaN, prev)` false-triggered a stack split — one physical acquisition would be shattered into arbitrary chunks. New helper `_split_by_echo_numbers` picks the smallest `EchoNumbers` bucket for stack detection so each position appears exactly once, then re-attaches the remaining echoes to the detected stacks by `ImagePositionPatient`. Also skips the direction update whenever `norm(delta) < 1e-6`, guarding against residual duplicates and genuinely repeated slices. No-op for single-echo series (0 or 1 distinct EchoNumbers value). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`_from_dicom_to_nii` only tried to split multi-echo output when the BIDS `part` entity resolved to "magnitude" or "phase". Any 4-D volume whose part was mapped to something else via `dixon_mapping` / `parts_mapping` (or none at all) escaped the split and sat on disk as an opaque 4-D file. Since `_split_multi_echo_dixon` is already a no-op on 3-D input, drop the gate and always try to split. Also switch the frame/TE iteration in `_split_multi_echo_dixon` to `zip_strict`. `tes` is padded to `n_echo` upstream, so `zip` is currently safe — but `zip_strict` makes the invariant explicit and traps future regressions instead of silently truncating. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The first `yield dcm_dirs` handed back the raw argument, which is a
`list[Path]` when the caller passed a list. Downstream code (in
`extract_dicom_folder`) then did `str(dicom_path).endswith(".pkl")` on that
list and eventually `_read_dicom_files(list)` — the list-repr string
match ate the guard and `.rglob` blew up with `AttributeError`.
Iterate the normalised `dcm_dirs` list instead, so we always yield `Path`
instances one at a time. Behaviour for the common single-Path case is
unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously `_read_dicom_files` handed every file under a source folder to
`pydicom.dcmread(force=True)` — README.txt, .DS_Store, thumbnail PNGs, JSON
sidecars from earlier extractions, and so on. Each dcmread cost real time
on top-level scanner exports that mix DICOMs with reports and metadata.
New helper `_looks_like_dicom(path)`:
1. Skips files with obviously non-DICOM extensions (json/txt/md/pdf, image
formats, csv/tsv/yaml/html/xml, already-extracted NIfTI / nrrd).
2. Reads the first 132 bytes and accepts the standard "DICM" magic at
offset 128.
3. Falls back to accepting extensionless files (and .dcm / .ima / .dicom),
matching the previous permissive behaviour for Philips/Siemens exports
that ship as `IM000001` and skip the preamble.
Archive extensions (`.zip` / `.tar` / …) are intentionally NOT in the block
list because top-level zips are unpacked by `extract_dicom_folder` before
`_read_dicom_files` sees them, and any residual archive would fail the
DICM-magic check anyway.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`_inc_key` incremented its key while `path_exists(keys)` remained True with no upper bound. A mis-configured `path_exists` callback (e.g. one that never sees the mutation because it captures stale state) would spin forever. Add a hard cap and raise `RuntimeError` with the offending key value instead, so the mistake surfaces immediately. 10 000 tries covers `-a` through `-mnw` letter suffixes — well beyond any realistic filename-collision run. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous one-liner claimed "Defaults to 1 (sequential)" but did not mention that `n_cpu=None` — a common thing to try when reaching for "use all cores" — hits the parallel path with `max_workers=None`, meaning `min(32, os.cpu_count() + 4)` Python threads. Spell all three cases out (1 / >1 / None) and note that this workload is I/O-heavy. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two related plane-detection tweaks: 1. The real caller inside `extract_keys_from_json` was passing `hires_threshold=1` while the docstring / function default advertised 0.8. Push the actual call to 0.8 so the behaviour matches the docs (and so submillimetre isotropic MRI still resolves cleanly after the multi-echo pre-filter took the "everything clamps to 1.0" collapse out of the equation). 2. The bare `except Exception: return None` at the end of `get_plane_dicom` swallowed every failure silently. A follow-up `acq-None` filename would then be unattributable. Log a warning with the exception type + message before returning None, guarding the log import itself so the fallback still returns None cleanly if the logger isn't importable. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Several common series descriptions were misclassified because the greedy ``.*t2.*`` / ``.*t1.*`` catch-alls matched first (first-match wins). Add specific patterns BEFORE them: * Quantitative maps: MP2RAGE, T2*, R2*, T1map, T2map, T2starmap. * SWI (susceptibility weighted). * Multi-echo VIBE variants for both NAKO Siemens (`ME_vibe_fatquant_*`) and the generic mevibe / fatquant labels. * DIXON auxiliary maps: fatfrac, PDFF, GE Ideal. * Philips localizer aliases the existing pilot/scout entries missed: survey, refscan, smartexam. Also add an explicit ``.*mdixon.*`` alongside the existing ``.*mdix.*`` so both spellings of the Philips protocol name resolve to "dixon". No behaviour change for series that were already correctly classified. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ypart
Three coupled additions so `extract_keys_from_json` stops raising on
non-CT/PET/MR/XA modalities.
* `bids_constants.formats`: add `xray`, `us`, `nm`, `sc`, `photo`,
`endoscopy`, `rtimage`, `rtstruct`, `rtdose`, `rtplan`, `ot`. Without
these entries, `BIDS_FILE.get_changed_bids` refuses the new formats
under strict mode.
* New modality branches in `extract_keys_from_json`. Instead of
`raise NotImplementedError`, dispatch:
- CR/DX/RG/PX/IO/MG → `xray`
- US → `us`
- NM → `nm` (Radiopharmaceutical → `ce` when set)
- SC → `sc`
- OP/XC → `photo`
- ES → `endoscopy`
- RTIMAGE/RTSTRUCT/RTDOSE/RTPLAN → matching lowercase format
- OT → `ot`
- unknown → warn and fall back to `modality.lower()`
so the pipeline can complete instead of aborting mid-tree.
* Two helpers extract the DICOM tags that actually differentiate these
files in a session:
- `_apply_view_keys` lifts `ViewPosition` into `acq` (only overwriting
a None/iso plane) and `ImageLaterality`/`Laterality` into `part`,
with ophthalmic OS/OD normalised to L/R.
- `_apply_bodypart_key` lifts `BodyPartExamined` into `desc` when
empty, matching common values (ABDOMEN, PELVIS, CHEST, HEAD…).
Called from CT, PET, and every new modality branch so files that only
differ by view (MG four-view screening), laterality (ophthalmic),
tracer (NM), or body region no longer collapse onto identical BIDS
filenames.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Benchmark comparisonSpeed (wall time per call)baseline Showing the 5 most-changed measurements per case; the rest are collapsed. Rows with a baseline below 3 ms are tagged
|
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
poi_calc_poi_from_subreg_vert |
10.27 ±0.10 | 10.63 ±0.40 | +3.5% | 0.037 |
poi_calc_centroids_nocrop |
5.43 ±0.04 | 5.58 ±0.02 | +2.7% | 0.000 |
nii_load_img |
12.72 ±0.09 | 12.86 ±0.07 | +1.1% | 0.228 |
nii_save |
10.57 ±0.06 | 10.68 ±0.08 | +1.0% | 0.087 |
nii_dilate_msk |
101.18 ±0.12 | 102.20 ±0.23 | +1.0% | 0.000 |
… 31 more measurements
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_fill_holes |
4.48 ±0.03 | 4.46 ±0.08 | -0.4% | 0.616 |
nii_rescale |
42.85 ±0.23 | 42.68 ±0.98 | -0.4% | 0.393 |
nii_filter_connected_components |
4.16 ±0.03 | 4.15 ±0.02 | -0.2% | 0.542 |
nii_erode_msk |
6.03 ±0.03 | 6.04 ±0.05 | +0.1% | 0.877 |
nii_resample_from_to |
44.00 ±0.16 | 44.04 ±0.22 | +0.1% | 0.118 |
metric_voxels |
250463.00 ±0.00 | 250463.00 ±0.00 | +0.0% | — |
metric_labels |
3.00 ±0.00 | 3.00 ±0.00 | +0.0% | — |
metric_foreground_pct |
14.51 ±0.00 | 14.51 ±0.00 | +0.0% | — |
nii_set_dtype |
0.41 ±0.01 | 0.37 ±0.02 | -9.6% (noise) | 0.050 |
poi_reorient |
0.30 ±0.02 | 0.31 ±0.01 | +5.9% (noise) | 0.841 |
poi_rescale |
0.18 ±0.01 | 0.17 ±0.01 | -5.5% (noise) | 0.467 |
nii_apply_crop |
0.50 ±0.01 | 0.52 ±0.02 | +3.3% (noise) | 0.380 |
nii_compute_crop |
0.28 ±0.01 | 0.28 ±0.00 | -2.4% (noise) | 0.484 |
poi_resample_from_to |
0.51 ±0.01 | 0.52 ±0.01 | +2.3% (noise) | 0.117 |
nii_center_of_masses |
1.54 ±0.02 | 1.51 ±0.01 | -1.9% (noise) | 0.054 |
poi_to_global |
0.16 ±0.01 | 0.16 ±0.01 | -1.7% (noise) | 0.235 |
nii_get_connected_components |
2.41 ±0.03 | 2.45 ±0.01 | +1.5% (noise) | 0.316 |
nii_pad_to |
0.53 ±0.01 | 0.52 ±0.02 | -1.4% (noise) | 0.473 |
nii_rescale_seg |
2.72 ±0.06 | 2.75 ±0.14 | +1.1% (noise) | 0.337 |
poi_local_to_global_arr |
0.27 ±0.01 | 0.27 ±0.01 | -1.0% (noise) | 0.877 |
nii_reorient |
0.64 ±0.03 | 0.64 ±0.02 | -1.0% (noise) | 0.496 |
poi_load |
0.77 ±0.01 | 0.78 ±0.04 | +0.9% (noise) | 0.299 |
nii_extract_label |
0.29 ±0.02 | 0.30 ±0.01 | +0.9% (noise) | 0.524 |
nii_map_labels |
1.01 ±0.01 | 1.02 ±0.01 | +0.8% (noise) | 0.330 |
poi_save |
0.30 ±0.02 | 0.30 ±0.02 | -0.7% (noise) | 0.952 |
nii_load_seg |
2.36 ±0.03 | 2.38 ±0.03 | +0.6% (noise) | 0.453 |
nii_unique |
0.83 ±0.01 | 0.83 ±0.01 | -0.6% (noise) | 0.177 |
nii_get_array |
0.04 ±0.00 | 0.04 ±0.00 | +0.6% (noise) | 0.783 |
poi_calc_centroids |
2.06 ±0.03 | 2.07 ±0.01 | +0.4% (noise) | 0.667 |
poi_map_labels |
0.14 ±0.01 | 0.14 ±0.01 | +0.4% (noise) | 0.996 |
nii_volumes |
1.64 ±0.01 | 1.63 ±0.02 | -0.2% (noise) | 0.523 |
ct_2d — shape (73, 47, 1)
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_rescale |
5.11 ±0.06 | 5.15 ±0.05 | +0.8% | 0.630 |
nii_resample_from_to |
6.54 ±0.05 | 6.52 ±0.07 | -0.3% | 0.857 |
nii_dilate_msk |
11.66 ±0.04 | 11.67 ±0.24 | +0.1% | 0.384 |
metric_voxels |
3431.00 ±0.00 | 3431.00 ±0.00 | +0.0% | — |
metric_labels |
3.00 ±0.00 | 3.00 ±0.00 | +0.0% | — |
… 30 more measurements
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
metric_foreground_pct |
37.60 ±0.00 | 37.60 ±0.00 | +0.0% | — |
nii_unique |
0.08 ±0.00 | 0.09 ±0.01 | +10.8% (noise) | 0.062 |
nii_set_dtype |
0.31 ±0.01 | 0.29 ±0.01 | -7.7% (noise) | 0.028 |
nii_volumes |
0.20 ±0.01 | 0.21 ±0.00 | +7.4% (noise) | 0.105 |
poi_reorient |
0.34 ±0.01 | 0.32 ±0.01 | -5.1% (noise) | 0.262 |
nii_center_of_masses |
0.13 ±0.01 | 0.14 ±0.01 | +4.9% (noise) | 0.095 |
poi_map_labels |
0.16 ±0.01 | 0.15 ±0.01 | -3.7% (noise) | 0.493 |
nii_apply_crop |
0.50 ±0.02 | 0.48 ±0.02 | -3.6% (noise) | 0.240 |
nii_compute_crop |
0.10 ±0.01 | 0.10 ±0.01 | +3.5% (noise) | 0.301 |
nii_extract_label |
0.29 ±0.02 | 0.30 ±0.01 | +3.3% (noise) | 0.961 |
nii_reorient |
0.62 ±0.02 | 0.60 ±0.02 | -3.2% (noise) | 0.175 |
poi_local_to_global_arr |
0.30 ±0.01 | 0.29 ±0.01 | -3.0% (noise) | 0.339 |
nii_load_img |
1.14 ±0.01 | 1.17 ±0.02 | +2.7% (noise) | 0.027 |
nii_pad_to |
0.41 ±0.02 | 0.40 ±0.01 | -2.4% (noise) | 0.145 |
poi_calc_centroids |
0.68 ±0.01 | 0.70 ±0.01 | +2.4% (noise) | 0.042 |
poi_resample_from_to |
0.55 ±0.01 | 0.56 ±0.01 | +1.6% (noise) | 0.594 |
nii_get_array |
0.01 ±0.00 | 0.01 ±0.00 | +1.5% (noise) | 0.384 |
nii_rescale_seg |
0.56 ±0.02 | 0.57 ±0.02 | +1.5% (noise) | 0.833 |
nii_load_seg |
1.18 ±0.03 | 1.20 ±0.03 | +1.5% (noise) | 0.465 |
poi_rescale |
0.19 ±0.01 | 0.20 ±0.01 | +1.4% (noise) | 0.671 |
nii_save |
0.99 ±0.02 | 1.01 ±0.03 | +1.4% (noise) | 0.762 |
poi_calc_centroids_nocrop |
0.86 ±0.02 | 0.87 ±0.01 | +1.2% (noise) | 0.476 |
nii_fill_holes |
0.64 ±0.01 | 0.64 ±0.02 | +1.1% (noise) | 0.829 |
poi_to_global |
0.17 ±0.01 | 0.17 ±0.01 | +0.8% (noise) | 0.215 |
nii_map_labels |
0.35 ±0.01 | 0.35 ±0.01 | -0.7% (noise) | 0.907 |
nii_get_connected_components |
0.48 ±0.01 | 0.48 ±0.02 | +0.7% (noise) | 0.363 |
nii_erode_msk |
0.76 ±0.03 | 0.76 ±0.01 | +0.6% (noise) | 0.900 |
poi_save |
0.34 ±0.01 | 0.34 ±0.01 | -0.3% (noise) | 0.889 |
poi_load |
0.81 ±0.01 | 0.81 ±0.02 | +0.2% (noise) | 0.903 |
nii_filter_connected_components |
0.62 ±0.03 | 0.62 ±0.02 | +0.2% (noise) | 0.652 |
mri_3d — shape (68, 52, 67)
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_filter_connected_components |
4.36 ±0.02 | 4.46 ±0.06 | +2.3% | 0.012 |
poi_calc_centroids_nocrop |
11.49 ±0.04 | 11.28 ±0.07 | -1.8% | 0.004 |
nii_erode_msk |
7.60 ±0.04 | 7.67 ±0.04 | +0.9% | 0.040 |
nii_resample_from_to |
41.57 ±0.62 | 41.85 ±0.98 | +0.7% | 0.299 |
nii_rescale |
40.09 ±0.72 | 40.35 ±0.97 | +0.6% | 0.473 |
… 31 more measurements
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
poi_calc_poi_from_subreg_vert |
9.52 ±0.06 | 9.56 ±0.05 | +0.5% | 0.100 |
nii_save |
8.50 ±0.07 | 8.54 ±0.07 | +0.5% | 0.267 |
nii_fill_holes |
5.49 ±0.03 | 5.51 ±0.02 | +0.4% | 0.330 |
nii_load_img |
10.15 ±0.04 | 10.17 ±0.06 | +0.2% | 0.994 |
nii_dilate_msk |
132.10 ±0.32 | 132.26 ±0.53 | +0.1% | 0.671 |
metric_voxels |
236912.00 ±0.00 | 236912.00 ±0.00 | +0.0% | — |
metric_labels |
8.00 ±0.00 | 8.00 ±0.00 | +0.0% | — |
metric_foreground_pct |
18.21 ±0.00 | 18.21 ±0.00 | +0.0% | — |
nii_get_array |
0.04 ±0.00 | 0.05 ±0.00 | +15.4% (noise) | 0.000 |
nii_pad_to |
0.55 ±0.01 | 0.63 ±0.01 | +14.2% (noise) | 0.000 |
nii_extract_label |
0.35 ±0.01 | 0.38 ±0.02 | +9.4% (noise) | 0.017 |
poi_local_to_global_arr |
0.29 ±0.01 | 0.32 ±0.01 | +9.2% (noise) | 0.017 |
nii_volumes |
1.84 ±0.02 | 1.68 ±0.01 | -8.5% (noise) | 0.000 |
nii_center_of_masses |
1.88 ±0.01 | 1.73 ±0.04 | -8.3% (noise) | 0.001 |
nii_set_dtype |
0.44 ±0.01 | 0.47 ±0.02 | +7.4% (noise) | 0.027 |
poi_to_global |
0.20 ±0.00 | 0.21 ±0.01 | +4.3% (noise) | 0.035 |
poi_rescale |
0.20 ±0.00 | 0.21 ±0.02 | +4.3% (noise) | 0.121 |
poi_resample_from_to |
0.58 ±0.04 | 0.60 ±0.01 | +3.8% (noise) | 0.927 |
nii_map_labels |
1.04 ±0.01 | 1.07 ±0.01 | +3.7% (noise) | 0.003 |
nii_get_connected_components |
2.32 ±0.03 | 2.38 ±0.04 | +2.5% (noise) | 0.031 |
poi_map_labels |
0.20 ±0.01 | 0.20 ±0.01 | -2.5% (noise) | 0.224 |
nii_unique |
0.85 ±0.02 | 0.86 ±0.02 | +1.9% (noise) | 0.038 |
nii_compute_crop |
0.30 ±0.02 | 0.30 ±0.02 | -1.4% (noise) | 0.793 |
poi_calc_centroids |
2.28 ±0.01 | 2.31 ±0.02 | +1.2% (noise) | 0.041 |
nii_load_seg |
2.32 ±0.02 | 2.34 ±0.11 | +1.0% (noise) | 0.240 |
poi_load |
1.78 ±0.01 | 1.79 ±0.00 | +0.4% (noise) | 0.029 |
nii_apply_crop |
0.53 ±0.03 | 0.54 ±0.00 | +0.3% (noise) | 0.589 |
poi_reorient |
0.36 ±0.01 | 0.36 ±0.01 | +0.2% (noise) | 0.256 |
nii_reorient |
0.67 ±0.01 | 0.67 ±0.01 | +0.1% (noise) | 0.975 |
nii_rescale_seg |
2.62 ±0.04 | 2.62 ±0.07 | -0.0% (noise) | 0.704 |
poi_save |
0.40 ±0.02 | 0.40 ±0.02 | -0.0% (noise) | 0.683 |
mri_2d — shape (68, 52, 1)
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_resample_from_to |
6.53 ±0.06 | 6.62 ±0.07 | +1.3% | 0.198 |
nii_rescale |
5.14 ±0.05 | 5.18 ±0.02 | +0.7% | 0.235 |
nii_dilate_msk |
15.46 ±0.09 | 15.53 ±0.07 | +0.4% | 0.229 |
metric_voxels |
3536.00 ±0.00 | 3536.00 ±0.00 | +0.0% | — |
metric_labels |
7.00 ±0.00 | 7.00 ±0.00 | +0.0% | — |
… 30 more measurements
| Measurement | baseline ms (median ±½·range) | head ms (median ±½·range) | Δ % | p |
|---|---|---|---|---|
metric_foreground_pct |
50.17 ±0.00 | 50.17 ±0.00 | +0.0% | — |
poi_to_global |
0.21 ±0.00 | 0.23 ±0.01 | +9.2% (noise) | 0.026 |
nii_extract_label |
0.33 ±0.02 | 0.30 ±0.01 | -9.0% (noise) | 0.060 |
nii_get_array |
0.01 ±0.00 | 0.01 ±0.00 | +8.5% (noise) | 0.079 |
poi_rescale |
0.22 ±0.01 | 0.21 ±0.01 | -8.1% (noise) | 0.011 |
poi_save |
0.38 ±0.00 | 0.41 ±0.03 | +7.6% (noise) | 0.093 |
nii_get_connected_components |
0.46 ±0.02 | 0.49 ±0.01 | +7.6% (noise) | 0.065 |
nii_pad_to |
0.41 ±0.02 | 0.39 ±0.01 | -4.5% (noise) | 0.099 |
nii_unique |
0.08 ±0.00 | 0.09 ±0.01 | +4.5% (noise) | 0.181 |
nii_center_of_masses |
0.31 ±0.01 | 0.30 ±0.00 | -4.3% (noise) | 0.059 |
nii_load_seg |
1.17 ±0.02 | 1.22 ±0.03 | +4.2% (noise) | 0.004 |
nii_rescale_seg |
0.58 ±0.02 | 0.56 ±0.01 | -4.0% (noise) | 0.174 |
poi_local_to_global_arr |
0.30 ±0.01 | 0.31 ±0.02 | +3.9% (noise) | 0.283 |
nii_reorient |
0.63 ±0.01 | 0.61 ±0.03 | -3.1% (noise) | 0.065 |
nii_load_img |
1.11 ±0.01 | 1.14 ±0.03 | +3.1% (noise) | 0.030 |
nii_volumes |
0.35 ±0.01 | 0.36 ±0.01 | +1.9% (noise) | 0.758 |
nii_apply_crop |
0.48 ±0.01 | 0.47 ±0.01 | -1.4% (noise) | 0.411 |
poi_load |
1.58 ±0.02 | 1.60 ±0.02 | +1.4% (noise) | 0.308 |
nii_compute_crop |
0.10 ±0.00 | 0.10 ±0.00 | -1.3% (noise) | 0.411 |
nii_set_dtype |
0.30 ±0.02 | 0.29 ±0.01 | -1.3% (noise) | 0.137 |
poi_map_labels |
0.19 ±0.01 | 0.19 ±0.01 | +1.2% (noise) | 0.703 |
nii_fill_holes |
0.80 ±0.02 | 0.79 ±0.02 | -1.2% (noise) | 0.913 |
nii_filter_connected_components |
0.65 ±0.06 | 0.64 ±0.01 | -1.1% (noise) | 0.398 |
poi_resample_from_to |
0.59 ±0.02 | 0.60 ±0.02 | +0.7% (noise) | 0.985 |
poi_calc_centroids |
0.87 ±0.01 | 0.86 ±0.02 | -0.7% (noise) | 0.547 |
nii_save |
0.99 ±0.01 | 0.98 ±0.02 | -0.7% (noise) | 0.872 |
nii_map_labels |
0.37 ±0.01 | 0.36 ±0.01 | -0.5% (noise) | 0.606 |
nii_erode_msk |
0.97 ±0.00 | 0.97 ±0.01 | -0.4% (noise) | 0.183 |
poi_reorient |
0.35 ±0.01 | 0.35 ±0.01 | -0.3% (noise) | 0.358 |
poi_calc_centroids_nocrop |
1.09 ±0.01 | 1.09 ±0.02 | +0.1% (noise) | 0.677 |
Gate: a measurement fails when the baseline is ≥ 1 ms, the median grows by ≥ 50%, and Welch's t-test gives p < 0.05. metric_* rows are context only.
Memory (peak RSS growth per call)
baseline 68adb2a vs head 68adb2a · python 3.11.16 · 5 repeats + 1 warmup · sampler proc-status, isolation fork · measurement floor ≈ 0.95 MiB
Showing the 5 most-changed measurements per case; the rest are collapsed. Rows with a baseline below 3 MiB are tagged (noise) — runner jitter on those swamps any real change.
ct_3d — shape (73, 47, 73)
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
poi_resample_from_to |
5.59 ±0.00 | 5.49 ±0.00 | -1.7% | — |
nii_reorient |
6.03 ±0.00 | 5.93 ±0.00 | -1.6% | — |
nii_load_seg |
6.13 ±0.00 | 6.04 ±0.00 | -1.5% | — |
nii_resample_from_to |
6.61 ±0.00 | 6.51 ±0.00 | -1.5% | — |
nii_rescale |
6.48 ±0.00 | 6.39 ±0.00 | -1.4% | — |
… 31 more measurements
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_load_img |
3.62 ±0.00 | 3.58 ±0.00 | -1.2% | — |
nii_dilate_msk |
6.12 ±0.00 | 6.05 ±0.00 | -1.1% | — |
nii_set_dtype |
5.43 ±0.00 | 5.37 ±0.00 | -1.1% | — |
nii_rescale_seg |
6.25 ±0.00 | 6.19 ±0.00 | -1.1% | — |
nii_pad_to |
5.55 ±0.00 | 5.49 ±0.00 | -1.1% | — |
nii_extract_label |
5.55 ±0.00 | 5.50 ±0.00 | -1.1% | — |
nii_map_labels |
5.62 ±0.00 | 5.56 ±0.00 | -1.0% | — |
nii_apply_crop |
5.99 ±0.00 | 5.93 ±0.00 | -1.0% | — |
poi_calc_centroids_nocrop |
6.11 ±0.00 | 6.05 ±0.00 | -1.0% | — |
nii_erode_msk |
6.12 ±0.00 | 6.06 ±0.00 | -1.0% | — |
poi_calc_centroids |
6.24 ±0.00 | 6.18 ±0.00 | -0.9% | — |
nii_get_connected_components |
6.74 ±0.00 | 6.68 ±0.00 | -0.9% | — |
nii_fill_holes |
6.47 ±0.00 | 6.41 ±0.00 | -0.8% | — |
nii_filter_connected_components |
7.05 ±0.00 | 7.00 ±0.00 | -0.8% | — |
poi_calc_poi_from_subreg_vert |
7.36 ±0.00 | 7.30 ±0.00 | -0.8% | — |
nii_compute_crop |
3.02 ±0.00 | 3.02 ±0.00 | -0.3% | — |
poi_local_to_global_arr |
3.08 ±0.00 | 3.07 ±0.00 | -0.3% | — |
poi_to_global |
3.21 ±0.00 | 3.20 ±0.00 | -0.2% | — |
nii_center_of_masses |
3.27 ±0.00 | 3.27 ±0.00 | -0.2% | — |
nii_volumes |
3.59 ±0.00 | 3.58 ±0.00 | -0.2% | — |
nii_save |
3.90 ±0.00 | 3.89 ±0.00 | -0.2% | — |
poi_reorient |
3.96 ±0.00 | 3.95 ±0.00 | -0.2% | — |
metric_voxels |
250463.00 ±0.00 | 250463.00 ±0.00 | +0.0% | — |
metric_labels |
3.00 ±0.00 | 3.00 ±0.00 | +0.0% | — |
metric_foreground_pct |
14.51 ±0.00 | 14.51 ±0.00 | +0.0% | — |
poi_save |
1.53 ±0.00 | 1.52 ±0.00 | -0.5% (noise) | — |
poi_load |
1.68 ±0.00 | 1.67 ±0.00 | -0.5% (noise) | — |
nii_get_array |
1.71 ±0.00 | 1.70 ±0.00 | -0.5% (noise) | — |
poi_map_labels |
1.84 ±0.00 | 1.83 ±0.00 | -0.4% (noise) | — |
poi_rescale |
2.67 ±0.00 | 2.66 ±0.00 | -0.3% (noise) | — |
nii_unique |
2.96 ±0.00 | 2.95 ±0.00 | -0.3% (noise) | — |
ct_2d — shape (73, 47, 1)
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
poi_resample_from_to |
5.53 ±0.00 | 5.43 ±0.00 | -1.8% | — |
nii_reorient |
5.97 ±0.00 | 5.87 ±0.00 | -1.6% | — |
nii_load_seg |
6.07 ±0.00 | 5.98 ±0.00 | -1.5% | — |
nii_resample_from_to |
6.55 ±0.00 | 6.45 ±0.00 | -1.5% | — |
nii_rescale |
6.42 ±0.00 | 6.32 ±0.00 | -1.5% | — |
… 30 more measurements
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_load_img |
3.56 ±0.00 | 3.52 ±0.00 | -1.2% | — |
nii_dilate_msk |
6.05 ±0.00 | 5.98 ±0.00 | -1.2% | — |
nii_set_dtype |
5.37 ±0.00 | 5.31 ±0.00 | -1.1% | — |
nii_rescale_seg |
6.19 ±0.00 | 6.12 ±0.00 | -1.1% | — |
nii_pad_to |
5.49 ±0.00 | 5.43 ±0.00 | -1.1% | — |
nii_extract_label |
5.49 ±0.00 | 5.43 ±0.00 | -1.1% | — |
nii_map_labels |
5.55 ±0.00 | 5.50 ±0.00 | -1.1% | — |
nii_apply_crop |
5.93 ±0.00 | 5.87 ±0.00 | -1.0% | — |
nii_erode_msk |
6.06 ±0.00 | 6.00 ±0.00 | -1.0% | — |
poi_calc_centroids_nocrop |
6.11 ±0.00 | 6.05 ±0.00 | -1.0% | — |
poi_calc_centroids |
6.18 ±0.00 | 6.12 ±0.00 | -0.9% | — |
nii_get_connected_components |
6.68 ±0.00 | 6.62 ±0.00 | -0.9% | — |
nii_fill_holes |
6.41 ±0.00 | 6.35 ±0.00 | -0.9% | — |
nii_filter_connected_components |
6.99 ±0.00 | 6.93 ±0.00 | -0.8% | — |
poi_local_to_global_arr |
3.02 ±0.00 | 3.01 ±0.00 | -0.3% | — |
poi_to_global |
3.14 ±0.00 | 3.14 ±0.00 | -0.2% | — |
nii_center_of_masses |
3.21 ±0.00 | 3.20 ±0.00 | -0.2% | — |
nii_volumes |
3.52 ±0.00 | 3.52 ±0.00 | -0.2% | — |
nii_save |
3.84 ±0.00 | 3.83 ±0.00 | -0.2% | — |
poi_reorient |
3.89 ±0.00 | 3.89 ±0.00 | -0.2% | — |
metric_voxels |
3431.00 ±0.00 | 3431.00 ±0.00 | +0.0% | — |
metric_labels |
3.00 ±0.00 | 3.00 ±0.00 | +0.0% | — |
metric_foreground_pct |
37.60 ±0.00 | 37.60 ±0.00 | +0.0% | — |
poi_save |
1.46 ±0.00 | 1.46 ±0.00 | -0.5% (noise) | — |
poi_load |
1.61 ±0.00 | 1.61 ±0.00 | -0.5% (noise) | — |
nii_get_array |
1.65 ±0.00 | 1.64 ±0.00 | -0.5% (noise) | — |
poi_map_labels |
1.77 ±0.00 | 1.77 ±0.00 | -0.4% (noise) | — |
poi_rescale |
2.61 ±0.00 | 2.60 ±0.00 | -0.3% (noise) | — |
nii_unique |
2.90 ±0.00 | 2.89 ±0.00 | -0.3% (noise) | — |
nii_compute_crop |
2.96 ±0.00 | 2.95 ±0.00 | -0.3% (noise) | — |
mri_3d — shape (68, 52, 67)
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
poi_resample_from_to |
5.53 ±0.00 | 5.43 ±0.00 | -1.8% | — |
nii_reorient |
5.97 ±0.00 | 5.87 ±0.00 | -1.6% | — |
nii_load_seg |
6.07 ±0.00 | 5.98 ±0.00 | -1.5% | — |
nii_resample_from_to |
6.55 ±0.00 | 6.45 ±0.00 | -1.5% | — |
nii_rescale |
6.42 ±0.00 | 6.32 ±0.00 | -1.5% | — |
… 31 more measurements
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_load_img |
3.62 ±0.00 | 3.58 ±0.00 | -1.2% | — |
nii_dilate_msk |
6.05 ±0.00 | 5.98 ±0.00 | -1.2% | — |
nii_set_dtype |
5.43 ±0.00 | 5.37 ±0.00 | -1.1% | — |
nii_rescale_seg |
6.19 ±0.00 | 6.12 ±0.00 | -1.1% | — |
nii_pad_to |
5.49 ±0.00 | 5.43 ±0.00 | -1.1% | — |
nii_extract_label |
5.49 ±0.00 | 5.43 ±0.00 | -1.1% | — |
nii_map_labels |
5.55 ±0.00 | 5.50 ±0.00 | -1.1% | — |
nii_apply_crop |
5.93 ±0.00 | 5.87 ±0.00 | -1.0% | — |
poi_calc_centroids_nocrop |
6.05 ±0.00 | 5.99 ±0.00 | -1.0% | — |
nii_erode_msk |
6.06 ±0.00 | 6.00 ±0.00 | -1.0% | — |
poi_calc_centroids |
6.18 ±0.00 | 6.12 ±0.00 | -0.9% | — |
nii_get_connected_components |
6.68 ±0.00 | 6.62 ±0.00 | -0.9% | — |
nii_fill_holes |
6.41 ±0.00 | 6.35 ±0.00 | -0.9% | — |
nii_filter_connected_components |
6.99 ±0.00 | 6.93 ±0.00 | -0.8% | — |
poi_calc_poi_from_subreg_vert |
7.36 ±0.00 | 7.30 ±0.00 | -0.8% | — |
poi_local_to_global_arr |
3.02 ±0.00 | 3.01 ±0.00 | -0.3% | — |
poi_to_global |
3.14 ±0.00 | 3.14 ±0.00 | -0.2% | — |
nii_center_of_masses |
3.21 ±0.00 | 3.20 ±0.00 | -0.2% | — |
nii_volumes |
3.52 ±0.00 | 3.52 ±0.00 | -0.2% | — |
nii_save |
3.84 ±0.00 | 3.83 ±0.00 | -0.2% | — |
poi_reorient |
3.89 ±0.00 | 3.89 ±0.00 | -0.2% | — |
metric_voxels |
236912.00 ±0.00 | 236912.00 ±0.00 | +0.0% | — |
metric_labels |
8.00 ±0.00 | 8.00 ±0.00 | +0.0% | — |
metric_foreground_pct |
18.21 ±0.00 | 18.21 ±0.00 | +0.0% | — |
poi_save |
1.53 ±0.00 | 1.52 ±0.00 | -0.5% (noise) | — |
poi_load |
1.61 ±0.00 | 1.61 ±0.00 | -0.5% (noise) | — |
nii_get_array |
1.65 ±0.00 | 1.64 ±0.00 | -0.5% (noise) | — |
poi_map_labels |
1.77 ±0.00 | 1.77 ±0.00 | -0.4% (noise) | — |
poi_rescale |
2.61 ±0.00 | 2.60 ±0.00 | -0.3% (noise) | — |
nii_unique |
2.90 ±0.00 | 2.89 ±0.00 | -0.3% (noise) | — |
nii_compute_crop |
2.96 ±0.00 | 2.95 ±0.00 | -0.3% (noise) | — |
mri_2d — shape (68, 52, 1)
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
poi_resample_from_to |
5.53 ±0.00 | 5.43 ±0.00 | -1.8% | — |
nii_reorient |
5.97 ±0.00 | 5.87 ±0.00 | -1.6% | — |
nii_load_seg |
6.07 ±0.00 | 5.98 ±0.00 | -1.5% | — |
nii_resample_from_to |
6.55 ±0.00 | 6.45 ±0.00 | -1.5% | — |
nii_rescale |
6.42 ±0.00 | 6.32 ±0.00 | -1.5% | — |
… 30 more measurements
| Measurement | baseline MiB (median ±½·range) | head MiB (median ±½·range) | Δ % | p |
|---|---|---|---|---|
nii_load_img |
3.62 ±0.00 | 3.58 ±0.00 | -1.2% | — |
nii_dilate_msk |
6.05 ±0.00 | 5.98 ±0.00 | -1.2% | — |
nii_set_dtype |
5.43 ±0.00 | 5.37 ±0.00 | -1.1% | — |
nii_rescale_seg |
6.19 ±0.00 | 6.12 ±0.00 | -1.1% | — |
nii_pad_to |
5.49 ±0.00 | 5.43 ±0.00 | -1.1% | — |
nii_extract_label |
5.49 ±0.00 | 5.43 ±0.00 | -1.1% | — |
nii_map_labels |
5.55 ±0.00 | 5.50 ±0.00 | -1.1% | — |
nii_apply_crop |
5.93 ±0.00 | 5.87 ±0.00 | -1.0% | — |
nii_erode_msk |
6.06 ±0.00 | 6.00 ±0.00 | -1.0% | — |
poi_calc_centroids_nocrop |
6.11 ±0.00 | 6.05 ±0.00 | -1.0% | — |
poi_calc_centroids |
6.18 ±0.00 | 6.12 ±0.00 | -0.9% | — |
nii_get_connected_components |
6.68 ±0.00 | 6.62 ±0.00 | -0.9% | — |
nii_fill_holes |
6.41 ±0.00 | 6.35 ±0.00 | -0.9% | — |
nii_filter_connected_components |
6.99 ±0.00 | 6.93 ±0.00 | -0.8% | — |
poi_local_to_global_arr |
3.02 ±0.00 | 3.01 ±0.00 | -0.3% | — |
poi_to_global |
3.14 ±0.00 | 3.14 ±0.00 | -0.2% | — |
nii_center_of_masses |
3.21 ±0.00 | 3.20 ±0.00 | -0.2% | — |
nii_volumes |
3.52 ±0.00 | 3.52 ±0.00 | -0.2% | — |
nii_save |
3.84 ±0.00 | 3.83 ±0.00 | -0.2% | — |
poi_reorient |
3.89 ±0.00 | 3.89 ±0.00 | -0.2% | — |
metric_voxels |
3536.00 ±0.00 | 3536.00 ±0.00 | +0.0% | — |
metric_labels |
7.00 ±0.00 | 7.00 ±0.00 | +0.0% | — |
metric_foreground_pct |
50.17 ±0.00 | 50.17 ±0.00 | +0.0% | — |
poi_save |
1.53 ±0.00 | 1.52 ±0.00 | -0.5% (noise) | — |
poi_load |
1.61 ±0.00 | 1.61 ±0.00 | -0.5% (noise) | — |
nii_get_array |
1.65 ±0.00 | 1.64 ±0.00 | -0.5% (noise) | — |
poi_map_labels |
1.77 ±0.00 | 1.77 ±0.00 | -0.4% (noise) | — |
poi_rescale |
2.61 ±0.00 | 2.60 ±0.00 | -0.3% (noise) | — |
nii_unique |
2.90 ±0.00 | 2.89 ±0.00 | -0.3% (noise) | — |
nii_compute_crop |
2.96 ±0.00 | 2.95 ±0.00 | -0.3% (noise) | — |
Gate: a measurement fails when the baseline is ≥ 1 MiB, the median grows by ≥ 50%, and Welch's t-test gives p < 0.05. metric_* rows are context only.
Summary
Cleanup pass on
TPTBox/core/dicom/. Splits into 10 focused commits (plus the pre-existing07aac40 fix multi echo bug). Every commit is reviewable on its own.Correctness fixes
88ab7e8—_extract_nii_from_dicomreturnsFalseon unknown exceptions, deletes the half-written sidecar, and logs. Previously the bareexcept Exceptionfell through toreturn True, so_add_grid_info_to_json/_split_multi_echo_dixonthen ran on a NIfTI that was never written.391a28e—_classic_get_grouped_dicomsno longer producesNaNdirections on multi-echo series. New helper_split_by_echo_numbersruns stack detection on a single-echo subset, then re-attaches remaining echoes byImagePositionPatient.f0c70f4—_split_multi_echo_dixontriggers on any 4-D output (not justpart in ("magnitude", "phase")). Also switched frame/TE iteration tozip_strict.0894e46—_find_all_filesyieldsPathinstances one at a time instead of yielding the rawlist[Path]first, which downstream code cast tostrand broke on.7f9e38b—_inc_keycaps its retry loop at 10 000 iterations and raisesRuntimeErrorwith the offending key instead of spinning forever on a brokenpath_existscallback.a09de6a—get_plane_dicom's real call now uses0.8(matches the docstring default); the bare-except now logs a warning so downstreamacq-Nonefiles are traceable.Perf / usability
4c63431—_read_dicom_filesgates on a_looks_like_dicomcheck (extension blocklist +DICMmagic at offset 128) beforepydicom.dcmread. Skips reports, thumbnails, JSON sidecars, etc. on mixed source trees.7a7ade2— Docs:n_cpubehaviour spelled out (1 sequential / >1 N threads / None →ThreadPoolExecutor-Default ≈min(32, cpu_count + 4)).Features
e1cddbc— Expandedmap_series_description_to_file_format_default:.*t2.*/.*t1.*catch-alls that were misclassifying them.ME_vibe_fatquant_*,mevibe,fatquant,fatfrac,pdff, GEideal).survey,refscan,smartexam).5280eb8— Modality fallback + BIDS format whitelist + view/laterality/bodypart:bids_constants.formatsgrows byxray,us,nm,sc,photo,endoscopy,rtimage,rtstruct,rtdose,rtplan,ot.extract_keys_from_jsonno longer raisesNotImplementedErrorfor CR/DX/RG/PX/IO/MG/US/NM/SC/OP/XC/ES/RT*/OT — sensible defaults per DICOM modality group, with a warning + tag-based fallback for unknowns._apply_view_keysand_apply_bodypart_keyliftViewPosition,Laterality/ImageLaterality, andBodyPartExaminedintoacq/part/desc, so MG four-view screenings, bilateral radiographs, ophthalmic photos, and multi-region NM/US sessions no longer collapse onto identical filenames.Test plan
pytest unit_tests/— 595 passed, 5 skipped (unchanged from base).acq-ax(notacq-iso) and the classic stack detector no longer over-splits it.🤖 Generated with Claude Code