test(py): RFC-3 conformance driver + reference-data integration - #612
Open
vboussot wants to merge 24 commits into
Open
test(py): RFC-3 conformance driver + reference-data integration#612vboussot wants to merge 24 commits into
vboussot wants to merge 24 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughRFC-3 support adds an opt-in ChangesRFC-3 Axis Support
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Dataset
participant from_ngff_zarr
participant validate_structural
participant to_ngff_zarr
participant ZarrStore
Dataset->>from_ngff_zarr: read 1.0-DEV metadata
from_ngff_zarr->>validate_structural: validate RFC-3 structure
validate_structural-->>from_ngff_zarr: preserve axis names and order
from_ngff_zarr->>to_ngff_zarr: provide metadata and image data
to_ngff_zarr->>ZarrStore: write version-gated metadata
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
July 23, 2026 14:17
506916f to
d9fa293
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
ts/test/structural_validation_test.ts (1)
129-163: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAdd an
undefined-type case to match the Python parity test.This shape list uses
type: "custom"for the "arbitrary/optional type" case, but the Python twin (test_structural_validation.py) covers the same case withtype=None— the actual RFC-3 "type is optional" scenario (Axis.type: string | undefinedinzarr_metadata.ts). This TS suite never exercises an axis withtype: undefined.♻️ Proposed addition
// Arbitrary names and a custom type (EBSD-like). [ { name: "foo", type: "arbitrary", unit: undefined }, - { name: "bar", type: "custom", unit: undefined }, + { name: "bar", type: undefined, unit: undefined }, { name: "x", type: "space", unit: undefined }, ],🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ts/test/structural_validation_test.ts` around lines 129 - 163, Add a shape case to the `shapes` list in the structural validation test with at least one axis whose `type` is `undefined`, matching the optional-type scenario covered by the Python parity test; retain the existing custom-type case and ensure the new case is passed through `validateAxisNamesUnique`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/rfc3.md`:
- Line 7: Update the dimensionality phrase in the RFC text to use the hyphenated
compound adjective “2-to-5-dimensional,” preserving the surrounding wording.
---
Nitpick comments:
In `@ts/test/structural_validation_test.ts`:
- Around line 129-163: Add a shape case to the `shapes` list in the structural
validation test with at least one axis whose `type` is `undefined`, matching the
optional-type scenario covered by the Python parity test; retain the existing
custom-type case and ensure the new case is passed through
`validateAxisNamesUnique`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2d97b174-c8c3-4502-835c-a2261df535bc
📒 Files selected for processing (18)
docs/index.mddocs/rfc3.mdpy/examples/validate_structural_demo.pypy/ngff_zarr/spec/0.4/schemas/image.schemapy/ngff_zarr/spec/0.5/schemas/image.schemapy/ngff_zarr/structural_validation.pypy/ngff_zarr/v04/zarr_metadata.pypy/ngff_zarr/v06/zarr_metadata.pypy/test/rfc3_conformance.pypy/test/test_rfc3_axes.pypy/test/test_structural_validation.pypy/test/test_structural_validation_parity.pypy/test/test_unknown_axis_fields.pyts/src/types/zarr_metadata.tsts/src/utils/from_zarr_attrs.tsts/src/utils/structural_validation.tsts/test/structural_validation_parity_test.tsts/test/structural_validation_test.ts
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
py/test/rfc3_conformance.py (1)
215-216: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the public driver entry points.
run_data_dir()andmain()are public functions but have no docstrings.As per coding guidelines, “Include docstrings for all public Python functions and classes.”
Also applies to: 243-263
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py/test/rfc3_conformance.py` around lines 215 - 216, Document the public functions run_data_dir and main with concise docstrings describing their purpose, inputs, and behavior. Add the docstrings directly inside each function without changing their existing execution logic.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@py/test/rfc3_conformance.py`:
- Around line 193-195: Update the RFC-3 metadata validation around
declared_version and declared_axes to compare the manifest’s expected version
and complete axis objects, including each axis name, type, and unit, rather than
only names. Normalize version values before comparison as required by the
driver, and classify any version or axis-list mismatch as malformed data while
preserving valid metadata handling.
- Around line 243-247: The main argument parsing in main must accept
RFC3_DATA_DIR as an alternative to --data-dir: make --data-dir optional with the
environment variable as its default, then call parser.error() only when both
sources are absent. Preserve the existing Path conversion and downstream
data-directory behavior.
- Around line 202-204: Update the report classification flow around
_check_read_result() so it preserves the failure stage or category returned by
that check instead of assigning RFC3_METADATA to every failed post-read result.
Distinguish storage/chunk failures, including np.asarray() errors, from metadata
failures and apply the appropriate category while retaining the existing
pass/fail status behavior.
- Around line 253-263: Update the exit-status logic in the conformance driver
after the passed/failed/skipped counts so it returns success only when every
manifest case produced a pass status. Treat any skipped or otherwise non-pass
report as failure, including directories containing only skipped cases, while
preserving the existing JSON output.
- Around line 88-97: Update the metadata reader and version-normalization flow
to use one format-aware helper that checks consolidated metadata first, then
detects Zarr v2 or v3 and reads the corresponding authoritative representation.
Ensure v2 stores with version suffixes use .zattrs (including consolidated v2
metadata) instead of unconditionally opening zarr.json, while preserving the
existing local read-failure handling.
---
Nitpick comments:
In `@py/test/rfc3_conformance.py`:
- Around line 215-216: Document the public functions run_data_dir and main with
concise docstrings describing their purpose, inputs, and behavior. Add the
docstrings directly inside each function without changing their existing
execution logic.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e408a179-5078-42e9-b621-d9c65a46aae2
📒 Files selected for processing (18)
docs/index.mddocs/rfc3.mdpy/examples/validate_structural_demo.pypy/ngff_zarr/spec/0.4/schemas/image.schemapy/ngff_zarr/spec/0.5/schemas/image.schemapy/ngff_zarr/structural_validation.pypy/ngff_zarr/v04/zarr_metadata.pypy/ngff_zarr/v06/zarr_metadata.pypy/test/rfc3_conformance.pypy/test/test_rfc3_axes.pypy/test/test_structural_validation.pypy/test/test_structural_validation_parity.pypy/test/test_unknown_axis_fields.pyts/src/types/zarr_metadata.tsts/src/utils/from_zarr_attrs.tsts/src/utils/structural_validation.tsts/test/structural_validation_parity_test.tsts/test/structural_validation_test.ts
🚧 Files skipped from review as they are similar to previous changes (16)
- docs/index.md
- py/examples/validate_structural_demo.py
- ts/src/types/zarr_metadata.ts
- py/ngff_zarr/spec/0.5/schemas/image.schema
- py/ngff_zarr/spec/0.4/schemas/image.schema
- py/test/test_unknown_axis_fields.py
- py/ngff_zarr/v06/zarr_metadata.py
- ts/src/utils/from_zarr_attrs.ts
- py/ngff_zarr/v04/zarr_metadata.py
- ts/test/structural_validation_test.ts
- py/test/test_structural_validation.py
- py/ngff_zarr/structural_validation.py
- py/test/test_structural_validation_parity.py
- py/test/test_rfc3_axes.py
- ts/src/utils/structural_validation.ts
- ts/test/structural_validation_parity_test.ts
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
July 23, 2026 14:26
d9fa293 to
f52c934
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/rfc3.md`:
- Around line 84-90: Update the statement in the RFC-3 validation summary to
clarify that the three datasets read at full dimensionality, validate, and
preserve axis order only after normalizing the version from 0.5+rfc3 to 0.5.
Keep the existing explanation that the raw generator suffix is rejected and
normalization is the documented workaround.
In `@py/test/rfc3_conformance.py`:
- Around line 215-216: The public function run_data_dir in
py/test/rfc3_conformance.py:215-216 needs a docstring describing manifest
execution and the returned reports; also add a docstring to the CLI entry point
at py/test/rfc3_conformance.py:243 describing its arguments and exit-status
semantics.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 662fbb25-32b4-4f62-b5bd-5e414795d254
📒 Files selected for processing (18)
docs/index.mddocs/rfc3.mdpy/examples/validate_structural_demo.pypy/ngff_zarr/spec/0.4/schemas/image.schemapy/ngff_zarr/spec/0.5/schemas/image.schemapy/ngff_zarr/structural_validation.pypy/ngff_zarr/v04/zarr_metadata.pypy/ngff_zarr/v06/zarr_metadata.pypy/test/rfc3_conformance.pypy/test/test_rfc3_axes.pypy/test/test_structural_validation.pypy/test/test_structural_validation_parity.pypy/test/test_unknown_axis_fields.pyts/src/types/zarr_metadata.tsts/src/utils/from_zarr_attrs.tsts/src/utils/structural_validation.tsts/test/structural_validation_parity_test.tsts/test/structural_validation_test.ts
🚧 Files skipped from review as they are similar to previous changes (15)
- docs/index.md
- ts/src/types/zarr_metadata.ts
- py/ngff_zarr/v04/zarr_metadata.py
- py/ngff_zarr/spec/0.5/schemas/image.schema
- py/ngff_zarr/spec/0.4/schemas/image.schema
- py/ngff_zarr/v06/zarr_metadata.py
- ts/src/utils/from_zarr_attrs.ts
- py/test/test_rfc3_axes.py
- ts/src/utils/structural_validation.ts
- py/test/test_structural_validation.py
- py/test/test_unknown_axis_fields.py
- ts/test/structural_validation_test.ts
- py/test/test_structural_validation_parity.py
- py/ngff_zarr/structural_validation.py
- ts/test/structural_validation_parity_test.ts
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
July 23, 2026 14:31
f52c934 to
18dd6dd
Compare
This feels like something we're going to keep running into. We should probably come up with a convention that we can use for all proposed test data... @vboussot let me know if you want/need any help getting this over the line! |
Member
Author
|
@jni Agreed. One data point: for RFC-4 I didn't tag the sample data at all. It ships as plain RFC-3 can't do that, since it removes restrictions. Its data genuinely isn't valid 0.5 any more ( bioformats2raw #330 now writes Suggestion: additive RFCs keep the base version, and RFCs that relax rules use the version that adopts them. Happy to send a PR retagging the sample data if that sounds right. |
vboussot
force-pushed
the
test/rfc3-conformance
branch
2 times, most recently
from
July 27, 2026 14:47
ae6c6b9 to
3a7cfdc
Compare
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
py/test/test_structural_validation_parity.py (1)
75-85: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winAdd the uniqueness rule to the expected evaluation order.
validate_structural()evaluatesAXIS_NAMES_UNIQUEafter spatial-axis order, but this list skips it. The parity/order assertion will diverge from the validator.Proposed fix
SpecRule.AXIS_ORDER, # validate_axis_order (class ordering) SpecRule.AXIS_ORDER, # validate_spatial_axis_order (spatial suffix) + SpecRule.AXIS_NAMES_UNIQUE, SpecRule.GLOBAL_COORD_TRANSFORM_AFTER_PER_LEVEL, # per-dataset scale count🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py/test/test_structural_validation_parity.py` around lines 75 - 85, Add SpecRule.AXIS_NAMES_UNIQUE to EXPECTED_EVALUATION_ORDER immediately after the spatial-axis-order AXIS_ORDER entry, matching validate_structural() before subsequent validation rules.ts/src/io/upgrade_ome_zarr_common.ts (1)
185-196: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftImplement the supported v0.4-to-v3 in-place upgrade path.
This rejects all cross-format in-place upgrades, including
0.4 → 0.5,0.6, and1.0-DEV. The Python implementation safely rewrites v2 metadata to v3 with v2 chunk-key encoding while preserving chunks, so the TypeScript API now has a materially weaker upgrade contract despite claiming alignment. Implement the equivalent guarded migration or explicitly narrow the public feature contract.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ts/src/io/upgrade_ome_zarr_common.ts` around lines 185 - 196, Update the cross-format branch in the upgrade flow around sourceZarrFormat and targetZarrFormat to support the guarded v0.4-to-v3 in-place migration, rewriting v2 metadata to v3 while preserving existing v2 chunk-key encoding and chunks. Keep unsupported cross-format transitions rejected, or explicitly narrow the exposed contract so it no longer claims parity for them.
🧹 Nitpick comments (3)
ts/test/write_gate_test.ts (1)
86-96: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueThis test doesn't actually prove the unique-name rule fires below 1.0-DEV.
[y, y, x]tripsvalidateSpatialAxisOrderfirst at 0.4/0.5/0.6 (spatial names must be the suffix of(z, y, x)), so only the 1.0-DEV iteration exercisesAxisNamesUnique. Assert the repeated-name message text to keep the intent explicit.♻️ Sharpen the assertion
- const axes = [space("y"), space("y"), space("x")]; + // Suffix-of-(z, y, x) compliant apart from the repetition, so the unique-name + // rule is the one that trips at every version. + const axes = [space("y"), space("y"), space("x")]; for (const version of [...PRE_RFC3, "1.0-DEV"] as TargetVersion[]) { const error = assertThrows( () => buildRootAttributes(buildMetadata(axes), version), Error, ); assertStringIncludes(error.message, "Cannot write OME-Zarr"); + if (version === "1.0-DEV") { + assertStringIncludes(error.message, "is repeated"); + } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ts/test/write_gate_test.ts` around lines 86 - 96, Update the repeated-axis test around buildRootAttributes to use axis ordering that passes validateSpatialAxisOrder for every targeted version, then assert the specific repeated-name error message produced by AxisNamesUnique. Keep coverage for PRE_RFC3 and 1.0-DEV while making each iteration verify the unique-name rule rather than only the generic write-gate error.py/ngff_zarr/upgrade_ome_zarr.py (1)
206-218: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUpdate the version docstrings
_normalize_target_versionand_validate_target_versionstill describe the supported target set as only 0.4/0.5/0.6.1.0-DEVis accepted too, so both docstrings should mention it to avoid confusion.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py/ngff_zarr/upgrade_ome_zarr.py` around lines 206 - 218, Update the docstrings of _normalize_target_version and _validate_target_version to explicitly include 1.0-DEV in the supported target-version set, matching the accepted behavior and existing validation message; leave the implementation unchanged.py/ngff_zarr/v04/zarr_metadata.py (1)
344-374: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDeclare the newly supported v1.0-DEV conversion paths. Runtime dispatch accepts
Metadata_v10, but the publicto_version()return unions andfrom_version()input unions exclude it.
py/ngff_zarr/v04/zarr_metadata.py#L344-L374: include forward-referencedMetadata_v10in both conversion annotations.py/ngff_zarr/v05/zarr_metadata.py#L30-L61: include forward-referencedMetadata_v10in both conversion annotations.py/ngff_zarr/v06/zarr_metadata.py#L202-L230: include forward-referencedMetadata_v10in both conversion annotations.As per coding guidelines,
**/*.py: “Use type hints in Python, especially for public APIs.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py/ngff_zarr/v04/zarr_metadata.py` around lines 344 - 374, Update the public conversion annotations for to_version() and from_version() in py/ngff_zarr/v04/zarr_metadata.py (lines 344-374), py/ngff_zarr/v05/zarr_metadata.py (lines 30-61), and py/ngff_zarr/v06/zarr_metadata.py (lines 202-230) to include the forward-referenced Metadata_v10 type in the relevant return and input unions, matching the existing runtime v1.0-DEV conversion support.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@py/ngff_zarr/__init__.py`:
- Line 7: Replace the relative imports in py/ngff_zarr/__init__.py lines 7-7 and
76-76 with absolute imports from ngff_zarr._supported_versions and
ngff_zarr.upgrade_ome_zarr respectively. Also update the NgffVersion import in
py/ngff_zarr/structural_validation.py lines 77-77 to use
ngff_zarr._supported_versions, preserving the standard library/third-party/local
import grouping.
In `@py/ngff_zarr/to_ngff_zarr.py`:
- Around line 453-467: Update _axis_views to distinguish flat axes metadata from
1.0-DEV coordinate-system metadata before using metadata.axes. Prefer the
coordinateSystems collection when present and return an _AxisView entry for
every system; retain the existing flat multiscales[0].axes behavior only when no
coordinate systems are available.
In `@py/ngff_zarr/v10/zarr_metadata.py`:
- Around line 99-123: Update _filter_axis_dict to assign a default type of None
to the filtered axis mapping when the input omits type, before it is passed to
v0.6 Axis construction. Preserve existing required-name validation and
unknown-field filtering, and use setdefault so explicitly provided types remain
unchanged.
- Around line 354-360: Update the legacy flat-axis handling around the
coordinateSystems normalization loop to also normalize v0.5-style
scale/translation metadata lacking a transform output before delegating to the
v0.6 reader. Route this shape through the existing v0.5 reader and convert its
result to v1.0-DEV, or synthesize the required RFC-5 transform identifiers so
coordinateTransformations[0].output.name is always available.
In `@py/test/rfc3_conformance.py`:
- Around line 24-31: The module docstring lists an unsupported-operation
classification that the implementation never produces. Update the documentation
near the classification constants and _classify_read_failure to remove
unsupported-operation, or implement and return the category consistently;
preserve the existing VERSION_STRING, RFC3_METADATA, and STORAGE
classifications.
- Around line 157-163: Update the validate_structural invocation in the RFC-3
conformance test to validate multiscales.metadata at the RFC-3 version rather
than downgrading it to 0.4. Preserve the existing result recording and failure
classification logic around validate_structural.
In `@ts/src/io/from_ngff_zarr.ts`:
- Line 24: Update the version dispatch in fromNgffZarr so NgffVersion.V10DEV is
recognized by the same v0.6 reader branch as the versions covered by
isV06Version(), routing 1.0-DEV inputs through the v0.6 attribute reader instead
of fromZarrAttrsV04 while preserving existing handling for all other versions.
In `@ts/src/types/zarr_metadata.ts`:
- Around line 24-27: Update the Axis interface to accept the full RFC-3 contract
by changing name from SupportedDims to string and making type an optional string
property. Keep version-specific legacy restrictions in the existing structural
validation rather than enforcing them in the TypeScript type.
---
Outside diff comments:
In `@py/test/test_structural_validation_parity.py`:
- Around line 75-85: Add SpecRule.AXIS_NAMES_UNIQUE to EXPECTED_EVALUATION_ORDER
immediately after the spatial-axis-order AXIS_ORDER entry, matching
validate_structural() before subsequent validation rules.
In `@ts/src/io/upgrade_ome_zarr_common.ts`:
- Around line 185-196: Update the cross-format branch in the upgrade flow around
sourceZarrFormat and targetZarrFormat to support the guarded v0.4-to-v3 in-place
migration, rewriting v2 metadata to v3 while preserving existing v2 chunk-key
encoding and chunks. Keep unsupported cross-format transitions rejected, or
explicitly narrow the exposed contract so it no longer claims parity for them.
---
Nitpick comments:
In `@py/ngff_zarr/upgrade_ome_zarr.py`:
- Around line 206-218: Update the docstrings of _normalize_target_version and
_validate_target_version to explicitly include 1.0-DEV in the supported
target-version set, matching the accepted behavior and existing validation
message; leave the implementation unchanged.
In `@py/ngff_zarr/v04/zarr_metadata.py`:
- Around line 344-374: Update the public conversion annotations for to_version()
and from_version() in py/ngff_zarr/v04/zarr_metadata.py (lines 344-374),
py/ngff_zarr/v05/zarr_metadata.py (lines 30-61), and
py/ngff_zarr/v06/zarr_metadata.py (lines 202-230) to include the
forward-referenced Metadata_v10 type in the relevant return and input unions,
matching the existing runtime v1.0-DEV conversion support.
In `@ts/test/write_gate_test.ts`:
- Around line 86-96: Update the repeated-axis test around buildRootAttributes to
use axis ordering that passes validateSpatialAxisOrder for every targeted
version, then assert the specific repeated-name error message produced by
AxisNamesUnique. Keep coverage for PRE_RFC3 and 1.0-DEV while making each
iteration verify the unique-name rule rather than only the generic write-gate
error.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ee42c4f0-79ef-497b-b668-a1edbce09dc7
📒 Files selected for processing (34)
docs/index.mddocs/rfc3.mddocs/validation/parity.mddocs/validation/rule-reference.mdpy/ngff_zarr/__init__.pypy/ngff_zarr/_supported_versions.pypy/ngff_zarr/from_ngff_zarr.pypy/ngff_zarr/structural_validation.pypy/ngff_zarr/to_ngff_zarr.pypy/ngff_zarr/upgrade_ome_zarr.pypy/ngff_zarr/v04/zarr_metadata.pypy/ngff_zarr/v05/zarr_metadata.pypy/ngff_zarr/v06/zarr_metadata.pypy/ngff_zarr/v10/__init__.pypy/ngff_zarr/v10/zarr_metadata.pypy/ngff_zarr/validate.pypy/test/rfc3_conformance.pypy/test/test_rfc3_axes.pypy/test/test_structural_validation.pypy/test/test_structural_validation_parity.pypy/test/test_unknown_axis_fields.pyts/src/io/from_ngff_zarr-browser.tsts/src/io/from_ngff_zarr.tsts/src/io/to_ngff_zarr-browser.tsts/src/io/to_ngff_zarr.tsts/src/io/to_ngff_zarr_ozx_common.tsts/src/io/upgrade_ome_zarr_common.tsts/src/types/supported_versions.tsts/src/types/zarr_metadata.tsts/src/utils/from_zarr_attrs.tsts/src/utils/structural_validation.tsts/test/structural_validation_parity_test.tsts/test/structural_validation_test.tsts/test/write_gate_test.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- docs/index.md
- ts/src/utils/from_zarr_attrs.ts
- py/test/test_unknown_axis_fields.py
- py/test/test_structural_validation.py
vboussot
force-pushed
the
test/rfc3-conformance
branch
2 times, most recently
from
July 28, 2026 09:23
e2e6992 to
c417b70
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
py/test/test_v10_metadata.py (1)
30-33: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winSkip marker deviates from the established Zarr v3 gate convention.
Based on learnings from prior PRs in this repo, Zarr v3 skip markers should use
version.parse(zarr.__version__) < version.parse("3.0.0b1"), not ahasattrduck-type check, to stay consistent withtest_structural_validation_reader.py,test_to_ngff_zarr_rfc2_zarr_v3.py,test_convert_ome_zarr_version.py, andtest_cli_orientation.py.♻️ Proposed fix
+from packaging import version zarr_v3 = pytest.mark.skipif( - not hasattr(zarr.storage, "LocalStore"), + version.parse(zarr.__version__) < version.parse("3.0.0b1"), reason="OME-Zarr 1.0-DEV is a Zarr v3 hierarchy", )Based on learnings, "use the skip-gate convention
version.parse(zarr.__version__) < version.parse("3.0.0b1")" for Zarr v3 gating inpy/test/.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py/test/test_v10_metadata.py` around lines 30 - 33, Update the zarr_v3 skip marker to gate on zarr.__version__ using version.parse, skipping when it is less than version.parse("3.0.0b1"). Replace the current LocalStore hasattr check and follow the established Zarr v3 convention used by the referenced tests.Source: Learnings
py/ngff_zarr/upgrade_ome_zarr.py (1)
16-28: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winModule docstring doesn't mention 1.0-DEV as a supported target.
"Every supported transition (0.4<->0.5<->0.6) works in this mode" and the Zarr v2↔v3 boundary example ("0.4->0.5 or 0.4->0.6") are now stale:
_validate_target_versionand the dispatch logic inupgrade_ome_zarrboth accept"1.0-DEV"as a valid target for in-place and write-to-new-store upgrades.📝 Proposed doc update
-In-place upgrades that cross the Zarr v2<->v3 boundary in the *upgrade* -direction (OME-Zarr 0.4->0.5 or 0.4->0.6) are also metadata-only: each array's +In-place upgrades that cross the Zarr v2<->v3 boundary in the *upgrade* +direction (OME-Zarr 0.4->0.5, 0.4->0.6, or 0.4->1.0-DEV) are also metadata-only: each array's Zarr v3 ``zarr.json`` is given a ``v2`` chunk-key encoding matching the source separator so the existing chunk binaries resolve unchanged (see :func:`_rewrite_v2_group_to_v3`). The reverse, an in-place *downgrade* across the boundary (0.5/0.6->0.4), cannot preserve chunk keys and is rejected with a clear ``ValueError``; pass an ``output`` store instead. **Write-to-new-store conversion.** When an `output` store distinct from `input` is given, the source is read lazily and re-written to `output` at the requested version through the standard, tested write pipeline. Every supported transition -(0.4<->0.5<->0.6) works in this mode. Array data streams from the source; the +(0.4<->0.5<->0.6<->1.0-DEV) works in this mode. Array data streams from the source; the source store is never erased.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py/ngff_zarr/upgrade_ome_zarr.py` around lines 16 - 28, Update the module docstring describing supported upgrade targets and transitions to include 1.0-DEV wherever applicable, including the boundary examples and write-to-new-store transition list. Keep the existing conversion behavior and clarify that 1.0-DEV is supported by the same in-place and output-store dispatch paths.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@py/test/rfc3_conformance.py`:
- Around line 229-231: The group metadata read in _read_group_attrs currently
occurs outside the error-handling path, allowing invalid JSON or I/O failures to
abort the conformance run. Move or wrap the _read_group_attrs call and related
report population in handling that catches json.JSONDecodeError as
malformed-data and OSError as storage, records the classification for the case,
and allows later cases to continue.
- Around line 56-85: The RFC3 conformance manifest and _check_read_result()
currently validate only dimensionality, allowing incorrect axis extents. Add an
expected shape entry for each MANIFEST dataset, then update _check_read_result()
to compare image.data.shape against that manifest value while retaining the
existing dimensionality checks.
- Around line 261-262: Update the VERSION_STRING branch in the report-generation
flow to carry manifest metadata failures from meta_problems into the result
returned by _version_normalized(path, case). Ensure retagged reports remain
invalid when declared axis type or unit metadata is malformed, while preserving
existing dimension and structural validation.
---
Nitpick comments:
In `@py/ngff_zarr/upgrade_ome_zarr.py`:
- Around line 16-28: Update the module docstring describing supported upgrade
targets and transitions to include 1.0-DEV wherever applicable, including the
boundary examples and write-to-new-store transition list. Keep the existing
conversion behavior and clarify that 1.0-DEV is supported by the same in-place
and output-store dispatch paths.
In `@py/test/test_v10_metadata.py`:
- Around line 30-33: Update the zarr_v3 skip marker to gate on zarr.__version__
using version.parse, skipping when it is less than version.parse("3.0.0b1").
Replace the current LocalStore hasattr check and follow the established Zarr v3
convention used by the referenced tests.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b913a6f6-990c-429d-a22a-0bf2b1377654
📒 Files selected for processing (35)
docs/index.mddocs/rfc3.mddocs/validation/parity.mddocs/validation/rule-reference.mdpy/ngff_zarr/__init__.pypy/ngff_zarr/_supported_versions.pypy/ngff_zarr/from_ngff_zarr.pypy/ngff_zarr/structural_validation.pypy/ngff_zarr/to_ngff_zarr.pypy/ngff_zarr/upgrade_ome_zarr.pypy/ngff_zarr/v04/zarr_metadata.pypy/ngff_zarr/v05/zarr_metadata.pypy/ngff_zarr/v06/zarr_metadata.pypy/ngff_zarr/v10/__init__.pypy/ngff_zarr/v10/zarr_metadata.pypy/ngff_zarr/validate.pypy/test/rfc3_conformance.pypy/test/test_rfc3_axes.pypy/test/test_structural_validation.pypy/test/test_structural_validation_parity.pypy/test/test_unknown_axis_fields.pypy/test/test_v10_metadata.pyts/src/io/from_ngff_zarr-browser.tsts/src/io/from_ngff_zarr.tsts/src/io/to_ngff_zarr-browser.tsts/src/io/to_ngff_zarr.tsts/src/io/to_ngff_zarr_ozx_common.tsts/src/io/upgrade_ome_zarr_common.tsts/src/types/supported_versions.tsts/src/types/zarr_metadata.tsts/src/utils/from_zarr_attrs.tsts/src/utils/structural_validation.tsts/test/structural_validation_parity_test.tsts/test/structural_validation_test.tsts/test/write_gate_test.ts
🚧 Files skipped from review as they are similar to previous changes (28)
- docs/index.md
- py/ngff_zarr/v10/init.py
- py/ngff_zarr/init.py
- ts/src/io/from_ngff_zarr-browser.ts
- ts/src/utils/from_zarr_attrs.ts
- py/ngff_zarr/v06/zarr_metadata.py
- ts/src/io/to_ngff_zarr-browser.ts
- docs/validation/parity.md
- ts/src/io/to_ngff_zarr.ts
- py/test/test_structural_validation_parity.py
- ts/src/io/upgrade_ome_zarr_common.ts
- py/ngff_zarr/_supported_versions.py
- ts/src/types/zarr_metadata.ts
- py/test/test_structural_validation.py
- py/ngff_zarr/to_ngff_zarr.py
- py/ngff_zarr/validate.py
- ts/src/types/supported_versions.ts
- py/test/test_unknown_axis_fields.py
- ts/test/structural_validation_test.ts
- docs/validation/rule-reference.md
- py/ngff_zarr/v04/zarr_metadata.py
- ts/test/structural_validation_parity_test.ts
- ts/src/io/to_ngff_zarr_ozx_common.ts
- py/ngff_zarr/structural_validation.py
- py/ngff_zarr/from_ngff_zarr.py
- py/test/test_rfc3_axes.py
- py/ngff_zarr/v10/zarr_metadata.py
- ts/src/utils/structural_validation.ts
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
July 28, 2026 14:01
c417b70 to
df7810a
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds opt-in support for OME-Zarr 1.0-DEV (RFC-3 axis model) across the Python and TypeScript implementations, plus an integration-style conformance driver that can be run locally against the official RFC-3 reference datasets without vendoring or downloading data.
Changes:
- Introduces
1.0-DEVas a supported (opt-in) version and makes structural axis validation version-aware (axis-count/type/order rules become inert only at1.0-DEV, while axis-name uniqueness remains enforced). - Adds write-gating in both languages to refuse serializing RFC-3 axis models to pre-
1.0-DEVtargets, with targeted tests covering the gate and structural-validation parity. - Adds RFC-3 documentation and an opt-in Python conformance driver + reference-data integration (via
RFC3_DATA_DIR/--data-dir).
Reviewed changes
Copilot reviewed 36 out of 36 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| ts/test/write_gate_test.ts | New tests covering the TS writer gate behavior across versions, including a 1.0-DEV round-trip path selection. |
| ts/test/structural_validation_test.ts | Adds tests for axis-name uniqueness and version-aware axis-rule enforcement. |
| ts/test/structural_validation_parity_test.ts | Updates canonical rule-id list to include axis-names-unique. |
| ts/src/utils/structural_validation.ts | Makes axis rules version-aware, adds validateAxisNamesUnique, and enforces min space-axis count (pre-1.0-DEV). |
| ts/src/utils/from_zarr_attrs.ts | Treats missing axis type as optional (keeps undefined instead of "undefined"). |
| ts/src/types/zarr_metadata.ts | Makes axis type optional in the TS metadata model. |
| ts/src/types/supported_versions.ts | Adds NgffVersion.V10DEV, includes it in supported versions, and adds RFC-3 axis-model allowance helper. |
| ts/src/io/upgrade_ome_zarr_common.ts | Extends upgrade APIs to accept "1.0-DEV" as a target version. |
| ts/src/io/to_ngff_zarr.ts | Extends writer options to accept "1.0-DEV". |
| ts/src/io/to_ngff_zarr-browser.ts | Extends browser-writer options to accept "1.0-DEV". |
| ts/src/io/to_ngff_zarr_ozx_common.ts | Adds TS write gate (shared by writers/upgrader) and writes ome.version="1.0-DEV" for 1.0-DEV targets. |
| ts/src/io/from_ngff_zarr.ts | Extends reader options to accept "1.0-DEV" and routes 1.0-DEV stores through the v0.6 reader shape. |
| ts/src/io/from_ngff_zarr-browser.ts | Extends browser-reader options to accept "1.0-DEV". |
| py/test/test_v10_metadata.py | New tests for Python 1.0-DEV metadata reading, conversion, and schema-reporting behavior. |
| py/test/test_unknown_axis_fields.py | Updates expectation: missing axis type is optional and should read as None. |
| py/test/test_structural_validation.py | Adds Python tests for axis-name uniqueness validation. |
| py/test/test_structural_validation_parity.py | Updates canonical rule-id list to include axis-names-unique. |
| py/test/test_rfc3_axes.py | New end-to-end Python tests for RFC-3 axis preservation, version-gated writing, and opt-in reference-data integration. |
| py/test/rfc3_conformance.py | New opt-in RFC-3 conformance driver against the official generated reference datasets. |
| py/ngff_zarr/validate.py | Explicitly reports missing JSON Schema availability for 1.0-DEV via NotImplementedError. |
| py/ngff_zarr/v10/zarr_metadata.py | New Python metadata model for 1.0-DEV (RFC-3) with permissive axis parsing and v0.6 structural compatibility. |
| py/ngff_zarr/v10/init.py | Adds the v10 Python package marker. |
| py/ngff_zarr/v06/zarr_metadata.py | Adds conversion paths between v0.6 and v1.0-DEV. |
| py/ngff_zarr/v05/zarr_metadata.py | Adds conversion paths between v0.5 and v1.0-DEV. |
| py/ngff_zarr/v04/zarr_metadata.py | Makes axis type optional on read, updates Axis typing, and adds conversion to v1.0-DEV. |
| py/ngff_zarr/upgrade_ome_zarr.py | Allows upgrading to 1.0-DEV and adds axis-model gating before destructive rewrites. |
| py/ngff_zarr/to_ngff_zarr.py | Accepts 1.0-DEV as a target and adds axis-model write gating aligned with structural rules. |
| py/ngff_zarr/structural_validation.py | Adds RFC-3 version-awareness to axis rules and introduces axis-name uniqueness rule. |
| py/ngff_zarr/multiscales.py | Extends NgffMultiscales.metadata union type to include v1.0-DEV metadata. |
| py/ngff_zarr/from_ngff_zarr.py | Adds 1.0-DEV read path and avoids downgrading 1.0-DEV metadata to v0.6 post-read. |
| py/ngff_zarr/_supported_versions.py | Adds NgffVersion.V10DEV and includes it in supported versions. |
| py/ngff_zarr/init.py | Re-exports NgffVersion from the package top-level. |
| docs/validation/rule-reference.md | Updates rule reference ordering and adds axis-names-unique, noting 1.0-DEV inertness for other axis rules. |
| docs/validation/parity.md | Updates canonical rule ordering to include axis-names-unique. |
| docs/rfc3.md | New documentation page describing RFC-3 support, limitations, and reference-data workflow. |
| docs/index.md | Adds RFC-3 to the docs index and feature list. |
Suppressed comments (1)
ts/src/utils/structural_validation.ts:376
- Same as above: this new
count < 2branch hard-codesv0.4, but it triggers for v0.5/v0.6 too (when not targeting 1.0-DEV). The message should be version-agnostic or mention the full applicable range.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
vboussot
force-pushed
the
test/rfc3-conformance
branch
2 times, most recently
from
August 4, 2026 12:19
988db23 to
382ecf7
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (1)
py/ngff_zarr/multiscales.py (1)
15-15: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse an absolute package import.
Replace the relative import with
from ngff_zarr.v10.zarr_metadata import Metadata as Metadata_v10. Apply the same import style consistently within this import block.As per coding guidelines, Python code must use absolute imports.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@py/ngff_zarr/multiscales.py` at line 15, Update the Metadata_v10 import in the multiscales module to use the absolute ngff_zarr.v10.zarr_metadata path, and apply absolute package-import style consistently to the surrounding import block.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/rfc3.md`:
- Around line 110-114: Update the RFC3 verification statement to explicitly say
that full-dimensional reads, structural validation, and axis-order preservation
are performed on copies retagged from the rejected 0.5+rfc3 version to 1.0-DEV.
Clarify that the raw datasets do not all pass these checks under their original
version declaration, while preserving the listed dataset examples and outcomes.
In `@py/test/rfc3_conformance.py`:
- Around line 94-105: The _read_group_attrs function must check consolidated v2
metadata in .zmetadata before falling back to .zattrs, while preserving the
existing v3 handling and checking consolidated metadata before store validation.
In the retagging logic for v2 stores around retagging symbols at
py/test/rfc3_conformance.py lines 184-200, update the matching consolidated
.zattrs metadata entry in .zmetadata as well as the standalone metadata so both
representations receive the new tag.
- Around line 63-64: Pin and verify the upstream generator revision and
reference-data extents before retaining the expected shape assertions in
py/test/rfc3_conformance.py at lines 63-64, 77-78, and 87-88; ensure each
assertion uses data generated from that pinned revision so valid upstream
updates cannot be reported as malformed-data.
- Around line 218-235: Update the normalized-check result construction around
_version_normalized() so it sets an explicit status field to PASS or FAIL based
on the read and validation outcome. Ensure the status is assigned before
returning the result and reflects any classified read, axis, or validation
problems.
In `@py/test/test_rfc3_axes.py`:
- Around line 206-209: Update the assertions around run_data_dir in the RFC-3
integration test so any report with status "fail" causes pytest to fail, while
retaining support for the valid "pass" and "skip" statuses and the
manifest-count assertion.
- Around line 22-25: Update the needs_zarr_v3 marker to compare against
packaging_version.parse("3.0.0b1") and change its reason text to state that Zarr
version >= 3.0.0b1 is required, keeping the existing skip condition and marker
structure unchanged.
In `@ts/src/utils/structural_validation.ts`:
- Around line 373-380: Update the AxisOrder validation around the count < 2
check to remove the lower-bound rejection, allowing one space axis for legacy
OME-Zarr versions. Preserve the existing upper-bound validation and rely on the
existing suffix validation to require the `(x,)` axis order for the one-axis
case.
---
Nitpick comments:
In `@py/ngff_zarr/multiscales.py`:
- Line 15: Update the Metadata_v10 import in the multiscales module to use the
absolute ngff_zarr.v10.zarr_metadata path, and apply absolute package-import
style consistently to the surrounding import block.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 0e744db7-33c0-4c87-a43c-351d1a5b4c59
📒 Files selected for processing (31)
docs/index.mddocs/rfc3.mddocs/validation/rule-reference.mdpy/ngff_zarr/__init__.pypy/ngff_zarr/_supported_versions.pypy/ngff_zarr/from_ngff_zarr.pypy/ngff_zarr/multiscales.pypy/ngff_zarr/structural_validation.pypy/ngff_zarr/to_ngff_zarr.pypy/ngff_zarr/upgrade_ome_zarr.pypy/ngff_zarr/v04/zarr_metadata.pypy/ngff_zarr/v05/zarr_metadata.pypy/ngff_zarr/v06/zarr_metadata.pypy/ngff_zarr/v10/__init__.pypy/ngff_zarr/v10/zarr_metadata.pypy/ngff_zarr/validate.pypy/test/rfc3_conformance.pypy/test/test_rfc3_axes.pypy/test/test_v10_metadata.pyts/src/io/from_ngff_zarr-browser.tsts/src/io/from_ngff_zarr.tsts/src/io/to_ngff_zarr-browser.tsts/src/io/to_ngff_zarr.tsts/src/io/to_ngff_zarr_ozx_common.tsts/src/io/upgrade_ome_zarr_common.tsts/src/types/supported_versions.tsts/src/types/units.tsts/src/types/zarr_metadata.tsts/src/utils/structural_validation.tsts/test/structural_validation_test.tsts/test/write_gate_test.ts
🚧 Files skipped from review as they are similar to previous changes (24)
- py/ngff_zarr/validate.py
- py/ngff_zarr/v05/zarr_metadata.py
- py/ngff_zarr/_supported_versions.py
- docs/validation/rule-reference.md
- ts/src/io/upgrade_ome_zarr_common.ts
- ts/test/structural_validation_test.ts
- py/ngff_zarr/v04/zarr_metadata.py
- ts/src/types/supported_versions.ts
- py/ngff_zarr/structural_validation.py
- py/ngff_zarr/v10/init.py
- py/ngff_zarr/init.py
- py/ngff_zarr/v06/zarr_metadata.py
- docs/index.md
- ts/src/io/from_ngff_zarr-browser.ts
- py/ngff_zarr/upgrade_ome_zarr.py
- ts/src/io/to_ngff_zarr.ts
- ts/test/write_gate_test.ts
- ts/src/io/from_ngff_zarr.ts
- ts/src/io/to_ngff_zarr_ozx_common.ts
- ts/src/io/to_ngff_zarr-browser.ts
- py/ngff_zarr/from_ngff_zarr.py
- py/ngff_zarr/to_ngff_zarr.py
- py/test/test_v10_metadata.py
- py/ngff_zarr/v10/zarr_metadata.py
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
August 4, 2026 12:54
382ecf7 to
cab997c
Compare
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
August 5, 2026 23:46
cab997c to
78a61c4
Compare
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
August 20, 2026 17:57
78a61c4 to
8c662d4
Compare
Member
Author
|
State of the CodeRabbit review after the rebase onto the current #611 head, checked against the code on this branch rather than against the diff the review ran on. Addressed by the commits that followed the review:
Not applied, with reasons: Absolute imports in
Keeping one spatial axis valid for legacy versions. The v0.4 The version string is now |
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
August 20, 2026 18:57
8c662d4 to
58ff6c8
Compare
Two corrections to axis metadata handling, both additive. `type` is optional. The v0.4 schema accepts an axis object that declares only `name` (`$defs.axes.items.oneOf` has a branch requiring `name` alone), but the reader raised `ValueError: Axis dictionary is missing required field 'type'`, so a schema-legal store could not be read. `_filter_axis_dict` now requires only `name` and defaults a missing `type` to `None`, and `Axis.type` widens from `AxesType` to `AxesType | None`. The TypeScript reader stops turning an absent type into the literal string "undefined", `Axis.type` becomes `AxesType | undefined`, and `validateAxisOrder` skips an axis with no type, mirroring the Python port's dict `.get()`. Axis names must be unique. RFC-3 states axis names MUST NOT repeat, and v0.6 already requires uniqueness within a coordinate system; the v0.4 schema's `uniqueItems` compares whole objects, so two axes sharing a name but differing in `unit` pass. `SpecRule.AXIS_NAMES_UNIQUE` and `validate_axis_names_unique` are added after `axis-order` in both ports, in the enum, the orchestrator, and the byte-pinned parity manifests. No existing rule is removed or renamed.
OME-Zarr 1.0-DEV is 0.6 plus RFC-3: an image may declare any number of axes, with any names, any type strings, and in any order. `NgffVersion.V10DEV` joins the enum and `SUPPORTED_VERSIONS`; `LATEST` stays `0.6.dev4`, so 1.0-DEV is opt-in and no caller's default target changes. `v10/zarr_metadata.py` defines only `Axis`, `CoordinateSystem` and `Metadata`, re-exporting every transform from v0.6. Those must be the same classes, not copies: `v06.Metadata._to_v05` dispatches on `isinstance(t, Scale)` and `isinstance(t, TransformSequence)`, so copies would fall through every branch and emit default transforms instead. `Metadata` does not subclass its v0.6 counterpart either, because `v05.Metadata.from_version` dispatches on `isinstance(metadata, Metadata_v06)` and would route a subclass through the v0.5 downgrade path. `_from_v06` re-instantiates every axis. `v06._from_v05` assigns the v0.5 axis list straight into a `CoordinateSystem`, so a v0.6 `Metadata` reached from a 0.4/0.5 store holds v0.4 `Axis` objects in the same list object; aliasing them would share mutable state with the source, drop fields under `dataclasses.asdict`, and break class-exact dataclass equality. `discrete` is read with `getattr` for the same reason. Reading is wired end to end: `from_ngff_zarr` dispatches 1.0-DEV to the v10 reader before the 0.6 branch, and the closing normalization keeps a 1.0-DEV store at 1.0-DEV instead of downgrading it to 0.6 and discarding its RFC-3 axis model. Every `to_version`/`from_version` chain gains a 1.0-DEV arm, and both the writer and `upgrade_ome_zarr` accept the new target. The CLI keeps its existing version choices. No 1.0-DEV JSON Schema is published, so no `spec/1.0-DEV` tree is bundled; `load_schema` reports that explicitly and `validate=True` on a 1.0-DEV read says the same rather than validating nothing.
The axis restrictions become version-conditional and the writer enforces them. `is_rfc3_axis_model_allowed` decides, and only `1.0-DEV` lifts the historical limits: the bundled 0.4, 0.5 *and* 0.6 axes schemas all cap the axis count at 5 and require 2-3 `space` axes, so 0.6 is RFC-5, not RFC-3. Each axis rule gains an optional `version` argument that defaults to today's behaviour, so every existing single-argument call is unchanged and the rule set, its identifiers and the fail-fast order are untouched -- the byte-pinned py/ts parity manifests do not move. `_gate_axis_model` runs those same rules against the *target* version in `_prepare_metadata`, after `to_version` and before the store is created, so a refused write never erases the destination. Reusing the validators rather than a private predicate is the point: the writer and `validate_structural` cannot disagree about what a version permits. The two `upgrade_ome_zarr` paths that bypass `_prepare_metadata` call it directly, the in-place one before `_rewrite_v2_group_to_v3` deletes the v2 sidecars. `validate_spatial_axis_order` gains the missing lower bound: every bundled axes schema states `minContains: 2`, but only the upper bound was enforced, so a metadata with fewer than two `space` axes passed validation and would have passed the gate. Type-class ordering is reported, not enforced. ngff-zarr represents vector and RGB data with the channel axis last -- itkwasm bin-shrink vector mode, dask-image, and the TIFF `S` sample axis all produce e.g. `(z, y, x, c)` -- which violates the spec's time -> channel -> space ordering. Raising would break those conversions, including RGB TIFF through the CLI, so the deviation is surfaced as a warning and the store is still written. The underlying non-conformance is pre-existing and worth fixing separately.
`NgffVersion.V10DEV` and `isRfc3AxisModelAllowed` mirror the Python members, and the eight `"0.4" | "0.5" | "0.6"` literal unions across the readers, writers and the upgrade path gain the new member -- without them 1.0-DEV is rejected at compile time before any runtime code sees it. `LATEST` stays `0.6.dev4`, as in Python. The four axis rules gain the same optional `version` argument and the same early return, so an existing single-argument call keeps today's behaviour and the rule identifiers, their evaluation order and both byte-pinned parity manifests are untouched. `validateAxisNamesUnique` takes the argument but is never inert: RFC-3 keeps unique axis names. `validateSpatialAxisOrder` gains the same missing `minContains: 2` lower bound as the Python port. `gateAxisModel` runs those rules against the target version at the top of `buildRootAttributes`, the single function behind all three TypeScript writers, and `buildRootAttributes` learns to tag a 1.0-DEV store -- unlike `"0.6"` that string is already the on-disk value, so it needs no shim. Type-class ordering warns rather than throws, matching the Python gate and for the same reason: vector and RGB data is represented with the channel axis last.
- v10 Metadata: resolve the intrinsic coordinate system with a clear ValueError instead of StopIteration, and route `axes` and `dimension_names` through it instead of assuming index zero. - ts: widen `Axis.name`/`Axis.type` to the RFC-3 free-form strings via `AxisName`/`AxisType`, dropping the `as never` casts; type the axis rules as `Pick<Metadata, "axes">` so the write gate calls them unchanged. - py,ts: the axis-rule messages no longer claim v0.4 for constraints that apply to 0.4, 0.5 and 0.6 alike. - ts: attach the misplaced `gateAxisModel` JSDoc to its function.
The bundled axes schemas state minContains: 2 for 'space'; the TS port enforced the lower bound but the Python validator only enforced the upper bound.
The development version that carries RFC-3 is 0.9.dev1. ome/ngff-spec#190 titles the specification 0.9.dev1 and its _version.schema enum holds that value; bioformats2raw#330 writes the same string. Renames the package py/ngff_zarr/v10 to v09 and its test module, the NgffVersion member V10DEV to V09dev1, and the version string in both ports. LATEST stays 0.6.dev4 and the version stays opt-in.
…is rule The v0.6 axes.schema is a oneOf: either 2 or 3 space axes, or two or more array axes. coordinate_systems.schema applies it to every coordinate system, and the write gate walks all of them. An RFC-5 array coordinate system takes the second arm and declares no space axis, so the space-axis floor must not fire on it. The floor is skipped when the axis list takes the array arm. Scoped to v0.6: the v0.4 and v0.5 schemas have no such arm and require 2 or 3 space axes unconditionally. is_v06_version mirrors the TypeScript isV06Version. It tests isinstance rather than str(): NgffVersion is a stdlib StrEnum from 3.11 and a str, Enum backport below it, and only the former renders as its value under str(), so a str()-based check returns False for enum members on the 3.10 zarr-python 2 matrix. validate_axis_names_unique states RFC-3 rule 5. No released schema carries it, so below 0.9.dev1 it is a strictness choice rather than a spec MUST of those versions.
…sion set The write gate is a second dispatcher, not the structural pass. It runs the five axis rules only, applies them to every coordinate system where the structural pass reads one flat axis list, and reports the axis class-ordering arm as a warning so channel-last vector and RGB stores still write. It runs after the target-version conversion, so it cannot refuse an axis model the downgrade has discarded. The axis-order warning names its coordinate system. The four axis rules take a version and are inert for the versions that adopt RFC-3, which both ports must agree on. CANONICAL_RFC3_VERSIONS pins that set as a literal shared by the two parity suites, asserted through the public orchestrator. axis-names-unique is absent from it: RFC-3 rule 5 applies at every version. rule-reference records the space-axis floor, the array arm, RFC-3 rule 5 as the source of axis-names-unique, and the write gate as a fourth dispatcher. The upgrade docstrings name 0.9.dev1 among the targets they accept.
upgrade_ome_zarr accepts 0.9.dev1, so the CLI in front of it offers the same set. OME publishes no JSON Schema for the version, so --validate cannot check a store at that version, which the help text states.
The README, the feature list, and the Python and TypeScript guides name the versions the library reads and writes, and the TypeScript guide reproduces the fromNgffZarr, toNgffZarr and upgradeOmeZarr option signatures. All of them now include 0.9.dev1, with its two caveats: it is opt-in, and schema validation is unavailable while OME publishes no JSON Schema for it.
…test to_version and from_version in v04, v05 and v06 each convert to and from Metadata_v09, so their unions name it. fromZarrAttrsV04 and fromZarrAttrsV06 keep the parsed axis name as a string: RFC-3 permits any name, which the closed SupportedDims union cannot express. test_rfc3_read_six_dimensional_data_values asserted ndim, shape and payload on a six-axis store, all of which test_rfc3_read_preserves_axis_order[ramp_6d] already covers.
fromZarrAttrsV06 records `0.6`, which is right for the whole 0.6 family but not for a 0.9.dev1 store: that string is the on-disk version, and callers read `metadata.version` to tell the two models apart. buildV06MultiscalesEntry serializes the first coordinate system from `metadata.axes` and the later ones verbatim, and MetadataInterface does not tie `axes` to `coordinateSystems[0].axes`. The gate reads the list the writer serializes, so every axis model that reaches disk is checked. The README lists 0.9.dev1 among the versions the Python side reads and writes, and the two upgrade guides state that a transition needs the target version to be able to express the axis model: an RFC-3 model only converts upward.
…r MUST Axis order is a spec MUST and validate_structural rejects it, so the writer rejects it too. The gate refuses exactly what the validator refuses. fideus-labs#623 normalizes generated axes to (t, c, z, y, x), so the Python pipeline produces a canonical order and a model that reaches the gate out of order comes from the caller. That normalization is Python only. Two TypeScript downsampling tests feed `(z, y, c, x)` and `(t, z, y, c, x)` to check that the channel axis survives in place, and write at 0.9.dev1, whose axis model allows the order they assert.
`(z, y, x, c)` satisfies every other axis rule: 4 axes, one channel, and the space names are the `(z, y, x)` suffix. Only the class ordering is wrong, so it reaches validate_axis_order and nothing else, and each port asserts the writer refuses it below 0.9.dev1 and accepts it at 0.9.dev1.
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
August 20, 2026 20:39
58ff6c8 to
118a1c2
Compare
The canonical rule table at the top of structural_validation.py asks maintainers to keep it in sync with SpecRule, and it lost axis-names-unique when that rule was added. Nothing detected it: neither the comment table nor docs/validation/rule-reference.md was checked against the manifest the parity test already locks. Two assertions now compare both tables to CANONICAL_SPEC_RULE_IDS, so a rule added to SpecRule without its table entries fails the suite.
RFC-3 widens `name` and `type` to arbitrary strings, but the spec-defined vocabulary is still the useful default: `AxisName` and `AxesType` are unions that keep v0.4's `SupportedDims` and v0.6's type literals alongside the free-form string, so editors complete the conventional names while an arbitrary axis stays legal. Mirrors the TypeScript port's `AxisType = AxesType | (string & Record<never, never>)`. The 0.9.dev1 axis model expands the supported count, names, types and order rather than removing a notion of validity, and the feature lists now say so. The metadata tests call `to_ome_zarr` and `from_ome_zarr`, the canonical entry points, instead of their backwards-compatible aliases.
`validateStructural` takes the OME-Zarr version, and the axis rules are inert
for the versions that adopt the RFC-3 axis model. The readers called it without
one, so `isRfc3AxisModelAllowed(undefined)` was false and every store was held
to the v0.4 axis caps.
Two documents this port can itself produce were refused on the way back in
under `validate: true`: a 0.9.dev1 store with six axes ("v0.4, v0.5 and v0.6
require between 2 and 5 axes", quoting versions the store does not claim), and
a v0.6 store whose only coordinate system takes the `array` arm of the axes
schema, which carries no `space` axis.
The v0.6 reader serves both 0.6 and 0.9.dev1, so it reads the group-level
`version` to tell them apart. The version it reports for the 0.6 family stays
`0.6`.
The write gate covered a different set of coordinate systems in each port, so the same metadata was written by one and refused by the other. Python gates after `Metadata.to_version`, which for a v0.4/v0.5 target leaves the single flat `axes` the writer serializes. TypeScript gated the version-agnostic metadata, so a coordinate system the downgrade discards blocked the write and the refusal named `multiscales[0].coordinateSystems[1].axes`, a node absent from the v0.4 document. `axisViews` now takes the target version and walks the systems only where they are serialized. Three divergences in the refusal text are gone. `ValidationError` prefixes its `message` with `Spec rule [...] violated: `, which the gate then prefixed again; it now carries `detail`, the bare text, matching Python's `exc.message`. Axes rendered through `JSON.stringify`, giving `type="space"` against Python's `type='space'` and `type=undefined` against `type=None`; both ports now render Python `repr`. `formatNameList` always single-quoted, which malforms a name containing an apostrophe, so it delegates to the same helper. The two messages are asserted byte-for-byte against a literal pinned in both suites. Reading an RFC-3 dataset also asserts axis types, which `dims` cannot carry: the EBSD fixture exists to exercise a custom `type` string, and a regression that coerced or dropped it left every assertion intact.
The rule runs fifth in both orchestrators, but neither parity suite proved it: `EXPECTED_EVALUATION_ORDER` had no entry for it, and moving the call to the front of either orchestrator left both suites green. The cascade now runs eleven stages, with stages 1 to 4 carrying a repeated `z` behind the rule they target and stage 5 isolating it, so moving the call in either direction fails. Widening `Axis.name` to `AxisName` removed the reason the TypeScript fixtures substituted `SupportedDims` members for the Python free-form names; they now use the same names, leaving the default-level check as the one adaptation. `is_v06_version` is pinned by a `str` subclass that renders as the backport's `str()` does, so the assertion bites on every interpreter rather than only on the 3.10 job where the backport is in play. The validation reference records the eleven-step cascade, the version argument both orchestrators take, and the fact that a v0.6 `array` coordinate system is exempt from the two-space-axis floor but not from the three-axis cap or the suffix check.
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
August 20, 2026 22:22
118a1c2 to
1b11d4b
Compare
`py/test/rfc3_conformance.py` drives the official RFC-3 sample datasets (`clbarnes/ome-zarr-rfc3-data`: ecg_1d, astronaut_xcy, ramp_6d) against an authored manifest and prints a per-case PASS/FAIL report. The data is generated, not checked in, so the driver is opt-in (`--data-dir` / `RFC3_DATA_DIR`), never runs in the default suite and never downloads. Each case records the declared version, Zarr format, axes and shape, then diffs what ngff-zarr does: reading, structural validation, reading every referenced array, and the scale/translation-length check. A read failure is classified -- version-string, rfc3-metadata, malformed-data, storage -- so a disagreement about the version tag is never confused with one about the axes. The reference data declares `0.5+rfc3`, which no specification defines. The driver reports a separate pass over a copy retagged `1.0-DEV`, the version that adopts RFC-3: those axis models are conformant there, whereas 0.5 rejects a 6-D image outright, so declaring `0.5` would claim conformance to a version the data violates.
- The manifest now authors the exact scale-0 shape of each reference dataset, and `_check_read_result` diffs against it rather than only the dimensionality. - Docstrings for the public `run_data_dir` and `main` entry points.
- The manifest records the ome-zarr-rfc3-data revision its ground truth describes, and the report header prints it. - The retagged 1.0-DEV check now carries its own pass/fail status. - Consolidated v2 metadata (.zmetadata) is read and retagged too. - The opt-in integration test only tolerates the undefined 0.5+rfc3 version tag, and requires the retagged copy to fully conform. - docs/rfc3.md states the checks run on the retagged copy.
The version that carries RFC-3 is 0.9.dev1, so the driver's RFC3_VERSION, the retagged-copy pass it reports, the upgrade-refusal test and the RFC-3 guide name it.
vboussot
force-pushed
the
test/rfc3-conformance
branch
from
August 21, 2026 08:12
1b11d4b to
dfa7df2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a small driver that checks the official RFC-3 sample data
(
clbarnes/ome-zarr-rfc3-data:ecg_1d,astronaut_xcy,ramp_6d) against what ngff-zarr actually does, andprints a PASS/FAIL report.
The data isn't checked in, so the driver is opt-in (
--data-dir), never runsin CI, and never downloads anything.
For each dataset it checks the version, axes, shape, reading, structural
validation and array access. Failures are classified, so a problem with the
version tag is never confused with a problem with the axes.
The version tag
The sample data declares
0.5+rfc3, which no spec defines and which ngff-zarrrejects. The driver reports a second pass over a copy retagged
1.0-DEV: a6-axis image is valid at 1.0-DEV and invalid at 0.5, so that's the correct
declaration rather than a workaround.
Also adds
docs/rfc3.md.Summary by CodeRabbit
1.0-DEV(RFC-3) support with flexible axis counts, ordering, names, and types.