-
Notifications
You must be signed in to change notification settings - Fork 17
WIP: 0.9.dev1 #190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 0.9dev
Are you sure you want to change the base?
WIP: 0.9.dev1 #190
Changes from 8 commits
9c1b163
08bab99
ce324f0
9eb0d96
ddffb3b
2ae554b
8b9b069
e042763
9df947d
e0e91fd
874c6d7
98a80c5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| __version__ = '0.6rc0' | ||
| __version__ = '0.9.dev1' |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,7 @@ | ||
| { | ||
| "multiscales": [ | ||
| { | ||
| "version": "0.6rc0", | ||
| "version": "0.9.dev1", | ||
| "name": "example", | ||
| "coordinateSystems" : [ | ||
| { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| { | ||
| "ome": { | ||
| "version": "0.6rc0", | ||
| "version": "0.9.dev1", | ||
| "multiscales": [ | ||
| { | ||
| "name": "Image", | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| { | ||
| "ome": { | ||
| "version": "0.6rc0", | ||
| "version": "0.9.dev1", | ||
| "name": "the_dfield", | ||
| "multiscales": [ | ||
| { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
| { | ||
| "ome": { | ||
| "version": "0.6rc0", | ||
| "version": "0.9.dev1", | ||
| "name": "some_image", | ||
| "multiscales": [ | ||
| { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,8 +4,8 @@ short_title: OME-Zarr | |
| authors: " " | ||
| --- | ||
|
|
||
| # Version 0.6rc0 | ||
| (ngff-spec:spec:0.6rc0)= | ||
| # Version 0.9.dev1 | ||
| (ngff-spec:spec:0.9.dev1)= | ||
|
|
||
| **Feedback:** [Forum](https://forum.image.sc/tag/ome-ngff), [Github](https://github.com/ome/ngff/issues) | ||
|
|
||
|
|
@@ -18,19 +18,18 @@ authors: " " | |
| ## Abstract | ||
|
|
||
| ```{warning} | ||
| This is the release candidate for version 0.6rc0 of the ngff-specification. | ||
| Upon release of 0.6, this warning will be removed and the version number in the document updated. | ||
|
|
||
| This is a development version of the specification. It includes RFC-3 and RFC-4 | ||
| which have not yet been accepted. | ||
| ``` | ||
|
|
||
| This document contains next-generation file format (NGFF) specifications for storing bioimaging data in the cloud. | ||
| All specifications are submitted to the <https://image.sc> community for review. | ||
|
|
||
| ## Status of This Document | ||
|
|
||
| The working title version of this specification is 0.6rc0. | ||
| The working title version of this specification is 0.9.dev1. | ||
| Migration scripts will be provided between numbered versions. | ||
| Data written with these latest changes (an "editor's draft") will not necessarily be supported. | ||
| Data written with these latest changes will not necessarily be supported. | ||
|
|
||
| The conventions and specifications defined in this document | ||
| are designed to enable next-generation file formats to represent | ||
|
|
@@ -66,7 +65,7 @@ but could equally be stored on a web server to be accessed via HTTP or in object | |
| ### Images | ||
|
|
||
| The following layout describes the expected Zarr hierarchy for images with multiple levels of resolutions and optionally associated labels. | ||
| Note that the number of dimensions is variable between 2 and 5 and that axis names are arbitrary, see [multiscales metadata](#multiscales-md) for details. | ||
| Note that the number of dimensions is variable and that axis names are arbitrary, see [multiscales metadata](#multiscales-md) for details. | ||
|
|
||
| ```text | ||
| ├── 123.zarr # One OME-Zarr image (id=123). | ||
|
|
@@ -188,14 +187,14 @@ The OME-Zarr Metadata is stored in the various `zarr.json` files throughout the | |
| The OME-Zarr Metadata version MUST be consistent within a hierarchy. | ||
|
|
||
| The group `attributes` MUST contain a key `ome`. The value of the `ome` key MUST be a JSON | ||
| object that MUST contain a `version` key, the value of which MUST be a string specifying the version of the OME-Zarr specification defined by [this document](#ngff-spec:spec:0.6rc0). | ||
| object that MUST contain a `version` key, the value of which MUST be a string specifying the version of the OME-Zarr specification defined by [this document](#ngff-spec:spec:0.9.dev1). | ||
|
|
||
| ```jsonc | ||
| { | ||
| // ... | ||
| "attributes": { | ||
| "ome": { | ||
| "version": "0.6rc0", | ||
| "version": "0.9.dev1", | ||
| // ... | ||
| } | ||
| } | ||
|
|
@@ -266,10 +265,56 @@ where each object describes a dimension (axis) and: | |
| - MAY contain the field `longName`. | ||
| The value MUST be a string, | ||
| and can provide a longer name or description of an axis and its properties. | ||
| - MAY contain an "orientation" field, itself containing a "type" and a | ||
| "value". Currently, "type" may only be "anatomical", and "value" may | ||
| only be one from the list given in [list of possible anatomical | ||
| orientation values](orientation-values). A specific orientation value or | ||
| its inverse MUST NOT appear more than once in a given coordinate system. | ||
|
|
||
| The values in the `name` fields MUST be unique within the same coordinate system. | ||
| The length of "axes" MUST be equal to the number of dimensions of the arrays that contain the image data. | ||
|
|
||
| ```{note} | ||
| :label: orientation-values | ||
|
|
||
| When a spatial axis is aligned with a specific orientation, this can be | ||
| indicated with the "orientation" field, which must have "type" | ||
| "anatomical" (with further types to be added to the specification or as | ||
| extensions in the future), and one of the values detailed below: | ||
|
|
||
| **Common to both bipeds and quadrupeds:** | ||
| - `left-to-right` | ||
| - `right-to-left` | ||
| - `proximal-to-distal` | ||
| - `distal-to-proximal` | ||
|
|
||
| **Primarily for bipeds (humans):** | ||
| - `anterior-to-posterior` (front-to-back) | ||
| - `posterior-to-anterior` (back-to-front) | ||
| - `inferior-to-superior` (feet-to-head) | ||
| - `superior-to-inferior` (head-to-feet) | ||
| - `dorsal-to-palmar` (back of hand to palm) | ||
| - `palmar-to-dorsal` (palm to back of hand) | ||
| - `dorsal-to-plantar` (top of foot to sole) | ||
| - `plantar-to-dorsal` (sole to top of foot) | ||
|
|
||
| **Primarily for quadrupeds:** | ||
| - `rostral-to-caudal` (nose/beak-to-tail, especially for nervous system) | ||
| - `caudal-to-rostral` (tail-to-nose/beak, especially for nervous system) | ||
| - `cranial-to-caudal` (head-to-tail) | ||
| - `caudal-to-cranial` (tail-to-head) | ||
| - `dorsal-to-ventral` (back/top-to-belly/bottom) | ||
| - `ventral-to-dorsal` (belly/bottom-to-back/top) | ||
|
|
||
| **For layered and polarized tissues (subject-local):** | ||
| - `superficial-to-deep` (outer surface to inner depth, e.g. skin, gut, cortex) | ||
| - `deep-to-superficial` (inner depth to outer surface) | ||
| - `apical-to-basal` (apical to basal surface, e.g. epithelial layers, polarized cells) | ||
| - `basal-to-apical` (basal to apical surface) | ||
| - `apex-to-base` (tip to broad base, e.g. heart, lungs) | ||
| - `base-to-apex` (broad base to tip) | ||
| ``` | ||
|
|
||
| Arrays are inherently discrete (see Array coordinate systems, below) | ||
| but are often used to store discrete samples of a continuous variable. | ||
| The continuous values "in between" discrete samples can be retrieved using an *interpolation* method. | ||
|
|
@@ -429,7 +474,7 @@ The following transformations are supported: | |
| | [`bijection`](#bijection-md) | `"forward":Transformation`<br>`"inverse":Transformation` | An invertible transformation providing an explicit forward transformation and its inverse. | | ||
| | [`byDimension`](#bydimension-md) | `"transformations":List[Transformation]`.<br>Transformations in the array MUST have<br>`"inputAxes": List[number]`, <br> and `"outputAxes": List[number]` | A high dimensional transformation using lower dimensional transformations on subsets of dimensions. | | ||
|
|
||
| The parameter values (e.g., `scale` for a [scale transformation](#scale-md)) MUST be compatible with input and output space dimensionality (see details). | ||
| The parameter values (e.g., `scale` for a [scale transformation](#scale-md)) MUST be compatible with input and output space dimensionality (see details). | ||
|
|
||
| The `input` and `output` fields are objects structured as follows: | ||
|
|
||
|
|
@@ -471,7 +516,7 @@ Depending on which, different constraints apply to the transformations, as descr | |
| - Both `input` and `output` MUST specify a coordinate system `name`. | ||
| - `path` is required when referencing a coordinate system in a multiscale image subgroup; | ||
| it MAY be omitted or null when referencing a coordinate system defined in the scene's own `coordinateSystems`. | ||
|
|
||
|
|
||
| In any context, the values given for `name` and `path` provide an unambiguous reference to a named coordinate system. | ||
| If the `path` field is null or omitted, this is to be interpreted as referring to a named coordinate system in the same `zarr.json` file. | ||
|
|
@@ -1107,7 +1152,7 @@ An exact reproducibility of pixel values for images transformed and resampled by | |
|
|
||
| The multiscale group at `path` MUST satisfy: | ||
| - **Dimensionality**: If the input coordinate system has `N` axes, the multiscale image at location `path` MUST have `N+1` dimensions. | ||
| - **Vector dimension length**: | ||
| - **Vector dimension length**: | ||
| - For `coordinates` transformations, the length of the array along the `coordinate` dimension (last axis) MUST equal `M`, | ||
| the number of axes in the output coordinate system. | ||
| - For `displacements` transformations, the length of the array along the `displacement` dimension (last axis) MUST equal `N`, | ||
|
|
@@ -1275,9 +1320,9 @@ the input and output of the `forward` and `inverse` transformations are understo | |
| (multiscales-md)= | ||
|
|
||
| Metadata about an image can be found under the `multiscales` key in the group-level OME-Zarr Metadata. | ||
| Here, "image" refers to 2 to 5 dimensional data representing image | ||
| or volumetric data with optional time or channel axes. | ||
| It is stored in a multiple resolution representation. | ||
| Here, "image" refers to data stored in a Zarr Array representing image, | ||
| volumetric, time lapse, or similar data. It MAY be stored in multiple | ||
| resolutions. | ||
|
|
||
| `multiscales` contains an array of objects where each entry describes a multiscale image. | ||
| Each object provides the following fields: | ||
|
|
@@ -1295,15 +1340,26 @@ Each object provides the following fields: | |
| : The `coordinateSystems` field is a JSON array containing [coordinate system metadata](#coordinate-systems-md) | ||
| The following conditions apply to all coordinate systems inside multiscales metadata: | ||
|
|
||
| - The length of `axes` must be between 2 and 5 and MUST be equal to the dimensionality of the Zarr arrays storing the image data (see `datasets:path`). | ||
| - `axes` MUST contain 2 or 3 entries of `type:space` | ||
| - `axes` MAY contain one additional entry of `type:time` | ||
| - `axes` MAY contain one additional entry of `type:channel` or a null / custom type. | ||
| - `axes` entries MUST be ordered by `type` where the `time` axis must come first (if present), | ||
| followed by the `channel` or custom axis (if present) and the axes of type `space`. | ||
| - If there are three spatial axes where two correspond to the image plane (`yx`) | ||
| and images are stacked along the other (anisotropic) axis (`z`), | ||
| the spatial axes SHOULD be ordered as `zyx`. | ||
| 0. The length of the axis names MUST match the number of axes of the array. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd be inclined towards 1-basing for ordered lists. The audience is obviously technical but in text 1-based is much more common.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ... But the wrong tick marks in point 5 should definitely be fixed 😂
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ok I fixed both the numbering and the quotes. |
||
| 1. *If* a dataset contains exactly 2 spatial dimensions, those dimensions | ||
| SHOULD be named `y` and `x`, except where rule 4 applies. | ||
| 2. *If* a dataset contains exactly 3 spatial dimensions, those dimensions | ||
| SHOULD be named 'z', 'y', and 'x', except where rule 4 applies. | ||
| 3. *If* a dataset contains exactly 1 time dimension, it should be named `t`. | ||
| 4. When image data axes map straightforwardly to axes with common names in | ||
| the relevant field of practice, those axes SHOULD be named according to | ||
| such conventions. For example, spatial frequency axes resulting from a | ||
| Fourier transformation of `z', 'y', and 'x' SHOULD be named 'w', 'v', and | ||
| `u`, respectively. Similarly, a temporal frequency axis resulting from | ||
| a Fourier transformation of the `t` axis SHOULD be named `w` or `ω`. | ||
| 5. Axis names MUST NOT be repeated within a dataset, and SHOULD NOT be | ||
| different only by upper/lower-case. For example, the same dataset SHOULD | ||
| NOT have both an `X` and an `x` axis. | ||
| 6. The order of the axes MUST match their ordering within the data if | ||
| applicable. For example, if the axes are ordered as `DZYX`, where `D` is a | ||
| field of displacement vectors, then the vectors must be ordered as `ZYX` | ||
| within the array. | ||
|
|
||
|
|
||
| ```{hint} | ||
| (spec:hint:multiscales-intrinsic-coordinate-system)= | ||
|
|
@@ -1320,8 +1376,7 @@ is the coordinate system that is referenced by all multiscale coordinate transfo | |
| - Each object in `datasets` MUST contain the field `path`, | ||
| whose value is a string containing the path to the Zarr array for this resolution relative to the current Zarr group. | ||
| - The `path`s MUST be ordered from largest (i.e. highest resolution) to smallest. | ||
| - Every Zarr array referred to by a `path` MUST have the same number of dimensions and datatype, | ||
| and MUST NOT have more than 5 dimensions. | ||
| - Every Zarr array referred to by a `path` MUST have the same number of dimensions and datatype. | ||
| - The number of dimensions and order MUST correspond to number and order of `axes`. | ||
|
|
||
| Each object in `datasets` MUST contain the field `coordinateTransformations`, | ||
|
|
@@ -1576,7 +1631,7 @@ The `zarr.json` under the `labels` group contains a JSON object with the key `la | |
| { | ||
| "attributes": { | ||
| "ome": { | ||
| "version": "0.6rc0", | ||
| "version": "0.9.dev1", | ||
| "labels": [ | ||
| "cell_segmentation" | ||
| ] | ||
|
|
@@ -1870,7 +1925,7 @@ If they do so, it is RECOMMENDED that the scene's first entry under the `coordin | |
| If no coordinate system is defined therein, but only in the respective linked multiscale groups, | ||
| viewers may want to expose a choice for the user to select a coordinate system for display when opening the dataset for the first time. | ||
|
|
||
|
|
||
|
|
||
| ``` | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,10 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "$id": "https://ngff.openmicroscopy.org/0.6rc0/schemas/_version.schema", | ||
| "$id": "https://ngff.openmicroscopy.org/0.9.dev1/schemas/_version.schema", | ||
| "title": "OME-Zarr version", | ||
| "description": "OME-Zarr version.", | ||
| "type": "string", | ||
| "enum": [ | ||
| "0.6rc0" | ||
| "0.9.dev1" | ||
| ] | ||
| } |

There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could RECOMMEND TCZYX here for backwards-compatibility, if there's any benefit to that? Or alternatively, add a
{hint}that axis ordering could affect performance of certain quries in combination with configuration of the array chunking/ sharding/ transposing.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@clbarnes I'm puzzled about the line you chose to make this comment on? I don't see why axis an orientation note would recommend anything about axis ordering?
At any rate, I think hints about performance can come in future iterations of the 0.9.devN series.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am also puzzled by my choice of line here. Maybe I fat-fingered the touchpad.