Skip to content

Commit ab5bd13

Browse files
committed
support composing input variables
1 parent 2f14f11 commit ab5bd13

File tree

7 files changed

+371
-61
lines changed

7 files changed

+371
-61
lines changed

docs/src/datahandling.md

Lines changed: 61 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ interface). For instance, if need arises, the `DataHandler` can be used (almost)
1717
directly to process files with a different format from NetCDF.
1818

1919
The key struct in `DataHandling` is the `DataHandler`. The `DataHandler`
20-
contains a `FileReader`, a `Regridder`, and other metadata necessary to perform
20+
contains one or more `FileReader`(s), a `Regridder`, and other metadata necessary to perform
2121
its operations (e.g., target `ClimaCore.Space`). The `DataHandler` can be used
2222
for static or temporal data, and exposes the following key functions:
2323
- `regridded_snapshot(time)`: to obtain the regridded field at the given `time`.
@@ -52,7 +52,16 @@ It is possible to pass down keyword arguments to underlying constructors in
5252
`DataHandler` with the `regridder_kwargs` and `file_reader_kwargs`. These have
5353
to be a named tuple or a dictionary that maps `Symbol`s to values.
5454

55-
## Example
55+
A `DataHandler` can contain information about a variable that we read directly from
56+
an input file, or about a variable that is produced by composing data from multiple
57+
input variables. In the latter case, the input variables may either all come from
58+
the same input file, or may each come from a separate input file. The user must
59+
provide the composing function, which operates pointwise on each of the inputs,
60+
as well as an ordered list of the variable names to be passed to the function.
61+
Composing multiple input variables is currently only supported with the
62+
`InterpolationsRegridder`, not with `TempestRegridder`.
63+
64+
## Example with a single data variable
5665

5766
As an example, let us implement a simple linear interpolation for a variable `u`
5867
defined in the `era5_example.nc` NetCDF file. The file contains monthly averages
@@ -68,6 +77,7 @@ import Interpolations
6877

6978
import Dates
7079

80+
# Define pre-processing function to convert units of input
7181
unit_conversion_func = (data) -> 1000. * data
7282

7383
data_handler = DataHandling.DataHandler("era5_example.nc",
@@ -93,6 +103,55 @@ function linear_interpolation(data_handler, time)
93103
end
94104
```
95105

106+
## Example with a multiple input data variables
107+
108+
As an example, let us take two input variables, `u` and `v` defined in the
109+
`era5_example.nc` NetCDF file. Suppose that we care only about the sum of these
110+
variables, rather than their individual values. We'll use a composing function
111+
and the `InterpolationsRegridder` to produce the data we want, `u + v`.
112+
The input file contains monthly averages for each variable starting from the year 2000.
113+
114+
We'll use the same linear interpolation function defined in the previous function.
115+
116+
```julia
117+
import ClimaUtilities.DataHandling
118+
import ClimaCore
119+
import NCDatasets
120+
# Loading ClimaCore and Interpolations automatically loads DataHandling
121+
import Interpolations
122+
# This will load InterpolationsRegridder
123+
124+
import Dates
125+
126+
# Define pre-processing function to convert units of input
127+
unit_conversion_func = (data) -> 1000. * data
128+
129+
# Define the pointwise composing function we want, a simple sum in this case
130+
compose_func = (x, y) -> x + y
131+
data_handler = DataHandling.DataHandler("era5_example.nc",
132+
["u", "v"],
133+
target_space,
134+
reference_date = Dates.DateTime(2000, 1, 1),
135+
regridder_type = :InterpolationsRegridder,
136+
file_reader_kwargs = (; preprocess_func = unit_conversion_func),
137+
compose_func)
138+
139+
function linear_interpolation(data_handler, time)
140+
# Time is assumed to be "simulation time", ie seconds starting from reference_date
141+
142+
time_of_prev_snapshot = DataHandling.previous_time(data_handler, time)
143+
time_of_next_snapshot = DataHandling.next_time(data_handler, time)
144+
145+
prev_snapshot = DataHandling.regridded_snaphsot(data_handler, time_of_prev_snapshot)
146+
next_snapshot = DataHandling.regridded_snaphsot(data_handler, time_of_next_snapshot)
147+
148+
# prev and next snapshots are ClimaCore.Fields defined on the target_space
149+
150+
return @. prev_snapshot + (next_snapshot - prev_snapshot) *
151+
(time - time_of_prev_snapshot) / (time_of_next_snapshot - time_of_prev_snapshot)
152+
end
153+
```
154+
96155
## API
97156

98157
```@docs

docs/src/inputs.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ regridding onto the computational domains (using [`Regridders`](@ref) and
2828
`TimeVaryingInputs` support:
2929
- analytic functions of time;
3030
- pairs of 1D arrays (for `PointSpaces`);
31-
- 2/3D NetCDF files;
31+
- 2/3D NetCDF files (including composing multiple variables from one or more files into one variable);
3232
- linear interpolation in time (default), nearest neighbors, and "period filling";
3333
- boundary conditions and repeating periodic data.
3434

@@ -153,10 +153,10 @@ albedo_tv = TimeVaryingInputs.TimeVaryingInput("cesem_albedo.nc", "alb", target_
153153
(chiefly the [`DataHandling`](@ref) module) to construct a `Field` from
154154
different sources.
155155
156-
`TimeVaryingInputs` support:
156+
`SpaceVaryingInputs` support:
157157
- analytic functions of coordinates;
158158
- pairs of 1D arrays (for columns);
159-
- 2/3D NetCDF files.
159+
- 2/3D NetCDF files (including composing multiple variables from one or more files into one variable).
160160
161161
In some ways, a `SpaceVaryingInput` can be thought as an alternative constructor
162162
for a `ClimaCore` `Field`.
@@ -202,4 +202,3 @@ ClimaUtilities.TimeVaryingInputs.extrapolation_bc
202202
Base.in
203203
Base.close
204204
```
205-

0 commit comments

Comments
 (0)