-
The
PlaceholderSpaceinfrastructure is replaced by a simplerPlaceholderGrid, which only stands in for grids in the arguments of GPU kernels launched from the host and is swapped out at the start of each kernel, so CPU and device-side broadcasts no longer pay for it. This fixes GPU compilation of some finite difference broadcasts (e.g., lazyFCTZalesakstencils), and spectral element operators now accept broadcasts that combine extruded fields with level fields (e.g.,@. gradh(field * level_field)). -
Remapping.PressureInterpolatoracceptsmethod = Remapping.LogLinearInPressure()to interpolate linearly in log pressure. The defaultRemapping.LinearInPressure()interpolates linearly in pressure.
Broadcasts over fields with different spaces now recursively traverse the entire expression tree to find the largest shared space. All input fields are checked against the shared space, with strict compatibility rules defined through
Grids.issubgridandSpaces.issubspace. Broadcast destinations can have different spaces from the inputs, as long as their sizes are consistent.
-
New spectral element operator
Operators.LumpedRestriction(quadrature_style): within each element, it restricts a field onto the nodal basis of a lower-degree quadrature with a lumped mass matrix and interpolates the result back to the element's own quadrature points, so the output lives on the input's space and composes in one broadcast with other spectral operators (e.g.@. lumped(norm_sqr(grad(f)))). The defaultGLL{2}gives the bilinear corner basis;GL{1}gives the element mean. It reproduces constants and conserves every element'sWJ-weighted integral exactly, and it removes the systematic excess of positive-definite gradient invariants likenorm_sqr(grad(f))at element edges and corners that comes from the end-of-interval error of polynomial differentiation and that DSS cannot remove. -
Reduced compilation latency (TTFX) across grid/space construction,
DataLayoutandFieldbroadcasts, spectral and finite-difference operators, andFieldVectorbroadcasts by eliminating internalCore.kwcallforwarding layers and duplicateThisThreadPoolloop specializations inDataLayouts, makingis_cpu_linear_compatiblea compile-time check inFieldVectorcopyto!, avoiding union-splitting inExtrudedFiniteDifferenceGrid, removing unusedLocalGeometryconstructions inMatrixFields.multiply_matrix_at_index, and precompiling single-threaded CPU contexts inprecompile_workload.jl. The package image also shrinks by roughly 20% (322 MB to 257 MB). -
The
copyto!methods forFields andDataLayouts, including those for spectral and column-stencil operator broadcasts, take the optionalDataMaskas a third positional argument,copyto!(dest, bc, mask), in place of themaskkeyword argument. A multi-element tuple broadcast into aDataLayoutwrites only the points that the mask marks as active. -
Broadcasting a single-element tuple into a
FieldVector, as inY .= (x,), no longer throws aDimensionMismatch. This is the formBase.broadcastableproduces for custom scalar types, so it now behaves the same asY .= Ref(x). TheFieldVector's ownBlockedOneToaxes were previously passed down to each leaf array unchanged.
Restored the device-side grid types
Grids.DeviceFiniteDifferenceGrid,Grids.DeviceExtrudedFiniteDifferenceGrid,Grids.DeviceSpectralElementGrid1D,Grids.DeviceSpectralElementGrid2DandTopologies.DeviceIntervalTopology, which v1.0.0 removed in favour of adapting the host grids into GPU kernels. Host grids are mutable again, so a grid sits behind one pointer in a space rather than inline: an extruded cubed sphere space goes from 176 bytes back to 8, and everyFieldandBroadcastednode holding one shrinks with it. Downstream code that builds operator andFieldVectorbroadcasts compiles roughly twice as fast, back to its v0.16.1 speed (perf/grid_compile_repro.jl). Run time and ClimaCore's own precompile time are unaffected.
ClimaCore's first stable release. Names that are neither exported nor documented remain internal and may change in any release. Apart from the removals listed below, v1.0.0 is API-identical to v0.16.2.
-
Removed the
ClimaCoreMakieandClimaCorePlotspackages underlib/. Both had been deprecated shims since v0.16.0: the Makie recipes live in ClimaCore'sClimaCoreMakieExtextension and the Plots recipes in itsClimaCoreRecipesBaseExtextension, and both extensions load automatically once ClimaCore and the corresponding plotting package are loaded. Drop the dependency and callClimaCore.Visualize.fieldheatmap(and friends) orPlots.plot(field)directly.ClimaCoreTempestRemapandClimaCoreSpectraare unaffected and remain inlib/. -
Removed the backwards-compatibility shims and deprecated aliases that the rewrites of the last few releases left behind. Every one of them had a direct replacement, listed below; nothing else about the API changes.
Geometry:AxisTensor{T, N, B, S}→Tensor{N, T, B, S}(theTandNparameters are swapped),AxisVector{T, A, S}→Tensor{1, T, Tuple{A}, S},Axis2Tensor{T, B, S}→Tensor{2, T, B, S},AxisTensor(bases, components)→Tensor(components, bases), andGeometry.components(x)→parent(x)for tensors (theAbstractPointmethod ofcomponentsis unchanged). The generic axis aliasesCovariantAxis{I},ContravariantAxis{I},LocalAxis{I}andCartesianAxis{I}are gone; use the concreteComponents{...}aliases that name their indices, e.g.Contravariant12AxisforContravariantAxis{(1, 2)}.DataLayouts:AbstractData{T}→DataLayout{T}; theIJFH{T, Nij}andIJHF{T, Nij}aliases forNv = 1layouts →VIJFH{T, 1, Nij, Nij, nothing}andVIJHF{T, 1, Nij, Nij, nothing}; and the "universal index" methods that accepted a 5-dimensionalCartesianIndex→ the 4-dimensionalCartesianIndex(v, i, j, h).AbstractData,IJFHandIJHFare no longer exported byDataLayouts.Operators:WeakDivergence/WeakGradient/WeakCurl→Divergence{WeakForm}/Gradient{WeakForm}/Curl{WeakForm};LeftBiasedC2F/LeftBiasedF2C/RightBiasedC2F/RightBiasedF2C→BottomBiasedC2F/BottomBiasedF2C/TopBiasedC2F/TopBiasedF2C; andFirstOrderOneSided/ThirdOrderOneSided→Extrapolate{0}/Extrapolate{1}(which are not numerically identical to the boundary conditions the old names originally named — see the v0.16.0 entry).Fields:ColumnField→FiniteDifferenceField, and the no-indexlevel(field)fallback →level(field, v).- The "point cloud" names deprecated in v0.16.0:
Grids.PointCloudGrid→Grids.MultiPointGrid,Grids.ExtrudedPointCloudGrid→Grids.ExtrudedMultiPointGrid,Spaces.PointCloudSpace→Spaces.MultiPointSpace,CommonGrids.PointColumnEnsembleGrid→CommonGrids.MultiColumnGrid, andCommonSpaces.PointColumnEnsembleSpace→CommonSpaces.MultiColumnSpace. MatrixFields: the⋅alias forMultiplyColumnwiseBandMatrixField()→*, which has denoted matrix multiplication since v0.14.27.⋅is no longer exported byMatrixFields.- The mesh-only constructors deprecated in v0.14.10:
Topologies.IntervalTopology(mesh),Spaces.FaceFiniteDifferenceSpace(mesh),Spaces.CenterFiniteDifferenceSpace(mesh)andGrids.FiniteDifferenceGrid(mesh)→ the two-argument forms that take aClimaComms.AbstractDevicefirst. - The
RecursiveApplymodule (⊞,⊟,⊠,radd,rsub,rmul,rdiv,rzero,rmin,rmax,rpromote_type), a compatibility layer over theAutoBroadcasterwrappers since v0.14.52. Wrap the arguments withUtilities.add_auto_broadcasters, apply the ordinary function, and unwrap the result withUtilities.drop_auto_broadcasters; e.g.x ⊞ ybecomesdrop_auto_broadcasters(add_auto_broadcasters(x) + add_auto_broadcasters(y)), andrzero(T)becomesdrop_auto_broadcasters(zero(add_auto_broadcasters(T))).
Operators.get_node, which lived at the end ofsrc/Operators/deprecated.jlbut was never deprecated, is also removed. Its only callers were theRemappinginterpolation kernels, which now read the field directly withfield[v, i, j, h]through the internal helperRemapping.node_value. -
#2642 Ensured that all names used in downstream packages are documented.
-
Geometry.mul_return_typepromotes the storage type of aNumber * Tensor(andTensor * Number) product along with its element type, so that the return type is the one*produces. Previously aFloattensor times aDualscalar reportedDualelements withFloatstorage, and a broadcast that had to be materialized into an intermediate field — for exampleWVector(∇ᵥ(I(θ))).components.data.:1over aDual-valuedθ, where the composed stencil becomes a band-matrix product withFloatcoefficients — allocated that intermediate with theFloatelement type and then failed to write theDualvalues into it. -
The device-side grid types
Grids.DeviceFiniteDifferenceGrid,Grids.DeviceExtrudedFiniteDifferenceGrid,Grids.DeviceSpectralElementGrid1D,Grids.DeviceSpectralElementGrid2DandTopologies.DeviceIntervalTopologyare removed. -
Spaces.MultiPointSpaceimplementsSpaces.quadrature_style,Spaces.node_horizontal_length_scale,Spaces.all_nodes,MatrixFields.all_columns, andMatrixFields.field2arrays. -
Grids.MultiPointGriduses an identity horizontal metric (∂x∂ξ = I,J = WJ = 1) instead of the sphere metric, and no longer rejects points at the poles.
- #2622 Rewrote the documentation as a Diátaxis suite (tutorials, how-to guides, explanation pages, and curated reference pages), revised the docstrings and code comments across the code base, and rewrote the README.
Remapping.PressureInterpolatorsupportsMultiColumnFiniteDifferenceSpace, so fields on multiple independent columns can be interpolated to pressure coordinates like fields on extruded or single-column spaces.
- Bugfixes for compatibilty with CUDA.jl v6 2628
-
The
CommonGridsandCommonSpacesconstructors that build a horizontal spectral-element grid (ExtrudedCubedSphereGrid,CubedSphereGrid,Box3DGrid,SliceXZGrid,RectangleXYGridand the matching*Spaceconstructors) accept adiscretizationkeyword and forward it to the grid, so a discontinuous space is built withdiscretization = Grids.DG(); when omitted, the discretization follows the quadrature as for the grid constructors. TheSpaces.SpectralElementSpace1D/SpectralElementSpace2Ddocstrings name the keywords they forward, andexamples/plane/bickleyjet.jlbuilds both of its spaces withCommonSpaces.RectangleXYSpaceinstead of assembling the domain, mesh and topology by hand. PR 2621 -
The two Bickley-jet examples
examples/plane/ bickleyjet_cg.jlandexamples/plane/bickleyjet_dg.jlare replaced by the single driverexamples/plane/bickleyjet.jl, which takes the discretization as its first argument (cgordg, then the numerical-flux name and an optionalnoslip). The prognostic equations, physical flux, initial condition and diagnostics are shared; the only structural difference is the space'sdiscretizationand theOperators.tendency_completionbuilt from it, so the tendency function is written once. -
The DG scalar Laplacian (
Operators.scalar_laplacianonGrids.DG()spaces, andOperators.sipg_laplacian_tendency) is now the symmetric interior-penalty (SIPG) discretization. It previously omitted the adjoint-consistency term−∫_f {{κ∇v}}·n̂[[q]], which made it the incomplete (IIPG) form - with symmetry enforced, the methods are now labelled SIPG. -
Operators.sipg_penalty_parameter(κ, space)returns aFieldrather than a scalar, takes aweightkeyword argument, and is joined by the in-placeOperators.sipg_penalty_parameter!(τ, κ; weight). -
The Galerkin discretization of a spectral-element grid is represented by the singleton types
Grids.CG()(the default) andGrids.DG(): constructSpectralElementGrid1D/SpectralElementGrid2D(and the correspondingSpacesconstructors) withdiscretization = Grids.DG(), and read it back withGrids.discretization(grid)/Spaces.discretization(space), which discretization-dependent code can dispatch on. This replaces the Booleandiscontinuous = truegrid keyword. The discretization is a type parameter of these grids rather than a field, so it is inferrable and the methods dispatching on it resolve statically; code that spells out their type parameters positionally needs one more parameter. Omitting the keyword follows the quadrature:CG()when its nodes are shared across element boundaries (Quadratures.requires_dss, e.g. GLL) andDG()otherwise, so a single-node horizontal space stays discontinuous. PassingCG()explicitly with a quadrature that cannot represent a continuous space raises anArgumentErrorrather than being silently downgraded. On aDG()gridSpaces.weighted_dss!(and itsstart!/internal!/ghost!split) is a no-op,Spaces.create_dss_bufferreturnsnothing, and no DSS weights are computed;Grids.is_continuous/Spaces.is_continuousreport the discretization, so models can gate DSS on the space itself rather than on the quadrature type. The discretization is part of the grid cache key, forwards throughGrids.LevelGrid, and is serialized byInputOutput(grids in files without the attribute read back as continuous). PR 2599 -
Added horizontal discontinuous-Galerkin (DG) operator support. The discretization-generic machinery lives in
src/Operators/numericalflux.jl: interior- and boundary-face numerical fluxes (Operators.add_numerical_flux_interior!andadd_numerical_flux_boundary!) and symmetric face liftings (add_lifting_flux_interior!) on pure-2D and extruded (1D and 2D horizontal) spectral-element spaces, the flux-differencing (split-form / FDDG) volume divergence of Souza et al. (2023), and the device-residentDGConnectivityface buffer. A generic flux library lives insrc/Operators/dg_fluxes.jl:CentralNumericalFlux,RusanovNumericalFlux, central-lifting and jump-penalty face functions, and an SIPG Laplacian for optional scale-selective dissipation. Equation-set-specific fluxes (compressible Euler with Cartesian momentum components) are example code. The face operators run on CPU and CUDA (with kernels in theClimaCoreCUDAExtextension) and on distributed (MPI)Topology2Dspaces, where each rank completes its rank-boundary (Topologies.ghost_faces) side through a face-strip halo exchange (Topologies.GhostFaceExchange) that ships only theNqface-node values per face. Several operators in one tendency evaluation can share a single exchange by passing the handle returned byOperators.start_dg_ghost_exchange(args...)as a leading argument. Covered by 2- and 3-rank CPU and CUDA tests. PRs 2602, 2603 -
Added the model-level CG↔DG switch
Operators.tendency_completion/Operators.complete_tendency!: a tendency written as an element-local weak form plus one completion call runs on both discretizations, with the completion object built once from the space —Spaces.weighted_dss!on continuous spaces, the mass-weighted DG interface (and optional boundary) numerical fluxes onGrids.DG()spaces. The tendency may be aFieldon either discretization, or aFieldVectoron a continuous one, where the completion is a single batchedSpaces.weighted_dss!over all components (which must agree on their discretization). On a discontinuous space it must be oneFieldwith a composite (e.g.NamedTuple) eltype, since the interface numerical flux is evaluated on the whole state at a face node. The CG↔DG Bickley-jet integration test runs one shallow-water tendency on both discretizations through this switch. -
Added
Operators.Outflow(; order = 0), a physically named convenience constructor for advection-operator outflow boundary conditions: it returnsOperators.Extrapolate{order}(), an order-orderextrapolation from the interior whose order-0 case is the zero-normal-gradient closure, soOutflow() === Extrapolate{0}()andOutflow(; order = 2) === Extrapolate{2}(). Additive only;Extrapolateremains the numerical primitive and nothing changes for existing code. -
Operators.AbstractBoundaryConditionis specialized intoOperators.VerticalBoundaryCondition(the boundary conditions of the vertical finite-difference operators, e.g.SetValue,Extrapolate) andOperators.HorizontalBoundaryCondition(those of the horizontal DG numerical-flux operators, e.g.ReflectingWallBC), so the two families are distinct at the type level. Passing a boundary condition to an operator of the other family now fails at dispatch with an error naming the mismatch. Custom boundary conditions must subtype the family they implement: the finite-difference stencil defaults now dispatch onVerticalBoundaryCondition, so a pre-existing direct subtype ofAbstractBoundaryConditionfails with aMethodError(inleft_interior_idx/right_interior_idx) or, whenboundary_widthis undefined, the generic invalid-boundary-condition error. -
Added the horizontal Laplacian atoms
Operators.scalar_laplacian,Operators.scalar_laplacian!andOperators.vector_laplacian, the building blocks of ∇⁴ hyperdiffusion. On continuous (CG) spaces the returning forms give lazy, element-local operator expressions that fuse into consuming broadcasts; materialized intermediates must be made continuous withSpaces.weighted_dss!between passes (batch several intermediates into one call to share the ghost exchange). On discontinuous (DG) spacesscalar_laplacianincludes the interior-penalty face corrections itself andweighted_dss!is a no-op, so one calling sequence serves both discretizations (vector_laplacianis not yet implemented for DG).Operators.scalar_laplacian!(out, χ; weight)writes the result into a caller-owned field and is allocation-free on both discretizations, so it is the form to use in a tendency;outmay aliasχ. Each call to the returningscalar_laplacianowns its result — freshly allocated on DG — so any number of results may be live at once. On DG the gradient and a materialized lazy argument live in scratch fields keyed on the space's grid (released by the zero-argumentUtilities.Cache.clean_cache!()), and the in-placeOperators.sipg_laplacian_tendency!writes the tendency into a caller-owned field. The hybrid example's hyperdiffusion is rewritten on top of these atoms. PR 2606 -
FieldVector broadcasts on GPU flatten to linear indexing when the destination and every array in the broadcast have the same shape and are
IndexLinear(dense arrays and contiguous views), bypassing CUDA.jl's N-dimensional Cartesian index computation (measured 3× faster vector updates in a baroclinic-wave benchmark); in-place updates likex .-= dxpreserve destination identity through the flattening, so they no longer allocate a defensive aliasing copy (previously 7.85 GB of device allocations per step on a 63-level sphere, now zero). CPU broadcasts keep the standard path, which guarantees zero allocations. Spectral-element slab loops reserve half the shared memory per block (MAX_SUBBLOCK_LAUNCH_THREADS256 → 128), and the eager finite-difference kernel packs 128 threads per block and queries the SM count from the device. The environment variablesCLIMA_CUDA_MAX_WAVES,CLIMA_FD_MAX_THREADS, andCLIMA_DSS_MAX_THREADS(read once at module load) override launch-tuning defaults for experiments (seeperf/sweep_kernel_configs.jl).CLIMA_COLLECT_KERNEL_STATStoggles the development-only per-launch statistics; it is a compile-time constant (read when the CUDA extension precompiles) so that the statistics block folds away and does not enter the kernel-launch type-stability contract. PR 2606 -
Spectral element operators are now evaluated through the slice-loop primitives (
foreach_slaband the otherDataLayouts.foreach_*loops), so they can appear inside fused loop bodies alongside pointwise broadcasts: one loop on CPUs and one kernel launch on GPUs per fused body, replacing the hand-written CUDA spectral kernels. Statements in a fused body may read the results of the statements before them, both pointwise and through operators, and temporaries materialized inside a body are stored in per-thread registers.Restrictcannot read a register-resident argument on GPUs and now throws an error asking for the argument to be materialized outside the fused loop. -
Fused spectral element broadcasts compile with lower memory and run faster than the replaced implementation on CPUs, and slightly faster on GPUs.
-
The advection operators' boundary treatment has been reworked around a generalized
Extrapolate{N}boundary condition (also writtenExtrapolate(N), withExtrapolate() == Extrapolate{0}()), which selects how the ghost points reached by the interior stencil are padded: every ghost point of a stencil takes the value extrapolated (with an order-Npolynomial,0 <= N <= 2) from theN + 1interior points closest to the boundary, and the order is reduced where the stencil has fewer interior points in range (at a boundary face itself only 2 are in range, so the order there is at most 1;UpwindBiasedProductC2F's 2-point stencil only ever has 1 in range, so every order reduces to closest-value padding for it).Extrapolateis now the only boundary condition the advection operators (UpwindBiasedProductC2F,Upwind3rdOrderBiasedProductC2F,LinVanLeerC2F,FCTBorisBook,FCTZalesak, andTVDLimitedFluxC2F) accept, andExtrapolate{0}is added to an operator'sbcswhen it is constructed with no boundary conditions. It replaces the one-sided conditions, which remain as deprecated aliases (FirstOrderOneSided == Extrapolate{0}andThirdOrderOneSided == Extrapolate{1}). The aliases are not numerically identical to the old conditions: the old conditions replaced the whole stencil with fixed one-sided reconstructions at the two faces nearest each boundary, whileExtrapolatekeeps the interior stencil's upwinding and only pads its ghost points. At the face one in from a boundary, the results coincide exactly when the velocity at that face points toward the boundary (the old downwind-biased reconstruction is then also the upwind choice), and differ when it points into the domain: with inflowv³at the second-lowest face and centersx₁, x₂, x₃counted up from the boundary,Upwind3rdOrderBiasedProductC2FwithThirdOrderOneSidedgavev³ (4x₁ + 10x₂ - 2x₃) / 12and now givesv³ (x₁ + x₂) / 2withExtrapolate{1}, while withFirstOrderOneSidedit gave the first-orderv³ x₁and now givesv³ (2x₁ + x₂) / 3withExtrapolate{0}. At the boundary face itself the old reconstructions reached one center beyond the boundary, so they were only meaningful under an enclosing operator that overrides the boundary face (e.g.DivergenceF2CwithSetValue);Extrapolate's ghost-point padding is well-defined there. User-supplied callable ghost-point reconstructions are no longer accepted. A boundary condition whose keyword name matches neither of the space's vertical boundary names is now an error at broadcast time (previously it was silently ignored), except for the default(bottom, top)pair ofExtrapolate{0}s, which applies at any boundary names. An advection operator is rewritten as an operator-matrix multiply exactly when its interior stencil is linear in the advected argument (Operators.has_linear_stencil), with the ghost-point extrapolations folded into its matrix's boundary rows by multiplying the interior row with the matrix of extrapolation weights; all others are evaluated pointwise. Note thatExtrapolateonUpwindBiasedProductC2Falso changes meaning: it used to replicate the operator's output at the closest interior face (U(v, x)[1/2] = U(v, x)[3/2]), while it now pads the ghost input, so the boundary face evaluates tov³[1/2] x[1]. As before, the flux through the boundary itself should be imposed by the enclosing operator, e.g.DivergenceF2Cwith aSetValueboundary.On non-periodic domains, this changes results at the two faces nearest each boundary (with the default
Extrapolate{0}padding):LinVanLeerC2Fpreviously used one-sided first-order upwind reconstructions there; it now uses the ghost-point-padded limited stencil.FCTBorisBookpreviously returned a zero antidiffusive flux there, and it still does, since the padded ghost points make the one-sided difference on the boundary side vanish, and that difference bounds the corrected flux; its results are unchanged everywhere.FCTZalesakandTVDLimitedFluxC2Fpreviously forced their corrected or limited fluxes to zero there; they now compute the ghost-point-padded stencil instead.
Two operators also change their calling convention. Since an advection operator advects a single center-valued argument,
FCTZalesaknow takes its two advected quantities as a single field with 2-tuple elements:FCTZalesak.(A, Φ, Φᵗᵈ)becomesFCTZalesak.(A, tuple.(Φ, Φᵗᵈ))(orFCTZalesak(A, tuple(Φ, Φᵗᵈ))inside@.). Broadcast arguments beyond the velocity and advected field are evaluated at the current face and passed through as is, soTVDLimitedFluxC2Fnow requires its upwinding velocity𝓊to be supplied as contravariant data: either aContravariant3Vectorfield, or a scalar field holding the contravariant3 component, e.g.Geometry.contravariant3.(u, Fields.local_geometry_field(face_space)).TVDLimitedFluxC2F's slope ratioris now zero whenever the upwind slope is zero, even if the denominatorϕ₊₁₂ - ϕ₋₁₂ + epsis also zero (which happens whenϕ₊₁₂ - ϕ₋₁₂is exactly-eps, e.g. in regions whereϕis flat up to roundoff, and previously producedNaNlimited fluxes from0 / 0).MatrixFields.operator_matrixnow reports every advection operator with a nonlinear stencil or boundary reconstruction (includingTVDLimitedFluxC2F, which previously threw aMethodError) as a nonlinear operator that cannot be represented by a matrix. 2544 -
Broadcasts over the linear one-argument finite difference operators are now evaluated as operator-matrix multiplies:
@. op(arg)is rewritten to multiplyargbyop's operator matrix (MatrixFields.operator_matrix), with value-fixing boundary conditions reapplied around the multiply by aSetBoundaryOperator. Two user-visible consequences:- A center-input operator (
InterpolateC2F,GradientC2F,DivergenceC2F,CurlC2F, ...) constructed without a boundary condition produces aNaNmatrix row at each boundary face, so the result there isNaN, just as it was on the pointwise stencil path; a forgotten boundary condition is still flagged at run time. This also applies to the explicitMatrixFields.operator_matrixAPI, whose boundary rows for a missing boundary condition were previously zero: they are nowNaNrows. CurlC2Fresults are nowContravariant12Vectors rather thanContravariant123Vectors with a structurally zero third component, since its operator matrix's rows produce the two nonzero components of the vertical curl contribution;SetCurlboundary values are projected onto theContravariant12axis accordingly. Code that read theu³component of aCurlC2Fresult must drop it (it was always zero).
Columns too short for a stencil's interior (e.g. a center-to-face operator on a single-level column, or a 4-point advection stencil on a 2-level column) are now supported:
Operators.window_boundsclamps the overlapping boundary windows instead of rejecting them with anAssertionError, and every face is computed with the appropriate boundary row (the ghost-point extrapolations reduce their order to the interior points actually in range). When the two boundary windows overlap on such a column, the bottom (left) boundary condition always takes precedence: e.g. a two-sidedSetDivergenceon a single-level column, which prescribes two values for the one output point, applies the bottom value there. On GPUs, finite difference broadcasts overMultiColumnFiniteDifferenceSpacenow use the eager one-thread-per-level kernel like the extruded and single-column families, instead of falling back to the one-thread-per-value kernel. 2544 - A center-input operator (
-
Removed the
Extrapolateboundary condition fromGradientF2C, where it replicated the operator's output at the closest interior point (G(x)[1] = G(x)[2]); it remains available onDivergenceF2C, where it keeps that replicate-output meaning (D(v)[1] = D(v)[2]), and on the interpolation operatorsInterpolateC2FandWeightedInterpolateC2F, where it copies the closest interior input.GradientF2Cnow acceptsSetGradient(v₀), which prescribes the gradient at the center closest to the boundary (withv₀projected onto the covariant 3 axis, as forGradientC2F). The finite-difference biased interpolation operators are named for the vertical direction they lean toward:BottomBiasedC2FandBottomBiasedF2Ctake the value below a node,TopBiasedC2FandTopBiasedF2Cthe value above.LeftBiasedC2F,LeftBiasedF2C,RightBiasedC2F, andRightBiasedF2Cremain available as compatibility aliases. 2544, 2608 -
Removed unused finite difference operators and boundary conditions 2521
- Removed
SetValuefromGradientC2F,DivergenceC2F,CurlC2FandUpwindBiasedProductC2F - Removed
SetGradientfromInterpolateC2FandWeightedInterpolateC2F - Removed the
AdvectionC2C,AdvectionF2F,FluxCorrectionC2CandFluxCorrectionF2Foperators - Removed the
UpwindBiasedGradientoperator, which had no downstream users - Removed the
LeftBiased3rdOrderC2F,LeftBiased3rdOrderF2C,RightBiased3rdOrderC2FandRightBiased3rdOrderF2Coperators, which had no downstream users (the C2F reconstructions survive as the interior stencil ofUpwind3rdOrderBiasedProductC2F, which applies the left- or right-biased variant according to the sign of its velocity argument)
Each of these can be written in terms of the remaining operators and boundary conditions;
test/Operators/finitedifference/unit_column.jlcontains a testset ("Boundary values and advection built from the primitive operators") that pins the replacement expressions forSetValueonGradientC2F,DivergenceC2FandUpwindBiasedProductC2F, and for theAdvectionC2C,AdvectionF2FandFluxCorrectionC2Cstencils, against the stencil each reproduces (the remaining removals follow the same patterns). For example, aSetValue(x₀)boundary onGradientC2Fis the same asSetGradient(Covariant3Vector(2 * (x[1] - x₀))), andAdvectionC2C(v, θ)isInterpolateF2C()(dot(Contravariant3Vector(v), GradientC2F()(θ))). The helper functionsOperators.gradient_c2f_dirichlet,Operators.divergence_c2f_dirichlet,Operators.curl_c2f_dirichletandOperators.upwind_biased_product_c2f_dirichletbuild the removedSetValuereplacements automatically (materializing their results, so they are migration aids rather than performance-sensitive building blocks), pass boundary values that are already boundary conditions through unchanged (so a Dirichlet value on one boundary can be combined with an explicit condition on the other), and requesting aSetValuefrom one of the four operators' constructors automatically returns anOperators.DirichletOperatorthat applies the matching helper when broadcast, so existingGradientC2F(bottom = SetValue(x₀)).(x)-style code keeps working. The replacement stencil is built lazily, so it fuses into an enclosing broadcast like a true operator application, but each application still materializes any lazy argument (the boundary rows need the argument's boundary levels) along with small level fields holding the boundary rows' values.MatrixFields.operator_matrixnow reportsLinVanLeerC2Fas a nonlinear operator instead of failing with aMethodError. - Removed
-
The "point cloud" types have been renamed to use the
Multiprefix:Grids.PointCloudGridis nowGrids.MultiPointGrid,Grids.ExtrudedPointCloudGridisGrids.ExtrudedMultiPointGrid,Spaces.PointCloudSpaceis nowSpaces.MultiPointSpace,CommonGrids.PointColumnEnsembleGridis nowCommonGrids.MultiColumnGrid,CommonSpaces.PointColumnEnsembleSpaceis nowCommonSpaces.MultiColumnSpace.- The old names are available, but deprecated. 2611
-
Support remapping and HDF5 reading/writing for
MultiColumnFiniteDifferenceSpace2596 -
The plotting companion packages have been folded into package extensions, and the
ClimaCoreVTKcompanion package has been deleted. Makie plotting has moved fromClimaCoreMakieinto theClimaCoreMakieExtextension, which loads automatically when one of theMakiebackends is loaded, and the Plots recipes have moved fromClimaCorePlotsinto theClimaCoreRecipesBaseExtextension, which loads automatically when bothClimaCoreandPlotsare loaded.ClimaCoreVTKwas unmaintained and unused, and has been removed with no replacement. 2595
-
Removed the SciML compatibility layer (
src/Fields/compat_diffeq.jl) and theRecursiveArrayToolsdependency. Nothing in ClimaCore, ClimaAtmos, ClimaLand, or ClimaCoupler uses it: all of them time-step with ClimaTimeSteppers, which dropped SciMLBase in 0.10. Passing aFieldorFieldVectorto an OrdinaryDiffEq integrator is no longer supported: the layer's eightRecursiveArrayToolsmethods are removed, andrecursive_bottom_eltype, the only one ClimaCore itself used, is nowClimaCore.Utilities.recursive_bottom_eltype. Of the fourBasemethods the layer also carried,any,similar(::F, ::Type{F})andveconFieldare kept and are used beyond OrdinaryDiffEq;vecreturns a vector over the field's values now, so its eltype is the field's rather than the backing array's.muladd(x, ::Field)is removed: it broadcast a two-argumentmuladdover scalars, which does not exist, so it has thrown aMethodErroron every call since it was added. 2576 -
Bugfix for indexing of reduced-dimension fields in GPU finite difference kernels 2584
-
Bugfix for mapreduce with
TensorvaluedField(s) on GPUs 2591
- Extend kernel fusion interface (
foreach_slice/reduce_points) toFieldarguments. 2571 - Add
PointCloudSpaceandMultiColumnFiniteDifferenceSpacefor running independent points/columns in parallel. 2525
-
Fixed a GPU compilation failure (
gpu_gc_pool_allocandjl_f__apply_iterateinInvalidIRError) for kernels over wide or complexly typed broadcast expressions, like the fused EDMFX cloud fraction broadcasts in ClimaAtmos AMIP runs. The broadcast argument traversal functions inDataLayoutsand the CUDA extension no longer filter arguments with reductions that push into an accumulator, whose recursively growing type could trigger inference's widening heuristics; they now collect layouts withunrolled_flatmap, whose intermediate types all stay concrete, andf_dimreduces with a fixed nothing-or-integer accumulator. All other kernel-reachable calls to the accumulator-basedunrolled_filter,unrolled_unique, andunrolled_allunique(inGeometrybasis computations and inUtilities.nested_broadcast) were converted to per-item checks orunrolled_flatmap, and a unit test now forbids these functions in kernel-reachable directories. Errors thrown from kernel-reachable code inDataLayouts,Geometry, andUtilitiesare now singleton exception types (InvalidSubscopeError,NonOverlappingScopesError,NoMetricError,ScalarBasisError,DuplicateComponentNamesError,UnequalNestedBroadcastLengthsError) whose messages include the relevant types but are only generated byshowerroron the host. All kernel-reachableunrolled_reducecalls pass their init values positionally, since kwcalls do not always specialize during GPU compilation of wide broadcast expressions. A unit test materializes wide fused broadcasts that mix 0-dimensional and 3-dimensional layouts of heterogeneous element types, and a 1-GPU Buildkite job compiles them through real CUDA kernels. GPU kernel launch latency improved by roughly 10%, and the latency benchmark baselines were lowered accordingly. -
Added
Fields.fieldvector2array!andFields.array2fieldvector!, allocation-free, GPU-safe block-wise copies between aFieldVectorand a flat vector (plus allocating versions without the!). Together withKrylov.ktypeof(::FieldVector)(defined inKrylovExt), these let iterative Krylov solvers keep their workspace vectors on the GPU while model code continues to operate onFieldVectors. -
Fixed
zero(::FieldVector)to preserve scalar (ScalarWrapper) components instead of turning them into 0-dimensionalArrays, and fixedClimaComms.array_type(::FieldVector)(and henceKrylov.ktypeof(::FieldVector)) forFieldVectors with plain-array or scalar components, which previously threw aMethodError. -
Removed the
Operators.columnwise!operator, along with its CUDA kernels (ext/cuda/operators_columnwise.jl), its tests, and its CI jobs. It was unused by downstream code. 2548 -
Removed the unused shared-memory finite difference operator code path on GPUs (
CUDAWithShmemColumnStencilStyleand theoperators_fd_shmem*.jlfiles in the CUDA extension), along with its tests, benchmarks, CI job, and theshmem_design.mddocumentation page. This path was already disabled by default (Operators.use_fd_shmem()returnsfalse), so default behavior and performance are unchanged. Downstream code that opted in by definingOperators.use_fd_shmem() = truewill no longer see any effect from doing so. 2526 -
Unified strong/weak spectral element operator variants via a
FormTypeparameter.Divergence,Gradient, andCurlnow carry a second type parameter (StrongFormorWeakForm), andWeakDivergence,WeakGradient, andWeakCurlare backwards-compatibility aliases for theWeakFormvariants (e.g.WeakDivergence{I} = Divergence{I, WeakForm}), collected insrc/Operators/deprecated.jlfor eventual removal. All operator names and their public constructors are unchanged. Results are bitwise identical to the previous implementations, with one exception: the strong-formDivergenceon GPUs in 2D now keeps one accumulator per dimension (matching the other unified operators) instead of summing both dimensions into a single accumulator, which changes results by accumulation roundoff only (verified identical in exact arithmetic; float differences are a fewepsof the accumulated terms). Two notes for downstream code:isachecks against the strong operator types now also match the weak variants (e.g.WeakDivergence() isa Divergenceistrue), so dispatch that must distinguish forms should use theFormTypeparameter (e.g.Divergence{I, StrongForm}); and the internalDivergence{()}(space)-style constructors used to reset operator axes have been replaced byrebuild_operator(op, space). As a side effect of unifying the CUDAGradientkernels,WeakGradientnow accepts rank-1 tensor fields on GPUs, which previously raised an "Unsupported input type" error even though both forms already supported them on CPUs. 2556 -
Refactor DataLayouts module 2522
- All data layout types are unified into a single
DataLayouttype, and all loops over data go through two communication primitives (foreach_sliceandreduce_points) that work the same way on CPUs, multi-threaded CPUs, and GPUs. Runtime performance on CPUs and GPUs matches the previous release. - Data in a
Fieldis now indexed in the order[v, i, j, h], and every parent array has a 5-dimensional shape, with one axis used to store struct fields. Code that indexes intoparent(field)arrays must be updated. - Several old names (
AbstractData,IJFH,IJHF) are available as aliases, and[i, j, f, v, h]indexing is still allowed for backward compatibility.
- All data layout types are unified into a single
-
Fixed multithreaded CPU loops over
ThisThreadPoolreading Julia's process-global threaded-region flag more than once per loop. A task that concurrently entered a threaded loop could flip the flag in between two of those reads, which either raised a spurious"Nested loops over ThisThreadPool are not supported"error or leftnum_threadsandthread_rankdisagreeing about how a loop's indices were divided among threads. Loops now resolve their scope once, up front, withresolve_pool_threads, and a thread's rank comes from task-local storage rather than from global state. 2557 -
Divided the CPU thread pool between multithreaded loops that run at the same time, instead of giving every thread to whichever loop starts first and leaving the rest to run on a single thread each. Broadcasting over
Fields from several tasks at once, asThreadsX.mapreducedoes, is about 2.6x faster with two concurrent tasks on 8 threads, and no slower with one. 2557 -
Cached the CUDA device attributes that every GPU launch configuration needs, instead of querying the CUDA driver for each of them on every launch, and reused a per-task device buffer for the intermediate results of GPU reductions instead of allocating one per reduction. 2557
-
Fixed DSS for MPI after DataLayouts refactor. 2572
- Add a
reverseoption to the column integral operators (column_reduce!andcolumn_accumulate!) so vertical reductions and accumulations can proceed from the top of a column downward. 2531
- Refactor Geometry module 2457
- Bug fix for broadcasting gpu errors 2517. 2518
- Make specialized shared memory-based tridiagonal solver mask aware 2518. 2518
- Add a specialized shared memory-based tridiagonal solver that uses PCR algorithm for CUDA backend. 2486
- Replace RecursiveApply with AutoBroadcaster wrappers. 2417
- Add support for nonuniform data structures 2464
- Add shared memory stencil support in CUDAExt (for spaces with non-periodic topologies and no masks) 2466
- Add support for bilinear interpolation for diagnostics to the Remapper. 2455
- Add
PressureInterpolator2422 - Add Buildkite pipeline to check flagship AMIP performance 2438
- Fix conservation of split divergence 2432
- Add
VerticalMassBorrowingLimiter2383
- Use space-filling curve for CubedSphereGrid and ExtrudedCubedSphereGrid 2429
- Restrict GPUCompiler to < v1.7.6 to avoid GPU errors 2430
- Add split divergence operator 2409
- Revert
to_devicedispatch change from #2375 2403
- Add support for horizontal operators on column spaces 2375
-
Add opt-in CUDA kernel renaming based on stacktrace for broadcasts over ClimaCore Fields 2376
To enable, set the following environment variable before starting Julia:
export CLIMA_NAME_CUDA_KERNELS_FROM_STACK_TRACE=trueWith default kernel names, profiling downstream packages results in names like
knl_copyto_(VIJFH<Float32, (int)63, (int)4, CuDeviceArray<Float32, (int)5, (int)1>>, Broadcasted<VIJFHStyle<(int)63, (int)4..., which are hard to read and often more than 50,000 characters long. With renaming enabled, profilers show names likeset_velocity_quantities__FILE_ClimaAtmos_jl_src_cache_precomputed_quantities_jl_L351, which gives more context about where the kernel was launched from.
- Add
projectfor both sides of 2-tensors (previously only left side was supported). Simplify implementation. 2379
- Store
reverse_modein theIntervalMeshstruct, so that we can access it when writing and reading the mesh for restarts
-
Upgrade compatibility with RootSolvers.jl, KrylovKit.jl, DataStructures.jl. Using the following versions of RootSolvers may result in slightly different
IntervalMesh(es) and results ofcolumn_integral_indefinite!(∂ϕ∂z, ᶠϕ)- < v0.45
- = v0.45
- > v0.45
- Extend level and field_values to work with lazy Fields 2356
- Fix unrolling over field broadcast arguments 2350
- minor fixes/Fix field2array error msg, level func bounds check 2347
- Define FiniteDifferenceSpace for extruded spaces 2344
- Implement a columnwise shmem operator 2328
- Dont reference localgeometry fields unnecessarily 2340
- Add missing methods for column spaces and FieldVectors 2330
- Move Δz_metric_component to Geometry module 2336
- Allow for more flexibly constructed LocalGeometry 2335
- Latitude and longitudes were flipped in the default target coordinates in the remapper, leading to non-sense values.
Add option to disable check on spaces. PR 2322
ClimaCore.DebugOnly now contains a function allow_mismatched_spaces_unsafe. When
this function is overridden to return true, ClimaCore will skip consistency
checks on spaces in broadcasted expressions. This is useful when debugging with
deepcopy. See
documentation
for more information.
- Expanded compat for Krylov to
0.10. 2318.
-
Disabled shared memory on finite difference operators. It was found that it has a severe performance penalty for more complex cases. 2317
-
Fixed missing method for
Topologies.mesh(Topology2D)2288.
-
GPU performance should now be more robust 2296.
-
Remapping is now protected for masked operations 2292.
-
Shmem support for InterpolateC2F was added 2290.
-
Some masked operations were fixed 2285.
-
Internal refactoring should help reduce latency and improve gpu performance 2284.
-
Masks now support restart 2212.
-
Fixed error messages in our DataLayouts 2275.
-
Bug fix in our finite difference shared memory implementation 2282.
-
Added support for shared memory in
GradientC2F2268.
Improvements to Remapping.interpolate PR 2272
PR 2272 introduces
convenience interpolate functions that are optimally suited for plotting.
Assuming field is a 3D Field,
import ClimaCore.Remapping: interpolate
array3D = interpolate(fields)array3D will be a 3D Array with values along the three dimensions.
By default, no interpolation in the vertical direction is performed. Linear
interpolation can be obtained by passing zresolution
array3D_linear_in_z = interpolate(fields; zresolution)Controlling the specific interpolation points is also possible, as shown in this example:
import ClimaCore.Geometry: LatLongPoint, ZPoint
import ClimaCore.Remapping: interpolate
target_hcoords = [LatLongPoint(45., 23.)]
target_zcoords = ZPoint.([100., 200., 1000.])
array3D_specific = interpolate(fields; target_hcoords, target_zcoords)- Fix empty interpolation with certain versions of CUDA+MPI 2256.
- Bugfix in ClimaCore FD stencil code PR 2251.
-
Added support for
map!PR 2248. -
The standard
*symbol now denotes matrix multiplication, and the aliasconst ⋅ = MultiplyColumnwiseBandMatrixField()is being deprecated PR 2232. Please replace use of⋅with*. -
Fields.Field(Bool, ::AbstractSpace)is now supported. PR 2239. -
SpectralElementSpace2Dconstructors now support nodal masks. PR 2201. See its documentation here. Note that it does not yet support restarts. -
Added support for InputOutput with PointSpaces PR 2162.
-
Reading a
FieldwithHDF5Readerwill not calleval ∘ Meta.parseon the rawvalue_typeattribute. Instead, the newread_typefunction is used, which prevents the execution of arbitrary code. PR 2162. -
Improved
showfor spaces, and addedshowfor grids. PR 2202. -
UnrolledFunctionsremoved and replaced withUnrolledUtilitiesPR 1713 -
Unused original implicit solver intereface removed PR 2200
-
Using incorrect boundary conditions will now error PR 2180.
-
Prior to this version,
CommonSpacescould not be created withClimaComms.MPICommsContext. This is now fixed with PR 2176. -
Fixed bug in distributed remapping with CUDA. Sometimes,
ClimaCorewould not properly fill the output arrays with the correct values. This is now fixed. PR 2169 -
Added support for 1D extruded spaces. PR 2172. Closed 1989, and 1383.
- A new
Adaptwrapper was added,to_device, which allows users to adapt datalayouts, spaces, fields, and fieldvectors between the cpu and gpu. PR 2159. - Remap interpolation cuda threading was improved. PR 2159.
center_spaceandface_spaceare now exported fromCommonSpaces. PR 2157.- Limiter debug printing can now be suppressed, and printed at the end of a simulation, using
Limiters.print_convergence_stats(limiter). PR 2152. - Fixed getidx for
GradientC2FwithSetValuebcs. PR 2148. - Added support
data2arrayforDataF(i.e.,PointFields). PR 2143. HDF5Reader/HDF5Writernow supportdo-syntax. PR 2147.
-
Added support for CPU<->GPU conversion of spaces, fields and fieldvectors, via Adapt. PR 2114.
-
Fixed numerics of column integrals for deep atmosphere. PR 2119.
-
Fixed high-resolution gpu space construction. PR 2100.
A new ClimaCore.DebugOnly module was added, which can help users find where
NaNs or Infs come from in a simulation, interactively. Documentation, with a
simple representative example can be found here.
Various improvements to Remapper 2060
The ClimaCore.Remapping module received two improvements. First, Remapper is
now compatible with purely vertical Spaces (performing a linear
interpolation), making it compatible with column setups. Second, a new set of
simplified interpolation functions are provided.
Now, interpolating a Field field is as easy as
import ClimaCore.Remapping: interpolate
output_array = interpolate(field)The target coordinates are automatically determined, but can also be customized. Refer to the documentation for more information.
- Fixed gpu support (adapt) for van-leer limiters. PR 2112.
- Support for new TVD limiters were added, PR [1662] (CliMA#1662).
- Fixed writing/reading purely vertical spaces. PR 2102
- Fixed correctness bug in reductions on GPUs. PR 2106
ClimaCore.Spaces now comes with two functions, face_space and
center_space, to convert a Space from being cell-centered to be
face-centered (and viceversa). These functions only work for vertical and
extruded spaces.
- We've added new convenience constructors for spaces PR 2082. Here are links to the new constructors:
-
We've added new convenience constructors for grids PR 1848. Here are links to the new constructors:
-
A
strict = truekeyword was added torcompare, which checks that the types match. Ifstrict = false, thenrcomparewill returntrueforFieldVectors andNamedTuples with the same properties but permuted order. For example:rcompare((;a=1,b=2), (;b=2,a=1); strict = true)will returnfalseandrcompare((;a=1,b=2), (;b=2,a=1); strict = false)will returntrue
-
We've added new datalayouts:
VIJHF,IJHF,IHF,VIHF, to explore their performance compared to our existing datalayouts:VIJFH,IJFH,IFH,VIFH. PR #2055, PR #2052. -
We've refactored some modules to use less internals. PR #2053, PR #2052, #2051, #2049.
-
Some work was done in attempt to reduce specializations and compile time. PR #2042, #2041
ClimaCore had incorrect lower bounds for certain packages. PR
#2078 fixes the lower bounds
and adds a GitHub Action workflow to test it. ClimaCore now requires Julia
1.10 or greater.
- Fixed world-age issue on Julia 1.11 issue Julia#54780, PR #2034.
PR #2034 fixes some undefined
behavior in the DataLayout module. This bug was manifesting itself as a world age error in some applications that are using Julia 1.11.
PR #2033 introduces new
constructors for DataLayouts. Instead of writing
array = rand(FT, Nv, Nij, Nij, 3, Nh)
data = VIJFH{S, Nv, Nij}(array)You can now write
data = VIJFH{S}(ArrayType{FT}, rand; Nv, Nij, Nh)and grab the array with parent(data) (if you need).
Note: These constructors are meant to be used in tests and interactive use, not in performance sensitive modules (due to their lack of inferrability).
- Fixed multiple-field solve for land simulations PR #2025.
- Fixed Julia 1.11 PR #2018.
Nhwas turned back into a dynamic parameter, in order to alleviate compile times PR #2005.- Defined some convenience methods #2012
Due to a bug, == was not recursively checking FieldVectors with different
types, which resulted in false positives. This is now fixed and FieldVectors
with different types are always considered different.
Prior to this change, the ClimaCore.InputOutput module did not save whether a
Space was constructed with deep = true. This meant that restarting a
simulation from a HDF5 file led to inconsistent and incorrect spaces and
Fields. This affected only extruded 3D spectral spaces.
We now expect Spaces read from a file to be bitwise identical to the original
one.
PR #2021.
- Fixed some type instabilities PR #2004
- More fixes to higher resolution column cases for the GPU #1854
-
Extended
create_dss_bufferandweighted_dss!forFieldVectors, rather than justFields. PR #2000. -
Fix restarting simulations from
Spaces withenable_bubble = true
Prior to this change, the ClimaCore.InputOutput module did not save whether a
Space was constructed with enable_bubble = true. This meant that restarting
a simulation from a HDF5 file led to inconsistent and incorrect spaces and
Fields. This affected only 2D spectral spaces (and extruded ones that have
this type of horizontal space).
We now expect Spaces read from a file to be bitwise identical to the original
one.
PR #1999.
- Added support for mixing extruded and horizontal spaces in GPU kernels. PR #1987.
- Inference was fixed for some broadcast expressions involving columns PR #1984.
-
CUDA kernel launch configurations have been tuned to improve performance, and now allows for high resolution in the vertical direction PR #1969, issue #1854 closed.
-
DSS was refactored, and machine precision changes can be expected. PR #1958.
- Added hyperbolic tangent stretching. PR #1930.
- Support for matrix fields on spectral and point spaces was added, PR #1884.
- Support for 3-component DSS transform was added, PR #1693.
- Support for column-wise "accumulate"/"reduce" operations were added, PR #1903. These abstractions will allow us to group, paralellize and optimize more column-wise work on the GPU.
- A new macro,
Fields.@rprint_diffwas added, which recursively print differences between twoFieldVectors (of the same type) (PR #1886). - Julia 1.11 fixes (PR #1883)
Nhhas been added to the type parameter space, which allows us to more flexibly write performant backend kernels (PR #1894). This was leveraged in PR #1898, and may result in slightly more performant kernels.
- Various performance tweaks (PRs #1840, #1837, #1843, #1839).
- CPU/GPU kernels are now determined by dispatching, instead of specializing, which should (hopefully) have generally fixed GPU dispatching issues (PR #1863).
- Matrix multiplication kernels have been improved (PR #1880).
- Support for the following methods have been deprecated (PR #1821, ):
IntervalTopology(::Mesh)in favor of usingIntervalTopology(::ClimaComms.AbstractDevice, ::Mesh)FaceFiniteDifferenceSpace(::Mesh)in favor of usingFaceFiniteDifferenceSpace(::ClimaComms.AbstractDevice, ::Mesh)CenterFiniteDifferenceSpace(::Mesh)in favor of usingCenterFiniteDifferenceSpace(::ClimaComms.AbstractDevice, ::Mesh)FiniteDifferenceGrid(::Mesh)in favor of usingFiniteDifferenceGrid(::ClimaComms.AbstractDevice, ::Mesh)
GPU dispatching with
copyto!andfill!have been fixed PR #1802.
Added
FieldMatrixWithSolver, a wrapper that helps defining implicit Jacobians. PR #1788
-
Added
array2field(::Field)andfield2array(::Field)convenience functions, to help facilitate use with RRTMGP. PR #1768 -
Nvis now a type parameter in DataLayouts that have vertical levels. As a result, users can useDataLayouts.nlevels(::AbstractData)to obtain a compile-time constant for the number of vertical levels. -
Added
interpolate(field, target_hcoords, target_zcoord)convenience function so that theRemapperdoes not have to be explicitely constructed. PR #1764
run_field_matrix_solver!was fixed for column spaces, and tests were added to ensure it doesn't break in the future. PR #1750We're now using local memory (MArrays) in the
band_matrix_solve!, which has improved performance. PR #1735.We've specialized some cases in
run_field_matrix_solver!, which results in more efficient kernels being launched. PR #1732.We've reduced memory reads in the
band_matrix_solve!for tridiagonal systems, improving its performance. PR #1731.We've added NVTX annotations in ClimaCore functions, so that we have a more granular trace of performance. PRs #1726, #1723.
Extend adapt_structure for all operator and boundary condition types. Also use
unrolled_mapinmultiply_matrix_at_indexto avoid the recursive inference limit when compiling nested matrix operations. PR #1684Remappers can now process multipleFields at the same time if created with somebuffer_lenght > 1. PR (#1669) Machine-precision differences are expected. This change is breaking because remappers now return the same array type as the input field.We inlined the
multiple_field_solvekernels, which should improve performance. PR #1715.We added support for MultiBroadcastFusion, which allows users to fuse similar space point-wise broadcast expressions via
Fields.@fused_direct. PR #1641.
We fixed some fieldvector broadcasting on Julia 1.9. PR #1658.
We fixed an inference failure with matrix field broadcasting. PR #1683.
We now always inline for all ClimaCore kernels. PR #1647. This can result in more brittle inference (due to compiler heuristics). Technically, this is not a breaking change, but some code changes may be needed in practice.
fixed array allocation for interpolation on CPU. PR #1643.
fixed edge case in interpolation that led to incorrect vertical interpolation. PR #1640.
fixed
interpolate!for MPI runs. PR #1642.
support for many deprecated methods have been dropped PR #1632.
Slight performance improvement by replacing
rdivwithrmul. PR (#1496) Machine-precision differences are expected.Rewritten
distributed_remapping. Newdistributed_remappingis non-allocating and up to 1000x faster (on GPUs). Newdistributed_remappingno longer supports thephysical_zargument (this option is still available inRemapping.interpolate_column). Newinterpolate!function is available for remapping in-place. The new preferred way to define aRampperisRemapper(space, target_hcoords, target_zcoords)(instead ofRemapper(target_hcoords, target_zcoords, space)). PR (#1630)