4/8 Feature GISA-new impervious surface

[Timothy-W-Hilton]

Add scalable GISA-new impervious-surface preprocessing

Summary

Add support for preparing GISA-new impervious-surface data on a VPRM satellite grid. The implementation is designed for high-resolution sources that can exceed the practical source-grid size handled by one ESMF_RegridWeightGen invocation.

What this adds

• A gisa_new satellite manager for GISA-new GeoTIFF tiles.
  • GISA-new values record first impervious-surface detection year as an index: 1 is 1985 and 37 is 2021.
  • The manager derives first-detection year and a target-year binary impervious-surface percentage field (0 or 100 percent).
  • It lazily opens source tiles, crops before mosaicking, and handles 180th-meridian-crossing geographic tile coordinates.
• vprm_preprocessor.add_impervious_surface_area().
  • Conservatively regrids an ISA percentage field to the active satellite grid and stores it as impervious_surface_area.
  • Preserves ISA data when compatible preprocessors are combined into a multi-tile domain.
• General partitioned conservative-regridding utilities.
  • Split an oversized regular source grid into contiguous y-axis partitions.
  • Generate and cache one ESMF weight file per partition.
  • Sum the disjoint destination-area-normalized contributions.
  • Exclude zero ISA source cells through SCRIP grid_imask, reducing ESMF memory use for sparse impervious-surface products.
  • Restore the destination CRS and affine transform on the returned dataset.

Why

For two New Zealand MODIS tiles, a full-resolution GISA-new source grid approached 391 million cells. A single ESMF weight-generation run failed near element ID 2^28 with a degenerate-element error. Splitting the source grid keeps each ESMF mesh below that practical limit while preserving conservative destination contributions. The mask is equally important because GISA is mostly zero over ocean and other non-impervious areas.

The resulting helper is intentionally general: any regular, high-resolution, sparse source field can use it when omitted source cells legitimately represent zero destination contribution.

Example

from pyVPRM.sat_managers.gisa_new import gisa_new

isa = gisa_new("/data/GISA-new", target_year=2021)
isa.load()
isa.crop_to_polygon(target_grid_polygon)

vprm_pre.add_impervious_surface_area(
    isa,
    regridder_save_path="/data/weights/gisa_to_satellite.nc",
    max_source_cells=150_000_000,
)

Validation

• Added self-contained tests for GISA detection-index conversion, source-mask propagation, source partitioning and contribution summation, and destination CRS/transform preservation.
• Test suite on this branch: 4 passed.
• The partitioning and masking approach was also exercised against full GISA source grids for all five New Zealand MODIS tiles outside this test suite.