Skip to content

Module setup#2

Open
jnnr wants to merge 60 commits into
mainfrom
feature-module-setup
Open

Module setup#2
jnnr wants to merge 60 commits into
mainfrom
feature-module-setup

Conversation

@jnnr

@jnnr jnnr commented Jun 18, 2025

Copy link
Copy Markdown
Collaborator

This sets up a first version of the data module for pv and wind capacity factors.

To test the module, download the zip file "module_pv_wind_files.zip" (https://surfdrive.surf.nl/s/NZDE4C2NKXj3EQG), unzip it and put it into a directory tests/files/. Then, cd tests/integration and run either snakemake nld_raster, snakemake nld_point for a small example, or snakemake europe_raster for a large example.

@irm-codebase

irm-codebase commented Sep 22, 2025

Copy link
Copy Markdown

Brief list of unrequested suggestions to increase compatibility with other modules.

  • avoid the netCDF ERA5 input by just accessing ERA5 data from xarray directly using the https://platform.destine.eu/ service. All you need is to request users to sign in and create a .netrc file. We can create this file as a SECRET in the calliope organisation to enable integration tests.
  • do not use GeoJSON. We use geo-parquet for efficiency purposes, and translating to GeoJSON bloats the process in general.
  • fill in INTERFACE.yaml. It's the primary way users can know what is expected from them input-wise.
  • Consider adding a short description / list of steps to README.md explaining how to use this module.

@sjpfenninger

sjpfenninger commented Nov 12, 2025

Copy link
Copy Markdown
Contributor

@jnnr I've made some changes to get the tests to run, but they still don't. Could you look into why the integration test fails? Basically, running pixi install; pixi run test-integration should work.

(I already updated to the latest template, so that's not needed here unlike in module_demand_electricity)

@sjpfenninger sjpfenninger left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good start - with a bit of cleaning up we can build on this further very well!

Comment thread LICENSE
Comment thread config/config.yaml
aggregate: true
cutout_params:
module: era5
x: [3.4, 7.2]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the x and y bounds come from the input geometries (I guess spatial_units/{name_spatial_units}.parquet)?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could be done. However, if you need Europe in NUTS0, NUTS2, NUTS3 and for several years, you will download a big dataset several times.

Comment thread .readthedocs.yaml Outdated
Comment thread mkdocs.yaml Outdated
Comment thread INTERFACE.yaml Outdated
Comment thread workflow/scripts/_backend_atlite.py
Comment thread workflow/scripts/_backend_atlite.py Outdated
import xarray as xr


def cf_agg_from_point_layout(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shares a lot of code with cf_agg_from_raster_layout so I would merge the two

Comment thread workflow/scripts/prepare_capacityfactors_point_layout.py Outdated
Comment thread workflow/scripts/prepare_capacityfactors_point_layout.py Outdated
@@ -0,0 +1,72 @@
"""Prepare PV capacityfactors, given a cutout, a layout, spatial units to aggregate to and technology specifications."""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to share a lot of code with prepare_capacityfactors_raster_layout.py so I would merge this into one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants