-
Notifications
You must be signed in to change notification settings - Fork 0
Module setup #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Module setup #2
Changes from 36 commits
d00838c
382e994
8995967
393d8f8
7f6689d
62f7c95
622861c
177c28b
1c2d22c
8f490a2
b22620a
7451085
c203a62
cd3c2b4
16e0d36
dccd8c0
88d9dfa
5251b4c
f9a6b00
ecf5c99
1f2e8f8
83b4780
3274159
51b9e08
0278af0
e88b910
b5d37b3
bcace4d
0579889
eec3901
9fe6e8b
7548372
9c88280
7edc44d
ad7e446
187e6e0
c358ceb
71268ac
16f85bc
212491c
9d83d7a
1ba5f13
1bfb4c3
f16dfe3
23f559c
bde2d46
b01f549
07caabb
2c9f739
5a24951
82eb839
1402f4e
a1b65c1
8c96e3e
c4ec7c6
dd43ce3
257b561
2e2d502
0a6bae8
b8c9dc2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Changes here will be overwritten by Copier | ||
| # DO NOT MANUALLY MODIFY THIS FILE | ||
| _commit: latest | ||
| _src_path: https://github.com/calliope-project/data-module-template.git | ||
| author_email: j.a.c.launer@tudelft.nl | ||
| author_family_name: Launer | ||
| author_given_name: Jann | ||
| github_org: calliope-project | ||
| license: MIT | ||
| module_description: This data module produces capacity factors for PV and wind at | ||
| arbitrary spatial resolution. | ||
| module_long_name: PV and wind capacity factors | ||
| module_short_name: module_pv_wind |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Documentation: https://EditorConfig.org | ||
| # Inspired by Django .editorconfig file | ||
|
|
||
| root = true | ||
|
|
||
| [*] | ||
| indent_style = space | ||
| indent_size = 4 | ||
| trim_trailing_whitespace = true | ||
| end_of_line = lf | ||
| insert_final_newline = true | ||
| charset = utf-8 | ||
|
|
||
| [*.{yaml,yml,.cff}] | ||
| indent_size = 2 | ||
|
|
||
| [*.md] | ||
| trim_trailing_whitespace = true | ||
| indent_size = 4 | ||
|
|
||
| [*.{diff,patch}] | ||
| trim_trailing_whitespace = false |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| ### OS-specific | ||
| .DS_Store | ||
|
|
||
| ### Editors | ||
| *~ | ||
| \#*\# | ||
| **/.vscode | ||
| .$* | ||
|
|
||
| ### Byte-compiled files | ||
| __pycache__ | ||
| *.pyc | ||
|
|
||
| ### Environments | ||
| .pixi/ | ||
|
|
||
| ### Snakemake | ||
| .snakemake/ | ||
| gurobi.log | ||
|
|
||
| ### Notebooks | ||
| .ipynb_checkpoints | ||
|
|
||
| # Ignore mutable folders | ||
| **/logs/* | ||
| **/resources/* | ||
| **/results/* | ||
| **/temp/* | ||
| **/tmp/* |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| default_language_version: | ||
| python: python3 | ||
|
|
||
| repos: | ||
| # Generic 'file quality' testing | ||
| - repo: https://github.com/pre-commit/pre-commit-hooks | ||
| rev: v5.0.0 | ||
| hooks: | ||
| # Machine-friendliness | ||
| - id: trailing-whitespace | ||
| - id: end-of-file-fixer | ||
| # Repo quality | ||
| - id: check-added-large-files | ||
| args: [--enforce-all] | ||
| - id: forbid-submodules | ||
| - id: check-case-conflict | ||
| - id: check-illegal-windows-names | ||
| - id: check-merge-conflict | ||
| - id: debug-statements | ||
| # Detect config. file issues | ||
| - id: check-json | ||
| - id: check-yaml | ||
| args: [--unsafe] | ||
|
|
||
| # Python file formatting | ||
| - repo: https://github.com/astral-sh/ruff-pre-commit | ||
| rev: v0.9.6 | ||
| hooks: | ||
| - id: ruff | ||
| args: [--fix, --exit-non-zero-on-fix] | ||
| - id: ruff-format | ||
|
|
||
| # Snakemake file formatting | ||
| - repo: https://github.com/snakemake/snakefmt | ||
| rev: v0.10.2 | ||
| hooks: | ||
| - id: snakefmt | ||
|
|
||
| # Spelling | ||
| - repo: https://github.com/codespell-project/codespell | ||
| rev: v2.4.1 | ||
| hooks: | ||
| - id: codespell | ||
| files: .*\.(py|smk|md)$|^Snakefile$ | ||
|
|
||
| ci: # https://pre-commit.ci/ | ||
| autoupdate_schedule: monthly |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Read the Docs configuration file | ||
| # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details | ||
| version: 2 | ||
|
|
||
| build: | ||
| os: ubuntu-22.04 | ||
| tools: | ||
| python: "3.12" | ||
| jobs: | ||
| install: | ||
| - asdf plugin add pixi | ||
| - asdf install pixi latest | ||
| - asdf global pixi latest | ||
| build: | ||
| html: | ||
| - pixi run build-docs -d $READTHEDOCS_OUTPUT/html | ||
|
|
||
| mkdocs: | ||
| configuration: mkdocs.yaml |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # Integration with https://snakemake.github.io/snakemake-workflow-catalog/# | ||
| usage: | ||
| software-stack-deployment: | ||
| conda: true | ||
| report: false |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| This is the list of contributors for copyright purposes. | ||
|
|
||
| This does not necessarily list everyone who has contributed to this software's | ||
| code or documentation. For a full contributor list, see: | ||
| <https://github.com/calliope-project/module_pv_wind/graphs/contributors> | ||
|
|
||
| Jann Launer, <j.a.c.launer@tudelft.nl> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # Citation file | ||
| # For more information, see: | ||
| # https://citation-file-format.github.io/ | ||
| cff-version: 1.2.0 | ||
| message: If you use this software or data produced by it, please cite it using the metadata from this file. | ||
| title: "clio - module_pv_wind: PV and wind area potentials and capacity factors " | ||
| repository: "https://github.com/calliope-project/module_pv_wind" | ||
| license: MIT | ||
| authors: | ||
| - given-names: Jann | ||
| family-names: Launer |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Module Input-Output structure for automated doc. generation | ||
| resources: | ||
| user: | ||
| "cutout_{name_cutout}.nc": Cutout of meteorological data provided by user. | ||
| "tech_specs/{name_tech}.yaml": Technology specification of PV/wind. | ||
| "layout/{name_layout}.tif": Raster layout file describing the spatial distribution of capacity. | ||
| "layout/{name_layout}.parquet": Point layout file describing the spatial distribution of capacity. | ||
| "spatial_units/{name_spatial_units}.parquet": Geometries of spatial units (e.g., administrative regions). | ||
|
jnnr marked this conversation as resolved.
Outdated
|
||
| automatic: | ||
|
jnnr marked this conversation as resolved.
Outdated
|
||
| "cutout_era5.nc": Cutout of meteorological data downloaded from ERA5. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shouldn't the auto-downloaded ERA5 data have at least one wildcard in its path?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, the filename could reflect both spatial and temporal scope. However, as I argued above, better not tie this to the shapefiles. You will re-download large files if you want to analyse the same spatial scope at different resolutions (NUTS0, NUTS3, etc.). Better: Use a wildcard, but take that wildcard from the config, as we just discussed in person. |
||
| results: | ||
| "{name_cutout}/{name_spatial_units}/{name_layout}/capacityfactors_{name_tech}.nc": Capacity factors for the specified technology. | ||
| wildcards: | ||
| name_cutout: Cutout name | ||
| name_tech: Technology name | ||
| name_layout: Layout name | ||
| name_spatial_units: Spatial units name | ||
|
jnnr marked this conversation as resolved.
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| MIT License | ||
|
|
||
| Copyright (c) 2025 AUTHORS | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # PV and wind capacity factors | ||
|
|
||
| This data module produces capacity factors for PV and wind at arbitrary spatial resolution. | ||
|
|
||
| A modular `snakemake` workflow built for [`clio`](https://clio.readthedocs.io/) data modules. | ||
|
|
||
| ## Using this module | ||
|
|
||
| This module can be imported directly into any `snakemake` workflow. | ||
| Please consult the integration example in `tests/integration/Snakefile` for more information. | ||
|
|
||
| ## Development | ||
|
|
||
| We use [`pixi`](https://pixi.sh/) as our package manager for development. | ||
| Once installed, run the following to clone this repo and install all dependencies. | ||
|
|
||
| ```shell | ||
| git clone git@github.com:calliope-project/module_pv_wind.git | ||
| cd module_pv_wind | ||
| pixi install --all | ||
| ``` | ||
|
|
||
| For testing, simply run: | ||
|
|
||
| ```shell | ||
| pixi run test-integration | ||
| ``` | ||
|
|
||
| To view the documentation locally, use: | ||
|
|
||
| ```shell | ||
| pixi run serve-docs | ||
| ``` | ||
|
|
||
| To test a minimal example of a workflow using this module: | ||
|
|
||
| ```shell | ||
| pixi shell # activate this project's environment | ||
| cd tests/integration/ # navigate to the integration example | ||
| snakemake --use-conda --cores 2 # run the workflow! | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Configuration | ||
|
|
||
| This workflow is part of the [clio project](https://clio.readthedocs.io/en/latest/). | ||
| Please consult our documentation for more details. | ||
| Other useful resources are: | ||
|
|
||
| - `INTERFACE.yaml`: user input files (placed in `resources/user`) and module output files (placed in `results`). | ||
| - `workflow/internal/config.schema.yaml`: general configuration options. | ||
| - `tests/integration/`: a simple example of how to use this module. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # A minimal example of how to configure this module | ||
| download_cutout: True | ||
| backend: atlite | ||
| layout: raster # raster or point | ||
| aggregate: true | ||
| cutout_params: | ||
| module: era5 | ||
| x: [3.4, 7.2] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shouldn't the x and y bounds come from the input geometries (I guess
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That could be done. However, if you need Europe in NUTS0, NUTS2, NUTS3 and for several years, you will download a big dataset several times. |
||
| y: [50.7, 53.5] | ||
| time: ["2017-01", "2017-02"] | ||
| features: ['height', 'wind', 'influx', 'temperature'] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shouldn't this be auto-populated based on what technologies need to be run?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ideally yes. But as the variables are all united in one |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # Home | ||
|
|
||
| Welcome to the documentation of the `module_pv_wind` data module! | ||
| Please consult the [specification guidelines](./specification.md) and the [`clio` documentation](https://clio.readthedocs.io/) for more information. |
|
jnnr marked this conversation as resolved.
Outdated
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # yaml-language-server: $schema=https://squidfunk.github.io/mkdocs-material/schema.json | ||
|
|
||
| site_name: PV and wind area potentials and capacity factors | ||
| site_url: https://module_pv_wind.readthedocs.io/ | ||
| repo_name: module_pv_wind | ||
| repo_url: https://github.com/calliope-project/module_pv_wind/ | ||
| edit_uri: edit/main/docs/ | ||
| theme: | ||
| name: material | ||
| features: | ||
| - navigation.footer | ||
| - navigation.instant | ||
| - navigation.tabs | ||
| - navigation.tabs.sticky | ||
| - navigation.indexes | ||
| - search.suggest | ||
| - search.highlight | ||
| - toc.follow | ||
| palette: | ||
| # Palette toggle for automatic mode | ||
| - media: "(prefers-color-scheme)" | ||
| toggle: | ||
| icon: material/brightness-auto | ||
| name: Switch to light mode | ||
| primary: red | ||
| accent: pink | ||
| # Palette toggle for light mode | ||
| - media: "(prefers-color-scheme: light)" | ||
| scheme: default | ||
| toggle: | ||
| icon: material/weather-sunny | ||
| name: Switch to dark mode | ||
| primary: red | ||
| accent: pink | ||
| # Palette toggle for dark mode | ||
| - media: "(prefers-color-scheme: dark)" | ||
| scheme: slate | ||
| toggle: | ||
| icon: material/weather-night | ||
| name: Switch to system preference | ||
| primary: red | ||
| accent: pink | ||
| markdown_extensions: | ||
| - admonition | ||
| - pymdownx.arithmatex: | ||
| generic: true | ||
| - pymdownx.betterem: | ||
| smart_enable: all | ||
| - pymdownx.details | ||
| - pymdownx.emoji: | ||
| emoji_index: !!python/name:material.extensions.emoji.twemoji | ||
| emoji_generator: !!python/name:material.extensions.emoji.to_svg | ||
| - pymdownx.highlight: | ||
| anchor_linenums: true | ||
| - pymdownx.snippets: | ||
| check_paths: true | ||
| restrict_base_path: true | ||
| - pymdownx.superfences: | ||
| custom_fences: | ||
| - name: mermaid | ||
| class: mermaid | ||
| format: !!python/name:mermaid2.fence_mermaid_custom | ||
| - toc: | ||
| anchorlink: true | ||
| plugins: | ||
| - search | ||
| copyright: Copyright © since 2025 <a href="https://github.com/calliope-project/module_pv_wind/blob/main/AUTHORS">module_pv_wind contributors</a> (MIT licensed) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| [mypy] | ||
| disable_error_code = import-untyped | ||
| exclude = (^|/)\.(snakemake|pixi)(/|$) |
Uh oh!
There was an error while loading. Please reload this page.