Skip to content

Enable creating graph from datastore YAML file#128

Closed
matschreiner wants to merge 11 commits intomllam:mainfrom
matschreiner:create_graph_from_datastore
Closed

Enable creating graph from datastore YAML file#128
matschreiner wants to merge 11 commits intomllam:mainfrom
matschreiner:create_graph_from_datastore

Conversation

@matschreiner
Copy link
Copy Markdown

@matschreiner matschreiner commented Feb 13, 2025

This PR adds support for creating graphs directly from datastores. The current approach introduces unnecessary complexity by requiring neural_lam_config, even though graph creation only needs a datastore, and the sole function of the neural_lam_config in this context is to be the owner of the datastore path.

The first positional argument in

python -m neural_lam.create_graph first_arg

still refers to a neurallam_config and if both datastore and neurallam_config are specified, neurallam_config takes precedence, ensuring this is a non-breaking change. However, I propose deprecating neurallam_config usage in the future.

Imports are now explicit instead of relative, allowing create_graph.py to be executed both as a standalone script (python create_graph.py) and as a module (python -m neural_lam.create_graph).
This change improves usability, as some users feel more confident running scripts the traditional way.

New Script Arguments:

--datastore: Path to the datastore.
--datastore_type: Type of datastore (defaults to mdp). The script will throw an error if an invalid type is provided.

The change doesn't introduce any new dependencies to the code.

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch - if not update your fork with the changes from the target branch (use pull with --rebase option if possible).
  • I have performed a self-review of my code
  • For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
  • I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
  • I have updated the README to cover introduced code changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have given the PR a name that clearly describes the change, written in imperative form (context).
  • I have requested a reviewer and an assignee (assignee is responsible for merging). This applies only if you have write access to the repo, otherwise feel free to tag a maintainer to add a reviewer and assignee.

Checklist for reviewers

Each PR comes with its own improvements and flaws. The reviewer should check the following:

  • the code is readable
  • the code is well tested
  • the code is documented (including return types and parameters)
  • the code is easy to maintain

Author checklist after completed review

  • I have added a line to the CHANGELOG describing this change, in a section
    reflecting type of change (add section where missing):
    • added: when you have added new functionality
    • changed: when default behaviour of the code has been changed
    • fixes: when your contribution fixes a bug

Checklist for assignee

  • PR is up to date with the base branch
  • the tests pass
  • author has added an entry to the changelog (and designated the change as added, changed or fixed)
  • Once the PR is ready to be merged, squash commits and merge the PR.

@matschreiner
Copy link
Copy Markdown
Author

@leifdenby

@matschreiner matschreiner requested a review from SimonKamuk March 7, 2025 15:05
@joeloskarsson
Copy link
Copy Markdown
Collaborator

As we are working towards #83, I am not sure how to prioritize updates to the current create_graph.py. But this is a small and useful change, however the PR would need to be made up to date with main. If you wan to do this we can go ahead and push this through, but if you feel that this is not worth putting effort to now we should probably just close this or change to a draft PR. Tagging also @leifdenby for visibility.

@sadamov sadamov marked this pull request as draft February 21, 2026 12:06
@sadamov sadamov added the enhancement New feature or request label Apr 13, 2026
@sadamov
Copy link
Copy Markdown
Collaborator

sadamov commented Apr 28, 2026

Thanks @matschreiner. Following up on @joeloskarsson's earlier suggestion - closing this in favour of the weather-model-graphs migration in #384 / #596, since create_graph.py won't see further development. Reopen if you'd like to revive against the new path once it lands.

@sadamov sadamov closed this Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants