Skip to content

Document the live config keys in the generated README #5

Description

@florian6973

Nine config keys are reachable only by reading the YAML. The generated README documents the five commands
and their artifacts, then says "any extra argument is a Hydra override" — true, and the reason nobody
discovers pipeline.

What to add

A Configuration section in README.md.jinja. Its content is roughly the live-key table from the audit
— the keys that select a branch and change what the commands actually do:

key commands non-default behaviour
do_resume pretrain, supervised_train reuse scratch checkpoints from a previous attempt, or discard them
work_dir pretrain, supervised_train relocate the scratch directory away from the output sibling
clean_work_dir pretrain, supervised_train keep the scratch tree instead of rmtreeing it
pipeline preprocess_data run an additional CLI (MEDS_transform-pipeline) before tensorization
pipeline_overrides preprocess_data Hydra overrides appended to that CLI
attach_labels predict emit predictions without boolean_value
seed pretrain, supervised_train null skips seed_everything entirely
splits infer, predict accepts null, a bare string, or a list
do_reshard preprocess_data false skips reshard_to_split, requiring split-sharded input
input_data_dir: null predict recover the workspace from the source artifact's manifest
do_overwrite all replace an existing artifact instead of refusing

The inert keys (max_seq_len, batch_size, num_workers, trainer.*, optimizer, model.*, the
output_* paths, the paths/ interpolations) need only a one-line pointer at configs/ — documenting
them individually documents Hydra, not this template.

Acceptance

Notes

One issue on purpose. Nine per-key issues would put nine agents in README.md.jinja and produce
nothing but conflicts.

Coordinate with #6, which adds determinism keys that belong in the same section.

From the Hydra configuration audit, §1, §2 and §6 (docs/CONFIG-AUDIT.md).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    configHydra configuration surfacedocumentationImprovements or additions to documentation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions