Nine config keys are reachable only by reading the YAML. The generated README documents the five commands
and their artifacts, then says "any extra argument is a Hydra override" — true, and the reason nobody
discovers pipeline.
What to add
A Configuration section in README.md.jinja. Its content is roughly the live-key table from the audit
— the keys that select a branch and change what the commands actually do:
| key |
commands |
non-default behaviour |
do_resume |
pretrain, supervised_train |
reuse scratch checkpoints from a previous attempt, or discard them |
work_dir |
pretrain, supervised_train |
relocate the scratch directory away from the output sibling |
clean_work_dir |
pretrain, supervised_train |
keep the scratch tree instead of rmtreeing it |
pipeline |
preprocess_data |
run an additional CLI (MEDS_transform-pipeline) before tensorization |
pipeline_overrides |
preprocess_data |
Hydra overrides appended to that CLI |
attach_labels |
predict |
emit predictions without boolean_value |
seed |
pretrain, supervised_train |
null skips seed_everything entirely |
splits |
infer, predict |
accepts null, a bare string, or a list |
do_reshard |
preprocess_data |
false skips reshard_to_split, requiring split-sharded input |
input_data_dir: null |
predict |
recover the workspace from the source artifact's manifest |
do_overwrite |
all |
replace an existing artifact instead of refusing |
The inert keys (max_seq_len, batch_size, num_workers, trainer.*, optimizer, model.*, the
output_* paths, the paths/ interpolations) need only a one-line pointer at configs/ — documenting
them individually documents Hydra, not this template.
Acceptance
Notes
One issue on purpose. Nine per-key issues would put nine agents in README.md.jinja and produce
nothing but conflicts.
Coordinate with #6, which adds determinism keys that belong in the same section.
From the Hydra configuration audit, §1, §2 and §6 (docs/CONFIG-AUDIT.md).
Nine config keys are reachable only by reading the YAML. The generated README documents the five commands
and their artifacts, then says "any extra argument is a Hydra override" — true, and the reason nobody
discovers
pipeline.What to add
A Configuration section in
README.md.jinja. Its content is roughly the live-key table from the audit— the keys that select a branch and change what the commands actually do:
do_resumework_dirclean_work_dirrmtreeing itpipelineMEDS_transform-pipeline) before tensorizationpipeline_overridesattach_labelsboolean_valueseednullskipsseed_everythingentirelysplitsnull, a bare string, or a listdo_reshardfalseskipsreshard_to_split, requiring split-sharded inputinput_data_dir: nulldo_overwriteThe inert keys (
max_seq_len,batch_size,num_workers,trainer.*,optimizer,model.*, theoutput_*paths, thepaths/interpolations) need only a one-line pointer atconfigs/— documentingthem individually documents Hydra, not this template.
Acceptance
pipeline+do_reshardinteraction is stated (see preprocess_data: do_reshard guard, split-table desync, and the untested default path #2) — it is the one combination a reader canotherwise only discover by hitting it.
Notes
One issue on purpose. Nine per-key issues would put nine agents in
README.md.jinjaand producenothing but conflicts.
Coordinate with #6, which adds determinism keys that belong in the same section.
From the Hydra configuration audit, §1, §2 and §6 (
docs/CONFIG-AUDIT.md).