Repository navigation
Add model.py: the whole speech LLM in one readable file - #154
Merged
Merged
Conversation
A deliberately simple, heavily commented teaching version of the model: frozen Whisper encoder, trained frame-stacking MLP projector, frozen LLM decoder, label masking, a greedy transcribe(), and a tiny overfit loop (`python model.py`). tests/test_model_py.py runs it end to end on the offline fixtures so the read-first file cannot silently rot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KL2vCbyZuBDbEM6iyKcPsx
Following the course-reframe plan, step 1: - README leads with the build-promise and the nanoGPT positioning, then a three-step quickstart (model.py on a laptop, the published model, the full recipe), a "Read this first" section pointing at model.py, a numbered chapter table linking into the existing course site, and the WER table under "Expected results". Only numbers already in the repo are used; the per-budget WER table waits on real runs. - The HTTP serving reference moves verbatim to docs/SERVING.md. - model.py is now linted and strict-type-checked with the rest of the code (CODE_PATHS, pyright include); build_prompt always returns labels. - pyproject description matches the new one-liner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KL2vCbyZuBDbEM6iyKcPsx
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KL2vCbyZuBDbEM6iyKcPsx
Keep one code path: a standalone teaching model would drift from the recipe that actually trains the published model. "Read this first" now sends readers to tiny_audio/projectors.py, then to _encode_audio / forward / generate in tiny_audio/asr_modeling.py. The quickstart is the published model plus the default training run; model.py's entries in CODE_PATHS, the pyright include list and CONTRIBUTING are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KL2vCbyZuBDbEM6iyKcPsx
configs/experiments/tiny.yaml layers the 73-clip LibriSpeech sample and portable hardware settings (adamw_torch, no bf16 autocast or tf32, no dataloader workers, no W&B/Hub, unfused CE allowed) onto the default recipe. The model block is inherited unchanged, so the free run trains exactly the encoder/projector/LoRA setup the published model uses, on a laptop or a free Colab GPU, and saves to outputs/tiny for `ta eval --local-code`. Smoke-tested on CPU with the offline fixture models standing in for Granite/Qwen3.5: the loop trains, the loss falls, and the saved checkpoint loads and transcribes through the working-tree pipeline. README, QUICKSTART, CONTRIBUTING and the course budget table point at it; a config test pins that it shares the default model block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KL2vCbyZuBDbEM6iyKcPsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A deliberately simple, heavily commented teaching version of the model:
frozen Whisper encoder, trained frame-stacking MLP projector, frozen LLM
decoder, label masking, a greedy transcribe(), and a tiny overfit loop
(
python model.py). tests/test_model_py.py runs it end to end on theoffline fixtures so the read-first file cannot silently rot.
Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01KL2vCbyZuBDbEM6iyKcPsx