Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ formats (m4a, video) need [ffmpeg](https://ffmpeg.org/download.html) installed.

```bash
# 2. Train your own for free, on your laptop: ~30 minutes on an Apple Silicon Mac, ~3 GB download
# Needs Python 3.12 and Poetry (pip install poetry)
git clone https://github.com/alexkroman/tiny-audio.git && cd tiny-audio
poetry install
poetry run ta train asr -e tiny
Expand All @@ -65,6 +66,14 @@ then drops below 1 around step 500 as the decoder starts reading the audio. The
unseen LibriSpeech audio at about 16% WER, with mistakes like "furnishings" for "furniture". It fits
in 16 GB of memory and also runs on a CUDA GPU, including a free Colab T4.

What runs where:

| Step | Apple Silicon (16 GB) | Any CUDA GPU (incl. Colab T4) | CPU only | One 80 GB GPU |
| ---------------------------- | --------------------- | ----------------------------- | --------- | ------------- |
| 1. Transcribe | Yes | Yes | Yes, slow | Yes |
| 2. Free run (`-e tiny`) | Yes, ~30 min | Yes | Slow | Yes |
| 3. Full run (default config) | No | Needs 80 GB | No | Yes, a day+ |

No install at all: the **[live demo](https://huggingface.co/spaces/mazesmazes/tiny-audio)** lets you
record yourself or upload a file.

Expand Down Expand Up @@ -185,6 +194,25 @@ Every training setting is a [Hydra](https://hydra.cc/) override, for example
`model.projector_hidden_dim=2048`. `ta push` publishes a model to the Hub and `ta deploy` puts a
demo like the one above on a Space.

## Stuck?

| Problem | Fix |
| ------------------------------------------- | ----------------------------------------------------- |
| `poetry install` refuses the Python version | Install Python 3.12, then `poetry env use python3.12` |
| Model download fails | `poetry run hf auth login`, or export `HF_TOKEN` |
| `flash_attention_2` warning | Expected off CUDA; the model falls back to SDPA |
| Out of GPU memory during the free run | Halve the batch and keep the effective batch of 4: |

```bash
poetry run python scripts/train.py +experiments=tiny \
training.per_device_train_batch_size=1 training.gradient_accumulation_steps=4
```

Anything else: check
[Common issues](https://alexkroman.github.io/tiny-audio/4-quick-reference/#common-issues), then
[open an issue](https://github.com/alexkroman/tiny-audio/issues) with the command you ran and its
output.

## Contributing

Want to try a new projector, add a dataset, or change the codebase? See
Expand Down
19 changes: 16 additions & 3 deletions docs/course/0-course-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@ A hands-on course: build, train, evaluate, and deploy your own ASR model with Ti
| **Cost** | Classes 1 and 3 are free. Class 2's cloud run costs GPU time (see [Budget](#budget)) |
| **Prerequisites** | Python, the command line, git |

> **Short on time? Three steps, no accounts, no cost.** (1) Transcribe a clip with the published
> model ([Class 1, Exercise 2](./1-introduction-and-setup.md#exercise-2-run-inference-15-min), 5
> minutes). (2) Train your own on your laptop with `ta train asr -e tiny`
> ([Class 2, Exercise 0](./2-training.md#exercise-0-the-free-run-optional-30-min-unattended), about
> 30 minutes unattended). (3) Read the
> [projector](https://github.com/alexkroman/tiny-audio/blob/main/tiny_audio/projectors.py), the only
> part trained from scratch, about twenty lines of code. The classes below explain what you just
> ran, then scale it up to a real training run.

## What You'll Build

```text
Expand Down Expand Up @@ -63,6 +72,7 @@ ______________________________________________________________________
**Time:** 1 hour, then training runs on its own

- What `scripts/train.py` does, step by step
- A free rehearsal on your laptop: the default recipe, scaled down
- Sizing a cloud GPU with `ta runpod plan`
- Writing your own experiment config and launching on RunPod
- Reading the loss and gradient curves
Expand Down Expand Up @@ -114,9 +124,12 @@ ______________________________________________________________________
**Development (Classes 1 and 3):** any modern laptop with 8 GB RAM and 20 GB of free disk. Loading
the published model downloads about 6 GB (its own weights plus the encoder's repo).

**Training (Class 2):** a RunPod account and a GPU. There is no laptop-sized training recipe: the
default config is a 2B decoder sized for one 80 GB GPU, so Class 2 trains in the cloud. The class
shows you how to size the GPU with `ta runpod plan`.
**Free run (optional, Class 2):** a 16 GB Apple Silicon Mac or any CUDA GPU, including a free Colab
T4. `ta train asr -e tiny` peaks at 8-10 GB of GPU memory and takes about 30 minutes on an M4 Max.

**Course run (Class 2):** a RunPod account and a GPU. The default config is a 2B decoder sized for
one 80 GB GPU, so the course run trains in the cloud. The class shows you how to size the GPU with
`ta runpod plan`.

______________________________________________________________________

Expand Down
29 changes: 26 additions & 3 deletions docs/course/2-training.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,9 +142,32 @@ ______________________________________________________________________

## Part B: Hands-On (40 min)

There is no laptop-sized smoke-test recipe: the default config is a 2B decoder sized for one 80 GB
80GB, so all training in this class happens on a rented GPU. Exercises 1 and 2 (sizing and writing
your config) still run on your laptop.
The course run uses the default 2B decoder, sized for one 80 GB GPU, so it trains on a rented GPU.
Exercises 1 and 2 (sizing and writing your config) still run on your laptop, and so does Exercise 0:
a free rehearsal of the whole loop before you spend anything.

### Exercise 0: The Free Run (optional, ~30 min unattended)

`configs/experiments/tiny.yaml` is the default recipe with the smaller Qwen3.5-0.8B decoder, trained
on LibriSpeech dev-clean (5.4 hours, a ~340 MB download). It peaks at 8-10 GB of GPU memory, so it
runs on a 16 GB Apple Silicon Mac or any CUDA GPU, including a free Colab T4. Nothing leaves your
machine: no W&B, no Hub, no RunPod.

```bash
poetry run ta train asr -e tiny
```

Watch the loss in the log. It sits near 4 while the projector's output is still noise to the
decoder, then drops below 1 somewhere around step 400-650 as the decoder starts reading the audio.
That drop is the moment the encoder and the decoder start talking to each other; the cloud run below
shows the same curve on far more data. When it finishes, read what it learned:

```bash
poetry run ta eval -m outputs/tiny --local-code -d librispeech -n 20
```

Expect about 16% WER, with mistakes like "furnishings" for "furniture". Start it before Exercise 1
and it trains while you work through the rest of the class.

### Exercise 1: Size Your Cloud Run (5 min)

Expand Down
6 changes: 5 additions & 1 deletion docs/course/4-quick-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,11 @@ poetry run ta dev format # black, ruff, mdformat
### Training (Local)

```bash
# Default recipe: Granite + Qwen3.5-2B + LoRA (80 GB GPU, terabyte of data). No laptop recipe.
# Free run: the default recipe with Qwen3.5-0.8B on LibriSpeech dev-clean (16 GB Mac or any GPU)
poetry run ta train asr -e tiny
poetry run ta eval -m outputs/tiny --local-code -d librispeech -n 20

# Default recipe: Granite + Qwen3.5-2B + LoRA (80 GB GPU, terabyte of data)
poetry run python scripts/train.py

# Your own experiment (configs/experiments/my_run.yaml), with overrides
Expand Down