[alexkroman]

Summary

• Noise: MUSAN's noise set (~930 files, 6 h) → DNS5 AudioSet + Freesound noise plus WHAM! (~240 h). WHAM! adds café, restaurant and street babble, the noise type URGENT's hardest clips carry.
• Rooms: the 270 MIT IR Survey responses plus synthetic rooms → DNS5's 60k room responses (OpenSLR 26 simulated rooms + 28 real ones). That is URGENT's RIR set. Synthetic reverb is removed, and reverb_prob now raises without a room corpus.
• What URGENT held out for its validation set (the leaderboard test) stays out of training:
  • WHAM!: train split only.
  • DNS5 noise: the alphabetically first quarter of files is skipped. URGENT held out the first 1/8 by name of each bandwidth group, and those groups aren't public, so the exclusion covers most of it but can't be exact.
  • Rooms: URGENT never split them, so all are used.
• Code:
  • AudioCorpus takes {source: glob}, so one corpus can pool several sources.
  • It unpacks parquet shards (the Hub mirrors philgzl/dns5 and philgzl/wham are Opus/FLAC in parquet) to one file per row, once, before DataLoader workers fork. The unpack writes to a temporary directory and renames it into place, so concurrent ranks are safe.
  • Config: noise_corpus and reverb_corpus become mappings. The *_glob fields and reverb_rt60 are removed.
  • pyarrow is now declared; it was already locked through datasets.
• Bug fix: bandlimit now rescales its output. Both the resampled and the brickwall low-pass ring past a full-scale input's peak (1.002 was seen after reverb).
• Disk: the pod needs about 8 GB for the shards plus about 8 GB unpacked, under HF_HOME.
• Licence: WHAM! is CC BY-NC 4.0. DNS5 is CC BY 4.0.

Why

URGENT is 1/9 of the leaderboard mean. More varied noise that matches the test should help that column, which ranks 24/56. I estimate 0.5–1.5 WER there (0.05–0.15 on the mean of 9) and no change on the clean sets. That estimate is not measured yet; the next training run will tell.

Test plan

• Unit tests: parquet unpacking (raw bytes and Hub Audio structs), each shard parsed only once, pooling across sources, reverb drawing only from the corpus, the no-corpus error, band-limit ringing on a full-scale square wave
• Real data: one shard each of DNS5 rooms, DNS5 noise and WHAM! decoded through AudioCorpus, then reverberated and mixed (finite, peak ≤ 1)
• ta dev check, ta dev test, pre-commit gate

🤖 Generated with Claude Code