Skip to content

Speed up CUDA inference on the RTX 5090 #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
praveenperera merged 13 commits into from
Oct 23, 2026
Merged

Speed up CUDA inference on the RTX 5090 #75

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
f86d176
Add RTX 5090 FP16 early trunk recipe
praveenperera Oct 22, 2026
d3933b1
Launch the next embedding batch before reading one
praveenperera Oct 22, 2026
29e5b13
Segment the next CUDA file during embedding
praveenperera Oct 22, 2026
d5d2ba9
Pair FP16 trunk taps into m16n8k16 on the 5090
praveenperera Oct 22, 2026
e3dddab
Run the whole 5090 trunk on FP16 at large batches
praveenperera Oct 22, 2026
17dc9a0
Mention the RTX 5090 kernel profile in the README
praveenperera Oct 22, 2026
21bb1a9
Add FP16 Winograd trunk kernels for the RTX 5090
praveenperera Oct 23, 2026
58cb839
Run 64-channel trunk layers on FP16 Winograd
praveenperera Oct 23, 2026
b387103
Add RTX 5090 rows to the CUDA benchmarks
praveenperera Oct 23, 2026
bb3bc55
Refresh the RTX 4090 CUDA benchmarks
praveenperera Oct 23, 2026
365ae4c
Scope the Fp16Policy test import to its test
praveenperera Oct 23, 2026
df8abb6
Move README benchmark edits into the crate docs
praveenperera Oct 23, 2026
0b0890a
Clear unfinished CUDA batches after a failed run
praveenperera Oct 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Fast Rust speaker diarization (who spoke when) with pyannote-level accuracy.

On VoxConverse dev, `speakrs` gets **7.0% DER at 1786x realtime** on an RTX 4090, so an hour of audio takes about 2.0 seconds. On an Apple M4 Pro with CoreML it gets 7.1% DER at 529x. pyannote gets 7.2% at 32x on an RTX 4090, and 24x on the M4 Pro. Full results are in [benchmarks/](https://github.com/avencera/speakrs/tree/master/benchmarks).
On VoxConverse dev, `speakrs` gets **7.0% DER at 1912x realtime** on an RTX 4090, so an hour of audio takes about 1.9 seconds. On an Apple M4 Pro with CoreML it gets 7.1% DER at 529x. pyannote gets 7.2% at 32x on an RTX 4090, and 24x on the M4 Pro. Full results are in [benchmarks/](https://github.com/avencera/speakrs/tree/master/benchmarks).

If you want a small end-to-end app using it, see [avencera/smrze](https://github.com/avencera/smrze).

Expand Down Expand Up @@ -162,18 +162,18 @@ meetings, lectures, podcasts, or bulk archives.

## Benchmarks

On VoxConverse dev on an RTX 4090, speakrs `cuda` is about 56 times faster than
an earlier pyannote CUDA run, with 7.0% versus 7.2% DER. `cuda-fast` is about 105
On VoxConverse dev on an RTX 4090, speakrs `cuda` is about 60 times faster than
an earlier pyannote CUDA run, with 7.0% versus 7.2% DER. `cuda-fast` is about 115
times faster, with 7.4% DER.
On an RTX 4090, an hour of audio takes about 2.0 seconds with `cuda` and
1.1 seconds with `cuda-fast`. On an Apple M4 Pro with CoreML it takes about 7 seconds.
On an RTX 4090, an hour of audio takes about 1.9 seconds with `cuda` and
1.0 second with `cuda-fast`. On an Apple M4 Pro with CoreML it takes about 7 seconds.

VoxConverse dev, collar=0ms:

| Platform | Implementation | DER | Time | RTFx |
|----------|----------------|-----|------|------|
| RTX 4090 | `speakrs` `cuda` | **7.0%** | 40.9s | 1786x |
| RTX 4090 | `speakrs` `cuda-fast` | 7.4% | 22.0s | **3325x** |
| RTX 4090 | `speakrs` `cuda` | **7.0%** | 38.2s | 1912x |
| RTX 4090 | `speakrs` `cuda-fast` | 7.4% | 20.0s | **3646x** |
| RTX 4090 | pyannote community-1 (CUDA) | 7.2% | 2301.3s | 32x |
| Apple M4 Pro | `speakrs` `coreml` | **7.1%** | 138s | **529x** |
| Apple M4 Pro | `speakrs` `coreml-fast` | 7.4% | 169s | 434x |
Expand All @@ -194,21 +194,21 @@ threads ahead of it; it clusters each finished file while the GPU runs the next.

| Dataset | Audio | `cuda` DER | `cuda` RTFx | `cuda-fast` DER | `cuda-fast` RTFx |
|---|---:|---:|---:|---:|---:|
| VoxConverse dev | 20.3 h | 7.0% | 1786x | 7.4% | 3325x |
| VoxConverse test | 43.5 h | 11.1% | 1921x | 11.2% | 3722x |
| AMI IHM | 18.7 h | 17.0% | 1955x | 17.4% | 3796x |
| AMI SDM | 18.7 h | 19.7% | 1874x | 20.6% | 3665x |
| AISHELL-4 | 12.7 h | 11.1% | 1916x | 11.4% | 3706x |
| Earnings-21 | 39.3 h | 9.7% | 1864x | 9.2% | 3578x |
| ICSI | 71.7 h | 33.3% | 1814x | 33.7% | 3628x |
| AVA-AVD | 4.4 h | 45.4% | 2083x | 48.9% | 3899x |
| VoxConverse dev | 20.3 h | 7.0% | 1912x | 7.4% | 3646x |
| VoxConverse test | 43.5 h | 11.1% | 1859x | 11.2% | 3618x |
| AMI IHM | 18.7 h | 17.0% | 1909x | 17.4% | 3563x |
| AMI SDM | 18.7 h | 19.7% | 1791x | 20.6% | 3754x |
| AISHELL-4 | 12.7 h | 11.1% | 1866x | 11.4% | 3534x |
| Earnings-21 | 39.3 h | 9.7% | 1876x | 9.2% | 3613x |
| ICSI | 71.7 h | 33.3% | 1937x | 33.7% | 3708x |
| AVA-AVD | 4.4 h | 45.4% | 2075x | 48.9% | 3927x |

That's about 229 hours of audio in under 8 minutes with `cuda`, or under 4
minutes with `cuda-fast`. The VoxConverse dev row comes from the runs above;
the other rows are single runs of the same version on RTX 4090 containers with
16 CPU cores and 8 GiB of RAM.
On VoxConverse test, `cuda` and pyannote CUDA both score 11.1% DER; pyannote
ran at 25x in an earlier container, about 78 times slower than `cuda`. On AMI
ran at 25x in an earlier container, about 75 times slower than `cuda`. On AMI
IHM and Earnings-21, `cuda` matches the earlier pyannote runs at 17.0% and 9.7% DER,
where pyannote ran at 15x and 18x. On macOS, `coreml` runs these datasets at
450x to 644x, with DER within 1.6 points of pyannote's. See
Expand Down Expand Up @@ -356,7 +356,7 @@ VoxConverse subset, and their times include process start-up and model loading:
| RTX 4090 | 1453x | 1438x |

Every run had 16 CPU cores, and each value is the median of 3 runs. The RTX 4060
Ti and RTX 5060 Ti also have tuned kernel profiles. In an earlier version, on the
Ti, RTX 5060 Ti and RTX 5090 also have tuned kernel profiles. In an earlier version, on the
same subset, speakrs's own kernels were 1.4x to 2.0x faster than cuDNN and cuBLAS
on the RTX 4090, L4, A10 and H100. See [benchmarks/](https://github.com/avencera/speakrs/tree/master/benchmarks)
for full results across all datasets.
Expand Down
77 changes: 50 additions & 27 deletions benchmarks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,17 @@ Hardware: Apple M4 Pro, macOS 26.3

## Linux (CUDA)

The speakrs CUDA rows use NVIDIA RTX 4090 containers with 16 CPU cores and
8 GiB of RAM. The cloud host doesn't report its CPU model. The VoxConverse
pyannote rows ran earlier in a container with the same GPU model, CPU count and
RAM, so those speedups compare separate runs.
The unmarked speakrs CUDA rows use NVIDIA RTX 4090 containers with 16 CPU
cores and 8 GiB of RAM. The cloud host doesn't report its CPU model. ICSI and
AVA-AVD ran on driver 595.99.02 (nvidia-smi CUDA 13.2) and the other datasets
on driver 580.126.18 (nvidia-smi CUDA 13.0). The VoxConverse pyannote rows ran
earlier in a container with the same GPU model, CPU count and RAM, so those
speedups compare separate runs.

The rows marked RTX 5090 ran on an NVIDIA RTX 5090 with an Intel Core Ultra 7
270K Plus, 24 CPU cores and 57 GiB of RAM; driver 595.99.02, nvidia-smi CUDA
13.2. Both GPUs ran the same speakrs version. On each GPU, VoxConverse Dev is
the median of 3 runs and the other datasets are one run each.

Audio and models were copied to the container's local disk before each dataset
run. Time and RTFx use the `speakrs-bm` timer: WAV loading, inference,
Expand All @@ -114,28 +121,32 @@ comparison.

### VoxConverse Dev (216 files, 1217.8 min)

Hardware: NVIDIA RTX 4090, 16 CPU cores (model not reported), 8 GiB RAM;
RTX 4090 hardware: 16 CPU cores (model not reported), 8 GiB RAM;
driver 580.126.18, nvidia-smi CUDA 13.0.
The speakrs rows are the median of 3 runs in one container. pyannote CUDA ran
earlier in a container with the same configuration.
The RTX 4090 speakrs rows are the median of 3 runs in one container. pyannote
CUDA ran earlier in a container with the same configuration.

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| **speakrs CUDA** | **7.0%** | 2.3% | 2.3% | 2.4% | 40.9s | 1786x |
| speakrs CUDA Fast | 7.4% | 2.3% | 2.3% | 2.8% | 22.0s | **3325x** |
| **speakrs CUDA** | **7.0%** | 2.3% | 2.3% | 2.4% | 38.2s | 1912x |
| speakrs CUDA Fast | 7.4% | 2.3% | 2.3% | 2.8% | 20.0s | 3646x |
| **speakrs CUDA (RTX 5090)** | **7.0%** | 2.3% | 2.3% | 2.4% | 25.0s | 2922x |
| speakrs CUDA Fast (RTX 5090) | 7.4% | 2.3% | 2.3% | 2.8% | 13.0s | **5633x** |
| pyannote CUDA | 7.2% | 2.3% | 2.3% | 2.6% | 2301.3s | 32x |

### VoxConverse Test (232 files, 2612.2 min)

Hardware: NVIDIA RTX 4090, 16 CPU cores (model not reported), 8 GiB RAM.
The speakrs rows are one run each, with driver 595.99.02 (nvidia-smi CUDA 13.2).
pyannote CUDA ran earlier in a container with driver 580.126.18 (nvidia-smi
CUDA 13.0) and otherwise the same configuration.
RTX 4090 hardware: 16 CPU cores (model not reported), 8 GiB RAM;
driver 580.126.18, nvidia-smi CUDA 13.0.
The RTX 4090 speakrs rows are one run each. pyannote CUDA ran earlier in a
container with the same configuration.

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| **speakrs CUDA** | **11.1%** | 3.4% | 4.1% | 3.7% | 81.6s | 1921x |
| speakrs CUDA Fast | 11.2% | 3.3% | 4.1% | 3.8% | 42.1s | **3722x** |
| **speakrs CUDA** | **11.1%** | 3.4% | 4.1% | 3.7% | 84.3s | 1859x |
| speakrs CUDA Fast | 11.2% | 3.3% | 4.1% | 3.8% | 43.3s | 3618x |
| **speakrs CUDA (RTX 5090)** | **11.1%** | 3.4% | 4.1% | 3.7% | 54.3s | 2888x |
| speakrs CUDA Fast (RTX 5090) | 11.2% | 3.3% | 4.1% | 3.8% | 27.6s | **5678x** |
| **pyannote CUDA** | **11.1%** | 3.4% | 4.1% | 3.7% | 6341.3s | 25x |

### AMI IHM (34 files, 1123.8 min)
Expand All @@ -144,47 +155,59 @@ Earlier pyannote hardware: NVIDIA L40S, AMD EPYC 9354.

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| **speakrs CUDA** | **17.0%** | 8.1% | 4.3% | 4.5% | 34.5s | 1955x |
| speakrs CUDA Fast | 17.4% | 8.2% | 4.3% | 4.9% | 17.8s | **3796x** |
| **speakrs CUDA** | **17.0%** | 8.1% | 4.3% | 4.5% | 35.3s | 1909x |
| speakrs CUDA Fast | 17.4% | 8.2% | 4.3% | 4.9% | 18.9s | 3563x |
| **speakrs CUDA (RTX 5090)** | **17.0%** | 8.1% | 4.3% | 4.5% | 23.1s | 2919x |
| speakrs CUDA Fast (RTX 5090) | 17.4% | 8.2% | 4.3% | 4.9% | 11.6s | **5791x** |
| **pyannote CUDA (L40S, earlier run)** | **17.0%** | 8.1% | 4.3% | 4.5% | 4388.1s | 15x |

### AMI SDM (34 files, 1123.8 min)

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| **speakrs CUDA** | **19.7%** | 9.6% | 4.3% | 5.7% | 36.0s | 1874x |
| speakrs CUDA Fast | 20.6% | 9.6% | 4.3% | 6.6% | 18.4s | **3665x** |
| **speakrs CUDA** | **19.7%** | 9.6% | 4.3% | 5.7% | 37.6s | 1791x |
| speakrs CUDA Fast | 20.6% | 9.6% | 4.3% | 6.6% | 18.0s | 3754x |
| **speakrs CUDA (RTX 5090)** | **19.7%** | 9.6% | 4.3% | 5.7% | 23.0s | 2927x |
| speakrs CUDA Fast (RTX 5090) | 20.6% | 9.6% | 4.3% | 6.6% | 11.5s | **5838x** |

### AISHELL-4 (20 files, 763.5 min)

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| **speakrs CUDA** | **11.1%** | 3.9% | 3.9% | 3.3% | 23.9s | 1916x |
| speakrs CUDA Fast | 11.4% | 3.9% | 3.9% | 3.6% | 12.4s | **3706x** |
| **speakrs CUDA** | **11.1%** | 3.9% | 3.9% | 3.3% | 24.6s | 1866x |
| speakrs CUDA Fast | 11.4% | 3.9% | 3.9% | 3.6% | 13.0s | 3534x |
| **speakrs CUDA (RTX 5090)** | **11.1%** | 3.9% | 3.9% | 3.3% | 16.1s | 2844x |
| speakrs CUDA Fast (RTX 5090) | 11.4% | 3.9% | 3.9% | 3.6% | 8.2s | **5621x** |

### Earnings-21 (44 files, 2355.8 min)

Earlier pyannote hardware: NVIDIA RTX 4090, AMD EPYC 7B13.

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| speakrs CUDA | 9.7% | 2.6% | 2.4% | 4.7% | 75.8s | 1864x |
| **speakrs CUDA Fast** | **9.2%** | 2.5% | 2.5% | 4.2% | 39.5s | **3578x** |
| speakrs CUDA | 9.7% | 2.6% | 2.4% | 4.7% | 75.3s | 1876x |
| **speakrs CUDA Fast** | **9.2%** | 2.5% | 2.5% | 4.2% | 39.1s | 3613x |
| speakrs CUDA (RTX 5090) | 9.7% | 2.6% | 2.4% | 4.7% | 49.4s | 2864x |
| **speakrs CUDA Fast (RTX 5090)** | **9.2%** | 2.5% | 2.5% | 4.2% | 24.9s | **5667x** |
| pyannote CUDA (RTX 4090, earlier run) | 9.7% | 2.6% | 2.4% | 4.7% | 8036.8s | 18x |

### ICSI (75 files, 4301.2 min)

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| **speakrs CUDA** | **33.3%** | 19.5% | 9.5% | 4.3% | 142.2s | 1814x |
| speakrs CUDA Fast | 33.7% | 19.4% | 9.6% | 4.7% | 71.1s | **3628x** |
| **speakrs CUDA** | **33.3%** | 19.5% | 9.5% | 4.3% | 133.2s | 1937x |
| speakrs CUDA Fast | 33.7% | 19.4% | 9.6% | 4.7% | 69.6s | 3708x |
| **speakrs CUDA (RTX 5090)** | **33.3%** | 19.5% | 9.5% | 4.3% | 89.4s | 2888x |
| speakrs CUDA Fast (RTX 5090) | 33.7% | 19.4% | 9.6% | 4.7% | 45.1s | **5725x** |

### AVA-AVD (54 files, 266.1 min)

| Implementation | DER | Missed | False Alarm | Confusion | Time | RTFx |
|---|---|---|---|---|---|---|
| **speakrs CUDA** | **45.4%** | 16.1% | 10.8% | 18.6% | 7.7s | 2083x |
| speakrs CUDA Fast | 48.9% | 15.9% | 11.3% | 21.8% | 4.1s | **3899x** |
| **speakrs CUDA** | **45.4%** | 16.1% | 10.8% | 18.6% | 7.7s | 2075x |
| speakrs CUDA Fast | 48.9% | 15.9% | 11.3% | 21.8% | 4.1s | 3927x |
| **speakrs CUDA (RTX 5090)** | **45.4%** | 16.1% | 10.8% | 18.6% | 5.2s | 3095x |
| speakrs CUDA Fast (RTX 5090) | 48.9% | 15.9% | 11.3% | 21.8% | 2.7s | **5971x** |

## Other implementations

Expand Down
18 changes: 15 additions & 3 deletions benchmarks/aishell-4.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,23 @@ speakrs CoreML 11.1% 3.9% 3.9% 3.3% 72.7s
speakrs CoreML Fast 12.1% 4.4% 3.8% 3.9% 51.5s 890x
SpeakerKit 11.7% 4.3% 4.1% 3.3% 56.5s 810x

Linux — speakrs: NVIDIA RTX 4090; 16 CPU cores (model not reported by the host)
Linux — speakrs: NVIDIA RTX 4090; 16 CPU cores (model not reported by the host); 8 GiB RAM
Driver 580.126.18; nvidia-smi CUDA 13.0
AISHELL-4 DER (20 files, 763.5 min, collar=0ms)
Native speakrs CUDA: 1s step; CUDA Fast: 2s step. Audio and models use local disk.

Implementation DER% Missed% FalseAlarm% Confusion% Time RTFx
──────────────────────────────────────────────────────────────────────────────────────────────────────────
speakrs CUDA 11.1% 3.9% 3.9% 3.3% 23.9s 1916x
speakrs CUDA Fast 11.4% 3.9% 3.9% 3.6% 12.4s 3706x
speakrs CUDA 11.1% 3.9% 3.9% 3.3% 24.6s 1866x
speakrs CUDA Fast 11.4% 3.9% 3.9% 3.6% 13.0s 3534x

Linux — NVIDIA RTX 5090; Intel Core Ultra 7 270K Plus, 24 CPU cores; 57 GiB RAM
Driver 595.99.02; nvidia-smi CUDA 13.2
AISHELL-4 DER (20 files, 763.5 min, collar=0ms)
speakrs CUDA and CUDA Fast: one run each, through the batch API.
Native speakrs CUDA: 1s step; CUDA Fast: 2s step. Audio and models use local disk.

Implementation DER% Missed% FalseAlarm% Confusion% Time RTFx
──────────────────────────────────────────────────────────────────────────────────────────────────────────
speakrs CUDA 11.1% 3.9% 3.9% 3.3% 16.1s 2844x
speakrs CUDA Fast 11.4% 3.9% 3.9% 3.6% 8.2s 5621x
18 changes: 15 additions & 3 deletions benchmarks/ami-ihm.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,25 @@ speakrs CoreML 17.0% 8.1% 4.3% 4.6% 149.8s
speakrs CoreML Fast 17.6% 7.8% 4.7% 5.1% 73.9s 912x
SpeakerKit 18.0% 8.5% 5.2% 4.3% 82.8s 814x

Linux — speakrs: NVIDIA RTX 4090; 16 CPU cores (model not reported by the host)
Linux — speakrs: NVIDIA RTX 4090; 16 CPU cores (model not reported by the host); 8 GiB RAM
Driver 580.126.18; nvidia-smi CUDA 13.0
Earlier pyannote hardware: NVIDIA L40S, AMD EPYC 9354
AMI IHM DER (34 files, 1123.8 min, collar=0ms)
Native speakrs CUDA: 1s step; CUDA Fast: 2s step. Audio and models use local disk.

Implementation DER% Missed% FalseAlarm% Confusion% Time RTFx
──────────────────────────────────────────────────────────────────────────────────────────────────────────
speakrs CUDA 17.0% 8.1% 4.3% 4.5% 34.5s 1955x
speakrs CUDA Fast 17.4% 8.2% 4.3% 4.9% 17.8s 3796x
speakrs CUDA 17.0% 8.1% 4.3% 4.5% 35.3s 1909x
speakrs CUDA Fast 17.4% 8.2% 4.3% 4.9% 18.9s 3563x
pyannote CUDA (L40S, earlier run) 17.0% 8.1% 4.3% 4.5% 4388.1s 15x

Linux — NVIDIA RTX 5090; Intel Core Ultra 7 270K Plus, 24 CPU cores; 57 GiB RAM
Driver 595.99.02; nvidia-smi CUDA 13.2
AMI IHM DER (34 files, 1123.8 min, collar=0ms)
speakrs CUDA and CUDA Fast: one run each, through the batch API.
Native speakrs CUDA: 1s step; CUDA Fast: 2s step. Audio and models use local disk.

Implementation DER% Missed% FalseAlarm% Confusion% Time RTFx
──────────────────────────────────────────────────────────────────────────────────────────────────────────
speakrs CUDA 17.0% 8.1% 4.3% 4.5% 23.1s 2919x
speakrs CUDA Fast 17.4% 8.2% 4.3% 4.9% 11.6s 5791x
18 changes: 15 additions & 3 deletions benchmarks/ami-sdm.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,23 @@ speakrs CoreML 19.7% 9.6% 4.4% 5.8% 120.2s
speakrs CoreML Fast 20.8% 9.4% 4.6% 6.9% 80.7s 835x
SpeakerKit 19.9% 9.8% 5.3% 4.9% 86.6s 778x

Linux — speakrs: NVIDIA RTX 4090; 16 CPU cores (model not reported by the host)
Linux — speakrs: NVIDIA RTX 4090; 16 CPU cores (model not reported by the host); 8 GiB RAM
Driver 580.126.18; nvidia-smi CUDA 13.0
AMI SDM DER (34 files, 1123.8 min, collar=0ms)
Native speakrs CUDA: 1s step; CUDA Fast: 2s step. Audio and models use local disk.

Implementation DER% Missed% FalseAlarm% Confusion% Time RTFx
──────────────────────────────────────────────────────────────────────────────────────────────────────────
speakrs CUDA 19.7% 9.6% 4.3% 5.7% 36.0s 1874x
speakrs CUDA Fast 20.6% 9.6% 4.3% 6.6% 18.4s 3665x
speakrs CUDA 19.7% 9.6% 4.3% 5.7% 37.6s 1791x
speakrs CUDA Fast 20.6% 9.6% 4.3% 6.6% 18.0s 3754x

Linux — NVIDIA RTX 5090; Intel Core Ultra 7 270K Plus, 24 CPU cores; 57 GiB RAM
Driver 595.99.02; nvidia-smi CUDA 13.2
AMI SDM DER (34 files, 1123.8 min, collar=0ms)
speakrs CUDA and CUDA Fast: one run each, through the batch API.
Native speakrs CUDA: 1s step; CUDA Fast: 2s step. Audio and models use local disk.

Implementation DER% Missed% FalseAlarm% Confusion% Time RTFx
──────────────────────────────────────────────────────────────────────────────────────────────────────────
speakrs CUDA 19.7% 9.6% 4.3% 5.7% 23.0s 2927x
speakrs CUDA Fast 20.6% 9.6% 4.3% 6.6% 11.5s 5838x
Loading
Loading