Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion BUILD.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@
owner_team: llm
dir: templates/audio-dataset-curation-llm-judge
cluster_env:
image_uri: anyscale/ray-llm:2.55.1-py311-cu128
image_uri: anyscale/ray-llm:2.56.0-py312-cu130
compute_config:
AWS: configs/audio-dataset-curation-llm-judge/aws.yaml
GCP: configs/audio-dataset-curation-llm-judge/gce.yaml
Expand Down
4 changes: 2 additions & 2 deletions dependencies/template.depsets.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -149,12 +149,12 @@ depsets:
- templates/audio-dataset-curation-llm-judge/requirements.txt
output: templates/audio-dataset-curation-llm-judge/python_depset.lock
append_flags:
- --index https://download.pytorch.org/whl/cu128
- --index https://download.pytorch.org/whl/cu130
- --python-version=${PYTHON_VERSION}
- --python-platform=x86_64-manylinux_2_31
- --unsafe-package ray
build_arg_sets:
- ray2551_py311_cu128
- ray2560_py312_cu130

- name: multi_agent_a2a_depset_${RAY_VERSION}_${PYTHON_VERSION}_${CUDA_VARIANT}
operation: expand
Expand Down
13 changes: 8 additions & 5 deletions templates/audio-dataset-curation-llm-judge/README.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
"# Audio batch inference\n",
"\n",
"<div align=\"left\">\n",
" <a target=\"_blank\" href=\"https://console.anyscale.com/template-preview/audio-dataset-curation-llm-judge\"><img src=\"https://img.shields.io/badge/🚀 Run_on-Anyscale-9hf\"></a>&nbsp;\n",
" <a target=\"_blank\" href=\"https://console.anyscale.com/template-preview/audio-dataset-curation-llm-judge\"><img src=\"https://img.shields.io/badge/\ud83d\ude80 Run_on-Anyscale-9hf\"></a>&nbsp;\n",
" <a href=\"https://github.com/anyscale/templates/tree/main/templates/audio-dataset-curation-llm-judge\" role=\"button\"><img src=\"https://img.shields.io/static/v1?label=&message=View%20On%20GitHub&color=586069&logo=github&labelColor=2f363d\"></a>&nbsp;\n",
"</div>\n",
"\n",
"**⏱️ Time to complete**: 30 min\n",
"**\u23f1\ufe0f Time to complete**: 30 min\n",
"\n",
"This tutorial demonstrates a batch inference pipeline that converts raw\n",
"audio files into a curated subset using two different ML models.\n",
Expand All @@ -21,7 +21,7 @@
"2. Resample each clip to 16 kHz for compatibility with Whisper.\n",
"3. Transcribe the audio with the `openai/whisper-large-v3-turbo` model.\n",
"4. Judge the educational quality of each transcription with a small Llama-3 model.\n",
"5. Persist only clips that score 3 to a Parquet dataset.\n",
"5. Persist only clips that score \u2265 3 to a Parquet dataset.\n",
"\n",
"Ray Data is particularly powerful for this use case because it:\n",
"- **Parallelizes work** across a cluster of machines automatically\n",
Expand Down Expand Up @@ -259,7 +259,10 @@
" self.processor = AutoProcessor.from_pretrained(TRANSCRIPTION_MODEL)\n",
"\n",
" def __call__(self, batch):\n",
" token_ids = batch.pop(\"token_ids\")\n",
" # Whisper's tokenizer probes token_ids with plain-Python truthiness\n",
" # (`if not token_ids`, `if has_prompt`, ...), so hand it plain nested lists\n",
" # rather than a pandas Series of numpy arrays.\n",
" token_ids = [ids.tolist() for ids in batch.pop(\"token_ids\")]\n",
" transcription = self.processor.batch_decode(token_ids, skip_special_tokens=True)\n",
" batch[\"transcription\"] = transcription\n",
" return batch\n",
Expand All @@ -277,7 +280,7 @@
"## LLM-based quality filter\n",
"\n",
"A Llama-3 model serves as a *machine judge* that scores each transcription\n",
"from 1 👎 to 5 👍 on its educational value. The **LLM Processor** API wraps the heavy\n",
"from 1 \ud83d\udc4e to 5 \ud83d\udc4d on its educational value. The **LLM Processor** API wraps the heavy\n",
"lifting of batching, prompt formatting, and vLLM engine interaction using a declarative API style.\n",
"\n",
"Ray Data provides a high-level API for integrating LLMs into data pipelines. The preprocessing and postprocessing functions handle data preparation and result parsing."
Expand Down
5 changes: 4 additions & 1 deletion templates/audio-dataset-curation-llm-judge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,10 @@ class Decoder:
self.processor = AutoProcessor.from_pretrained(TRANSCRIPTION_MODEL)

def __call__(self, batch):
token_ids = batch.pop("token_ids")
# Whisper's tokenizer probes token_ids with plain-Python truthiness
# (`if not token_ids`, `if has_prompt`, ...), so hand it plain nested lists
# rather than a pandas Series of numpy arrays.
token_ids = [ids.tolist() for ids in batch.pop("token_ids")]
transcription = self.processor.batch_decode(token_ids, skip_special_tokens=True)
batch["transcription"] = transcription
return batch
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
--index-url https://pypi.org/simple
--extra-index-url https://download.pytorch.org/whl/cu128
--extra-index-url https://download.pytorch.org/whl/cu130

cffi==2.0.0 \
--hash=sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb \
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
soundfile==0.13.1
# Match the base image's numpy (anyscale/ray-llm:2.55.1 ships numpy 1.26.4). The test
# Match the base image's numpy (anyscale/ray-llm:2.56.0-py312-cu130 ships numpy 1.26.4). The test
# layers this lock on the stock image with `uv pip install --no-deps`, so a higher numpy
# would force-upgrade it and break the image's numpy-1.x-compiled pandas/scipy (ABI).
numpy==1.26.4
Loading