1- Run the full TIRX test suite.
1+ ---
2+ name : tir-test
3+ description : Run TVM's in-tree TIRx tests and tirx-kernels import and correctness checks.
4+ ---
5+
6+ Run the compiler suite in ` 3rdparty/tvm-tirx/tests/python/tirx/ ` , using the ` tvm_tirx ` package
7+ in ` 3rdparty/tvm-tirx/python/ ` , together with the downstream correctness test in
8+ ` tests/python/tirx/test_tirx_kernels_registry_correctness.py ` .
9+
10+ Both tirx-kernels import gates and its correctness coverage are enabled by
11+ default. Use ` TIR_TEST_WITH_KERNELS=0 ` only for an explicitly requested
12+ compiler-only run, and report that reduced coverage.
213
314## Steps
415
516Run all commands below in the same Bash shell from the TVM repository root.
617
7- 1 . Point Python at this workspace's repos and select every Blackwell GPU.
18+ 1 . Point Python at this worktree and select every Blackwell GPU.
819 ` TIR_TEST_GPUS ` may explicitly provide physical GPU IDs, and
920 ` TIR_TEST_NUM_GPUS ` may optionally cap the automatically selected count. Otherwise,
1021 selection uses all Blackwell GPUs, ordered by least memory in use.
1122 ``` bash
12- export WORKSPACE=/path/to/workspace
23+ export TIR_TEST_TVM_ROOT= " $( pwd -P ) "
1324 export TIR_TEST_NUM_GPUS=" ${TIR_TEST_NUM_GPUS:- } "
1425 export TIR_TEST_XDIST_WORKERS=" ${TIR_TEST_XDIST_WORKERS:- 16} "
26+ export TIR_TEST_WITH_KERNELS=" ${TIR_TEST_WITH_KERNELS:- 1} "
27+
28+ [[ -d " $TIR_TEST_TVM_ROOT /3rdparty/tvm-tirx/python/tvm_tirx" &&
29+ -d " $TIR_TEST_TVM_ROOT /3rdparty/tvm-tirx/tests/python/tirx" ]] || {
30+ echo " run from the TVM repository root containing 3rdparty/tvm-tirx/" >&2
31+ exit 2
32+ }
33+ [[ " $TIR_TEST_WITH_KERNELS " == 0 || " $TIR_TEST_WITH_KERNELS " == 1 ]] || {
34+ echo " TIR_TEST_WITH_KERNELS must be 0 or 1" >&2
35+ exit 2
36+ }
1537
1638 [[ -z " $TIR_TEST_NUM_GPUS " || " $TIR_TEST_NUM_GPUS " =~ ^[1-9][0-9]* $ ]] || {
1739 echo " TIR_TEST_NUM_GPUS must be empty or a positive integer" >&2
@@ -66,22 +88,44 @@ Run all commands below in the same Bash shell from the TVM repository root.
6688
6789 export CUDA_VISIBLE_DEVICES
6890 CUDA_VISIBLE_DEVICES=" $( IFS=,; printf ' %s' " ${TIR_TEST_SELECTED_GPUS[*]} " ) "
69- # Registry correctness tests skip a device with less than this much free
70- # memory; 0 runs them regardless of what else is on the GPU.
71- export TIRX_KERNEL_TEST_MIN_FREE_GB=0
72- export PYTHONPATH=" ${WORKSPACE} /tirx-kernels:${WORKSPACE} /tvm/python"
73- export TVM_LIBRARY_PATH=" ${WORKSPACE} /tvm/build/lib"
91+ # Preserve reference packages and other paths supplied by branch-shell.
92+ export PYTHONPATH=" ${TIR_TEST_TVM_ROOT} /.local/python:${TIR_TEST_TVM_ROOT} /3rdparty/tvm-tirx/python:${TIR_TEST_TVM_ROOT} /python:${PYTHONPATH:- } "
93+ export TVM_LIBRARY_PATH=" ${TIR_TEST_TVM_ROOT} /build/lib"
94+ export TVM_TIRX_LIBRARY_PATH=" ${TIR_TEST_TVM_ROOT} /build/lib"
7495
7596 PYTHON_BIN=" ${PYTHON_BIN:- $(command -v python || true)} "
7697 [[ -n " $PYTHON_BIN " && -x " $PYTHON_BIN " ]] || {
7798 echo " set PYTHON_BIN to the test environment's Python executable" >&2
7899 exit 1
79100 }
80101 export PATH=" $( dirname " $PYTHON_BIN " ) :$PATH "
81- " $PYTHON_BIN " -c ' import pytest, torch, tvm' > /dev/null || {
82- echo " PYTHON_BIN must provide pytest and torch and import this TVM worktree" >&2
102+ " $PYTHON_BIN " - << 'PY '
103+ import os
104+ from pathlib import Path
105+
106+ import pytest
107+ import torch
108+ import tvm_ffi
109+ import tvm_tirx
110+ import xdist
111+ from tvm_tirx.base import _LOADED_LIBS, _RUNTIME_ONLY
112+
113+ root = Path(os.environ["TIR_TEST_TVM_ROOT"])
114+ assert Path(tvm_tirx.__file__).resolve().is_relative_to(root / "3rdparty/tvm-tirx/python"), (
115+ f"imported another TIRx worktree: {tvm_tirx.__file__}"
116+ )
117+ assert not _RUNTIME_ONLY, "build the TIRx compiler before running its tests"
118+ for name, library in _LOADED_LIBS.items():
119+ assert Path(library._name).resolve().is_relative_to(root / "build/lib"), (
120+ f"loaded another worktree's {name}: {library._name}"
121+ )
122+ print(f"tvm_tirx: {tvm_tirx.__file__}")
123+ print(f"tvm_ffi: {tvm_ffi.__file__}")
124+ PY
125+ if [[ $? -ne 0 ]]; then
126+ echo "PYTHON_BIN must provide pytest, xdist, torch and this worktree's TIRx build" >&2
83127 exit 1
84- }
128+ fi
85129 if ! command -v ninja >/dev/null; then
86130 NINJA_BIN_DIR="$("$PYTHON_BIN" -c 'import ninja; print(ninja.BIN_DIR)' 2>/dev/null || true)"
87131 if [[ -n "$NINJA_BIN_DIR" && -x "$NINJA_BIN_DIR/ninja" ]]; then
@@ -103,33 +147,50 @@ Run all commands below in the same Bash shell from the TVM repository root.
103147 `TIR_TEST_NUM_GPUS=4` to cap the count or `TIR_TEST_GPUS=3,4,5,6` to override
104148 selection with exact physical IDs.
105149
106- 2 . Import gate: bench workloads. Fail fast if any kernel listed in
107- ` workloads.yaml ` fails to import.
108- ``` bash
109- " $PYTHON_BIN " -m tirx_kernels.bench_suite --check-imports
110- ```
111- A non-zero exit means a pinned workload kernel failed to import. Fix it before
112- proceeding.
150+ Use dependencies compatible with `3rdparty/tvm-tirx/pyproject.toml`. A non-editable FFI
151+ installation in `.local/python/` takes priority over the inherited environment.
152+ If native libraries are missing or stale, build this worktree with the same
153+ `PYTHONPATH` before testing.
113154
114- 3 . Full kernel import gate for correctness-suite coverage:
155+ 2. Run downstream import gates and select the full test coverage:
115156 ```bash
116- " $PYTHON_BIN " -m tirx_kernels.registry --strict
157+ TIR_TEST_PATHS=(3rdparty/tvm-tirx/tests/python/tirx/)
158+ if [[ "$TIR_TEST_WITH_KERNELS" == 1 ]]; then
159+ TIR_TEST_KERNELS_ROOT="${TIR_TEST_KERNELS_ROOT:-$(dirname "$TIR_TEST_TVM_ROOT")/tirx-kernels}"
160+ [[ -d "$TIR_TEST_KERNELS_ROOT/tirx_kernels" ]] || {
161+ echo "set TIR_TEST_KERNELS_ROOT to the tirx-kernels repository" >&2
162+ exit 2
163+ }
164+ export PYTHONPATH="${TIR_TEST_KERNELS_ROOT}:${PYTHONPATH:-}"
165+ "$PYTHON_BIN" -m tirx_kernels.bench_suite --check-imports || exit 1
166+ "$PYTHON_BIN" -m tirx_kernels.registry --strict || exit 1
167+ TIR_TEST_PATHS+=(tests/python/tirx/test_tirx_kernels_registry_correctness.py)
168+ export TIRX_KERNEL_TEST_MIN_FREE_GB="${TIRX_KERNEL_TEST_MIN_FREE_GB:-0}"
169+ else
170+ echo "compiler-only run: tirx-kernels import and correctness checks disabled"
171+ fi
117172 ```
118173
119- 4 . Run the full test suite with xdist parallelism:
174+ 3 . Run the selected test suites with xdist parallelism and save the full log :
120175 ```bash
121- " $PYTHON_BIN " -m pytest tests/python/tirx/ -n " $TIR_TEST_XDIST_WORKERS "
176+ export TIR_TEST_LOG="${TIR_TEST_LOG:-/tmp/tir-test-$(date +%Y%m%d-%H%M%S).log}"
177+ set -o pipefail
178+ "$PYTHON_BIN" -m pytest "${TIR_TEST_PATHS[@]}" -n "$TIR_TEST_XDIST_WORKERS" 2>&1 | tee "$TIR_TEST_LOG"
122179 ```
123180
124- ` tests/python/tirx/conftest.py ` assigns each xdist worker's PyTorch current
125- device round-robin across the visible GPUs. Registry correctness tests also
126- take a per-device lock, so at most one large registered kernel runs on each
127- GPU concurrently. Tests that explicitly use ` tvm.cuda(0) ` still run on the
128- first selected physical GPU. MegaMoE remains skipped because it requires its
129- dedicated multi-process scheduler, not ordinary xdist device assignment.
181+ `3rdparty/tvm-tirx/tests/python/tirx/conftest.py` uses `tvm_tirx.testing._pytest` to assign
182+ each xdist worker's PyTorch current device round-robin across the visible
183+ GPUs. Tests that explicitly use `tvm_tirx.cuda(0)` still run on the first
184+ selected physical GPU.
185+
186+ Registry correctness tests take a per-device lock to serialize large workloads
187+ on each GPU. MegaMoE remains skipped because it requires its dedicated
188+ multi-process scheduler.
130189
131- 5 . Report results: selected physical GPUs, elapsed time, total passed, failed,
132- skipped, and errors, plus both import-gate results.
190+ 4. Report results: selected physical GPUs, elapsed time, total passed, failed,
191+ skipped, and errors, plus the full log path. Report both downstream gates as
192+ passed, failed, or explicitly disabled; disabled integration checks are not
193+ passes.
133194
134195## Failure triage rules
135196
@@ -139,7 +200,8 @@ Always capture and read full logs.**
139200Classify every failure into one of these categories:
140201
141202- **A - Environment/import error**: Module not found, missing dependency,
142- collection error. These are not caused by code changes.
203+ collection error. Check the source and native-library paths, FFI dependency,
204+ and old `tvm` versus `tvm_tirx` namespaces before assigning a cause.
143205- **B - Real kernel correctness regression**: Assertion failures (`cosine_sim`,
144206 numerical diff), `CUDA: unspecified launch failure`, or wrong results. These
145207 MUST be investigated and fixed if caused by current changes.
0 commit comments