Skip to content

[no-ci] toolshed: modernize conda_create_for_pathfinder_testing scripts #2013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rwgk merged 4 commits into from
May 17, 2026
Merged

[no-ci] toolshed: modernize conda_create_for_pathfinder_testing scripts #2013

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0df4909
toolshed: modernize conda_create_for_pathfinder_testing scripts
rwgk May 4, 2026
293bdcd
Merge branch 'main' into toolshed_conda_create_for_pathfinder_moderni…
rwgk May 17, 2026
76d2524
toolshed: batch the conda installs into a single solve
rwgk May 17, 2026
2d7dfe9
toolshed: take python_major_minor as an argument and align variable n…
rwgk May 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions toolshed/conda_create_for_pathfinder_testing.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION
# SPDX-License-Identifier: Apache-2.0

param(
Expand All @@ -7,22 +7,30 @@ param(
)

$ErrorActionPreference = "Stop"
Set-StrictMode -Version Latest

@rparolin rparolin May 5, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to spell out more what "modernization's" you are doing. I'm not sure what you are modernizing based on the code.

@rwgk rwgk May 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: See the massively expanded PR description.


$cudaMajor = $CudaVersion.Split(".", 2)[0]
switch ($cudaMajor) {
"12" { $pythonVersion = "3.12" }
"13" { $pythonVersion = "3.14" }
default {
throw "Unsupported CUDA major version for this helper: $cudaMajor. Expected a 12.x or 13.x toolkit version."
}
}

& "$env:CONDA_EXE" "shell.powershell" "hook" | Out-String | Invoke-Expression

conda create --yes -n "pathfinder_testing_cu$CudaVersion" python=3.13 "cuda-toolkit=$CudaVersion"
conda create --yes -n "pathfinder_testing_cu$CudaVersion" "python=$pythonVersion" "cuda-toolkit=$CudaVersion"
conda activate "pathfinder_testing_cu$CudaVersion"

# Keep this list aligned with the Windows-installable subset of
# cuda_pathfinder/pyproject.toml.
$cpkgs = @(
"cusparselt-dev",
"cutensor",
"libcublasmp-dev",
"cutlass",
"libcudss-dev",
"libcufftmp-dev",
"libmathdx-dev",
"libnvshmem3",
"libnvshmem-dev",
"libnvpl-fft-dev"
"libmathdx-dev"
)

foreach ($cpkg in $cpkgs) {
Expand Down
65 changes: 51 additions & 14 deletions toolshed/conda_create_for_pathfinder_testing.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,65 @@
#!/bin/bash

# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

set -euo pipefail

if [[ $# -ne 1 ]]; then
echo "Usage: $(basename "$0") ctk-major-minor-patch" 1>&2
exit 1
fi

cuda_version="$1"

@rparolin rparolin May 5, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: Refactoring these two scripts into a single python script to reduce maintenance headaches.

@rwgk rwgk May 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See last paragraph in the new expanded PR description.

I more-or-less went into the opposite direction: this feedback prompted me to simplify the scripts, by removing the "derive Python version from CUDA version" logic and simply requiring the Python version to be passed in.

cuda_major="${cuda_version%%.*}"
uname_m="$(uname -m)"
case "$cuda_major" in
12)
python_version=3.12
;;
13)
python_version=3.14
;;
*)
echo "Unsupported CUDA major version for this helper: $cuda_major" 1>&2
echo "Expected a 12.x or 13.x toolkit version." 1>&2
exit 1
;;
esac

eval "$(conda shell.bash hook)"

conda create --yes -n "pathfinder_testing_cu$1" python=3.13 cuda-toolkit="$1"
conda activate "pathfinder_testing_cu$1"

for cpkg in \
cusparselt-dev \
cutensor \
libcublasmp-dev \
libcudss-dev \
libcufftmp-dev \
libmathdx-dev \
libnvshmem3 \
libnvshmem-dev \
libnvpl-fft-dev; do
conda create --yes -n "pathfinder_testing_cu$cuda_version" "python=$python_version" cuda-toolkit="$cuda_version"
set +u
conda activate "pathfinder_testing_cu$cuda_version"
set -u

# Keep this list aligned with the Linux-installable subset of
# cuda_pathfinder/pyproject.toml.
cpkgs=(
"cusparselt-dev"
"cutensor"
"cutlass"
"libcublasmp-dev"
"libcudss-dev"
"libcufftmp-dev"
"libcusolvermp-dev"
"libmathdx-dev"
"libnvshmem3"
"libnvshmem-dev"
)

# Keep the conda environment aligned with platform-scoped pyproject groups.
if [[ "$uname_m" == "aarch64" ]]; then
cpkgs+=("libnvpl-fft-dev")
if [[ "$cuda_major" == "13" ]]; then
cpkgs+=("libcudla-dev")
fi
fi

for cpkg in "${cpkgs[@]}"; do
echo "CONDA INSTALL: $cpkg"
set +u
conda install -y -c conda-forge "$cpkg"
set -u
done
Loading