feat(anc): add check-hotfix subcommand to read hotfix pointer from LPS#8696
Merged
Azure Pipelines / Agentbaker GPU E2E
failed
Jul 4, 2026 in 32m 34s
Build #20260704.9 had test failures
Details
- Failed: 47 (20.52%)
- Passed: 182 (79.48%)
- Other: 0 (0.00%)
- Total: 229
Annotations
Check failure on line 4520 in Build log
azure-pipelines / Agentbaker GPU E2E
Build log #L4520
Script failed with exit code: 1
Check failure on line 1 in Test_ACL_GPUA100
azure-pipelines / Agentbaker GPU E2E
Test_ACL_GPUA100
Failed
Raw output
=== RUN Test_ACL_GPUA100
=== PAUSE Test_ACL_GPUA100
=== CONT Test_ACL_GPUA100
--- FAIL: Test_ACL_GPUA100 (0.02s)
Check failure on line 1 in Test_Ubuntu2404_NvidiaDevicePluginRunning/scriptless_nbc
azure-pipelines / Agentbaker GPU E2E
Test_Ubuntu2404_NvidiaDevicePluginRunning/scriptless_nbc
Failed
Raw output
=== RUN Test_Ubuntu2404_NvidiaDevicePluginRunning/scriptless_nbc
=== PAUSE Test_Ubuntu2404_NvidiaDevicePluginRunning/scriptless_nbc
=== CONT Test_Ubuntu2404_NvidiaDevicePluginRunning/scriptless_nbc
test_helpers.go:418: [40.830s] TAGS {Name:Test_Ubuntu2404_NvidiaDevicePluginRunning/scriptless_nbc ImageName:2404gen2containerd OS:ubuntu Arch:amd64 NetworkIsolated:false NonAnonymousACR:false GPU:true WASM:false BootstrapTokenFallback:false KubeletCustomConfig:false Scriptless:false VHDCaching:false MockAzureChinaCloud:false VMSeriesCoverageTest:false}
test_helpers.go:229: [40.830s] → running scenario...
test_helpers.go:246: [40.830s] using cluster abe2e-kubenet-v5-150ee in rg=abe2e-westus3 sub=8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8
test_helpers.go:247: [40.830s] portal: https://portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/abe2e-westus3/providers/Microsoft.ContainerService/managedClusters/abe2e-kubenet-v5-150ee/overview
test_helpers.go:279: [40.845s] → preparing AKS node...
vmss.go:531: [40.846s] → creating VMSS d6gy-2026-07-04-ubuntu2404nvidiadevicepluginrunningscript...
vmss.go:435: [41.607s] VMSS portal link: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-150ee_westus3/providers/Microsoft.Compute/virtualMachineScaleSets/d6gy-2026-07-04-ubuntu2404nvidiadevicepluginrunningscript/overview
vmss.go:441: [41.608s] Managed cluster portal link: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-150ee_westus3/providers/Microsoft.ContainerService/managedClusters/abe2e-kubenet-v5-150ee/overview
2026/07/04 23:13:40 Using VM extension version 1.465 for extension type Compute.AKS.Linux.AKSNode in region westus3
vmss.go:564: [59.240s] VM will be automatically deleted after the test finishes, to preserve it for debugging purposes set KEEP_VMSS=true or pause the test with a breakpoint before the test finishes or failed
vmss.go:568: [59.240s] SSH Instructions: (may take a few minutes for the VM to be ready for SSH)
========================
az network bastion ssh --target-resource-id "/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-150ee_westus3/providers/Microsoft.Compute/virtualMachineScaleSets/d6gy-2026-07-04-ubuntu2404nvidiadevicepluginrunningscript/virtualMachines/0" --name "abe2e-shared-bastion" --resource-group abe2e-westus3 --auth-type ssh-key --username azureuser --ssh-key /tmp/private-key-2139427548
bastionssh.go:304: [341.671s] Attempt 1/5 establishing SSH over bastion to 10.220.112.111
vmss.go:618: [343.320s] VM reached running state
vmss.go:588: [343.320s] ✓ creating VMSS d6gy-2026-07-04-ubuntu2404nvidiadevicepluginrunningscript done (302.5s)
kube.go:160: [343.321s] → waiting for node d6gy-2026-07-04-ubuntu2404nvidiadevicepluginrunningscript to be ready...
kube.go:182: [343.433s] node d6gy-2026-07-04-ubuntu2404nvidiadevicepluginrunningscript000000 is ready. Taints: [{"key":"node.kubernetes.io/network-unavailable","effect":"NoSchedule","timeAdded":"2026-07-04T23:18:31Z"}] Conditions: [{"type":"NetworkUnavailable","status":"True","lastHeartbeatTime":"2026-07-04T23:18:31Z","lastTransitionTime":"2026-07-04T23:18:31Z","reason":"NodeInitialization","message":"Waiting for cloud routes"},{"type":"VMEventScheduled","status":"False","lastHeartbeatTime":"2026-07-04T23:18:30Z","lastTransitionTime":"2026-07-04T23:18:30Z","reason":"NoVMEventScheduled","message":"VM has no scheduled event"},{"type":"FrequentDockerRestart","status":"False","last
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]
Check failure on line 1 in Test_Ubuntu2204_NvidiaDevicePluginRunning/scriptless_nbc
azure-pipelines / Agentbaker GPU E2E
Test_Ubuntu2204_NvidiaDevicePluginRunning/scriptless_nbc
Failed
Raw output
=== RUN Test_Ubuntu2204_NvidiaDevicePluginRunning/scriptless_nbc
=== PAUSE Test_Ubuntu2204_NvidiaDevicePluginRunning/scriptless_nbc
=== CONT Test_Ubuntu2204_NvidiaDevicePluginRunning/scriptless_nbc
test_helpers.go:418: [9.271s] TAGS {Name:Test_Ubuntu2204_NvidiaDevicePluginRunning/scriptless_nbc ImageName:2204gen2containerd OS:ubuntu Arch:amd64 NetworkIsolated:false NonAnonymousACR:false GPU:true WASM:false BootstrapTokenFallback:false KubeletCustomConfig:false Scriptless:false VHDCaching:false MockAzureChinaCloud:false VMSeriesCoverageTest:false}
test_helpers.go:229: [9.274s] → running scenario...
test_helpers.go:246: [9.274s] using cluster abe2e-kubenet-v5-150ee in rg=abe2e-westus3 sub=8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8
test_helpers.go:247: [9.274s] portal: https://portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/abe2e-westus3/providers/Microsoft.ContainerService/managedClusters/abe2e-kubenet-v5-150ee/overview
test_helpers.go:279: [9.306s] → preparing AKS node...
vmss.go:531: [9.309s] → creating VMSS 72wo-2026-07-04-ubuntu2204nvidiadevicepluginrunningscript...
vmss.go:435: [10.830s] VMSS portal link: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-150ee_westus3/providers/Microsoft.Compute/virtualMachineScaleSets/72wo-2026-07-04-ubuntu2204nvidiadevicepluginrunningscript/overview
vmss.go:441: [10.835s] Managed cluster portal link: https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/resource/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-150ee_westus3/providers/Microsoft.ContainerService/managedClusters/abe2e-kubenet-v5-150ee/overview
2026/07/04 23:13:10 Using VM extension version 1.465 for extension type Compute.AKS.Linux.AKSNode in region westus3
vmss.go:564: [30.149s] VM will be automatically deleted after the test finishes, to preserve it for debugging purposes set KEEP_VMSS=true or pause the test with a breakpoint before the test finishes or failed
vmss.go:568: [30.149s] SSH Instructions: (may take a few minutes for the VM to be ready for SSH)
========================
az network bastion ssh --target-resource-id "/subscriptions/8ecadfc9-d1a3-4ea4-b844-0d9f87e4d7c8/resourceGroups/MC_abe2e-westus3_abe2e-kubenet-v5-150ee_westus3/providers/Microsoft.Compute/virtualMachineScaleSets/72wo-2026-07-04-ubuntu2204nvidiadevicepluginrunningscript/virtualMachines/0" --name "abe2e-shared-bastion" --resource-group abe2e-westus3 --auth-type ssh-key --username azureuser --ssh-key /tmp/private-key-2139427548
bastionssh.go:304: [342.238s] Attempt 1/5 establishing SSH over bastion to 10.220.112.11
vmss.go:618: [343.896s] VM reached running state
vmss.go:588: [343.896s] ✓ creating VMSS 72wo-2026-07-04-ubuntu2204nvidiadevicepluginrunningscript done (334.6s)
kube.go:160: [343.897s] → waiting for node 72wo-2026-07-04-ubuntu2204nvidiadevicepluginrunningscript to be ready...
kube.go:182: [344.005s] node 72wo-2026-07-04-ubuntu2204nvidiadevicepluginrunningscript000000 is ready. Taints: [{"key":"node.kubernetes.io/network-unavailable","effect":"NoSchedule","timeAdded":"2026-07-04T23:18:33Z"}] Conditions: [{"type":"NetworkUnavailable","status":"True","lastHeartbeatTime":"2026-07-04T23:18:33Z","lastTransitionTime":"2026-07-04T23:18:33Z","reason":"NodeInitialization","message":"Waiting for cloud routes"},{"type":"FilesystemCorruptionProblem","status":"False","lastHeartbeatTime":"2026-07-04T23:18:30Z","lastTransitionTime":"2026-07-04T23:18:29Z","reason":"FilesystemIsOK","message":"Filesystem is healthy"},{"type":"FrequentContainerdRestart","status":"False","last
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]
Loading