Skip to content

Add external control bridge related infra #851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kostyanf14 wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Add external control bridge related infra #851

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e340d8d
Bump rtoolsHCK and update project with API
kostyanf14 Dec 9, 2025
40f6189
Add a lot of logs for NS creation process
kostyanf14 Dec 9, 2025
2f54395
Propagate cli options to setup manager enter
kostyanf14 Jan 5, 2026
9bea930
Add external control bridge related infra
kostyanf14 Jan 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ gem 'httpclient'
gem 'mono_logger'
gem 'octokit'
gem 'openssl', require: false
gem 'rtoolsHCK', git: 'https://github.com/HCK-CI/rtoolsHCK.git', ref: 'v0.6.1'
gem 'rtoolsHCK', git: 'https://github.com/HCK-CI/rtoolsHCK.git', ref: 'VIRTWINKVM-1865'

@gemini-code-assist gemini-code-assist Bot Jan 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For reproducible builds, it's better to pin a dependency to a specific commit hash rather than a branch name. Branch refs can be updated, leading to different versions of the gem being installed over time. Please use the full commit hash from Gemfile.lock to ensure that builds are consistent.

gem 'rtoolsHCK', git: 'https://github.com/HCK-CI/rtoolsHCK.git', ref: 'f59ae987141d8d7192ede862802dbf3d9d0d9105'

gem 'rubyzip'
gem 'sentry-ruby'
gem 'sorbet-runtime'
Expand Down
8 changes: 4 additions & 4 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
GIT
remote: https://github.com/HCK-CI/rtoolsHCK.git
revision: c59b7e6f7366d630313111fd0ad380a235b8b2ef
ref: v0.6.1
revision: f59ae987141d8d7192ede862802dbf3d9d0d9105
ref: VIRTWINKVM-1865
specs:
rtoolsHCK (0.6.1)
bundler
Expand Down Expand Up @@ -132,7 +132,7 @@ GEM
sorbet-runtime (>= 0.5.9204)
rbs (3.9.2)
logger
rdoc (6.15.1)
rdoc (6.17.0)
erb
psych (>= 4.0.0)
tsort
Expand Down Expand Up @@ -189,7 +189,7 @@ GEM
rbi (>= 0.2.3)
sorbet-static-and-runtime (>= 0.5.10187)
thor (>= 0.19.2)
stringio (3.1.8)
stringio (3.1.9)
tapioca (0.16.11)
benchmark
bundler (>= 2.2.25)
Expand Down
2 changes: 1 addition & 1 deletion bin/ns
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,6 @@ module AutoHCK
workspace_path = File.absolute_path(workspace_path, chdir)
end

QemuHCK::Ns.enter workspace_path, chdir, *argv
QemuHCK::Ns.enter workspace_path, chdir, {}, *argv
end
end
2 changes: 2 additions & 0 deletions bin/ns_unshared
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ module AutoHCK
'/proc/sys/net/bridge/bridge-nf-call-ip6tables',
'/proc/sys/net/bridge/bridge-nf-call-iptables'
].each do |file|
$stderr.write "[ns_unshared] Writing '0' to #{file}\n"
File.write file, '0'
rescue Errno::ENOENT
# br_netfilter is not loaded
Expand Down Expand Up @@ -50,6 +51,7 @@ module AutoHCK
%w[ip link add br_debug type bridge],
%w[ip link set br_debug up]
].each do |cmd|
$stderr.write "[ns_unshared] Running (wait spawn): #{cmd.join(' ')}\n"
Process.wait spawn(*cmd, out: :err)
raise $CHILD_STATUS.to_s unless $CHILD_STATUS.success?
rescue StandardError
Expand Down
46 changes: 46 additions & 0 deletions bin/ns_veth
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
#!/usr/bin/env ruby
# frozen_string_literal: true

require_relative '../lib/auto_hck'

module AutoHCK
run do
require 'English'
require 'fileutils'

side = ARGV[0]
case side
when 'host'
pid = ARGV[1]
run_id = ARGV[2]
veth_bridge = ARGV[3]

command = [
# *veth* pair for *control* *external* communication bridge
# one end (vce_#{run_id}) in host connected to veth_bridge
# other in namespace (ctrl_ext_ns) connected to br_ctrl_ext
%W[ip link add vce_#{run_id} type veth peer ctrl_ext_ns netns #{pid}],
%W[ip link set vce_#{run_id} master #{veth_bridge}],
%W[ip link set vce_#{run_id} up]
]
when 'ns'
command = [
# External control bridge for communication with Controller / Studio PCs in lab
%w[ip link set ctrl_ext_ns up],
%w[ip link add br_ctrl_ext type bridge],
%w[ip link set ctrl_ext_ns master br_ctrl_ext],
%w[ip link set br_ctrl_ext up]
]
else
raise "Unknown side: #{side}"
end

command.each do |cmd|
$stderr.write "[ns_veth #{side}] Running (wait spawn): #{cmd.join(' ')}\n"
Process.wait spawn(*cmd, out: :err)
raise $CHILD_STATUS.to_s unless $CHILD_STATUS.success?
rescue StandardError
raise "#{cmd}: command failed"
end
Comment on lines +38 to +44

@gemini-code-assist gemini-code-assist Bot Jan 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

There are a couple of issues with this error handling block:

  1. The rescue is placed after the command.each loop. In this position, the cmd variable from the loop is not in scope, which will lead to a NameError if an exception occurs within the loop.
  2. The error message raise "#{cmd}: command failed" is not very informative. It converts an array to a string (e.g., ["ip", "link", ...]), which is not readable, and it swallows the original exception details.

To fix this, the rescue should be inside the loop, and the error message should be improved to be more descriptive and include the original error.

    command.each do |cmd|
      begin
        $stderr.write "[ns_veth #{side}] Running (wait spawn): #{cmd.join(' ')}\n"
        Process.wait spawn(*cmd, out: :err)
        raise "command failed with status #{$CHILD_STATUS.exitstatus}" unless $CHILD_STATUS.success?
      rescue StandardError => e
        raise "Command '#{cmd.join(' ')}' failed: #{e.message}"
      end
    end

end
end
16 changes: 12 additions & 4 deletions bin/nsd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,11 @@ module AutoHCK
argv << File.join(__dir__, 'ns_unshared')

e_read, e_write = IO.pipe
$stderr.write "[nsd] Running (e pipe): #{e_read.fileno} => #{e_write.fileno}\n"
r_read, r_write = IO.pipe
$stderr.write "[nsd] Running (r pipe): #{r_read.fileno} => #{r_write.fileno}\n"
parent = Process.pid.to_s
$stderr.write "[nsd] Parent PID: #{parent}\n"

fork do
e_write.close
Expand All @@ -60,19 +63,24 @@ module AutoHCK
# Call newgidmap so that virtiofsd can use setgroups() to drop subgroups.
# unshare command in util-linux 2.38 can call newgidmap by its own, but
# unfortunately it is not available on some supported systems.
system 'newgidmap', parent, '0', Process.gid.to_s, '1',
subid_start, subid_start, subid_count, exception: true, out: :err
command = ['newgidmap', parent, '0', Process.gid.to_s, '1', subid_start, subid_start, subid_count]
$stderr.write "[nsd child] Running (system): #{command.join(' ')}\n"
system(*command, exception: true, out: :err)
end

e_read.close_on_exec = false
r_write.close_on_exec = false
exec 'slirp4netns', '-e', e_read.fileno.to_s, '-r', r_write.fileno.to_s,
'-a', 'slirp.sock', parent, 'tap_host', out: :err

command = ['slirp4netns', '-e', e_read.fileno.to_s, '-r', r_write.fileno.to_s,
'-a', 'slirp.sock', parent, 'tap_host']
$stderr.write "[nsd child] Running (exec): #{command.join(' ')}\n"
exec(*command, out: :err)
end

e_read.close
r_write.close

$stderr.write "[nsd] Running (exec, 3 => #{e_write.fileno}, 4 => #{r_read.fileno}): #{argv.join(' ')}\n"
exec(*argv, 3 => e_write, 4 => r_read)
end
end
2 changes: 1 addition & 1 deletion bin/run_dump
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ module AutoHCK
workspace = File.dirname(ARGV[0])
chdir = File.dirname(__dir__)
program = File.join(__dir__, 'run_dump_unshared')
QemuHCK::Ns.enter workspace, chdir, program, *ARGV
QemuHCK::Ns.enter workspace, chdir, {}, program, *ARGV
end
end
10 changes: 9 additions & 1 deletion lib/cli.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ def initialize
# class CommonOptions
class CommonOptions
attr_accessor :verbose, :config, :client_world_net, :id, :share_on_host_path, :workspace_path,
:client_ctrl_net_dev, :attach_debug_net
:client_ctrl_net_dev, :attach_debug_net, :control_bridge_external

def create_parser(sub_parser)
OptionParser.new do |parser|
Expand All @@ -47,6 +47,7 @@ def define_options(parser)
@id = 2
@share_on_host_path = nil
@attach_debug_net = false
@control_bridge_external = nil

parser.on('--share-on-host-path <path>', String,
'For using Transfer Network specify the directory to share on host machine') do |share_on_host_path|
Expand Down Expand Up @@ -83,6 +84,13 @@ def define_options(parser)
exit
end

parser.on('--control-bridge-external <control-bridge-external>', String,
'External control bridge name.',
'This bridge will be used to connect clients VM to ',
'external Controller / Studio PC network.',
'This is part of dedicated studio feature. (experimental)',
&method(:control_bridge_external=))

parser.on('-w <path>', String,
'Internal use only',
&method(:workspace_path=))
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win10_2004x64.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 4,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",

@YanVugenfirer YanVugenfirer Jan 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The json update doesn't match the commit message. Maybe should be separate commit

"image": "HLK2004-C1-Win10_2004x64.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 4,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK2004-C2-Win10_2004x64.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win10_2004x86.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 4,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK2004-C1-Win10_2004x86.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 4,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK2004-C2-Win10_2004x86.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win10_22H2x64.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 4,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK2004-C1-Win10_22H2x64.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 4,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK2004-C2-Win10_22H2x64.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win11_22H2x64_host_viommu.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK11_22H2-C1-Win11_22H2x64-viommu.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK11_22H2-C2-Win11_22H2x64-viommu.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win11_23H2x64_host_viommu.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK11_23H2-C1-Win11_23H2x64-viommu.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK11_23H2-C2-Win11_23H2x64-viommu.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win11_24H2x64_host.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK11_24H2-C1-Win11_24H2x64.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK11_24H2-C2-Win11_24H2x64.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win11_24H2x64_host_viommu.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK11_24H2-C1-Win11_24H2x64-viommu.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK11_24H2-C2-Win11_24H2x64-viommu.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win11_25H2x64_host.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK11_25H2-C1-Win11_25H2x64.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK11_25H2-C2-Win11_25H2x64.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win11_25H2x64_host_viommu.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK11_25H2-C1-Win11_25H2x64-viommu.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK11_25H2-C2-Win11_25H2x64-viommu.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win11nextx64_host_viommu.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK11_next-C1-Win11nextx64-viommu.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 8,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK11_next-C2-Win11nextx64-viommu.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win2016x64.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 2,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK1607-C1-Win2016x64.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 2,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK1607-C2-Win2016x64.qcow2"
}
}
Expand Down
4 changes: 2 additions & 2 deletions lib/engines/hcktest/platforms/Win2016x64_bios.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,14 @@
"name": "CL1",
"cpus": 4,
"memory_gb": 2,
"winrm_port": 4002,
"winrm_addr": "192.168.100.2",
"image": "HLK1607-C1-Win2016x64-bios.qcow2"
},
"c2": {
"name": "CL2",
"cpus": 4,
"memory_gb": 2,
"winrm_port": 4003,
"winrm_addr": "192.168.100.3",
"image": "HLK1607-C2-Win2016x64-bios.qcow2"
}
}
Expand Down
Loading