Expose VmFd::enable_cap on all architectures - #382
Conversation
9e688e0 to
ce22abb
Compare
ce22abb to
9e7db54
Compare
9e7db54 to
fb63118
Compare
| /// let kvm = Kvm::new().unwrap(); | ||
| /// let vm = kvm.create_vm().unwrap(); | ||
| /// let mut cap: kvm_enable_cap = Default::default(); | ||
| /// cap.cap = KVM_CAP_SPLIT_IRQCHIP; |
There was a problem hiding this comment.
This is the part that is architecture specific I guess.
Is there a capability that we can enable on all platforms? Alternatively, can we define a per platform capability?
There was a problem hiding this comment.
Thanks for raising this, @andreeaflorescu. I looked into it further and KVM_CAP_HALT_POLL is documented as supporting all architectures and can be enabled through VmFd, making it suitable for this example. I also verified that KVM_CHECK_EXTENSION reports it as supported and that KVM_ENABLE_CAP succeeds on both aarch64 and x86_64.
To remain compatible with older kernels, the example could check for support first:
use kvm_bindings::{kvm_enable_cap, KVM_CAP_HALT_POLL};
use kvm_ioctls::Kvm;
let kvm = Kvm::new().unwrap();
let vm = kvm.create_vm().unwrap();
if kvm.check_extension_raw(KVM_CAP_HALT_POLL.into()) > 0 {
let cap = kvm_enable_cap {
cap: KVM_CAP_HALT_POLL,
args: [100_000, 0, 0, 0],
..Default::default()
};
vm.enable_cap(&cap).unwrap();
}Since this PR has now merged, I’m happy to submit this as a small follow-up if you think it would improve the example.
| /// // Because an IOAPIC supports 24 pins, that's the reason why this test | ||
| /// // picked this number as reference. | ||
| /// cap.args[0] = 24; | ||
| /// # #[cfg(target_arch = "x86_64")] |
There was a problem hiding this comment.
The first # is not needed
There was a problem hiding this comment.
Thanks both, @ShadowCurse and @rbradford. Fixed now by removing the unnecessary leading #, and I’ve rebased the PR onto the latest main.
| /// // Because an IOAPIC supports 24 pins, that's the reason why this test | ||
| /// // picked this number as reference. | ||
| /// cap.args[0] = 24; | ||
| /// # #[cfg(target_arch = "x86_64")] |
fb63118 to
a32409c
Compare
The KVM_ENABLE_CAP vm ioctl is architecture independent, but both the ioctl definition and VmFd::enable_cap were compiled only for x86_64, s390x and powerpc. aarch64 now needs them: Linux 6.15 gates writes to the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) behind KVM_CAP_ARM_WRITABLE_IMP_ID_REGS, which a VMM must enable on the VM before creating vCPUs. Without enable_cap, VMMs fall back to issuing the raw ioctl themselves (see firecracker-microvm/firecracker#6116). Remove the cfg gates and guard the x86 specific portion of the doc example so it builds and runs on every architecture. Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) writable from userspace, gated behind a capability that the VMM must enable on the VM before any vCPU is created. Firecracker never enables it, so a custom CPU template with a modifier for one of these registers fails at boot with EINVAL even on kernels that support the write: Failed to set register 0x603000000013c000 to value 0x410fd0c0: Invalid argument (os error 22) Enable the capability when the VM is created, whenever the host kernel offers it. This has no effect on guest visible state by itself: the registers keep their host values unless a template writes them, and writes of unchanged values (snapshot restore) were already accepted before the capability existed. A failure to enable it is logged rather than fatal: a VM whose template does not touch these registers is unaffected, and one that does still fails loudly when the template is applied. VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl is issued directly, with the same definition kvm-ioctls uses on other architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is released). Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) writable from userspace, gated behind a capability that the VMM must enable on the VM before any vCPU is created. Firecracker never enables it, so a custom CPU template with a modifier for one of these registers fails at boot with EINVAL even on kernels that support the write: Failed to set register 0x603000000013c000 to value 0x410fd0c0: Invalid argument (os error 22) Enable the capability when the VM is created, whenever the host kernel offers it. This has no effect on guest visible state by itself: the registers keep their host values unless a template writes them, and writes of unchanged values (snapshot restore) were already accepted before the capability existed. A failure to enable it is logged rather than fatal: a VM whose template does not touch these registers is unaffected, and one that does still fails loudly when the template is applied. VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl is issued directly, with the same definition kvm-ioctls uses on other architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is released). Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) writable from userspace, gated behind a capability that the VMM must enable on the VM before any vCPU is created. Firecracker never enables it, so a custom CPU template with a modifier for one of these registers fails at boot with EINVAL even on kernels that support the write: Failed to set register 0x603000000013c000 to value 0x410fd0c0: Invalid argument (os error 22) Enable the capability when the VM is created, whenever the host kernel offers it. This has no effect on guest visible state by itself: the registers keep their host values unless a template writes them, and writes of unchanged values (snapshot restore) were already accepted before the capability existed. A failure to enable it is logged rather than fatal: a VM whose template does not touch these registers is unaffected, and one that does still fails loudly when the template is applied. VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl is issued directly, with the same definition kvm-ioctls uses on other architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is released). Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) writable from userspace, gated behind a capability that the VMM must enable on the VM before any vCPU is created. Firecracker never enables it, so a custom CPU template with a modifier for one of these registers fails at boot with EINVAL even on kernels that support the write: Failed to set register 0x603000000013c000 to value 0x410fd0c0: Invalid argument (os error 22) Enable the capability when the VM is created, whenever the host kernel offers it. This has no effect on guest visible state by itself: the registers keep their host values unless a template writes them, and writes of unchanged values (snapshot restore) were already accepted before the capability existed. A failure to enable it is logged rather than fatal: a VM whose template does not touch these registers is unaffected, and one that does still fails loudly when the template is applied. VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl is issued directly, with the same definition kvm-ioctls uses on other architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is released). Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
Summary of the PR
The
KVM_ENABLE_CAPvm ioctl is architecture independent, but both the ioctl definition andVmFd::enable_capwere compiled only for x86_64, s390x and powerpc. This PR removes the cfg gates so they are available on every architecture. The doc example is unchanged as rendered; two hidden doctest lines gate the x86 specific call so the example builds and runs cleanly on every architecture.Motivation: aarch64 now needs the vm level
enable_cap. Linux 6.15 gates writes to the implementation ID registers (MIDR_EL1,REVIDR_EL1,AIDR_EL1) behindKVM_CAP_ARM_WRITABLE_IMP_ID_REGS, which a VMM must enable on the VM before creating any vCPU. Rewriting these registers is how a VMM masks the host CPU identity for guests (the reason the capability exists in KVM), and withoutenable_capa VMM has to issue the raw ioctl itself, as Firecracker currently does in firecracker-microvm/firecracker#6116.Verified: builds and passes clippy and rustfmt on aarch64. The doc example compiles for aarch64 and skips the x86 specific call at runtime; on hosts without
/dev/kvmit fails the same way as every other doctest in the crate, so CI coverage is unchanged. The underlying ioctl path (enable the capability, then successfully writeMIDR_EL1viaKVM_SET_ONE_REG) was exercised end to end on an EC2 c7g.metal running a 7.0 kernel as part of the Firecracker work linked above.Requirements
Before submitting your PR, please make sure you addressed the following requirements:
git commit -s), and the commit message has max 60 characters for the summary and max 75 characters for each description line.unsafecode is properly documented. (No new unsafe.)