Skip to content

Expose VmFd::enable_cap on all architectures - #382

Merged
ShadowCurse merged 1 commit into
rust-vmm:mainfrom
rogersnm:enable-cap-all-arches
Aug 24, 2026
Merged

Expose VmFd::enable_cap on all architectures#382
ShadowCurse merged 1 commit into
rust-vmm:mainfrom
rogersnm:enable-cap-all-arches

Conversation

@rogersnm

@rogersnm rogersnm commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Summary of the PR

The KVM_ENABLE_CAP vm ioctl is architecture independent, but both the ioctl definition and VmFd::enable_cap were compiled only for x86_64, s390x and powerpc. This PR removes the cfg gates so they are available on every architecture. The doc example is unchanged as rendered; two hidden doctest lines gate the x86 specific call so the example builds and runs cleanly on every architecture.

Motivation: aarch64 now needs the vm level enable_cap. Linux 6.15 gates writes to the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) behind KVM_CAP_ARM_WRITABLE_IMP_ID_REGS, which a VMM must enable on the VM before creating any vCPU. Rewriting these registers is how a VMM masks the host CPU identity for guests (the reason the capability exists in KVM), and without enable_cap a VMM has to issue the raw ioctl itself, as Firecracker currently does in firecracker-microvm/firecracker#6116.

Verified: builds and passes clippy and rustfmt on aarch64. The doc example compiles for aarch64 and skips the x86 specific call at runtime; on hosts without /dev/kvm it fails the same way as every other doctest in the crate, so CI coverage is unchanged. The underlying ioctl path (enable the capability, then successfully write MIDR_EL1 via KVM_SET_ONE_REG) was exercised end to end on an EC2 c7g.metal running a 7.0 kernel as part of the Firecracker work linked above.

Requirements

Before submitting your PR, please make sure you addressed the following requirements:

  • All commits in this PR have Signed-Off-By trailers (with git commit -s), and the commit message has max 60 characters for the summary and max 75 characters for each description line.
  • All added/changed functionality has a corresponding unit/integration test. (Covered by the existing doc example, which now compiles and runs on every architecture.)
  • All added/changed public-facing functionality has entries in the "Upcoming Release" section of CHANGELOG.md (if no such section exists, please create one).
  • Any newly added unsafe code is properly documented. (No new unsafe.)

Comment thread kvm-ioctls/src/ioctls/vm.rs Outdated
@rogersnm
rogersnm force-pushed the enable-cap-all-arches branch from 9e7db54 to fb63118 Compare August 20, 2026 22:23

@RuoqingHe RuoqingHe left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codewise looking good, while those changes are already in #344

/// let kvm = Kvm::new().unwrap();
/// let vm = kvm.create_vm().unwrap();
/// let mut cap: kvm_enable_cap = Default::default();
/// cap.cap = KVM_CAP_SPLIT_IRQCHIP;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the part that is architecture specific I guess.

Is there a capability that we can enable on all platforms? Alternatively, can we define a per platform capability?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for raising this, @andreeaflorescu. I looked into it further and KVM_CAP_HALT_POLL is documented as supporting all architectures and can be enabled through VmFd, making it suitable for this example. I also verified that KVM_CHECK_EXTENSION reports it as supported and that KVM_ENABLE_CAP succeeds on both aarch64 and x86_64.

To remain compatible with older kernels, the example could check for support first:

use kvm_bindings::{kvm_enable_cap, KVM_CAP_HALT_POLL};
use kvm_ioctls::Kvm;

let kvm = Kvm::new().unwrap();
let vm = kvm.create_vm().unwrap();

if kvm.check_extension_raw(KVM_CAP_HALT_POLL.into()) > 0 {
    let cap = kvm_enable_cap {
        cap: KVM_CAP_HALT_POLL,
        args: [100_000, 0, 0, 0],
        ..Default::default()
    };

    vm.enable_cap(&cap).unwrap();
}

Since this PR has now merged, I’m happy to submit this as a small follow-up if you think it would improve the example.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thanks!

ShadowCurse
ShadowCurse previously approved these changes Aug 24, 2026

@ShadowCurse ShadowCurse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall lgtm

Comment thread kvm-ioctls/src/ioctls/vm.rs Outdated
/// // Because an IOAPIC supports 24 pins, that's the reason why this test
/// // picked this number as reference.
/// cap.args[0] = 24;
/// # #[cfg(target_arch = "x86_64")]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first # is not needed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, please fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks both, @ShadowCurse and @rbradford. Fixed now by removing the unnecessary leading #, and I’ve rebased the PR onto the latest main.

Comment thread kvm-ioctls/src/ioctls/vm.rs Outdated
/// // Because an IOAPIC supports 24 pins, that's the reason why this test
/// // picked this number as reference.
/// cap.args[0] = 24;
/// # #[cfg(target_arch = "x86_64")]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, please fix.

The KVM_ENABLE_CAP vm ioctl is architecture independent, but both the
ioctl definition and VmFd::enable_cap were compiled only for x86_64,
s390x and powerpc. aarch64 now needs them: Linux 6.15 gates writes to
the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) behind
KVM_CAP_ARM_WRITABLE_IMP_ID_REGS, which a VMM must enable on the VM
before creating vCPUs. Without enable_cap, VMMs fall back to issuing
the raw ioctl themselves (see firecracker-microvm/firecracker#6116).

Remove the cfg gates and guard the x86 specific portion of the doc
example so it builds and runs on every architecture.

Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>

@rbradford rbradford left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ShadowCurse
ShadowCurse merged commit 615babb into rust-vmm:main Aug 24, 2026
27 checks passed
rogersnm added a commit to rogersnm/firecracker that referenced this pull request Aug 24, 2026
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1,
AIDR_EL1) writable from userspace, gated behind a capability that the
VMM must enable on the VM before any vCPU is created. Firecracker never
enables it, so a custom CPU template with a modifier for one of these
registers fails at boot with EINVAL even on kernels that support the
write:

  Failed to set register 0x603000000013c000 to value 0x410fd0c0:
  Invalid argument (os error 22)

Enable the capability when the VM is created, whenever the host kernel
offers it. This has no effect on guest visible state by itself: the
registers keep their host values unless a template writes them, and
writes of unchanged values (snapshot restore) were already accepted
before the capability existed. A failure to enable it is logged rather
than fatal: a VM whose template does not touch these registers is
unaffected, and one that does still fails loudly when the template is
applied.

VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl
is issued directly, with the same definition kvm-ioctls uses on other
architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is
released).

Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
rogersnm added a commit to rogersnm/firecracker that referenced this pull request Aug 24, 2026
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1,
AIDR_EL1) writable from userspace, gated behind a capability that the
VMM must enable on the VM before any vCPU is created. Firecracker never
enables it, so a custom CPU template with a modifier for one of these
registers fails at boot with EINVAL even on kernels that support the
write:

  Failed to set register 0x603000000013c000 to value 0x410fd0c0:
  Invalid argument (os error 22)

Enable the capability when the VM is created, whenever the host kernel
offers it. This has no effect on guest visible state by itself: the
registers keep their host values unless a template writes them, and
writes of unchanged values (snapshot restore) were already accepted
before the capability existed. A failure to enable it is logged rather
than fatal: a VM whose template does not touch these registers is
unaffected, and one that does still fails loudly when the template is
applied.

VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl
is issued directly, with the same definition kvm-ioctls uses on other
architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is
released).

Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
rogersnm added a commit to rogersnm/firecracker that referenced this pull request Aug 25, 2026
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1,
AIDR_EL1) writable from userspace, gated behind a capability that the
VMM must enable on the VM before any vCPU is created. Firecracker never
enables it, so a custom CPU template with a modifier for one of these
registers fails at boot with EINVAL even on kernels that support the
write:

  Failed to set register 0x603000000013c000 to value 0x410fd0c0:
  Invalid argument (os error 22)

Enable the capability when the VM is created, whenever the host kernel
offers it. This has no effect on guest visible state by itself: the
registers keep their host values unless a template writes them, and
writes of unchanged values (snapshot restore) were already accepted
before the capability existed. A failure to enable it is logged rather
than fatal: a VM whose template does not touch these registers is
unaffected, and one that does still fails loudly when the template is
applied.

VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl
is issued directly, with the same definition kvm-ioctls uses on other
architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is
released).

Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
philipparndt pushed a commit to philipparndt/firecracker that referenced this pull request Aug 26, 2026
Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1,
AIDR_EL1) writable from userspace, gated behind a capability that the
VMM must enable on the VM before any vCPU is created. Firecracker never
enables it, so a custom CPU template with a modifier for one of these
registers fails at boot with EINVAL even on kernels that support the
write:

  Failed to set register 0x603000000013c000 to value 0x410fd0c0:
  Invalid argument (os error 22)

Enable the capability when the VM is created, whenever the host kernel
offers it. This has no effect on guest visible state by itself: the
registers keep their host values unless a template writes them, and
writes of unchanged values (snapshot restore) were already accepted
before the capability existed. A failure to enable it is logged rather
than fatal: a VM whose template does not touch these registers is
unaffected, and one that does still fails loudly when the template is
applied.

VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl
is issued directly, with the same definition kvm-ioctls uses on other
architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is
released).

Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants