openvmmCI: enable remaining backend-native VMM tests #135

Context

#132 expands NVX's required OpenVMM VMM-test matrix and makes the backend-specific filters explicit. To keep the pull-request matrix reliable, it leaves several tests disabled after bare-metal and virtual-runner validation.

This issue tracks diagnosing those failures and incrementally removing the corresponding exclusions from scripts/nvx_tools/ci.py.

Scope

Keep each backend native to its host platform. This issue does not cover cross-compiling Windows guest payloads on Linux or enabling Linux KVM/MSHV tests that require the Windows GNU toolchain.

The following Linux exclusions are intentionally out of scope here:

• windows_datacenter_core_2022_x64
• virtio_net_windows
• no_vmbus_prepped_boot_no_vmbus_windows

Tests to enable

Linux / KVM

PCAT tests currently excluded by !test(openvmm_pcat_x64):

• multiarch::openvmm_pcat_x64_freebsd_13_2_x64_boot_no_agent (#183)
• multiarch::openvmm_pcat_x64_freebsd_13_2_x64_iso_boot_no_agent (#183)
• multiarch::openvmm_pcat_x64_ubuntu_2404_server_x64_boot (#183)
• multiarch::openvmm_pcat_x64_ubuntu_2504_server_x64_boot (#183)
• multiarch::openvmm_pcat_x64_ubuntu_2504_server_x64_boot_heavy (#183)

KVM-specific exclusions:

• x86_64_exclusive::openvmm_linux_x64_apicid_offset (#157)
• x86_64_exclusive::openvmm_linux_x64_legacy_xapic (#152)

Linux / MSHV

PCAT tests currently excluded by !test(openvmm_pcat_x64):

• multiarch::openvmm_pcat_x64_freebsd_13_2_x64_boot_no_agent (#183)
• multiarch::openvmm_pcat_x64_freebsd_13_2_x64_iso_boot_no_agent (#183)
• multiarch::openvmm_pcat_x64_ubuntu_2404_server_x64_boot (#183)
• multiarch::openvmm_pcat_x64_ubuntu_2504_server_x64_boot (#183)
• multiarch::openvmm_pcat_x64_ubuntu_2504_server_x64_boot_heavy (#183)

MSHV-specific exclusions:

• multiarch::pcie::openvmm_linux_x64_pcie_save_restore (#184)
• ttrpc::test_ttrpc_interface (#156)
• x86_64::openvmm_linux_x64_virtio_blk_device (#181)

Windows / WHP

Exact tests currently listed in OPENVMM_WHP_EXCLUDED_TESTS:

• multiarch::openvmm_pcat_x64_freebsd_13_2_x64_boot_no_agent (#158)
• multiarch::openvmm_pcat_x64_freebsd_13_2_x64_iso_boot_no_agent (#162)
• multiarch::openvmm_pcat_x64_windows_datacenter_core_2022_x64_boot (#177)
• multiarch::openvmm_pcat_x64_windows_datacenter_core_2022_x64_boot_heavy (#182)
• multiarch::pcie::openvmm_uefi_x64_windows_datacenter_core_2022_x64_pcie_nvme_boot (#146)

Existing evidence

Validation performed for #132 showed that runner type matters:

• Some tests passed on configured bare-metal hosts but failed on the Azure virtual-machine CI runners.
• KVM openvmm_linux_x64_apicid_offset failed on the virtual runner.
• MSHV test_ttrpc_interface failed intermittently and openvmm_linux_x64_virtio_blk_device hung until the test timeout on the virtual runner, despite passing bare-metal validation.
• WHP FreeBSD PCAT and the non-heavy Windows PCAT boot passed bare-metal validation but failed on the virtual runner.
• WHP Windows PCAT heavy boot and PCIe NVMe boot also failed during bare-metal validation.

Preserve the failure mode and runner type in any focused follow-up PR so a bare-metal-only pass is not treated as sufficient evidence for required CI.

Investigation areas

• Compare virtual-machine runner capabilities with the configured bare-metal hosts.
• Verify PCAT firmware availability and guest-image behavior on each host OS.
• Investigate APIC topology/emulation differences for the KVM xAPIC cases.
• Investigate save/restore and storage teardown behavior for the MSHV failures.
• Capture complete Petri/OpenVMM logs for hangs and early worker-launch failures.
• Confirm cold-cache and warm-cache behavior where guest images or prepared disks are involved.

Acceptance criteria

For each checkbox enabled:

• The exact test name is selected; no substring matcher silently broadens coverage.
• The test runs and passes on the corresponding required self-hosted virtual-machine CI runner.
• The test does not pass through an implicit runtime skip or missing-prerequisite early return.
• The implementation remains backend-native and does not add cross-platform toolchains to the runner.
• Cold-cache and warm-cache runs both succeed when external artifacts are required.
• Failure logs preserve enough Petri/OpenVMM detail to diagnose regressions.
• scripts/test_nvx_tools.py is updated to document the removed exclusion.
• The full Enable backend-specific OpenVMM VMM tests #132 OpenVMM, microVM, platform, quality, and performance matrix remains green.

Tests may be enabled in separate focused pull requests; this issue can remain open until all applicable checkboxes are complete or explicitly moved to narrower follow-ups.

Related work

• Enable backend-specific OpenVMM VMM tests #132 establishes the expanded backend-specific matrix and the initial exclusion policy.
• OpenVMM VMM test: enable test_ttrpc_interface #81, OpenVMM VMM test: enable openvmm_linux_x64_apicid_offset #93, and OpenVMM VMM test: enable openvmm_linux_x64_legacy_xapic #94 are earlier per-test issues closed while the integration approach was being reconsidered; this issue is their consolidated replacement under the now-established Enable backend-specific OpenVMM VMM tests #132 design.
• OpenVMM CI: add scheduled upstream-standard x64 VMM coverage #100 is the earlier scheduled-standard-suite proposal and remains out of scope for this required pull-request matrix follow-up.
• OpenVMM CI: provision PCAT firmware for Linux KVM and MSHV tests #183 tracks authorized PCAT firmware provisioning for the remaining Linux KVM/MSHV cases.
• OpenVMM CI: fix MSHV PCIe save/restore pulse hang #184 tracks the remaining MSHV PCIe save/restore pulse hang.