Virtual Interrupt¶
This section introduces the ACRN guest virtual interrupt management, which includes:
vCPU request for virtual interrupt kickoff
vPIC, vIOAPIC, and vLAPIC for virtual interrupt injection interfaces
physical-to-virtual interrupt mapping for a passthrough device
the process of VMX interrupt and exception injection
A standard VM never owns any physical interrupts. All interrupts received by the guest OS come from a virtual interrupt injected by vLAPIC, vIOAPIC, or vPIC. Such virtual interrupts are triggered either from a passthrough device or from I/O mediators in the Service VM via hypercalls. The Interrupt Remapping section describes how the hypervisor manages the mapping between physical and virtual interrupts for passthrough devices. However, a hard RTVM with LAPIC passthrough does own the physical maskable external interrupts. On its physical CPUs, interrupts are disabled in VMX root mode. While in VMX non-root mode, physical interrupts are delivered to the RTVM directly.
Devices are emulated inside the Service VM Device Model, i.e.,
acrn-dm
. However, for performance consideration, vLAPIC, vIOAPIC, and vPIC
are emulated inside the hypervisor directly.
From the guest OS point of view, vPIC is Virtual Wire Mode via vIOAPIC. The symmetric I/O Mode is shown in Figure 134 later in this section.
The following command-line options to a Linux guest affect whether it uses PIC or IOAPIC:
Kernel boot parameter with vPIC: Add
maxcpu=0
. The guest OS will use PIC.Kernel boot parameter with vIOAPIC: Add
maxcpu=1
(as long as not “0”). The guest OS will use IOAPIC and keep IOAPIC pin 2 as the source of PIC.
vCPU Request for Interrupt Injection¶
The vCPU request mechanism (described in Pending Request Handlers) is used to inject interrupts to a certain vCPU. As mentioned in IPI Management, physical vector 0xF0 is used to kick the vCPU out of its VMX non-root mode. Physical vector 0xF0 is also used to make a request for virtual interrupt injection or other requests such as flush EPT.
Note
The IPI-based vCPU request mechanism doesn’t work for the hard RTVM.
The eventid supported for virtual interrupt injection includes:
-
ACRN_REQUEST_EXCP¶
Request for exception injection.
-
ACRN_REQUEST_EVENT¶
Request for vLAPIC event.
-
ACRN_REQUEST_EXTINT¶
Request for external interrupt from vPIC.
-
ACRN_REQUEST_NMI¶
Request for non-maskable interrupt.
-
ACRN_REQUEST_EOI_EXIT_BITMAP_UPDATE¶
Request for EOI exit bitmap update.
-
ACRN_REQUEST_EPT_FLUSH¶
Request for EPT flush.
-
ACRN_REQUEST_TRP_FAULT¶
Request for triple fault.
-
ACRN_REQUEST_VPID_FLUSH¶
Request for VPID TLB flush.
-
ACRN_REQUEST_INIT_VMCS¶
Request for initilizing VMCS.
-
ACRN_REQUEST_WAIT_WBINVD¶
Request for sync waiting WBINVD.
-
ACRN_REQUEST_SPLIT_LOCK¶
Request for split lock operation.
The vcpu_make_request is necessary for a virtual interrupt injection. If the target vCPU is running under VMX non-root mode, it sends an IPI to kick it out, which leads to an external-interrupt VM-Exit. In some cases, there is no need to send an IPI when making a request, because the CPU making the request itself is the target vCPU. For example, the #GP exception request always happens on the current CPU when it finds an invalid emulation has happened. An external interrupt for a passthrough device always happens on the vCPUs of the VM that the device belongs to. After it triggers an external-interrupt VM-Exit, the current CPU is the target vCPU.
Virtual LAPIC¶
LAPIC is virtualized for all guest types: Service VM and User VMs. Given support by the physical processor, APICv virtual interrupt delivery (VID) is enabled and supports the Posted-Interrupt feature. Otherwise, it falls back to the legacy virtual interrupt injection mode.
vLAPIC provides the same features as the native LAPIC:
Vector mask/unmask
Virtual vector injections (Level or Edge trigger mode) to vCPU
vIOAPIC notification of EOI processing
TSC Timer service
vLAPIC support CR8 to update TPR
INIT/STARTUP handling
vLAPIC APIs¶
APIs are invoked when an interrupt source from vLAPIC needs to inject an interrupt, for example:
from LVT like LAPIC timer
from vIOAPIC for a passthrough device interrupt
from an emulated device for an MSI
These APIs finish by making a vCPU request.
-
void vlapic_inject_intr(struct acrn_vlapic *vlapic, bool guest_irq_enabled, bool injected)¶
-
void vlapic_set_intr(struct acrn_vcpu *vcpu, uint32_t vector, bool level)¶
-
int32_t vlapic_set_local_intr(struct acrn_vm *vm, uint16_t vcpu_id_arg, uint32_t lvt_index)¶
Triggers LAPIC local interrupt(LVT).
- Preconditions
vm != NULL
- Parameters
vm – [in] Pointer to VM data structure
vcpu_id_arg – [in] ID of vCPU, BROADCAST_CPU_ID means triggering interrupt to all vCPUs.
lvt_index – [in] The index which LVT would to be fired.
- Return values
0 – on success.
-EINVAL – on error that vcpu_id_arg or vector of the LVT is invalid.
-
int32_t vlapic_inject_msi(struct acrn_vm *vm, uint64_t addr, uint64_t data)¶
Inject MSI to target VM.
- Preconditions
vm != NULL
- Parameters
vm – [in] Pointer to VM data structure
addr – [in] MSI address.
data – [in] MSI data.
- Return values
0 – on success.
-1 – on error that addr is invalid.
-
void vlapic_receive_intr(struct acrn_vm *vm, bool level, uint32_t dest, bool phys, uint32_t delmode, uint32_t vec, bool rh)¶
EOI Processing¶
If APICv virtual interrupt delivery is supported, EOI virtualization is enabled. Except for level triggered interrupts, the VM will not exit in case of EOI.
If APICv virtual interrupt delivery is not supported, vLAPIC requires EOI from the guest OS whenever a vector is acknowledged and processed by the guest. vLAPIC behavior is the same as hardware LAPIC. Once an EOI is received, it clears the highest priority vector in ISR, and updates PPR status. vLAPIC sends an EOI message to vIOAPIC if the TMR bit is set to indicate that is a level triggered interrupt.
LAPIC Passthrough Based on vLAPIC¶
LAPIC passthrough is supported based on vLAPIC. The guest OS first boots with vLAPIC in xAPIC mode and then switches to x2APIC mode to enable the LAPIC passthrough.
If LAPIC passthrough is based on vLAPIC, the system has the following characteristics:
IRQs received by the LAPIC can be handled by the guest VM without
vmexit
.Guest VM always sees virtual LAPIC IDs for security consideration.
Most MSRs are directly accessible from the guest VM except for
XAPICID
,LDR
, andICR
. Write operations toICR
are trapped to avoid malicious IPIs. Read operations toXAPIC
andLDR
are trapped, so that the guest VM always sees the virtual LAPIC IDs instead of the physical ones.
Virtual IOAPIC¶
The hypervisor emulates vIOAPIC when the guest accesses the MMIO GPA range: 0xFEC00000-0xFEC01000. vIOAPIC for the Service VM should match the native hardware IOAPIC pin numbers. vIOAPIC for a guest VM provides 48 pins. As the vIOAPIC is always associated with vLAPIC, the virtual interrupt injection from vIOAPIC triggers a request for a vLAPIC event by calling vLAPIC APIs.
Supported APIs:
-
void vioapic_set_irqline_lock(const struct acrn_vm *vm, uint32_t vgsi, uint32_t operation)¶
Set vIOAPIC IRQ line status.
- Preconditions
irqline < vioapic_pincount(vm)
- Parameters
vm – [in] Pointer to target VM
vgsi – [in] GSI for the virtual interrupt
operation – [in] Action options: GSI_SET_HIGH/GSI_SET_LOW/ GSI_RAISING_PULSE/GSI_FALLING_PULSE
- Returns
None
-
void vioapic_set_irqline_nolock(const struct acrn_vm *vm, uint32_t vgsi, uint32_t operation)¶
Set vIOAPIC IRQ line status.
Similar with vioapic_set_irqline_lock(),but would not make sure operation be done with ioapic lock.
- Preconditions
irqline < vioapic_pincount(vm)
- Parameters
vm – [in] Pointer to target VM
vgsi – [in] GSI for the virtual interrupt
operation – [in] Action options: GSI_SET_HIGH/GSI_SET_LOW/ GSI_RAISING_PULSE/GSI_FALLING_PULSE
- Returns
None
Virtual PIC¶
vPIC is required for TSC calculation. Normally the guest OS boots with vIOAPIC and vPIC as the source of external interrupts. On every VM Exit, the hypervisor checks for pending external PIC interrupts. Usage of vPIC APIs is similar to that of vIOAPIC.
ACRN hypervisor emulates a vPIC for each VM based on I/O range 0x20~0x21, 0xa0~0xa1, and 0x4d0~0x4d1.
If an interrupt source from vPIC needs to inject an interrupt, the
following APIs need be called, which will finally make a request for
ACRN_REQUEST_EXTINT
or ACRN_REQUEST_EVENT
:
-
void vpic_set_irqline(struct acrn_vpic *vpic, uint32_t vgsi, uint32_t operation)¶
Set vPIC IRQ line status.
- Parameters
vpic – [in] Pointer to target VM’s vpic table
vgsi – [in] GSI for the virtual interrupt
operation – [in] action options:GSI_SET_HIGH/GSI_SET_LOW/ GSI_RAISING_PULSE/GSI_FALLING_PULSE
- Returns
None
The following APIs are used to query the vector that needs to be injected and ACK the service (move the interrupt from request service - IRR to in service - ISR):
-
void vpic_pending_intr(struct acrn_vpic *vpic, uint32_t *vecptr)¶
Get pending virtual interrupts for vPIC.
- Parameters
vpic – [in] Pointer to target VM’s vpic table
vecptr – [inout] Pointer to vector buffer and will be filled with eligible vector if any.
- Returns
None
-
void vpic_intr_accepted(struct acrn_vpic *vpic, uint32_t vector)¶
Accept virtual interrupt for vPIC.
- Preconditions
vm != NULL
- Parameters
vpic – [in] Pointer to target VM’s vpic table
vector – [in] Target virtual interrupt vector
- Returns
None
Virtual Exception¶
When doing emulation, an exception may need to be triggered in the hypervisor for these reasons:
The guest accesses an invalid vMSR register.
The hypervisor needs to inject a #GP.
The hypervisor needs to inject a #PF when an instruction accesses a non-existent page from
rip_gva
during instruction emulation.
ACRN hypervisor implements virtual exception injection using these APIs:
-
int32_t vcpu_queue_exception(struct acrn_vcpu *vcpu, uint32_t vector_arg, uint32_t err_code_arg)¶
Queue exception to guest.
This exception may be injected immediately or later, depends on the exeception class.
- Preconditions
vcpu != NULL
- Parameters
vcpu – [in] Pointer to vCPU.
vector_arg – [in] Vector of the exeception.
err_code_arg – [in] Error Code to be injected.
- Return values
0 – on success
-EINVAL – on error that vector is invalid.
-
void vcpu_inject_gp(struct acrn_vcpu *vcpu, uint32_t err_code)¶
Inject general protection exeception(GP) to guest.
- Preconditions
vcpu != NULL
- Parameters
vcpu – [in] Pointer to vCPU.
err_code – [in] Error Code to be injected.
- Returns
None
-
void vcpu_inject_pf(struct acrn_vcpu *vcpu, uint64_t addr, uint32_t err_code)¶
Inject page fault exeception(PF) to guest.
- Preconditions
vcpu != NULL
- Parameters
vcpu – [in] Pointer to vCPU.
addr – [in] Address that result in PF.
err_code – [in] Error Code to be injected.
- Returns
None
-
void vcpu_inject_ud(struct acrn_vcpu *vcpu)¶
Inject invalid opcode exeception(UD) to guest.
- Preconditions
vcpu != NULL
- Parameters
vcpu – [in] Pointer to vCPU.
- Returns
None
-
void vcpu_inject_ss(struct acrn_vcpu *vcpu)¶
Inject stack fault exeception(SS) to guest.
- Preconditions
vcpu != NULL
- Parameters
vcpu – [in] Pointer to vCPU.
- Returns
None
ACRN hypervisor uses the vcpu_inject_gp
and vcpu_inject_pf
functions to
queue an exception request. The hypervisor follows Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3, Section 6.15, Table 6-5, to
generate a double fault if the condition is met.
ACRN hypervisor can inject extint
and nmi
using similar vCPU APIs:
-
void vcpu_inject_extint(struct acrn_vcpu *vcpu)¶
Inject external interrupt to guest.
- Preconditions
vcpu != NULL
- Parameters
vcpu – [in] Pointer to vCPU.
- Returns
None
-
void vcpu_inject_nmi(struct acrn_vcpu *vcpu)¶
Inject NMI to guest.
- Preconditions
vcpu != NULL
- Parameters
vcpu – [in] Pointer to vCPU.
- Returns
None
Virtual Interrupt Injection¶
Virtual interrupts come from the DM or assigned devices.
For Service VM assigned devices: Whenever a physical interrupt is from an assigned device, the corresponding virtual interrupt is injected to the Service VM via vLAPIC/vIOAPIC. See Device Assignment.
For User VM assigned devices: Only PCI devices can be assigned to User VMs. For the standard VM and soft RTVM, the virtual interrupt injection process is the same way as for the Service VM. A virtual interrupt injection operation is triggered when a device’s physical interrupt occurs. For the hard RTVM, the physical interrupts are delivered to the VM directly without causing VM-exit.
For User VM emulated devices: DM manages the interrupt lifecycle of emulated devices. DM knows when an emulated device needs to assert a virtual IOAPIC/PIC pin or needs to send a virtual MSI vector to the guest. The logic is entirely handled by DM. Hard RTVMs should not have emulated devices.
Before APICv virtual interrupt delivery, a virtual interrupt can be
injected only if the guest interrupt is allowed. In many cases,
the guest RFLAGS.IF
gets cleared and does not accept any further
interrupts. The hypervisor checks for the available guest IRQ windows before
injection.
NMI is an unmaskable interrupt and its injection is always allowed
regardless of the guest IRQ window status. If the current IRQ
window is not present, the hypervisor enables
MSR_IA32_VMX_PROCBASED_CTLS_IRQ_WIN (PROCBASED_CTRL.bit[2])
and
VM Enter directly. The injection will be done on the next VM Exit once the guest
issues STI (GuestRFLAG.IF=1)
.
Data Structures and Interfaces¶
No data structure is exported to the other components in the hypervisor for virtual interrupts. The APIs listed in the previous sections are meant to be called whenever a virtual interrupt should be injected or acknowledged.