Using GRUB to Boot ACRN¶

GRUB is a multiboot bootloader used by many popular Linux distributions. It also supports booting the ACRN hypervisor. See http://www.gnu.org/software/grub/grub-download.html to get the latest GRUB source code and https://www.gnu.org/software/grub/grub-documentation.html for detailed documentation.

The ACRN hypervisor can boot from multiboot protocol or multiboot2 protocol. Comparing with multiboot protocol, the multiboot2 protocol adds UEFI support.

The multiboot protocol is supported by the ACRN hypervisor natively. The multiboot2 protocol is supported when hv.FEATURES.MULTIBOOT2 is enabled in configuration. The hv.FEATURES.MULTIBOOT2 is enabled by default. Which boot protocol is used depends on the hypervisor is loaded by GRUB’s multiboot command or multiboot2 command. The guest kernel or ramdisk must be loaded by the GRUB module command or module2 command accordingly when different boot protocol is used.

The ACRN hypervisor binary is built with two formats: acrn.32.out in ELF format and acrn.bin in RAW format. The GRUB multiboot command support ELF format only and does not support binary relocation, even if hv.FEATURES.RELOC is set. The GRUB multiboot2 command supports ELF format when hv.FEATURES.RELOC is not set, or RAW format when hv.FEATURES.RELOC is set.

Note

hv.FEATURES.RELOC is set by default, so use acrn.32.out in multiboot protocol and acrn.bin in multiboot2 protocol.
Per ACPI specification, the RSDP pointer is described in the EFI System Table instead of legacy ACPI RSDP area on a UEFI enabled platform. To make sure ACRN hypervisor gets the correct ACPI RSDP info, we recommend using acrn.bin with multiboot2 protocol to load hypervisor on a UEFI platform.

Using Pre-Installed GRUB¶

Most Linux distributions use GRUB version 2 by default. If its version 2.02 or higher, we can re-use the pre-installed GRUB to load the ACRN hypervisor.

Here’s an example using Ubuntu to load ACRN on a scenario with two pre-launched VMs (the SOS_VM is also a kind of pre-launched VM):

Copy ACRN hypervisor binary acrn.32.out (or acrn.bin) and the pre-launched VM kernel images to /boot/;
Modify the /etc/default/grub file as follows to make the GRUB menu visible when booting:
```
# GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=false
```
Append the following configuration in the /etc/grub.d/40_custom file:

Configuration template for multiboot protocol:
```
menuentry 'Boot ACRN hypervisor from multiboot' {
   insmod part_gpt
   insmod ext2
   echo 'Loading ACRN hypervisor ...'
   multiboot --quirk-modules-after-kernel /boot/acrn.32.out $(HV bootargs) $(Service VM bootargs)
   module /boot/kernel4vm0 xxxxxx $(VM0 bootargs)
   module /boot/kernel4vm1 yyyyyy $(VM1 bootargs)
}
```
Configuration template for multiboot2 protocol:
```
menuentry 'Boot ACRN hypervisor from multiboot2' {
   insmod part_gpt
   insmod ext2
   echo 'Loading ACRN hypervisor ...'
   multiboot2 /boot/acrn.bin $(HV bootargs) $(Service VM bootargs)
   module2 /boot/kernel4vm0 xxxxxx $(VM0 bootargs)
   module2 /boot/kernel4vm1 yyyyyy $(VM1 bootargs)
}
```
Note

The module /boot/kernel4vm0 is the VM0 kernel file. The param xxxxxx is VM0’s kernel file tag and must exactly match the kernel_mod_tag of VM0 configured in the misc/vm_configs/scenarios/$(SCENARIO)/vm_configurations.c file. The multiboot module /boot/kernel4vm1 is the VM1 kernel file and the param yyyyyy is its tag and must exactly match the kernel_mod_tag of VM1 in the misc/vm_configs/scenarios/$(SCENARIO)/vm_configurations.c file.

The guest kernel command-line arguments are configured in the hypervisor source code by default if no $(VMx bootargs) is present. If $(VMx bootargs) is present, the default command-line arguments are overridden by the $(VMx bootargs) parameters.

The $(Service VM bootargs) parameter in the multiboot command is appended to the end of the Service VM kernel command line. This is useful to override some Service VM kernel cmdline parameters because the later one would be used if the same parameters were configured in the Linux kernel cmdline. For example, adding root=/dev/sda3 will override the original root device to /dev/sda3 for the Service VM kernel.

All parameters after a # character are ignored since GRUB treats them as comments.

\, $, # are special characters in GRUB. An escape character \ must be added before these special characters if they are included in $(HV bootargs) or $(VM bootargs). For example, memmap=0x200000$0xE00000 for guest kernel cmdline must be written as memmap=0x200000\$0xE00000
Update GRUB:
```
$ sudo update-grub
```
Reboot the platform. On the platform’s console, Select the Boot ACRN hypervisor xxx entry to boot the ACRN hypervisor. The GRUB loader will boot the hypervisor, and the hypervisor will start the VMs automatically.

Installing Self-Built GRUB¶

If the GRUB version on your platform is outdated or has issues booting the ACRN hypervisor, you can have a try with self-built GRUB binary. Get the latest GRUB code and follow the GRUB Manual to build and install your own GRUB, and then follow the steps described earlier in pre-installed-grub.

Here we provide another simple method to build GRUB in EFI application format:

Make GRUB efi application:

$ git clone https://git.savannah.gnu.org/git/grub.git
$ cd grub
$ ./bootstrap
$ ./configure --with-platform=efi --target=x86_64
$ make
$ ./grub-mkimage -p /EFI/BOOT -d ./grub-core/ -O x86_64-efi -o grub_x86_64.efi \
      boot efifwsetup efi_gop efinet efi_uga lsefimmap lsefi lsefisystab \
      exfat fat multiboot2 multiboot terminal part_msdos part_gpt normal \
      all_video aout configfile echo file fixvideo fshelp gfxterm gfxmenu \
      gfxterm_background gfxterm_menu legacycfg video_bochs video_cirrus \
      video_colors video_fb videoinfo video net tftp

This will build a grub_x86_64.efi binary in the current directory, copy it to /EFI/boot/ folder on the EFI partition (it is typically mounted under /boot/efi/ folder on rootfs).

Create /EFI/boot/grub.cfg file containing the following:

set default=0
set timeout=5
# set correct root device which stores acrn binary and kernel images
set root='hd0,gpt3'

menuentry 'Boot ACRN hypervisor from multiboot' {
   insmod part_gpt
   insmod ext2
   echo 'Loading ACRN hypervisor ...'
   multiboot --quirk-modules-after-kernel /boot/acrn.32.out $(HV bootargs) $(Service VM bootargs)
   module /boot/kernel4vm0 xxxxxx $(VM0 bootargs)
   module /boot/kernel4vm1 yyyyyy $(VM1 bootargs)
}

menuentry 'Boot ACRN hypervisor from multiboot2' {
   insmod part_gpt
   insmod ext2
   echo 'Loading ACRN hypervisor ...'
   multiboot2 /boot/acrn.bin $(HV bootargs) $(Service VM bootargs)
   module2 /boot/kernel4vm0 xxxxxx $(VM0 bootargs)
   module2 /boot/kernel4vm1 yyyyyy $(VM1 bootargs)
}

Copy the ACRN binary and guest kernel images to the GRUB-configured folder, e.g. /boot/ folder on /dev/sda3/;
Run /EFI/boot/grub_x86_64.efi in the EFI shell.