Using PREEMPT_RT-Linux for real-time UOS

The ACRN project uses various techniques to support a User OS (UOS) running as virtual machine (VM) with real-time characteristics, also called a “RTVM” in ACRN terminology. Some of these techniques include device passthrough and cache allocation technology (CAT), as shown in Figure 230.

To benefit from these techniques, the OS running in the VM should also behave as a real-time system. In this tutorial, we explain how to run a Privileged VM based on a modified Clear Linux* OS using PREEMPT_RT real-time kernel patches from the Real-Time Linux collaborative project.

The PREEMPT_RT patch add preemption models to the mainline Linux kernel. The “Fully Preemptible Kernel” model is the one that turns Linux into an RTOS, allowing the system to react to an external event such as an interrupt within a defined time frame.

Figure 230 Real-Time Linux (PREEMPT_RT) VM on ACRN

The RTVM exclusively owns its passthrough devices, so in addition to the controller and file system used by the SOS, a dedicated storage controller and device are needed to host the RTVM’s root filesystem. The two storage devices should be under different PCI controllers because the system can only pass through PCI-based devices to a guest OS. The Intel NUC7ixDNHE NUC (KBL) is a good platform to set up a real-time system because it has both an NVMe and a SATA controller. You will need both NVMe and SATA storage devices in order to proceed.

The following procedures show an example for setting up a real-time system on Intel KBL NUC with a SATA SSD as /dev/sda and an NVME SSD as /dev/nvme0n1p.

1. Follow the Set up a Clear Linux Operating System instructions in the Getting started guide for Intel NUC to:

   1. Install Clear Linux (version 26800 or higher) onto the NVMe
   2. Install Clear Linux (version 26800 or higher) onto the SATA SSD
   3. Set up Clear Linux on the SATA SSD as the Service OS (SOS) following the Add the ACRN hypervisor to the EFI Partition instructions in the same guide.

2. Patch and build the Real-Time kernel

   1. Download Linux kernel real-time patch:

      $ wget https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/4.19/patch-4.19.31-rt18.patch.xz
      

   2. Sync the kernel code to acrn-2019w17.4-160000p:

      $ git clone https://github.com/projectacrn/acrn-kernel.git
      $ git checkout acrn-2019w17.4-160000p
      $ cd acrn-kernel
      $ wget https://raw.githubusercontent.com/projectacrn/acrn-hypervisor/master/doc/tutorials/rt_linux.patch
      $ git apply rt_linux.patch
      $ xzcat ../patch-4.19.31-rt18.patch.xz | patch -p1
      

   3. Edit the kernel_config_uos config file: search for the keyword “NVME Support”, delete # CONFIG_BLK_DEV_NVME is not set and add two lines under “NVME Support” to enable the NVME driver in RT kernel:

      CONFIG_NVME_CORE=y
      CONFIG_BLK_DEV_NVME=y
      

   4. Build the RT kernel:

      $ cp kernel_config_uos .config
      $ make targz-pkg
      

      Choose “Fully Preemptible Kernel (RT)” when prompted, and choose default for all the other options.

   5. Copy the generated package to SOS:

      $ scp linux-4.19.28-rt18-quilt-2e5dc0ac-dirty-x86.tar.gz <user name>@<SOS ip>:~/
      

3. Configure the system on SOS

   1. Extract kernel boot and lib modules from the package:

      $ cd ~/
      $ tar xzvf linux-4.19.28-rt18-quilt-2e5dc0ac-dirty-x86.tar.gz
      

   2. Copy the extracted lib modules to NVME SSD:

      $ mount /dev/nvme0n1p3 /mnt
      $ cp -r ~/lib/modules/4.19.28-rt18-quilt-2e5dc0ac-dirty /mnt/lib/modules
      

   3. Edit and run the launch_hard_rt_vm.sh script to launch the UOS. A sample launch_hard_rt_vm.sh is included in the Clear Linux release, and is also available in the acrn-hypervisor/devicemodel GitHub repo (in the samples folder).

      You’ll need to modify two places:

      1. Replace /root/rt_uos_kernel with ~/boot/vmlinuz-4.19.28-rt18-quilt-2e5dc0ac-dirty
      2. Replace root=/dev/sda3 with root=/dev/nvme0n1p3

   4. Run the launch script:

      $ sudo ./launch_hard_rt_vm.sh
      

4. At this point, you’ve successfully launched the real-time VM and Guest OS. You can verify a preemptible kernel was loaded using the uname -a command:

   root@rtvm-02 ~ # uname -a
   Linux rtvm-02 4.19.8-rt6+ #1 SMP PREEMPT RT Tue Jan 22 04:17:40 UTC 2019 x86_64 GNU/Linux
   

5. Now you can run all kinds of performance tools to experience real-time performance. One popular tool is cyclictest. You can install this tool and run it with:

   swupd bundle-add dev-utils
   cyclictest -N -p80 -D300