Getting Started Guide for ACRN Industry Scenario

Verified version

  • Clear Linux version: 32030
  • ACRN-hypervisor tag: v1.5 (acrn-2020w01.1-140000p)
  • ACRN-Kernel (Service VM kernel): 4.19.78-98.iot-lts2018-sos

Prerequisites

The example below is based on the Intel Kaby Lake NUC platform with two disks, a SATA disk for the Clear Linux-based Service VM and an NVMe disk for the RTVM.

  • Intel Kaby Lake (aka KBL) NUC platform with two disks inside (refer to the tables for detailed information).
  • If you need to enable the serial port on the KBL NUC, navigate to the troubleshooting section that discusses how to prepare the cable.
  • Follow the steps below to install Clear Linux OS (ver: 31670) onto the SATA disk of the KBL NUC. In our example, we install Clear Linux with version 31670; the subsequent ACRN quick setup script will upgrade Clear Linux to version 32030:
  1. Create a bootable USB drive on Linux*:

    1. Download the Clear Linux OS Server image.

    2. Plug in the USB drive.

    3. Use the lsblk command line to identify the USB drive:

      $ lsblk | grep sd*
      sda         8:0    0 931.5G  0 disk
      ├─sda1      8:1    0   512M  0 part /boot/efi
      ├─sda2      8:2    0 930.1G  0 part /
      └─sda3      8:3    0   977M  0 part [SWAP]
      sdc         8:32   1  57.3G  0 disk
      └─sdc1      8:33   1  57.3G  0 part
      
    4. Unmount all the /dev/sdc partitions and burn the image onto the USB drive:

      $ umount /dev/sdc* 2>/dev/null
      $ sudo dd if=./clear-31670-live-server.iso of=/dev/sdc oflag=sync status=progress bs=4M
      
  2. Plug in the USB drive to the KBL NUC and boot from USB.

  3. Launch the Clear Linux OS installer boot menu.

  4. With Clear Linux OS highlighted, select Enter.

  5. Log in with your root account and new password.

  6. Run the installer using the following command:

    # clr-installer
    
  7. From the Main menu, select Configure Installation Media and set Destructive Installation to your desired hard disk.

  8. Select Telemetry to set Tab to highlight your choice.

  9. Press A to show the Advanced options.

  10. Select Select additional bundles and add bundles for network-basic, and user-basic.

  11. Select Automatic OS Updates and choose No [Disable].

  12. Select Manage User and choose Add New User.

  13. Select Install.

  14. Select Confirm Install in the Confirm Installation window to start the installation.

Note

Refer to these step-by-step instructions from the Clear Linux OS installation guide.

Hardware Setup

Table 2 Hardware Setup
Platform (Intel x86) Product/kit name Hardware Descriptions
Kaby Lake NUC7i7DNH Processor
  • Intel® Core™ i7-8650U CPU @ 1.90GHz
Graphics
  • UHD Graphics 620
  • Two HDMI 2.0a ports supporting 4K at 60 Hz
System memory
  • 8GiB SODIMM DDR4 2400 MHz [1]
Storage capabilities
  • SATA: 1TB WDC WD10SPZX-22Z
  • NVMe: 256G Intel Corporation SSD Pro 7600p/760p/E 6100p
[1]The maximum supported memory size for ACRN is 16GB. If you are using 32GB memory, follow the How do I configure ACRN’s memory size? instructions to make a customized ACRN hypervisor that can support 32GB memory. For more detailed information about how to build ACRN from the source code, refer to this guide.

Set up the ACRN Hypervisor for industry scenario

The ACRN industry scenario environment can be set up in several ways. The two listed below are recommended:

Use the pre-installed industry ACRN hypervisor

Note

Skip this section if you choose Using the ACRN industry out-of-the-box image.

  1. Boot Clear Linux from SATA disk.

  2. Login as root and download ACRN quick setup script:

    # wget https://raw.githubusercontent.com/projectacrn/acrn-hypervisor/master/doc/getting-started/acrn_quick_setup.sh
    # chmod +x acrn_quick_setup.sh
    
  3. Run the script to set up Service VM:

    # ./acrn_quick_setup.sh -s 32030 -d -i
    

    Note

    -i option means the industry scenario efi image will be used, e.g. acrn.nuc7i7dnb.industry.efi. For the detailed usage of the acrn_quick_setup.sh script, refer to the quick setup ACRN guide or simply type ./acrn_quick_setup.sh -h.

  4. Use efibootmgr -v command to check the ACRN boot order:

    BootCurrent: 000C
    Timeout: 1 seconds
    BootOrder: 0001,0002,000C,000D,0008,000E,000B,0003,0000,0004,0007
    Boot0000* Windows Boot Manager    VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.4.7.9.5.}...o................
    Boot0001* ACRN    HD(1,GPT,c6715698-0f6e-4e27-bb1b-bf7779c1486d,0x800,0x47000)/File(\EFI\acrn\acrn.efi)u.a.r.t.=.d.i.s.a.b.l.e.d.
    Boot0002* Linux bootloader        HD(3,GPT,b537f16f-d70f-4f1b-83b4-0f11be83cd83,0xc1800,0xded3000)/File(\EFI\org.clearlinux\bootloaderx64.efi)
    Boot0003* CentOS  VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
    Boot0004* CentOS Linux    VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
    Boot0007* Linux bootloader        VenHw(99e275e7-75a0-4b37-a2e6-c5385e6c00cb)
    Boot0008* UEFI : Built-in EFI Shell       VenMedia(5023b95c-db26-429b-a648-bd47664c8012)..BO
    Boot000B* LAN : IBA CL Slot 00FE v0110    BBS(Network,,0x0)..BO
    Boot000C* SATA : PORT 0 : KINGSTON SUV500120G : PART 0 : Boot Drive       BBS(HD,,0x0)..BO
    Boot000D* INTEL SSDPEKKW256G8 : PART 0 : Boot Drive       BBS(HD,,0x0)..BO
    Boot000E* UEFI : INTEL SSDPEKKW256G8 : PART 0 : OS Bootloader     PciRoot(0x0)/Pci(0x1d,0x0)/Pci(0x0,0x0)/NVMe(0x1,00-00-00-00-00-00-00-00)/HD(1,GPT,8aa992f8-8149-4f6b-8b64-503998c776c1,0x800,0x47000)..BO
    

    Note

    Ensure that ACRN is first in the boot order, or you may use the efibootmgr -o 1 command to move it to the first position. If you need to enable the serial port, run the following command before rebooting:

    efibootmgr -c -l '\EFI\acrn\acrn.efi' -d /dev/sda -p 1 -L ACRN -u "uart=port@0x3f8 "

    Note the extra space at the end of the EFI command-line options string. This is a workaround for a current efi-stub bootloader name issue. It ensures that the end of the string is properly detected.

  5. Reboot KBL NUC.

  6. Use the dmesg command to ensure that the Service VM boots:

    # dmesg | grep ACRN
    [    0.000000] Hypervisor detected: ACRN
    [    1.252840] ACRNTrace: Initialized acrn trace module with 4 cpu
    [    1.253291] ACRN HVLog: Failed to init last hvlog devs, errno -19
    [    1.253292] ACRN HVLog: Initialized hvlog module with 4
    

Use the ACRN industry out-of-the-box image

Note

If you are following the section above to set up the Service VM, jump to the next section.

  1. Boot Clear Linux from NVMe disk.

  2. Download the Service VM industry image:

    # wget https://github.com/projectacrn/acrn-hypervisor/releases/download/acrn-2020w01.1-140000p/sos-industry-32030.img.xz
    
  3. Decompress the .xz image:

    # xz -d sos-industry-32030.img.xz
    
  4. Burn the Service VM image onto the SATA disk:

    # dd if=sos-industry-32030.img of=/dev/sda bs=4M oflag=sync status=progress iflag=fullblock seek=0 conv=notrunc
    
  5. Configure the EFI firmware to boot the ACRN hypervisor by default:

    # efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/sda -p 1 -L "ACRN" -u "uart=disabled "
    

    Or use the following command to enable the serial port:

    # efibootmgr -c -l "\EFI\acrn\acrn.efi" -d /dev/sda -p 1 -L "ACRN" -u "uart=port@0x3f8 "
    

    Note

    Note the extra space at the end of the EFI command-line options strings above. This is a workaround for a current efi-stub bootloader name issue. It ensures that the end of the string is properly detected.

  6. Reboot the test machine. After the Clear Linux OS boots, log in as “root” for the first time.

Install and launch the Preempt-RT VM

  1. Log in to the Service VM with root privileges.

  2. Download the Preempt-RT VM image:

    # wget https://github.com/projectacrn/acrn-hypervisor/releases/download/acrn-2020w01.1-140000p/preempt-rt-32030.img.xz
    
  3. Decompress the xz image:

    # xz -d preempt-rt-32030.img.xz
    
  4. Burn the Preempt-RT VM image onto the NVMe disk:

    # dd if=preempt-rt-32030.img of=/dev/nvme0n1 bs=4M oflag=sync status=progress iflag=fullblock seek=0 conv=notrunc
    
  5. Use the lspci command to ensure that the correct NVMe device IDs will be used for the passthru before launching the script:

    # lspci -v | grep -iE 'nvm|ssd'
    02:00.0 Non-Volatile memory controller: Intel Corporation Device f1a6 (rev 03) (prog-if 02 [NVM Express])
    
    # lspci -nn | grep "Non-Volatile memory controller"
    02:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a6] (rev 03)
    
  6. Modify the script to use the correct NVMe device IDs and bus number.

    # vim /usr/share/acrn/samples/nuc/launch_hard_rt_vm.sh
    
    passthru_vpid=(
    ["eth"]="8086 156f"
    ["sata"]="8086 9d03"
    ["nvme"]="8086 f1a6"
    )
    passthru_bdf=(
    ["eth"]="0000:00:1f.6"
    ["sata"]="0000:00:17.0"
    ["nvme"]="0000:02:00.0"
    )
    
    /usr/bin/acrn-dm -A -m $mem_size -s 0:0,hostbridge \
       --lapic_pt \
       --rtvm \
       --virtio_poll 1000000 \
       -U 495ae2e5-2603-4d64-af76-d4bc5a8ec0e5 \
       -s 2,passthru,02/00/0 \
       -s 3,virtio-console,@stdio:stdio_port \
       -s 8,virtio-net,tap0 \
       $pm_channel $pm_by_vuart \
       --ovmf /usr/share/acrn/bios/OVMF.fd \
       hard_rtvm
    
    }
    
  7. Upon deployment completion, launch the RTVM directly onto your KBL NUC:

    # /usr/share/acrn/samples/nuc/launch_hard_rt_vm.sh
    

RT Performance Test

Cyclictest introduction

The cyclictest is most commonly used for benchmarking RT systems. It is one of the most frequently used tools for evaluating the relative performance of real-time systems. Cyclictest accurately and repeatedly measures the difference between a thread’s intended wake-up time and the time at which it actually wakes up in order to provide statistics about the system’s latencies. It can measure latencies in real-time systems that are caused by hardware, firmware, and the operating system. The cyclictest is currently maintained by Linux Foundation and is part of the test suite rt-tests.

Pre-Configurations

Firmware update on the NUC

If you need to update to the latest UEFI firmware for the NUC hardware. Follow these BIOS Update Instructions for downloading and flashing an updated BIOS for the NUC.

Configure RDT

In addition to setting the CAT configuration via HV commands, we allow developers to add CAT configurations to the VM config and configure automatically at the time of RTVM creation. Refer to RDT Configuration for details on RDT configuration and RDT Allocation Feature Supported by Hypervisor for details on RDT high-level design.

Set up the core allocation for the RTVM

In our recommended configuration, two cores are allocated to the RTVM: core 0 for housekeeping and core 1 for RT tasks. In order to achieve this, follow the below steps to allocate all housekeeping tasks to core 0:

  1. Launch RTVM:

    # /usr/share/acrn/samples/nuc/launch_hard_rt_vm.sh
    
  2. Log in to RTVM as root and run the script as below:

    #!/bin/bash
    # Copyright (C) 2019 Intel Corporation.
    # SPDX-License-Identifier: BSD-3-Clause
    # Move all IRQs to core 0.
    for i in `cat /proc/interrupts | grep '^ *[0-9]*[0-9]:' | awk {'print $1'} | sed 's/:$//' `;
    do
        echo setting $i to affine for core zero
        echo 1 > /proc/irq/$i/smp_affinity
    done
    
    # Move all rcu tasks to core 0.
    for i in `pgrep rcu`; do taskset -pc 0 $i; done
    
    # Change realtime attribute of all rcu tasks to SCHED_OTHER and priority 0
    for i in `pgrep rcu`; do chrt -v -o -p 0 $i; done
    
    # Change realtime attribute of all tasks on core 1 to SCHED_OTHER and priority 0
    for i in `pgrep /1`; do chrt -v -o -p 0 $i; done
    
    # Change realtime attribute of all tasks to SCHED_OTHER and priority 0
    for i in `ps -A -o pid`; do chrt -v -o -p 0 $i; done
    
    echo disabling timer migration
    echo 0 > /proc/sys/kernel/timer_migration
    

    Note

    You can ignore the error messages during the script running.

Run cyclictest

  1. Refer to the troubleshooting section below that discusses how to enable the network connection for RTVM.

  2. Launch RTVM and log in as root.

  3. Install the cyclictest tool:

    # swupd bundle-add dev-utils
    
  4. Use the following command to start cyclictest:

    # cyclictest -a 1 -p 80 -m -N -D 1h -q -H 30000 --histfile=test.log
    

    Parameter descriptions:

    -a 1:to bind the RT task to core 1
    -p 80:to set the priority of the highest prio thread
    -m:lock current and future memory allocations
    -N:print results in ns instead of us (default us)
    -D 1h:to run for 1 hour, you can change it to other values
    -q:quiee mode; print a summary only on exit
    -H 30000 –histfile=test.log:
     dump the latency histogram to a local file

Troubleshooting

Use serial port on KBL NUC

You can enable the serial console on the KBL NUC (NUC7i7DNH). The KBL NUC has a serial port header you can expose with a serial DB9 header cable. You can build this cable yourself; refer to the KBL NUC product specification as shown below:

../_images/KBL-serial-port-header.png

Figure 21 KBL serial port header details

Or you can purchase such a cable.

You’ll also need an RS232 DB9 female to USB cable, or an RS232 DB9 female/female (NULL modem) cross-over cable to connect to your host system.

Note that If you want to use the RS232 DB9 female/female cable, choose the cross-over type rather than straight-through type.

EFI image doesn’t exist

You might see the error message if you are running the acrn_quick_setup.sh script on an older Clear Linux OS ( < 31470 ):

/usr/lib/acrn/acrn.nuc7i7dnb.industry.efi doesn't exist.
Use one of these efi images from /usr/lib/acrn.
------
/usr/lib/acrn/acrn.kbl-nuc-i7.industry.efi
------
Copy the efi image to /usr/lib/acrn/acrn.nuc7i7dnb.industry.efi, then run the script again.

To fix it, just rename the existing efi image to /usr/lib/acrn/acrn.nuc7i7dnb.industry.efi and then run the script again:

# cp -r /usr/lib/acrn/acrn.kbl-nuc-i7.industry.efi /usr/lib/acrn/acrn.nuc7i7dnb.industry.efi
# ./acrn_quick_setup.sh -s <target version> -i -d

Enabling the network on RTVM

If you need to access the internet, you must add the following command line to the launch_hard_rt_vm.sh script before launch it:

/usr/bin/acrn-dm -A -m $mem_size -s 0:0,hostbridge \
   --lapic_pt \
   --rtvm \
   --virtio_poll 1000000 \
   -U 495ae2e5-2603-4d64-af76-d4bc5a8ec0e5 \
   -s 2,passthru,02/0/0 \
   -s 3,virtio-console,@stdio:stdio_port \
   -s 8,virtio-net,tap0 \
   $pm_channel $pm_by_vuart \
   --ovmf /usr/share/acrn/bios/OVMF.fd \
   hard_rtvm
}