Trusty TEE on ACRN

Introduction

Trusty is a set of software components supporting a Trusted Execution Environment (TEE). TEE is commonly known as an isolated processing environment in which applications can be securely executed irrespective of the rest of the system. For more information about TEE, please visit the Trusted Execution Environment wiki page. Trusty consists of:

1. An operating system (the Trusty OS) that runs on a processor intended to provide a TEE
2. Drivers for the Android kernel (Linux) to facilitate communication with applications running under the Trusty OS
3. A set of libraries for Android/Linux systems software to facilitate communication with trusted applications executed within the Trusty OS using the kernel drivers

LK (Little Kernel) is a tiny operating system suited for small embedded devices, bootloaders, and other environments where OS primitives such as threads, mutexes, and timers are needed, but there’s a desire to keep things small and lightweight. LK has been chosen as the Trusty OS kernel.

Trusty Architecture

Note

Trusty OS is running in Secure World in the architecture drawing above.

Trusty specific Hypercalls

There are a few Hypercall APIs that are related to Trusty.

int32_t hcall_world_switch(struct vcpu *vcpu)

Switch vCPU state between Normal/Secure World.

• The hypervisor uses this hypercall to do the world switch
• The hypervisor needs to:
  • save current world vCPU contexts, and load the next world vCPU contexts
  • update rdi, rsi, rdx, rbx to next world vCPU contexts

Return

0 on success, non-zero on error.

Parameters

• vcpu: Pointer to VCPU data structure

int32_t hcall_initialize_trusty(struct vcpu *vcpu, uint64_t param)

Initialize environment for Trusty-OS on a vCPU.

• It is used by the User OS bootloader (UOS_Loader) to request ACRN to initialize Trusty
• The Trusty memory region range, entry point must be specified
• The hypervisor needs to save current vCPU contexts (Normal World)

Return

0 on success, non-zero on error.

Parameters

• vcpu: Pointer to vCPU data structure
• param: guest physical address. This gpa points to trusty_boot_param structure

int64_t hcall_save_restore_sworld_ctx(struct vcpu *vcpu)

Save/Restore Context of Secure World.

Return

0 on success, non-zero on error.

Parameters

• vcpu: Pointer to VCPU data structure

Trusty Boot flow

By design, the User OS bootloader (UOS_Loader) will trigger the Trusty boot process. The complete boot flow is illustrated below.

Trusty Boot Flow

As shown in the above figure, here are some details about the Trusty boot flow processing:

1. UOS_Loader
   1. Load and verify Trusty image from virtual disk
   2. Allocate runtime memory for trusty
   3. Do ELF relocation of trusty image and get entry address
   4. Call hcall_initialize_trusty with trusty memory base and entry address
2. ACRN (hcall_initialize_trusty)
   1. Save World context for Normal World
   2. Init World context for Secure World (RIP, RSP, EPT, etc.)
   3. Resume to Secure World
3. Trusty
   1. Booting up
   2. Call hcall_world_switch to switch back to Normal World if boot completed
4. ACRN (hcall_world_switch)
   1. Save World context for the World which caused this vmexit (Secure World)
   2. Restore World context for next World (Normal World (UOS_Loader))
   3. Resume to next World (UOS_Loader)
5. UOS_Loader
   1. Continue to boot

EPT Hierarchy

As per the Trusty design, Trusty can access Normal World’s memory, but Normal World cannot access Secure World’s memory. Hence it means Secure World EPTP page table hierarchy must contain normal world GPA address space, while Trusty world’s GPA address space must be removed from the Normal world EPTP page table hierarchy.

Design

Put Secure World’s GPA to very high position: 511 GB - 512 GB. The PML4/PDPT for Trusty World are separated from Normal World. PD/PT for low memory (< 511 GB) are shared in both Trusty World’s EPT and Normal World’s EPT. PD/PT for high memory (>= 511 GB) are valid for Trusty World’s EPT only.

Benefit

This design will benefit the EPT changes of Normal World. There are requirement to modify Normal World’s EPT during runtime such as increasing memory, changing attributes, etc. If such behavior happened, only PD and PT for Normal World need to be updated.

API

static void create_secure_world_ept(struct vm *vm, uint64_t gpa_orig, uint64_t size, uint64_t gpa_rebased)

Create Secure World EPT hierarchy.

Create Secure World EPT hierarchy, construct new PML4/PDPT, reuse PD/PT parse from vm->arch_vm->ept

Parameters

• vm: pointer to a VM with 2 Worlds
• gpa_orig: original gpa allocated from vSBL
• size: LK size (16M by default)
• gpa_rebased: gpa rebased to offset xxx (511G_OFFSET)

void destroy_secure_world(struct vm *vm, bool need_clr_mem)

static void save_world_ctx(struct vcpu *vcpu, struct ext_context *ext_ctx)

static void load_world_ctx(struct vcpu *vcpu, struct ext_context *ext_ctx)

static void copy_smc_param(struct run_context *prev_ctx, struct run_context *next_ctx)

void switch_world(struct vcpu *vcpu, int next_world)

static bool setup_trusty_info(struct vcpu *vcpu, uint32_t mem_size, uint64_t mem_base_hpa)

static bool init_secure_world_env(struct vcpu *vcpu, uint64_t entry_gpa, uint64_t base_hpa, uint32_t size)

bool initialize_trusty(struct vcpu *vcpu, uint64_t param)

void trusty_set_dseed(void *dseed, uint8_t dseed_num)

void save_sworld_context(struct vcpu *vcpu)

void restore_sworld_context(struct vcpu *vcpu)