Vmware, Inc.

Andrei Warkentin

Cyprien Laplace

Alexander Fainkichen

Ye Li

Regis Duchesne

Harvey Tuch

Sunil Kumar Kotian

Shruthi Muralidhara Hiriyuru

Jared McNeill

David Ott

Timothy P Mann

William Lam

Adrian Drzewiecki

Daniel Muller

Irfan Ulla Khan

Renaud Benjamin Voltz

Karthik Ramachandra

David A Dunn

Aravinda Haryadi

John L Manferdelli

Anjaneya Prasad Gondi

Doug Covelli

Anoop Jaishankar

Lingyuan He

Renukanthan Raman

Kalaiselvi Sengottuvel

Suman Boro

Srihari Venkatesan

Anil Veliyankaramadam

Grant Foudree

Cyrien Laplace

Disclosed are various examples of loading management hypervisors from a system control processor. In some examples, a host device executes a first stage bootloader of a management hypervisor from a system control processor. The first stage bootloader loads management hypervisor data and firmware instructions into a main processor memory of a main processor, and initializes the main processor to execute the firmware instructions. The system then jumps to a second stage bootloader that configures and launches the management hypervisor using the management hypervisor data.

Loading management hypervisors from a system control processor

An example method of virtualizing a host virtual counter and timer in a central processing unit (CPU) of a virtualized host computer includes: creating, by a hypervisor of the host computer in response to power on of a virtual machine (VM), a guest virtual counter, the guest virtual counter comprising a data structure including scaling factors; mapping a shared memory page having the data structure into an address space of memory allocated to the VM; and notifying a guest operating system (OS) of the VM of a location in the address space for the shared memory page having the data structure, the guest OS being paravirtualized to scale the host virtual counter and timer based on the scaling factors of the guest virtual counter.

Scaling a host virtual counter and timer in a virtualized computer system

Disclosed are various examples of lifecycle and recovery management for virtualized data processing unit (DPU) management operating systems. A DPU device executes a DPU management hypervisor that communicates with a management service over a network. The DPU management hypervisor virtualizes DPU hardware resources and passes control of the virtualized DPU hardware resources to a DPU management operating system (OS) virtual machine (VM). The DPU management hypervisor maintains control of a management network interface card (NIC) of the DPU device.

Lifecycle and recovery for virtualized DPU management operating systems

Systems and methods are included for causing a computing device to boot by retrieving hardware information from a device tree and further properties by utilizing a native access method call identified in the device tree. The access method can allow for getting a property, getting a property length, or setting a property. A table within firmware can identify the method, which then can retrieve the property information from memory. This Device tree Runtime ('DTRT') mechanism can allow the computing device to retrieve the hardware configuration and act as a power management interface for turning on the correct hardware and hardware properties on the computing device.

Device tree runtime mechanism

A hybrid scheme is provided for performing translation lookaside buffer (TLB) shootdowns in a computer system whose processing cores support both inter-processor interrupt (IPI) and broadcast TLB invalidate (TLBI) shootdown mechanisms. In one set of embodiments, this hybrid scheme dynamically determines, for each instance where a TLB shootdown is needed, whether to use the IPI mechanism or the broadcast TLBI mechanism to optimize shootdown performance (or otherwise make the TLB shootdown operation functional/practical).

Hybrid scheme for performing translation lookaside buffer (TLB) shootdowns

An example method of managing guest time for a virtual machine (VM) supported by a hypervisor of a virtualized host computer includes: configuring, by the hypervisor, a central processing unit (CPU) of the host computer to trap, to the hypervisor, access by guest code in the VM to a physical counter and timer of the CPU; configuring, by the hypervisor, the guest code in the VM to use the physical counter and timer of the CPU rather than a virtual counter and timer of the CPU; trapping, at the hypervisor, an access to the physical counter and timer by the guest code; and executing, by the hypervisor, the access to the physical counter and timer on behalf of the guest code while compensating for an adjustment of a system count of the physical counter and timer to maintain the guest time as scaled with respect to frequency of the physical counter and timer.

Guest time scaling for a virtual machine in a virtualized computer system

Disclosed are various examples of provisioning a data processing unit (DPU) management operating system (OS). A management hypervisor installer executed on a host device launches or causes a server component to provide a management operating system (OS) installer image at a particular URI accessible over a network internal to the host device. A baseboard management controller (BMC) transfers the DPU management OS installer image to the DPU device. A volatile memory based virtual disk is created using the DPU management OS installer image. The DPU device is booted to a DPU management OS installer on the volatile memory based virtual disk. The DPU management OS installer installs a DPU management operating system to a nonvolatile memory of the DPU device on reboot of the DPU device.

Provisioning DPU management operating systems

Examples provide for automatically provisioning hosts in a cloud environment. A cloud daemon generates a cloud host-state configuration, for a given cloud instance of a host, stored on a cloud metadata service prior to first boot of the given cloud instance of the host. A first boot of a plurality of cloud instances of hosts is performed using a stateless, master boot image lacking host-specific configuration data. On completion of the first boot of a given cloud instance of a host, the cloud host-state configuration is installed on the master boot image to generate a self-configured boot image including host-specific configuration data for the given cloud instance of the host. A second boot is performed on the given cloud instance of the host by executing the self-configured boot image to automatically provision the given cloud instance of the host in the cloud environment.

Remote provisioning of hosts in public clouds

Disclosed are various examples of provisioning a data processing unit (DPU) management operating system using a capsule. A management hypervisor installer executed on a host device receives a listing DPU device from a baseboard management controller (BMC). A preinstalled DPU management operating system image is identified for a DPU device from the listing, and is wrapped with a capsule that specifies the capsule as a DPU management operating system image capsule. A server component provides the DPU management operating system image capsule at a particular URI, and the URI is transmitted to the BMC.

Provisioning DPU management operating systems using firmware capsules

Disclosed are various examples of provisioning a data processing unit (DPU) management operating system (OS). A host device boots a host provisioning image, which executes a host provisioning agent. The host provisioning agent launches a server component that serves a DPU management OS. A provisioning command is transmitted to a DPU device installed to the host device. The server component transmits the DPU management OS from the host device to the DPU device. A host OS is executed once an indication that the DPU device is executing on the DPU management OS is received.

Provisioning DPU management operating systems using host and DPU boot coordination

The disclosure herein describes deploying a Virtual Secure Enclave (VSE) using a universal enclave binary and a Trusted Runtime (TR). A universal enclave binary is generated that includes a set of binaries of Instruction Set Architectures (ISAs) associated with Trusted Execution Environment (TEE) hardware backends. A TEE hardware backend is identified in association with a VSE-compatible device. A VSE that is compatible with the identified TEE hardware backend is generated on the VSE-compatible device and an ISA binary that matches the TEE hardware backend is selected from the universal enclave binary. The selected binary is linked to a runtime library of the TR and loads the linked binary into memory of the generated VSE. The execution of a trusted application is initiated in the generated VSE using a set of interfaces of the TR. The trusted application depends on the TR interfaces rather than the selected ISA binary.

Deploying enclaves on different tee backends using a universal enclave binary

An example method of exception handling in a computer system is described. The computer system includes a physical central processing unit (PCPU) and a system memory, the system memory storing a first stack, a second stack, and a double fault stack associated with the PCPU. The method includes: storing, by an exception handler executing in the computer system, an exception frame on the double fault stack in response to a stack overflow condition of the first stack; switching, by the exception handler, a first stack pointer of the PCPU from pointing to the first stack to pointing to the double fault stack; setting a current stack pointer of the PCPU to the first stack pointer; and executing software on the PCPU with the current stack pointer pointing to the double fault stack.

Behavioral implementation of a double fault stack in a computer system

Disclosed are various examples of providing provide efficient waiting for detection of memory value updates for Advanced RISC Machines (ARM) architectures. An ARM processor component instructs a memory agent to perform a processing action, and executes a waiting function. The waiting function ensures that the processing action is completed by the memory agent. The waiting function performs an exclusive load at a memory location, and a wait for event (WFE) instruction that causes the ARM processor component to wait in a low-power mode for an event register to be set. Once the event register is set, the waiting function completes and a second processing action is executed by the ARM processor component.

Power efficient memory value updates for arm architectures

Disclosed are various approaches for exposing peripheral component interconnect express (PCIe) configuration space implementations as Enhanced Configuration Access Mechanism (ECAM)-compatible. In some examples, a bridge device is identified on a segment corresponding to a root complex of a computing device. An endpoint device is connected to a bus downstream from the bridge device. A synthetic segment identifier is assigned to the bus once the endpoint device is identified as connected to the bus. Synthetic address data is generated for the endpoint device. The synthetic address data includes the synthetic segment identifier for the bus and sets a bus identifier of the bus to zero regardless of a hierarchical position of the bus in a standard peripheral component interconnect express (PCIe) bus hierarchy.

Exposing PCIE configuration spaces as ECAM compatible

System and method for creating and managing trusted execution environments (TEEs) using different underlying hardware TEE mechanisms use a virtual secure enclave device which runs in a virtualized environment in a computer system. The device enables an enclave command transmitted to the virtual secure enclave device to be retrieved and parsed to extract an enclave operation to be executed. A TEE backend module is used to interact with a particular hardware TEE mechanism among those available in the computer system. The module ensures the enclave operation for the software process is executed by the particular hardware TEE mechanism, or the TEE scheme based on a particular hardware TEE mechanism.

Growing community of inventors

Wakefield, MA, United States of America

Andrei Warkentin