Hewlett Packard Enterprise Development LP

Sai Rahul Chalamalasetti

Dejan S Milojicic

Martin Foltin

John Paul Strachan

Sergey Serebryakov

Paolo Faraboschi

Catherine Graves

Andrew Christopher Walton

Amit S Sharma

Cong Xu

Qiong Cai

Ping Chi

Marco Fiorentino

Harumi Kuno

Duncan Roweth

Zhikui Wang

William J Walker

Craig William Warner

Alan Lynn Davis

Geoffrey Ndu

Antonio Lain

Naysen J Robertson

Torsten Wilde

John L Byrne

Ryan Dean Menhusen

Aditya Dhakal

Daniel William Dauwe

Eun Sub Lee

Gourav Rattihalli

Peter Jin Rhim

Anshuman Goswami

Pedro H R Bruel

Vipin Kumar Kukkala

Hyunmin Jeong

Systems and methods are provided for an accelerator system that includes a baseline (production) accelerator, optimizing accelerator, and control hardware accelerator, and an operation of alternatingly switching the production/optimizing accelerators between production and optimizing. With two production/optimizing accelerators, at any given point in time, one accelerator adapts while another accelerator processes data. Once the second accelerator starts doing a better job (e.g., has adapted to data drift), the accelerators change their modes, and the trainable accelerator becomes the 'optimized' one. The accelerators do this non-stop, thus maintaining redundancy, providing expected quality of service (QoS) and adapting to data/concept drift.

Self-adaptable accelerators having alternating production/optimizing modes

A first NIC monitors a key-value cache associated with an LLM executed by a compute node that includes the first NIC and an accelerator. The key-value cache is stored in a memory associated with the accelerator. Responsive to detecting that the key-value cache is updated by the accelerator, the first NIC transfers a copy of the key-value cache update to a remote storage node. The key-value cache is deleted from the memory after the query is inferred. Responsive to receiving a follow-up user query, the first NIC determines a storage location on the remote storage node that stores the key-value cache corresponding to the user query and sends a KV-cache-transfer request to a second NIC on the remote storage node, the KV-cache-transfer request specifying the storage location, thereby facilitating the second NIC to transfer the key-value cache corresponding to the user query from the specified storage location to the memory.

Efficient key-value cache management for large language models

A crossbar array includes a number of memory elements. An analog-to-digital converter (ADC) is electronically coupled to the vector output register. A digital-to-analog converter (DAC) is electronically coupled to the vector input register. A processor is electronically coupled to the ADC and to the DAC. The processor may be configured to determine whether division of input vector data by output vector data from the crossbar array is within a threshold value, and if not within the threshold value, determine changed data values as between the output vector data and the input vector data, and write the changed data values to the memory elements of the crossbar array.

Acceleration of model/weight programming in memristor crossbar arrays

In some examples, a device includes a first processing core comprising a resistive memory array to perform an analog computation, and a digital processing core comprising a digital memory programmable with different values to perform different computations responsive to respective different conditions. The device further includes a controller to selectively apply input data to the first processing core and the digital processing core.

Resistive and digital processing cores

Systems and methods are provided for a multi-die dot-product engine (DPE) to provision large-scale machine learning inference applications. The multi-die DPE leverages a multi-chip architecture. For example, a multi-chip interface can include a plurality of DPE chips, where each DPE chip performs inference computations for performing deep learning operations. A hardware interface between a memory of a host computer and the plurality of DPE chips communicatively connects the plurality of DPE chips to the memory of the host computer system during an inference operation such that the deep learning operations are spanned across the plurality of DPE chips. Due to the multi-die architecture, multiple silicon devices are allowed to be used for inference, thereby enabling power-efficient inference for large-scale machine learning applications and complex deep neural networks. The multi-die DPE can be used to build a multi-device DNN inference system performing specific applications, such as object recognition, with high accuracy.

Multi-die dot-product engine to provision large scale machine learning inference applications

Examples described herein relate to a method for synchronizing a wavelength of light in an optical device. In some examples, a heater voltage may be predicted for a heater disposed adjacent to the optical device in a photonic chip. The predicted heater voltage may be applied to the heater to cause a change in the wavelength of the light inside the optical device. In response to applying the heater voltage, an optical power inside the optical device may be measured. Further, a check may be performed to determine whether the measured optical power is a peak optical power. If it is determined that measured optical power is the peak optical power, the application of the predicted heater voltage to the heater may be continued.

Proactive wavelength synchronization

In exemplary aspects, a golden data structure can be used to validate the stability of machine learning (ML) models and weights. The golden data structure includes golden input data and corresponding golden output data. The golden output data represents the known correct results that should be output by a ML model when it is run with the golden input data as inputs. The golden data structure can be stored in a secure memory and retrieved for validation separately or together with the deployment of the ML model for a requested ML operation. If the golden data structure is used to validate the model and/or weights concurrently with the performance of the requested operation, the golden input data is combined with the input data for the requested operation and run through the model. Relevant outputs are compared with the golden output data to validate the stability of the model and weights.

Resiliency for machine learning workloads

One embodiment provides a system and method for predicting network power usage associated with workloads. During operation, the system configures a simulator to simulate operations of a plurality of network components, which comprises embedding one or more event counters in each simulated network component. A respective event counter is configured to count a number of network-power-related events. The system collects, based on values of the event counters, network-power-related performance data associated with one or more sample workloads applied to the simulator; and trains a machine-learning model with the collected network-power-related performance data and characteristics of the sample workloads as training data 1, thereby facilitating prediction of network-power-related performance associated with a to-be-evaluated workload.

System and method for predicting power usage of network components

In some examples, an apparatus includes a management controller for use in a computer system having a processing resource for executing an operating system (OS) of the computer system, the management controller being separate from the processing resource and to perform, based on operation of the management controller within a cryptographic boundary, management of components of the computer system, the management of components comprising power control of the computer system. The management controller is to receive sensor data, perform facial recognition based on the sensor data, and determine whether to initiate a security action responsive to the facial recognition.

Facial recognition based security by a management controller

Disclosed techniques provide for dynamically changing precision of a multi-stage compute process. For example, changing neural network (NN) parameters on a per-layer basis depending on properties of incoming data streams and per-layer performance of an NN among other considerations. NNs include multiple layers that may each be calculated with a different degree of accuracy and therefore, compute resource overhead (e.g., memory, processor resources, etc.). NNs are usually trained with 32-bit or 16-bit floating-point numbers. Once trained, an NN may be deployed in production. One approach to reduce compute overhead is to reduce parameter precision of NNs to 16 or 8 for deployment. The conversion to an acceptable lower precision is usually determined manually before deployment and precision levels are fixed while deployed. Disclosed techniques and implementations address automatic rather than manual determination or precision levels for different stages and dynamically adjusting precision for each stage at run-time.

Adjustable precision for multi-stage compute processes

A method for allocating resources includes determining that an initial allocation of memory bandwidth for one or more computing jobs fails a performance metric. The memory bandwidth provides access to a global memory pool for multiple legacy processors across a memory fabric. The method also includes determining a new allocation of memory bandwidth for the computing jobs that meets the performance metric. Additionally, the method includes assigning the new allocation of memory bandwidth to the computing jobs. The method further includes executing the computing jobs using the new allocation of memory bandwidth.

Allocating resources of a memory fabric

An example system includes at least one memristive dot product engine (DPE) having at least one resource, the DPE further having a physical interface and a controller, the controller being communicatively coupled to the physical interface, the physical interface to communicate with the controller to access the DPE, and at least one replicated interface, each replicated interface being associated with a virtual DPE, the replicated interface with communicatively coupled to the controller. The controller is to allocate timeslots to the virtual DPE through the associated replicated interface to allow the virtual DPE access to the at least one resource.

Memristive dot product engine virtualization

Techniques for reallocating a memory pending queue based on stalls are provided. In one aspect, it may be determined at a memory stop of a memory fabric that at least one class of memory access is stalled. It may also be determined at the memory stop of the memory fabric that there is at least one class of memory access that is not stalled. At least a portion of a memory pending queue may be reallocated from the class of memory access that is not stalled to the class of memory access that is stalled.

Growing community of inventors

Palo Alto, CA, United States of America

Sai Rahul Chalamalasetti