Hewlett-packard Development Company, L.P.

Google Inc.

Hewlett Packard Enterprise Development LP

Parthasarathy Ranganathan

Jichuan Chang

Liqun Cheng

Norman Paul Jouppi

Vanish Talwar

Naveen Muralimanohar

Rama Krishna Govindaraju

Keith Farkas

Kevin T Lim

Doe Hyun Yoon

Cullen Edwin Bash

Sheng Li

Robert Nelson Mayo

Justin Moore

David Andrew Roberts

Puneet Sharma

Robert S Schreiber

Sujata Banerjee

Xiaoyun Zhu

Zhikui Wang

Khaldoun Alzien

Milad Olia Hashemi

David Lo

Richard Yoo

Justin James Meza

Benjamin Charles Serebrin

Sanjay Kumar

Jeffrey Clifford Mogul

Robert J Stets, Jr

Niraj Tolia

Gargi Adhav

Deborah Anne Wallach

Phillip A Leech

Shrikant S Kelkar

Lakshmi Sharma

Nidhi Aggarwal

Parveen Patel

Manoj Jayadevan

Ratnesh Kumar Sharma

William Riis Hamburgen

Paolo Faraboschi

Mehul A Shah

Manish Marwah

Amip J Shah

Nathan Lorenzo Binkert

Michael Roger Marty

Janet Lynn Wiener

Nishant P Patil

Jayaram Mudigonda

Charles N Shaver

Alvin AuYoung

Matteo Monchiero

Indrajit Roy

Chih C Shih

Andrew Gallatin

Harsh Satija

Chandrakant D Patel

Martin Guy Dixon

Quoc V Le

Abdlmonem H Beitelmal

Sridhar Lakshmanamurthy

Mark David Lillibridge

Marcos Kawazoe Aguilera

Yoshio Frank Turner

Mustafa Uysal

Ram Swaminathan

Sriganesh Madhvanath

Thomas J Flynn

Priya Mahadevan

Alistair Veitch

Dwight L Barron

Gregory P Imwalle

Luis E Luciani, Jr

Jeffrey R Hilland

William C Hallowell

Clifford A McCarthy

Jeffrey S Autor

Terence Patrick Kelly

Jose Renato G Santos

Daniel Stodolsky

Mitchel E Wright

Daniel J Magenheimer

Varun Sakalkar

Kevin Jordan Swersky

Shailesh N Joshi

Troy Don Miller

John L Byrne

Andrew James Phelan

Jishen Zhao

Chengwei Wang

Bruce J Walker

Ramakrishna Anne

Jamie Alexander Smith

Binayak Roy

Jacob B Leverich

Christopher Thomas Kennelly

Shivaram Venkataraman

Michael Mihn-Jong Lee

Ke Chen

Ramya Raghavendra

The technology is generally directed to a management framework that uses a predictive feedback control loop to reduce energy consumption of a datacenter. The framework determines how to place a series of jobs or workloads on the available pool of machines in a datacenter. For example, the framework may consider the current workload profile and the workload demand forecast of the datacenter to determine an updated workload profile. The updated workload profile may include a redistribution of the workloads or jobs onto a first subset of the machines such that a second subset of the machines can enter an idle state.

Efficient datacenter energy management based on compute capacity and fleet management

Generally disclosed herein is an approach for dynamically modulating a hardware prefetcher using system software. The hardware prefetcher may be disabled when the amount of the memory bandwidth utilization level is above an upper threshold. The hardware prefetcher may be enabled when the amount of the memory bandwidth utilization level is below a lower threshold. To avoid constantly toggling the hardware prefetcher on and off and allow for smoother transitions, the amount of the memory bandwidth utilization level is observed until it stays above the upper threshold or below the lower threshold for a particular time before the system software changes the latest state of the hardware prefetcher.

Hardware control system to modulate prefetchers based on runtime telemetry

The present disclosure includes systems, methods, and computer-readable mediums for discovering capabilities of local and remote hardware (HW) accelerator cards. A local hardware (HW) accelerator card may provide, via a communication interface, a listing of acceleration services from the local HW accelerator card. The listing of acceleration services may include a first set of acceleration services provided by one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by one or more accelerators of a remote HW accelerator card. A workload instruction defining a workload for processing by at least one of the acceleration services of the second set of acceleration services may be received from a processor of a computing device. The workload instruction may be forwarded to the remote HW accelerator card.

Hardware accelerator service aggregation

Aspects of the disclosure are directed to a heterogeneous machine learning accelerator system with compute and memory nodes connected by high speed chip-to-chip interconnects. While existing remote/disaggregated memory may require memory expansion via remote processing units, aspects of the disclosure add memory nodes into machine learning accelerator clusters via the chip-to-chip interconnects without needing assistance from remote processing units to achieve higher performance, simpler software stack, and/or lower cost. The memory nodes may support prefetch and intelligent compression to enable the use of low cost memory without performance degradation.

Heterogeneous ML accelerator cluster with flexible system resource balance

The subject matter described herein provides systems and techniques to address the challenges of growing hardware and workload heterogeneity using a Warehouse-Scale Computer (WSC) design that improves the efficiency and utilization of WSCs. The WSC design may include an abstraction layer and an efficiency layer in the software stack of the WSC. The abstraction layer and the efficiency layer may be designed to improve job scheduling, simplify resource management, and drive hardware-software co-optimization using machine learning techniques and automation in order to customize the WSC for applications at scale. The abstraction layer may embrace platform/hardware and workload diversity through greater coordination between hardware and higher layers of the WSC software stack in the WSC design. The efficiency layer may employ machine learning techniques at scale to realize hardware/software co-optimizations as a part of the autonomous WSC design.

Autonomous warehouse-scale computers

The present disclosure includes systems, methods, and computer-readable mediums for discovering capabilities of a hardware (HW) accelerator card. A processor may communicate a request for a listing of acceleration services to a HW accelerator card connected to the processor via the communication interface. The HW accelerator card may retrieve the listing from memory and provide a response to the processor that includes a listing of the HW acceleration services provided by the HW accelerator card.

Hardware accelerator service discovery

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for pre-fetching data from memory using neural networks. One example system receives a sequence of prior program counter addresses of a computer program and corresponding delta values. The system creates an input representation based on the sequence. The system provides the input representation as input to a recurrent neural network. The system receives from the recurrent neural network an output that defines a probability distribution over future delta values. Each probability in the distribution represents a likelihood that execution of a future instruction of the computer program will cause data to be fetched from a particular future memory address.

Computer system prediction machine learning models

Methods, systems, and computer storage media storing instructions for managing processing system efficiency. One of the methods includes obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

Managing processing system efficiency

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for disaggregating latent causes for computer system optimization. In one aspect, a method includes accessing a data stream for data values resulting from operations performed by a computer system; providing the data values as input to a data disaggregation machine learning model that generates descriptors of latent causes of the data values; providing the data values and the descriptors of the latent causes of the data values as inputs to a control system model that generates embedded representations of commands to modify the operations performed by the computer system; determining commands to modify the operations performed by the computer system based on the embedded representations of commands to modify the operations performed by the computer system; and providing the commands to the computer system.

Disaggregating latent causes for computer system optimization

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for caching data not frequently accessed. One of the methods includes receiving a request for data from a component of a device, determining that the data satisfies an infrequency condition, in response to determining that the data satisfies the infrequency condition: determining a target cache level which defines a cache level within a cache level hierarchy of a particular cache at which to store infrequently accessed data, the target cache level being lower than a highest cache level in the cache level hierarchy, requesting and receiving the data from a memory that is not a cache of the device, and storing the data in a level of the particular cache that is at or below the target cache level in the cache level hierarchy, and providing the data to the component.

Data caching

A method for compressing and compacting memory on a memory device is described. The method includes organizing a number of compressed memory pages referenced in a number of compaction table entries based on a size of the number of compressed memory pages and compressing the number of compaction table entries, in which a compaction table entry comprise a number of fields.

Compressing and compacting memory on a memory device wherein compressed memory pages are organized by size

A first memory controller receives an access command from a second memory controller, where the access command is timing non-deterministic with respect to a timing specification of a memory. The first memory controller sends at least one access command signal corresponding to the access command to the memory, wherein the at least one access command signal complies with the timing specification. The first memory controller determines a latency of access of the memory. The first memory controller sends feedback information relating to the latency to the second memory controller.

Separate memory controllers to access data in memory

Growing community of inventors

San Jose, CA, United States of America

Parthasarathy Ranganathan