Google Inc.

Aakanksha Chowdhery

Jeffrey Adgate Dean

Sanjay Ghemawat

Yonghui Wu

Ruoming Pang

Michael Isard

Chandramohan Amyangot Thekkath

Paul Ronald Barham

Ryan Sepassi

Sudip Roy

Laurent El Shafey

Parker Edward Schuh

Brennan Saeta

Daniel William Hurt

Hyeontaek Lim

Quoc V Le

Dengyong Zhou

Ed Huai-Hsin Chi

David J Bieber

Augustus Quadrozzi Odena

Jacob Austin

Maarten Paul Bosma

David Martin Dohan

Dale Eric Schuurmans

Jason Weng Wei

Sharan Narang, Jr

Henryk Michalewski

Aitor Lewkowycz

Jacob Daniel Devlin

Nathanael Schärli

Maxwell Nye

Sharan Narang

Nathan Scales

Olivier Jean Andrè Bousquet

Le Hou

Charles Aloysius Sutton

Guy Gur-Ari Krakover

Jiageng Luan

Anders Johan Andreassen

Xuezhi Wang

An example technique for image analysis is provided. An example image analysis method includes obtaining an instructive sequence descriptive of an instructive query, an instructive response, and an instructive trace of intermediate states from the instructive query to the instructive response. The example image analysis method includes inputting, to a machine-learned model, the instructive sequence and an operative image processing query that comprises image data, wherein the machine-learned model is configured to process the operative query with attention over the instructive sequence. The example method can include generating, using the machine-learned model and responsive to the operative query, an operative image processing response that comprises an analysis of the image data.

Image analysis by prompting of machine-learned models using chain of thought

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing machine learning workloads, e.g., computations for training a neural network or computing an inference using a neural network, across multiple hardware accelerators. One of the systems comprises a plurality of accelerator islands, each hardware accelerator island comprising a respective plurality of hardware devices that include a plurality of hardware accelerators and a corresponding host for each of the plurality of hardware accelerators; and a respective scheduler for each of the accelerator islands that is configured to schedule workloads across the plurality of accelerators and corresponding hosts in the accelerator island, wherein the system is configured to: receive data representing a machine learning workload; and assign a respective portion of the machine learning workload to each of the plurality of accelerator islands for scheduling by the respective scheduler for the accelerator island.

Asynchronous distributed data flow for machine learning workloads

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on a network input to generate a network output. One of the systems comprises an attention neural network configured to perform the machine learning task, the attention neural network comprising a plurality of attention layers, each attention layer comprising an attention sub-layer that is arranged in parallel with a feed-forward sub-layer.

Attention neural networks with parallel attention and feed-forward layers

Growing community of inventors

Mountain View, CA, United States of America

Aakanksha Chowdhery