Google Inc.

Waymo LLC

Mingxing Tan

Quoc V Le

Ruoming Pang

Vijay Vasudevan

Bo Chen

Dragomir Dimitrov Anguelov

Golnaz Ghiasi

Hanxiao Liu

Zihang Dai

Norman Paul Jouppi

Anelia Angelova

Sheng Li

Liqun Cheng

Wei Yu

Xiaodan Song

Reza Mahjourian

Benjamin Sapp

Pei Sun

Yuning Chai

Chenxi Liu

Tsung-Yi Lin

Hieu Hy Pham

Andrew Li

Anthony Jacob Piergiovanni

Jinkyu Kim

Pengchong Jin

Michael Sahngwon Ryoo

Weiyue Wang

Fei Xia

Zhaoqi Leng

Xianzhi Du

Yin Cui

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using memory-optimized contrastive learning to train image encoder and text encoder neural networks.

Memory-optimized contrastive learning

Methods, systems, and apparatus for processing point clouds using neural networks to perform a machine learning task. In one aspect, a system comprises one or more computers configured to obtain a set of point clouds captured by one or more sensors. Each point cloud includes a respective plurality of three-dimensional points. The one or more computers assign the three-dimensional points to respective voxels in a voxel grid, where the grid of voxels includes non-empty voxels to which one or more points are assigned and empty voxels to which no points are assigned. For each non-empty voxel, the one or more computers generate initial features based on the points that are assigned to the non-empty voxel. The one or more computers generate multi-scale features of the voxel grid, and the one or more computers generate an output for a point cloud processing task using the multi-scale features of the voxel grid.

Performing point cloud tasks using multi-scale features generated through self-attention

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting the future movement of agents in an environment. In particular, the future movement is predicted through occupancy flow fields that specify, for each future time point in a sequence of future time points and for each agent type in a set of one or more agent types: an occupancy prediction for the future time step that specifies, for each grid cell, an occupancy likelihood that any agent of the agent type will occupy the grid cell at the future time point, and a motion flow prediction that specifies, for each grid cell, a motion vector that represents predicted motion of agents of the agent type within the grid cell at the future time point.

Predicting the future movement of agents in an environment using occupancy flow fields

The present disclosure is directed to an automated neural architecture search approach for designing new neural network architectures such as, for example, resource-constrained mobile CNN models. In particular, the present disclosure provides systems and methods to perform neural architecture search using a novel factorized hierarchical search space that permits layer diversity throughout the network, thereby striking the right balance between flexibility and search space size. The resulting neural architectures are able to be run relatively faster and using relatively fewer computing resources (e.g., less processing power, less memory usage, less power consumption, etc.), all while remaining competitive with or even exceeding the performance (e.g., accuracy) of current state-of-the-art mobile-optimized models.

Neural architecture search with factorized hierarchical search space

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining an architecture for a task neural network that is configured to perform a particular machine learning task on a target set of hardware resources. When deployed on a target set of hardware, such as a collection of datacenter accelerators, the task neural network may be capable of performing the particular machine learning task with enhanced accuracy and speed.

Hardware-optimized neural architecture search

A computer-implemented method of generating scale-permuted models can generate models having improved accuracy and reduced evaluation computational requirements. The method can include defining, by a computing system including one or more computing devices, a search space including a plurality of candidate permutations of a plurality of candidate feature blocks, each of the plurality of candidate feature blocks having a respective scale. The method can include performing, by the computing system, a plurality of search iterations by a search algorithm to select a scale-permuted model from the search space, the scale-permuted model based at least in part on a candidate permutation of the plurality of candidate permutations.

Scale-permuted machine learning architecture

Systems and methods of the present disclosure can include a computer-implemented method for efficient machine-learned model training. The method can include obtaining a plurality of training samples for a machine-learned model. The method can include, for one or more first training iterations, training, based at least in part on a first regularization magnitude configured to control a relative effect of one or more regularization techniques, the machine-learned model using one or more respective first training samples of the plurality of training samples. The method can include, for one or more second training iterations, training, based at least in part on a second regularization magnitude greater than the first regularization magnitude, the machine-learned model using one or more respective second training samples of the plurality of training samples.

Systems and methods for progressive learning for machine-learned models to optimize training speed

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining one or more neural network architectures of a neural network for performing a video processing neural network task. In one aspect, a method comprises: at each of a plurality of iterations: selecting a parent neural network architecture from a set of neural network architectures; training a neural network having the parent neural network architecture to perform the video processing neural network task, comprising determining trained values of connection weight parameters of the parent neural network architecture; generating a new neural network architecture based at least in part on the trained values of the connection weight parameters of the parent neural network architecture; and adding the new neural network architecture to the set of neural network architectures.

Connection weight learning for guided architecture evolution

A method for determining a final architecture for a neural network to perform a particular machine learning task is described. The method includes receiving a baseline architecture for the neural network, wherein the baseline architecture has a network width dimension, a network depth dimension, and a resolution dimension; receiving data defining a compound coefficient that controls extra computational resources used for scaling the baseline architecture; performing a search to determine a baseline width, depth and resolution coefficient that specify how to assign the extra computational resources to the network width, depth and resolution dimensions of the baseline architecture, respectively; determining a width, depth and resolution coefficient based on the baseline width, depth, and resolution coefficient and the compound coefficient; and generating the final architecture that scales the network width, network depth, and resolution dimensions of the baseline architecture based on the corresponding width, depth, and resolution coefficients.

Compound model scaling for neural networks

A computer-implemented method for performing computer vision with reduced computational cost and improved accuracy can include obtaining, by a computing system including one or more computing devices, input data comprising an input tensor having one or more dimensions, providing, by the computing system, the input data to a machine-learned convolutional attention network, the machine-learned convolutional attention network including two or more network stages, and, in response to providing the input data to the machine-learned convolutional attention network, receiving, by the computing system, a machine-learning prediction from the machine-learned convolutional attention network. The convolutional attention network can include at least one attention block, wherein the attention block includes a relative attention mechanism, the relative attention mechanism including the sum of a static convolution kernel with an adaptive attention matrix. This provides for improved generalization, capacity, and efficiency of the convolutional attention network relative to some existing models.

Systems and methods for machine-learned models having convolution and attention

Growing community of inventors

Newark, CA, United States of America

Mingxing Tan