Meta Platforms, Inc.

Bichen Wu

Peter Vajda

Peizhao Zhang

Yanghan Wang

Xiaoliang Dai

Yuandong Tian

Computer implemented systems are described that implement a differentiable neural architecture search (DNAS) engine executing on one or more processors. The DNAS engine is configured with a stochastic super net defining a layer-wise search space having a plurality of candidate layers, each of the candidate layers specifying one or more operators for a neural network architecture. Further, the DNAS engine is configured to process training data to train weights for the operators in the stochastic super net based on a loss function representing a latency of the respective operator on a target platform, and to select a set of candidate neural network architectures from the trained stochastic super net. The DNAS engine may, for example, be configured to train the stochastic super net by traversing the layer-wise search space using gradient-based optimization of network architecture distribution.

Growing community of inventors

Menlo Park, CA, United States of America

Bichen Wu