Nvidia Corporation

Microsoft Technology Licensing, LLC

Stan Birchfield

Nikolai Smolyanskiy

Jan Kautz

Jonathan Tremblay

Artem Molchanov

Sumit Gulwani

Dieter Fox

Alexey Kamenev

Stephen Walter Tyree

Thang Hong To

Charles T Loop

Varun Jampani

Ke Chen

Alexander A Popov

Hao Dang

Ashley Nathan Feniello

Ankur Handa

Balakumar Sundaralingam

Nathan Donald Ratliff

Byron Boots

Yiran Zhong

Jialiang Wang

Alexander Lambert

Apparatuses, systems, and techniques are presented to determine distance for one or more objects. In at least one embodiment, a disparity network is trained to determine distance data from input stereoscopic images using a loss function that includes at least one of a gradient loss term and an occlusion loss term.

Distance determinations using one or more neural networks

Various embodiments enable a robot, or other autonomous or semi-autonomous device or system, to receive data involving the performance of a task in the physical world. The data can be provided as input to a perception network to infer a set of percepts about the task, which can correspond to relationships between objects observed during the performance. The percepts can be provided as input to a plan generation network, which can infer a set of actions as part of a plan. Each action can correspond to one of the observed relationships. The plan can be reviewed and any corrections made, either manually or through another demonstration of the task. Once the plan is verified as correct, the plan (and any related data) can be provided as input to an execution network that can infer instructions to cause the robot, and/or another robot, to perform the task.

Learning robotic tasks using one or more neural networks

In various examples, methods and systems are provided for estimating depth values for images (e.g., from a monocular sequence). Disclosed approaches may define a search space of potential pixel matches between two images using one or more depth hypothesis planes based at least on a camera pose associated with one or more cameras used to generate the images. A machine learning model(s) may use this search space to predict likelihoods of correspondence between one or more pixels in the images. The predicted likelihoods may be used to compute depth values for one or more of the images. The predicted depth values may be transmitted and used by a machine to perform one or more operations.

Predicting image warping for structure from motion using neural networks

Various examples of the present disclosure include a stereoscopic deep neural network (DNN) that produces accurate and reliable results in real-time. Both LIDAR data (supervised training) and photometric error (unsupervised training) may be used to train the DNN in a semi-supervised manner. The stereoscopic DNN may use an exponential linear unit (ELU) activation function to increase processing speeds, as well as a machine learned argmax function that may include a plurality of convolutional layers having trainable parameters to account for context. The stereoscopic DNN may further include layers having an encoder/decoder architecture, where the encoder portion of the layers may include a combination of three-dimensional convolutional layers followed by two-dimensional convolutional layers.

Stereo depth estimation using deep neural networks

Robotic task program synthesis embodiments are presented that generally synthesize a robotic task program based on received examples of repositioning tasks. In one implementation, the exemplary repositioning tasks are human demonstrations of object manipulation in an actual or displayed robot workspace. A domain specific language (DSL) designed for object repositioning tasks is employed for the robotic control program. In general, candidate robotic task programs are generated from the example tasks. Each candidate program includes instructions for causing the robot to reposition objects, and represents a different permutation of instructions consistent with the received example tasks. The candidate programs are ranked, and whenever the top ranking program accomplishes the repositioning specified in each example task, it is designated as the synthesized robotic task program.

Program synthesis for robotic tasks

Growing community of inventors

Sammamish, WA, United States of America

Stan Birchfield