Nvidia Corporation

Weiyu Liu

Dieter Fox

Christopher Jason Paxton

Tucker Ryer Hermans

A robotic system is provided for performing rearrangement tasks guided by a natural language instruction. The system can include a number of neural networks used to determine a selected rearrangement of the objects in accordance with the natural language instruction. A target object predictor network processes a point cloud of the scene and the natural language instruction to identify a set of query objects that are to-be-rearranged. A language conditioned prior network processes the point cloud, natural language instruction, and the set of query objects to sample a distribution of rearrangements to generate a number of sets of pose offsets for the set of query objects. A discriminator network then processes the samples to generate scores for the samples. The samples may be refined until a score for at least one of the sample generated by the discriminator network is above a threshold value.

Growing community of inventors

Atlanta, GA, United States of America

Weiyu Liu