Nvidia Corporation

Yuke Zhu

Jonathan Tremblay

Fabio Tozeto Ramos

Anima Anandkumar

Guanya Shi

Stanley Thomas Birchfield

Animashree Anandkumar

Christopher Bongsoo Choy

Bokui Shen

Youngwoon Lee

Yifeng Zhu

A manipulation task may include operations performed by one or more manipulation entities on one or more objects. This manipulation task may be broken down into a plurality of sequential sub-tasks (policies). These policies may be fine-tuned so that a terminal state distribution of a given policy matches an initial state distribution of another policy that immediately follows the given policy within the plurality of policies. The fine-tuned plurality of policies may then be chained together and implemented within a manipulation environment.

Fine-tuning policies to facilitate chaining

Apparatuses, systems, and techniques are described that solve task and motion planning problems. In at least one embodiment, a task and motion planning problem is modeled using a geometric scene graph that records positions and orientations of objects within a playfield, and a symbolic scene graph that represents states of objects within context of a task to be solved. In at least one embodiment, task planning is performed using symbolic scene graph, and motion planning is performed using a geometric scene graph.

Machine learning model for task and motion planning

One or more machine learning models (MLMs) may learn implicit 3D representations of geometry of an object and of dynamics of the object from performing an action on the object. Implicit neural representations may be used to reconstruct high-fidelity full geometry of the object and predict a flow-based dynamics field from one or more images, which may provide a partial view of the object. Correspondences between locations of an object may be learned based at least on distances between the locations on a surface corresponding to the object, such as geodesic distances. The distances may be incorporated into a contrastive learning loss function to train one or more MLMs to learn correspondences between locations of the object, such as a correspondence embedding field. The correspondences may be used to evaluate state changes when evaluating one or more actions that may be performed on the object.

Action-conditional implicit dynamics of deformable objects

Apparatuses, systems, and techniques generate poses of an object based on image data of the object obtained from a first viewpoint of the object and a second viewpoint of the object. The poses can be evaluated to determine a portion of the image data usable by an estimator to generate a pose of the object.

Data selection based on uncertainty quantification

Apparatuses, systems, and techniques generate poses of an object based on data of the object observed from a first viewpoint and a second viewpoint. The poses can be evaluated to determine a portion of the data usable by an estimator to generate a pose of the object.

Growing community of inventors

Austin, TX, United States of America

Yuke Zhu