Nvidia Corporation

University of Washington

Wei Yang

Dieter Fox

Yu-Wei Chao

Shaoyi Jiang

Hong Xue

Louisa Ruth Carr

Christopher Jason Paxton

Maya Cakmak

Ankur Handa

Balakumar Sundaralingam

Yu Xiang

Karl Van Wyk

Chris Paxton

Qian Wan

Approaches presented herein provide for predictive control of a robot or automated assembly in performing a specific task. A task to be performed may depend on the location and orientation of the robot performing that task. A predictive control system can determine a state of a physical environment at each of a series of time steps, and can select an appropriate location and orientation at each of those time steps. At individual time steps, an optimization process can determine a sequence of future motions or accelerations to be taken that comply with one or more constraints on that motion. For example, at individual time steps, a respective action in the sequence may be performed, then another motion sequence predicted for a next time step, which can help drive robot motion based upon predicted future motion and allow for quick reactions.

Reactive interactions for robotic applications and other automated systems

A system may simulate human motion for human-robot interactions, such as may involve a handover of an object. Motion capture can be performed for a hand grasping and moving an object to a location and orientation appropriate for a handover, without a need for a robot to be present or an actual handover to occur. This motion data can be used to separately model the hand and the object for use in a handover simulation, where a component such as a physics engine may be used to ensure realistic modeling of the motion or behavior. During a simulation, a robot control model or algorithm can predict an optimal location and orientation to grasp an object, and an optimal path to move to that location and orientation, using a control model or algorithm trained, based at least in part, using the motion models for the hand and object.

Simulating physical interactions for automated systems

A human pilot controls a robotic arm and gripper by simulating a set of desired motions with the human hand. In at least one embodiment, one or more images of the pilot's hand are captured and analyzed to determine a set of hand poses. In at least one embodiment, the set of hand poses is translated to a corresponding set of robotic-gripper poses. In at least one embodiment, a set of motions is determined that perform the set of robotic-gripper poses, and the robot is directed to perform the set of motions.

Vision-based teleoperation of dexterous robotic system

A robotic control system directs a robot to take an object from a human grasp by obtaining an image of a human hand holding an object, estimating the pose of the human hand and the object, and determining a grasp pose for the robot that will not interfere with the human hand. In at least one example, a depth camera is used to obtain a point cloud of the human hand holding the object. The point cloud is provided to a deep network that is trained to generate a grasp pose for a robotic gripper that can take the object from the human's hand without pinching or touching the human's fingers.

Machine learning control of object handovers

Zwitterionic crosslinking agents, crosslinked zwitterionic hydrogels prepared from copolymerization of zwitterionic monomers with the zwitterionic crosslinking agent, methods for making crosslinked zwitterionic hydrogels, and devices that include and methods that use the crosslinked zwitterionic hydrogels.

Crosslinked mixed charge hydrogels

Crosslinked zwitterionic hydrogels

Growing community of inventors

Seattle, WA, United States of America

Wei Yang