Dexterity, Inc.

Google Inc.

Kevin Jose Chavez

Samir Menon

Zhouwen Sun

Rohit Arka Pidaparthi

Talbot Morris-Downing

Harry Zhe Su

Yuan Gao

Ben Varkey Benjamin Pottayil

William Arthur Clary

Robert Holmberg

Ayush Sharma

Raphael Georg Wirth

Shitij Kumar

Cuthbert Sun

Jonathan Kuck

Neeraja Abhyankar

Andrew Nguyen

David Kliger Elson

Sudeep Gandhe

Robert Hegdahl

Benjamin Ross

David Eisenberg

Raj Agarwal

Samuel Tay

Emilie Peck

Christa Wimberley

David Leo Tondreau, Iii

Cyril Nader

Alberto Leyva Arvayo

Toby Leonard Baker

Roger Lam Hau

Andrew Lovett

Gerald Brantner

Farshid Farhat

Rohun Kulkarni

Ivan Alberto Trujillo Priego

Jeesu Baek

Salvador Perez

Robert Moreno

Varun Uday Nayak

Michael Root

Arth Beladiya

Rohit Pidaparthi

Ben Varkey Pottayil

A robotic system is disclosed. The system includes a communication interface configured to receive, from one or more sensors deployed in a workspace, sensor data indicative of a current state of the workspace, the workspace comprising a pallet or other receptacle and a plurality of items stacked on or in the receptacle. The system includes one or more processors that use a geometric model based at least in part on past item placements in combination with the sensor data to estimate a state of the pallet or other receptacle and one or more items stacked on or in the pallet or other receptacle, and use the estimated state to generate or update a plan to control a robotic arm to place a next item on or in, or remove a next item from, the pallet or other receptacle in a manner that avoids having the next item collide with any other item stacked on or in the pallet or other receptacle.

Collision avoidance based on vision data, geometric data and physics engine evaluation

A kitting system is disclosed. In various embodiments, the kitting system includes a conveyance structure configured to impart to an item a first net resultant force substantially in a first direction, wherein the first direction is associated with a direction of flow from a source end of the conveyance structure to a destination end opposite the source end and associated with a pick-up zone from which the item is to be retrieved; a sensor configured to provide a sensor output associated with the pick-up zone; and a processor configured to provide a control input to one or both of the conveyance structure and a disrupter device associated with the conveyance structure based at least in part on the sensor output.

Robotic kitting system

A robotic system is disclosed. The system includes a memory that stores a machine learning-based model to provide a scoring function value for a candidate item placement on a pallet on which are plurality of items are to be stacked given a current state value of the pallet and a set of zero or more items placed previously. The system includes one or more processors that use the model to determine a corresponding score for each of a plurality of candidate placements for a next item to be placed and the current state value associated with the current state of the pallet and a set of zero or more items placed previously, select a selected placement based at least in part on the respective scores, control a robotic arm to place the next item according to the selected placement.

Workflow for using learning based approach for placing boxes on pallets

A robotic system is disclosed. The system includes a communication interface that receives, from one or more sensors deployed in a workspace, sensor data indicative of a current state of the workspace. The system includes one or more processors that use the sensor data to estimate a state of one or both of the pallet or other receptacle and the set of zero or more items stacked on or in the receptacle, and use the estimated state to generate or update a plan to control a robotic arm to place a next set of items on or in, or remove the next set of items from, the pallet or other receptacle, the plan comprising an ordered sequence of item placements or removals. The plan is generated or updated based at least in part by performing a bounded tree search in which a subset of possible ordered sequences is explored.

Workflow for using tree search-based approach for placing boxes on pallet with limited knowledge of future sequence

A set of one or more potentially graspable features for one or more objects present in a workspace area are determined based on visual data received from a plurality of cameras. For each of at least a subset of the one or more potentially graspable features one or more corresponding grasp strategies are determined to grasp the feature with a robotic arm and end effector. A score associated with a probability of a successful grasp of a corresponding feature is determined with respect to each of a least a subset of said grasp strategies. A first feature of the one or more potentially graspable features is selected to be grasped using a selected grasp strategy based at least in part on a corresponding score associated with the selected grasp strategy with respect to the first feature. The robotic arm and the end effector are controlled to attempt to grasp the first feature using the selected grasp strategy.

Autonomous unknown object pick and place

Techniques are disclosed to use robotic system simulation to control a robotic system. In various embodiments, a communication indicating an action to be performed by a robotic element is received from a robotic control system. Performance of the action by the robotic element is simulated. A state tracking data is updated to reflect a virtual change to one or more state variables as a result of simulated performance of the action. Successful completion of the action by the robotic element is reported to the robotic control system.

Robotic system simulation engine

Systems and methods provides for dialog management in real-time rather than turn taking. An example method included generating first candidate responses to triggering event. The triggering event may be receipt of a live stream chunk for the dialog or receipt of a backend response to a previous backend request for a dialog shema. The method also includes updating a list of candidate responses that are accepted or pending with at least on of the first candidate responses, and determining, for the triggering event, whether the list of candidate responses includes a candidate response that has a confidence score that meets a triggering threshold. The method also includes waiting for a next triggering event without providing a candidate response when the list does not include a candidate response that has a confidence score that meets the triggering threshold.

Streaming real-time dialog management

A kitting machine is disclosed. In various embodiments, the machine includes an actuator configured to move a surface or receptacle to a position associated with item retrieval; and a controller configured to control operation of the actuator to position an item in the position associated with item retrieval, in a manner that is synchronized at least in part with operation of a robotic retrieval device configured to retrieve the item from the position associated with item retrieval.

Kitting machine

Data indicating a corresponding safety state information indicating an extent to which that operating zone currently is or is not available to the robotic system to perform tasks using one or more robotic instrumentalities associated with a robotic system is used, for each of a plurality of operating zones, to dynamically schedule and perform tasks associated with a higher level objective. The corresponding safety state information associated with the first operating zone indicating one of a plurality of safety states associated with a presence of a human worker in the first operating zone is received from one or more sensors associated with a first operating zone of the plurality of operating zones. Autonomous behavior of the one or more robotic instrumentalities is altered based on the one of the plurality of safety states associated with the presence of the human worker in the first operating zone indicated by the corresponding safety state information.

Robotic system with zone-based control

A robotic system is disclosed. The system includes a memory that stores for each of a plurality of items a set of attribute values. The system includes a processor(s) that uses the attribute values to simulate the placement of items, including by determining, iteratively, for each next item a placement location at which to place the item on a simulated stack of items on the pallet, using the attribute values and a geometric model of where items have been simulated to have been placed to estimate a state of the stack after each of a subset of simulated placements, and using the estimated state to inform a next placement decision. The steps of determining for each next item a placement location and estimating the state of the stack until all of at least a subset of the plurality of items have been simulated as having been placed on the stack.

Simulated box placement for algorithm evaluation and refinement

A robotic kitting machine is disclosed. In various embodiments, a robotic arm is used to move an item to a location in proximity to a slot into which the item is to be inserted. Force information generated by a force sensor is received via a communication interface. The force sensor information is used to align a structure comprising the item with a corresponding cavity comprising the slot, and the item is inserted into the slot.

Robotic kitting machine

A robotic singulation system is disclosed. In various embodiments, sensor data including image data associated with a workspace is received. The sensor data is used to generate a three dimensional view of at least a portion of the workspace, the three dimensional view including boundaries of a plurality of items present in the workspace. A grasp strategy is determined for each of at least a subset of items, and for each grasp strategy a corresponding probability of grasp success is computed. The grasp strategies and corresponding probabilities of grasp success are used to determine and implement a plan to autonomously operate a robotic structure to pick one or more items from the workplace and place each item singly in a corresponding location in a singulation conveyance structure.

Coordinating multiple robots to meet workflow and avoid conflict

A robotic system is disclosed. The system includes a communication interface that receives, from a sensor(s) deployed in a workspace, sensor data indicative of a current state of the workspace, the workspace comprising a pallet or other receptacle and a plurality of items stacked on or in the receptacle. The system includes one or more processors that control a robotic arm to place a first set of items on or in, or remove the first set of items from, the pallet or other receptacle, update a geometric model based on the first set of items placed on or in a receptacle, use the geometric model in combination with the sensor data to estimate a stack of one or more items on or in the receptacle, and use the estimated state to generate or update a plan to control the robotic arm to place a second set of items.

State estimation using geometric data and vision system for palletizing

A multicamera image processing system is disclosed. In various embodiments, image data is received from each of a plurality of sensors associated with a workspace, the image data comprising for each sensor in the plurality of sensors one or both of visual image information and depth information. Image data from the plurality of sensors is merged to generate a merged point cloud data. Segmentation is performed based on visual image data from at least a subset of the sensors in the plurality of sensors to generate a segmentation result. One or both of the merged point cloud data and the segmentation result is/are used to generate a merged three dimensional and segmented view of the workspace.

Multicamera image processing

A robotic line kitting system is disclosed. In various embodiments, a sensor reading associated with a force sensor associated with a robotic instrumentality comprising the robotic line kitting system is received. It is determined based at least in part on the sensor reading that a condition requiring human intervention has been detected. A task to be performed by a human worker to correct the condition is scheduled.

Autonomous and safe integration of human task in robotic operation

Growing community of inventors

Redwood City, CA, United States of America

Kevin Jose Chavez