Veo Robotics, Inc.

Symbotic LLC

Alberto Moel

Clara Vu

Scott A Denenberg

Patrick Sobalvarro

Lev Persits

Marek Wartenberg

Patrick Barragan

Ilya A Kriveshko

Valentina Chamorro

Gene Malkin

Paul Jakob Schroeder

Richard A Kelsey

Brad C Mello

Thomas C Ferree

Patrick Joseph Foy

Robert Craig Randall

Matthew Galligan

Joshua Downer

Paulo Ferreira

Elliot Simon

Mehmet Ali Guney

Justin Bronder

Pratik Devendra Dalvi

Dmitriy Dedkov

Nicole Aucoin

Patrick J Foy

Antonio Fiol-mahon

Eric Cobane

Ilya Kriveshko

Paul Jacob Schroeder

In various embodiments, systems and methods for safely operating machinery such as industrial robots signal intrusions into and/or above computationally defined safeguarded volumes in a workspace during sensor monitoring of the workspace. Various related actions may be taken in response thereto, including restricting operation of the machinery and/or issuing safety or occlusion signals.

Sensor for safe detection of intrusions in a 3D volume

Spatial regions potentially occupied by a robot (or other machinery) or portion thereof and a human operator during performance of all or a defined portion of a task or an application are computationally estimated. These 'potential occupancy envelopes' (POEs) may be based on the states (e.g., the current and expected positions, velocities, accelerations, geometry and/or kinematics) of the robot and the human operator. Once the POEs of human operators in the workspace are established, they can be used to guide or revise motion planning for task execution.

Motion planning and task execution using potential occupancy envelopes

A method of safely operating machinery in a workspace includes recording images of a portion of a workspace. The method also includes generating a three-dimensional (3D) representation of the portion of the workspace based on the recorded images, where the 3D representation includes one or more volumes that correspond to the portion of the workspace. Additionally, the method includes identifying one or more of the volumes as being either occupied or unoccupied. Further, the method includes mapping one or more safe zones based on the one or more identified volumes, where the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.

Safe motion planning for machinery operation

One or more computational strategies is used to reduce the complexity of handling detected point clusters appearing at a portal of a monitored workcell. If the other side of the portal is a second, adjacent workcell, the monitoring system for a first workcell predicts when a human in that workcell may pass through the portal to a second workcell and alerts the monitoring system for the second workcell. The prediction may be based on absolute proximity of a human to the portal or movement toward it. Other strategies are employed if the workcells overlap, or if the portal leads to an unmonitored space.

Intelligent monitoring of entry points in multi-cell workspaces

In various embodiments, systems and methods for acquiring depth images utilize an architecture suited to safety-rated applications, and may include more than sensor (such as time-of-flight sensors) operating along different optical paths and a comparison module for ensuring proper sensor operation. Error metrics may be associated with pixel-level depth values for purposes of allowing safe control based on imperfectly known depths.

Depth-sensing computer vision system

An integrated system and method for monitoring a workspace, classifying regions therein, dynamically identifying safe states, and identifying and tracking workpieces utilizes semantic analysis of a robot in the workspace and identification of the workpieces with which it interacts, as well as a semantic understanding of entity properties both governing and arising from interaction with other entities for control purposes. These properties may be stored in a profile corresponding to the entity; depending on the implementation, the profile and methods associated with the entity may be stored in a data structure corresponding to the entity 'class.' Volumetric regions (e.g., voxels) corresponding to each entity may be tracked and used to implement object methods and/or control methods. For example, machinery may be controlled in accordance with the attributes specified by the objects corresponding to identified entities in a monitored workspace.

Workplace monitoring and semantic entity identification for safe machine operation

Systems and methods monitor a workspace for safety purposes using sensors distributed about the workspace. The sensors are registered with respect to each other, and this registration is monitored over time. Occluded space as well as occupied space is identified, and this mapping is frequently updated. Based on the mapping, a constrained motion plan of machinery can be generated to ensure safety.

Crosstalk mitigation among cameras in neighboring monitored workcells is achieved by computationally defining a noninterference scheme that respects the independent monitoring and operation of each workcell. The scheme may involve communication between adjacent cells to adjudicate non-interfering camera operation or system-wide mapping of interference risks and mitigation thereof. Mitigation strategies can involve time-division and/or frequency-division multiplexing.

Crosstalk mitigation for multi-cell workspace monitoring

Control systems for industrial machinery (e.g., robots) or other devices such as medical devices utilize a safety processor (SP) designed for integration into safety applications and computational components that are not necessarily safety-rated. The SP monitors performance of the non-safety computational components, including latency checks and verification of identical outputs. One or more sensors send data to the non-safety computational components for sophisticated processing and analysis that the SP cannot not perform, but the results of this processing are sent to the SP, which then generates safety-rated signals to the machinery or device being controlled by the SP. As a result, the system may qualify for a safety rating despite the ability to perform complex operations beyond the scope of safety-rated components.

System architecture for safety applications

Embodiments of the present invention provide automated robotic system identification and stopping time and distance estimation, significantly improving on existing ad-hoc methods of robotic system identification. Systems and methods in accordance herewith can be used by end users, system integrators, and the robot manufacturers to estimate the dynamic parameters of a robot on an application-by-application basis.

System identification of industrial robot dynamics for safety-critical applications

Systems and methods for optimizing factory scheduling, layout or both which represent active factory elements (human and machine) as computational objects and simulate factory operation to optimize a solution. This enables the efficient assembly of customized products, accommodates variable demand, and mitigates unplanned events (floor blockages, machines/IMRs/workcell/workers downtime, variable quantity, location, and destination of supply parts).

Optimized factory schedule and layout generation

Safety systems in distributed factory workcells intercommunicate or communicate with a central controller so that when a person, robot or vehicle passes from one workcell or space into another on the same factory floor, the new workcell or space need not repeat the tasks of analysis and classification and can instead immediately integrate the new entrant into the existing workcell or space-monitoring schema. The workcell or space can also communicate attributes such as occlusions, unsafe areas, movement speed, and object trajectories, enabling rapid reaction by the monitoring system of the new workcell or space.

Growing community of inventors

Cambridge, MA, United States of America

Alberto Moel