Veo Robotics, Inc.

Irobot Corporation

Symbotic LLC

Harvest Automation, Inc.

Nextshift Robotics, Inc.

Clara Vu

Scott A Denenberg

Alberto Moel

Patrick Sobalvarro

Daniel N Ozick

David A Cohen

Tony L Campbell

Joseph Lee Jones

Paul Eric Sandin

James P Lynch

Philip R Mass

Matthew Cross

Jeffrey W Mammen

Zivthan A Dubrovsky

John Bergman

Charles M Grinnell

Joshua B Preneta

David M Lewis, Jr

Ilya A Kriveshko

Lev Persits

Paul Jakob Schroeder

Marek Wartenberg

Duane Leigh Gilbert, Jr

Patrick Barragan

Valentina Chamorro

Gene Malkin

Brad C Mello

Stephen C Toebes

Andrew Ziegler

Andrew Jones

Matthew J Aprea

Colin Angle

Richard A Kelsey

Mary Ellen Sparrow

John Kawola

Abraham K Feldman

Mark Joseph Chiappetta

Duane Gilbert

Thomas C Ferree

Larry B Gray

Todd Comins

Clive Bolton

Ken Sinclair

Joe Jones

Patrick Joseph Foy

Robert Craig Randall

Amanda Gruber

Matthew Galligan

Joshua Downer

Pratik Devendra Dalvi

Paulo Ferreira

Mehmet Ali Guney

Dmitriy Dedkov

Justin Bronder

Elliot Simon

Tim Bickmore

Ilya Kriveshko

Nicole Aucoin

Eric Cobane

Antonio Fiol-mahon

Zeynep Şeker

Paul Jacob Schroeder

Patrick J Foy

Duane L Gilbert

Jeffry W Mammen

David M Lewis Jr

Ken Singlair

John Goetsch

Pace Williston

Lorin Wilde

In various embodiments, systems and methods for safely operating machinery such as industrial robots signal intrusions into and/or above computationally defined safeguarded volumes in a workspace during sensor monitoring of the workspace. Various related actions may be taken in response thereto, including restricting operation of the machinery and/or issuing safety or occlusion signals.

Sensor for safe detection of intrusions in a 3D volume

Spatial regions potentially occupied by a robot (or other machinery) or portion thereof and a human operator during performance of all or a defined portion of a task or an application are computationally estimated. These 'potential occupancy envelopes' (POEs) may be based on the states (e.g., the current and expected positions, velocities, accelerations, geometry and/or kinematics) of the robot and the human operator. Once the POEs of human operators in the workspace are established, they can be used to guide or revise motion planning for task execution.

Motion planning and task execution using potential occupancy envelopes

A method of safely operating machinery in a workspace includes recording images of a portion of a workspace. The method also includes generating a three-dimensional (3D) representation of the portion of the workspace based on the recorded images, where the 3D representation includes one or more volumes that correspond to the portion of the workspace. Additionally, the method includes identifying one or more of the volumes as being either occupied or unoccupied. Further, the method includes mapping one or more safe zones based on the one or more identified volumes, where the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.

Safe motion planning for machinery operation

One or more computational strategies is used to reduce the complexity of handling detected point clusters appearing at a portal of a monitored workcell. If the other side of the portal is a second, adjacent workcell, the monitoring system for a first workcell predicts when a human in that workcell may pass through the portal to a second workcell and alerts the monitoring system for the second workcell. The prediction may be based on absolute proximity of a human to the portal or movement toward it. Other strategies are employed if the workcells overlap, or if the portal leads to an unmonitored space.

Intelligent monitoring of entry points in multi-cell workspaces

In various embodiments, systems and methods for acquiring depth images utilize an architecture suited to safety-rated applications, and may include more than sensor (such as time-of-flight sensors) operating along different optical paths and a comparison module for ensuring proper sensor operation. Error metrics may be associated with pixel-level depth values for purposes of allowing safe control based on imperfectly known depths.

Depth-sensing computer vision system

Systems and methods for determining safe and unsafe zones in a workspace—where safe actions are calculated in real time based on all relevant objects (e.g., some observed by sensors and others computationally generated based on analysis of the sensed workspace) and on the current state of the machinery (e.g., a robot) in the workspace—may utilize a variety of workspace-monitoring approaches as well as dynamic modeling of the robot geometry. The future trajectory of the robot(s) and/or the human(s) may be forecast using, e.g., a model of human movement and other forms of control. Modeling and forecasting of the robot may, in some embodiments, make use of data provided by the robot controller that may or may not include safety guarantees.

Dynamic, interactive signaling of safety-related conditions in a monitored environment

Systems and methods for identifying a robot end effector in a processing environment may utilize one or more sensors for digitally recording visual information and providing that information to an industrial workflow. The sensor(s) may be positioned to record at least one image of the robot including the end effector. A processor may determine the identity of the end effector from the recorded image(s) and a library or database stored digital models.

Robot end-effector sensing and identification

An integrated system and method for monitoring a workspace, classifying regions therein, dynamically identifying safe states, and identifying and tracking workpieces utilizes semantic analysis of a robot in the workspace and identification of the workpieces with which it interacts, as well as a semantic understanding of entity properties both governing and arising from interaction with other entities for control purposes. These properties may be stored in a profile corresponding to the entity; depending on the implementation, the profile and methods associated with the entity may be stored in a data structure corresponding to the entity 'class.' Volumetric regions (e.g., voxels) corresponding to each entity may be tracked and used to implement object methods and/or control methods. For example, machinery may be controlled in accordance with the attributes specified by the objects corresponding to identified entities in a monitored workspace.

Workplace monitoring and semantic entity identification for safe machine operation

Systems and methods monitor a workspace for safety purposes using sensors distributed about the workspace. The sensors are registered with respect to each other, and this registration is monitored over time. Occluded space as well as occupied space is identified, and this mapping is frequently updated. Based on the mapping, a constrained motion plan of machinery can be generated to ensure safety.

Crosstalk mitigation among cameras in neighboring monitored workcells is achieved by computationally defining a noninterference scheme that respects the independent monitoring and operation of each workcell. The scheme may involve communication between adjacent cells to adjudicate non-interfering camera operation or system-wide mapping of interference risks and mitigation thereof. Mitigation strategies can involve time-division and/or frequency-division multiplexing.

Crosstalk mitigation for multi-cell workspace monitoring

Control systems for industrial machinery (e.g., robots) or other devices such as medical devices utilize a safety processor (SP) designed for integration into safety applications and computational components that are not necessarily safety-rated. The SP monitors performance of the non-safety computational components, including latency checks and verification of identical outputs. One or more sensors send data to the non-safety computational components for sophisticated processing and analysis that the SP cannot not perform, but the results of this processing are sent to the SP, which then generates safety-rated signals to the machinery or device being controlled by the SP. As a result, the system may qualify for a safety rating despite the ability to perform complex operations beyond the scope of safety-rated components.

System architecture for safety applications

Systems and methods for identifying a workpiece in a processing environment may utilize one or more sensors for digitally recording visual information and providing that information to an industrial workflow. The sensor(s) may be positioned to record at least one image of the workpiece at a location where a specified position and orientation thereof is required. A processor may determine, from the recorded image(s) and a stored digital model, whether the workpiece conforms to the specified position and orientation.

Workpiece sensing for process management and orchestration

Embodiments of the present invention provide automated robotic system identification and stopping time and distance estimation, significantly improving on existing ad-hoc methods of robotic system identification. Systems and methods in accordance herewith can be used by end users, system integrators, and the robot manufacturers to estimate the dynamic parameters of a robot on an application-by-application basis.

System identification of industrial robot dynamics for safety-critical applications

Growing community of inventors

Cambridge, MA, United States of America

Clara Vu