Veo Robotics, Inc.

Jentek Sensors, Incorporated

Symbotic LLC

Scott A Denenberg

Clara Vu

Alberto Moel

Patrick Sobalvarro

Neil J Goldfine

Yanko K Sheiretov

Todd M Dunford

Lev Persits

Ilya A Kriveshko

Paul Jakob Schroeder

Marek Wartenberg

Gene Malkin

Valentina Chamorro

Stuart D Chaplan

Patrick Barragan

Kevin P Dixon

Brad C Mello

Saber Bahranifard

Abraham K Feldman

Andrew P Washabaugh

Karen E Walrath

Mark D Windoloski

Don Straney

Robert Craig Randall

Darrell E Schlicker

David C Grundy

Richard A Kelsey

Patrick Joseph Foy

Dmitriy Dedkov

Matthew Galligan

Elliot Simon

Justin Bronder

Pratik Devendra Dalvi

Christopher T Martin

Mark Windoloski

Ilya Kriveshko

Leon B Kristal

Eric Cobane

Brian L Manning

Antonio Fiol-mahon

Todd A Dunford

Zeynep Şeker

Nicole Aucoin

Paul Jacob Schroeder

Brian Manning

In various embodiments, systems and methods for safely operating machinery such as industrial robots signal intrusions into and/or above computationally defined safeguarded volumes in a workspace during sensor monitoring of the workspace. Various related actions may be taken in response thereto, including restricting operation of the machinery and/or issuing safety or occlusion signals.

Sensor for safe detection of intrusions in a 3D volume

Spatial regions potentially occupied by a robot (or other machinery) or portion thereof and a human operator during performance of all or a defined portion of a task or an application are computationally estimated. These 'potential occupancy envelopes' (POEs) may be based on the states (e.g., the current and expected positions, velocities, accelerations, geometry and/or kinematics) of the robot and the human operator. Once the POEs of human operators in the workspace are established, they can be used to guide or revise motion planning for task execution.

Motion planning and task execution using potential occupancy envelopes

Systems and methods utilize one or more 3D cameras (e.g., ToF cameras) in industrial safety applications. The 3D camera generates a depth map that may be used by external hardware and software to classify objects in a workcell and generate control signals for machinery. To facilitate sensor-specific calibration and coordination among sensors in a workcell, the sensors may store calibration data in a boot file that is loaded upon start-up. During initialization, the calibration data is loaded and, as the sensor operates, corrections are made to sensed data (e.g., pixel depth values) using the calibration data.

Calibration of depth-sensing computer vision systems

A method of safely operating machinery in a workspace includes recording images of a portion of a workspace. The method also includes generating a three-dimensional (3D) representation of the portion of the workspace based on the recorded images, where the 3D representation includes one or more volumes that correspond to the portion of the workspace. Additionally, the method includes identifying one or more of the volumes as being either occupied or unoccupied. Further, the method includes mapping one or more safe zones based on the one or more identified volumes, where the safe zones correspond to one or more regions within the portion of the workspace for safe operation of machinery.

Safe motion planning for machinery operation

A measurement system, its assembly and use are disclose. The system may include an instrument for making sensor measurements. The instrument has a substantially cylindrical housing. The shape and size allow the instrument to easily fit in an average hand enabling handheld operation. The housing houses a board stack of electronic boards. These electronics drive an electrical signal in at least one drive channel and measure responses from at least two sensing channels. These responses are provided to a processor for analysis. The instrument has a sensor connector that enables simultaneous electrical and mechanical attachment of an end effector.

Measurement system and method of use

One or more computational strategies is used to reduce the complexity of handling detected point clusters appearing at a portal of a monitored workcell. If the other side of the portal is a second, adjacent workcell, the monitoring system for a first workcell predicts when a human in that workcell may pass through the portal to a second workcell and alerts the monitoring system for the second workcell. The prediction may be based on absolute proximity of a human to the portal or movement toward it. Other strategies are employed if the workcells overlap, or if the portal leads to an unmonitored space.

Intelligent monitoring of entry points in multi-cell workspaces

In various embodiments, systems and methods for acquiring depth images utilize an architecture suited to safety-rated applications, and may include more than sensor (such as time-of-flight sensors) operating along different optical paths and a comparison module for ensuring proper sensor operation. Error metrics may be associated with pixel-level depth values for purposes of allowing safe control based on imperfectly known depths.

Depth-sensing computer vision system

Systems and methods for determining safe and unsafe zones in a workspace—where safe actions are calculated in real time based on all relevant objects (e.g., some observed by sensors and others computationally generated based on analysis of the sensed workspace) and on the current state of the machinery (e.g., a robot) in the workspace—may utilize a variety of workspace-monitoring approaches as well as dynamic modeling of the robot geometry. The future trajectory of the robot(s) and/or the human(s) may be forecast using, e.g., a model of human movement and other forms of control. Modeling and forecasting of the robot may, in some embodiments, make use of data provided by the robot controller that may or may not include safety guarantees.

Dynamic, interactive signaling of safety-related conditions in a monitored environment

Systems and methods for identifying a robot end effector in a processing environment may utilize one or more sensors for digitally recording visual information and providing that information to an industrial workflow. The sensor(s) may be positioned to record at least one image of the robot including the end effector. A processor may determine the identity of the end effector from the recorded image(s) and a library or database stored digital models.

Robot end-effector sensing and identification

Systems and methods for continuously monitoring a workcell during operation of industrial machinery are disclosed. The system may comprise a safety system that includes at least one sensor and supporting software and/or hardware for acquiring image data associated with the workcell; a monitoring system for detecting a parameter value associated with the safety system; and a controller configured to determine a status of the safety system based at least in part on the detected parameter value and cause an alert to be issued if the status of the safety system does not satisfy a target objective.

Continuous monitoring of a workcell safety system

An integrated system and method for monitoring a workspace, classifying regions therein, dynamically identifying safe states, and identifying and tracking workpieces utilizes semantic analysis of a robot in the workspace and identification of the workpieces with which it interacts, as well as a semantic understanding of entity properties both governing and arising from interaction with other entities for control purposes. These properties may be stored in a profile corresponding to the entity; depending on the implementation, the profile and methods associated with the entity may be stored in a data structure corresponding to the entity 'class.' Volumetric regions (e.g., voxels) corresponding to each entity may be tracked and used to implement object methods and/or control methods. For example, machinery may be controlled in accordance with the attributes specified by the objects corresponding to identified entities in a monitored workspace.

Workplace monitoring and semantic entity identification for safe machine operation

Systems and methods monitor a workspace for safety purposes using sensors distributed about the workspace. The sensors are registered with respect to each other, and this registration is monitored over time. Occluded space as well as occupied space is identified, and this mapping is frequently updated. Based on the mapping, a constrained motion plan of machinery can be generated to ensure safety.

System and method for characterizing material condition. The system includes a sensor, impedance instrument and processing unit to collect measurements and assess material properties. A model of the system may be used to enable accurate measurements of multiple material properties. A cylindrical model for an electromagnetic field sensor is disclosed for modeling substantially cylindrically symmetric material systems. Sensor designs and data processing approaches are provided to focus the sensitivity of the sensor to localize material conditions. Improved calibration methods are shown. Sizing algorithms are provided to estimate the size of defects such as cracks and corrosion. Corrective measures are provided where the actual material configuration differs from the data processing assumptions. Methods are provided for use of the system to characterize material condition, and detailed illustration is given for corrosion, stress, weld, heat treat, and mechanical damage assessment.

Method and apparatus for measurement of material condition

Crosstalk mitigation among cameras in neighboring monitored workcells is achieved by computationally defining a noninterference scheme that respects the independent monitoring and operation of each workcell. The scheme may involve communication between adjacent cells to adjudicate non-interfering camera operation or system-wide mapping of interference risks and mitigation thereof. Mitigation strategies can involve time-division and/or frequency-division multiplexing.

Crosstalk mitigation for multi-cell workspace monitoring

Growing community of inventors

Somerville, MA, United States of America

Scott A Denenberg