Siemens Aktiengesellschaft

University of California

Siemens Corporation

Eugen Solowjow

Juan L Aparicio Ojea

Chengtao Wen

Heiko Claussen

Martin Sehr

Yash Shahapurkar

Richard Gary McDaniel

Matthias Loskyll

Jianlan Luo

Swen Elpelt

Ines Ugalde Diaz

Eduardo Moura Cirilo Rocha

Husnu Melih Erdogan

Xiaowen Yu

Avinash Kumar

Changtao Wen

Wei Xi Xia

A computer-implemented method for building an object detection module uses mesh representations of objects belonging to specified object classes of interest to render images by a physics-based simulator. Each rendered image captures a simulated environment containing objects belonging to multiple object classes of interest placed in a bin or on a table. The rendered images are generated by randomizing a set of parameters by the simulator to render a range of simulated environments. The randomized parameters include environmental and sensor-based parameters. A label is generated for each rendered image, which includes a two-dimensional representation indicative of location and object classes of objects in that rendered image frame. Each rendered image and the respective label constitute a data sample of a synthetic training dataset. A deep learning model is trained using the synthetic training dataset to output object classes from an input image of a real-world physical environment.

Synthetic dataset creation for object detection and classification with deep learning

Autonomous operations, such as robotic grasping and manipulation, in unknown or dynamic environments present various technical challenges. For example, three-dimensional (3D) reconstruction of a given object often focuses on the geometry of the object without considering how the 3D model of the object is used in solving or performing a robot operation task. As described herein, in accordance with various embodiments, models are generated of objects and/or physical environments based on tasks that autonomous machines perform with the objects or within the physical environments. Thus, in some cases, a given object or environment may be modeled differently depending on the task that is performed using the model. Further, portions of an object or environment may be modeled with varying resolutions depending on the task associated with the model.

Task-oriented 3D reconstruction for autonomous robotic operations

It is recognized herein that robots or autonomous systems can lose time when computing grasp scores for empty bins. Further, when grasps are attempted on empty bins, for instance due to the related grasp score computations, the robot can lose additional time through being used unnecessarily to attempt the grasp. Such usage can wear on the robot, or damage the robot, in some cases. An autonomous system can classify or determine whether a bin contains an object or is empty, for example, such that a grasp computation is not performed when the bin is empty. In some examples, a system classifies a given bin at runtime before each grasp computation is performed. Thus, systems described herein can avoid performing unnecessary grasp computations, thereby conserving processing time and overheard, among addressing other technical problems.

Detecting empty workspaces for robotic material handling

Systems and methods for controlling robots including industrial robots. A method includes executing () a program () to control a robot () by the robot control system (). The method includes receiving () robot state information (). The method includes receiving () force torque feedback () inputs from a sensor () on the robot (). The method includes producing () a robot control command for the robot () based on the robot state information () and the force torque feedback () inputs. The method includes controlling () the robot () using the robot control command.

Reinforcement learning for contact-rich tasks in automation systems

A system controller for visual servoing includes a technology module with dedicated hardware acceleration for deep neural network that retrieves a desired configuration of a workpiece object being manipulated by a robotic device and receives visual feedback information from one or more sensors on or near the robotic device that includes a current configuration of the workpiece object. The hardware accelerator executes a machine learning model trained to process the visual feedback information and determine a configuration error based on a difference between the current configuration of the workpiece object and the desired configuration of the workpiece object. A servo control module adapts a servo control signal to the robotic device for manipulation of the workpiece object in response to the configuration error.

Machine learning enabled visual servoing with dedicated hardware acceleration

Computerized system and method are provided. A robotic manipulator () is arranged to grasp objects (). A gripper () is attached to robotic manipulator (), which includes an imaging sensor (). During motion of robotic manipulator (), imaging sensor () is arranged to capture images providing different views of objects in the environment of the robotic manipulator. A processor () is configured to find, based on the different views, candidate grasp locations and trajectories to perform a grasp of a respective object in the environment of the robotic manipulator. Processor () is configured to calculate respective values indicative of grasp quality for the candidate grasp locations, and, based on the calculated respective values indicative of grasp quality for the candidate grasp locations, processor () is configured to select a grasp location likely to result in a successful grasp of the respective object.

Computerized system and method using different image views to find grasp locations and trajectories for robotic pick up

Growing community of inventors

Berkeley, CA, United States of America

Eugen Solowjow