Siemens Aktiengesellschaft

University of California

Siemens Industry, Inc.

Siemens Corporation

Siemens Aktiengesell Schaft

Juan L Aparicio Ojea

Chengtao Wen

Heiko Claussen

Eugen Solowjow

Martin Sehr

Lingyun Wang

Justinian Rosca

Bradley Wehrwein

Brian Collum

Max A Fahrenkopf

Andrew Valdez

Steffen Lamparter

George Lo

Richard Gary McDaniel

Volkmar Sterzing

Ramu Sharat Chandra

Michael Unkelbach

Kenneth Y Goldberg

Mathias Maurmaier

Tao Cui

Ken Goldberg

Swen Elpelt

Matthias Loskyll

Jianlan Luo

Mohamed El Amine Houyou

Jeffrey Brian Mahler

Ines Ugalde Diaz

Yash Shahapurkar

Xiaowen Yu

Florian Till Pokorny

Wei Xi Xia

Avinash Kumar

Changtao Wen

Ramu Chandra

A computer-implemented method for building an object detection module uses mesh representations of objects belonging to specified object classes of interest to render images by a physics-based simulator. Each rendered image captures a simulated environment containing objects belonging to multiple object classes of interest placed in a bin or on a table. The rendered images are generated by randomizing a set of parameters by the simulator to render a range of simulated environments. The randomized parameters include environmental and sensor-based parameters. A label is generated for each rendered image, which includes a two-dimensional representation indicative of location and object classes of objects in that rendered image frame. Each rendered image and the respective label constitute a data sample of a synthetic training dataset. A deep learning model is trained using the synthetic training dataset to output object classes from an input image of a real-world physical environment.

Synthetic dataset creation for object detection and classification with deep learning

Autonomous operations, such as robotic grasping and manipulation, in unknown or dynamic environments present various technical challenges. For example, three-dimensional (3D) reconstruction of a given object often focuses on the geometry of the object without considering how the 3D model of the object is used in solving or performing a robot operation task. As described herein, in accordance with various embodiments, models are generated of objects and/or physical environments based on tasks that autonomous machines perform with the objects or within the physical environments. Thus, in some cases, a given object or environment may be modeled differently depending on the task that is performed using the model. Further, portions of an object or environment may be modeled with varying resolutions depending on the task associated with the model.

Task-oriented 3D reconstruction for autonomous robotic operations

Systems and methods for controlling robots including industrial robots. A method includes executing () a program () to control a robot () by the robot control system (). The method includes receiving () robot state information (). The method includes receiving () force torque feedback () inputs from a sensor () on the robot (). The method includes producing () a robot control command for the robot () based on the robot state information () and the force torque feedback () inputs. The method includes controlling () the robot () using the robot control command.

Reinforcement learning for contact-rich tasks in automation systems

Examples of techniques for threat detection in an industrial process system are described herein. An aspect includes determining a plurality of subsystems of an industrial process system. Another aspect includes, for each of the plurality of subsystems, constructing and training a respective deep autoencoder (DAE) model of the subsystem based on data corresponding to the industrial process system. Another aspect includes monitoring the industrial process system using the plurality of DAE models corresponding to the plurality of subsystems. Another aspect includes, based on the plurality of DAE models, determining a cyberattack in a subsystem of the plurality of subsystems.

Industrial process system threat detection

A system and method are disclosed for orchestrating the execution of computing tasks. An orchestration engine can receive task requests over a network from a plurality of process engines. The process engines may correspond to respective edge or field devices that are remotely located as compared to the orchestration engine. Each task request may indicate at least one task requirement for executing a respective computing task. A plurality of computing instances that have available computing resources can be selected from a set of computing instances. A predicted runtime can be generated for each of the computing tasks. In an example, based on the predicted runtimes, task requirements, available computing resources, and associated network conditions, a schedule and allocation scheme are determined by the orchestration engine. The schedule and allocation scheme define when each of the plurality of computing tasks is performed, and which of the plurality of selected computing instances performs each of the plurality of computing tasks. The selected computing instances execute the plurality of computing tasks according to the schedule and allocation scheme.

Orchestration of containerized applications

For manufacturing process control, closed-loop control is provided () based on a constrained reinforcement learned network (). The reinforcement is constrained () to account for the manufacturing application. The constraints may be for an amount of change, limits, or other factors reflecting capabilities of the controlled device and/or safety.

Manufacturing process control using constrained reinforcement machine learning

A system controller for visual servoing includes a technology module with dedicated hardware acceleration for deep neural network that retrieves a desired configuration of a workpiece object being manipulated by a robotic device and receives visual feedback information from one or more sensors on or near the robotic device that includes a current configuration of the workpiece object. The hardware accelerator executes a machine learning model trained to process the visual feedback information and determine a configuration error based on a difference between the current configuration of the workpiece object and the desired configuration of the workpiece object. A servo control module adapts a servo control signal to the robotic device for manipulation of the workpiece object in response to the configuration error.

Machine learning enabled visual servoing with dedicated hardware acceleration

Computerized system and method are provided. A robotic manipulator () is arranged to grasp objects (). A gripper () is attached to robotic manipulator (), which includes an imaging sensor (). During motion of robotic manipulator (), imaging sensor () is arranged to capture images providing different views of objects in the environment of the robotic manipulator. A processor () is configured to find, based on the different views, candidate grasp locations and trajectories to perform a grasp of a respective object in the environment of the robotic manipulator. Processor () is configured to calculate respective values indicative of grasp quality for the candidate grasp locations, and, based on the calculated respective values indicative of grasp quality for the candidate grasp locations, processor () is configured to select a grasp location likely to result in a successful grasp of the respective object.

Computerized system and method using different image views to find grasp locations and trajectories for robotic pick up

Over the past several decades, rapid advances in semiconductors, automation, and control systems have resulted in the adoption of programmable logic controllers (PLCs) in an immense variety of environments. Machine learning techniques help train replacement PLCs when a legacy PLC must be replaced, e.g., due to aging or failure. The techniques facilitate the efficient adoption and correct operation of replacement PLCs in the industrial environment.

Method for replacing legacy programmable logic controllers

A system for analyzing geometric properties for an object includes designing the object in a first computer process and producing information relating to the geometric properties of the object, and receiving the information in a second computer processor which identifies a first portion of the geometric property information as masked or private and second portion identified as public or shared, analysis is performed by the second processor on the public/shared portion of the geometric property information. An output based on the analysis may be provided to an industrial system performing processes on the object. A binary privacy label may be assigned to each triangle in a set of triangles representing the surfaces of the object in a 3D object mesh. The privacy label denotes an associated triangle as being private or shared. The system may be used to produce a set of planned grasps for a robotic gripper.

Method and system for preserving privacy for cloud-based manufacturing analysis services

Systems and methods for automatic generation of robot control policies include a CAD-based simulation engine for simulating CAD-based trajectories for the robot based on cost function parameters, a demonstration module configured to record demonstrative trajectories of the robot, an optimization engine for optimizing a ratio of CAD-based trajectories to demonstrative trajectories based on computation resource limits, a cost learning module for learning cost functions by adjusting the cost function parameters using a minimized divergence between probability distribution of CAD-based trajectories and demonstrative trajectories; and a deep inverse reinforcement learning engine for generating robot control policies based on the learned cost functions.

Method and system for automatic robot control policy generation via CAD-based deep inverse reinforcement learning

A method of generating events based on automation system data in an intelligent programmable logic controller includes the intelligent programmable logic controller collecting automation system data and applying a context model to the automation system data to create contextualized data. The intelligent programmable logic controller generates one or more events based on the contextualized data.

Generating events using contextual information on an intelligent programmable logic controller

A method of operating an intelligent programmable logic controller over a plurality of scan cycles includes the intelligent programmable logic controller selecting one or more soft-sensors available in a control program corresponding to a production unit, each soft-sensor comprising a local parameter or variable used by the control program. The intelligent programmable logic controller determines updated soft-sensor values corresponding to the one or more soft-sensors during each scan cycle and stores those values during each scan cycle on a non-volatile computer-readable storage medium operably coupled to the intelligent programmable logic controller. Additionally, the intelligent programmable logic controller annotates the updated soft-sensor values with automation system context information to generate contextualized data.

Using soft-sensors in a programmable logic controller

A method of compressing signal data in an intelligent programmable logic controller includes the controller creating a process image area in a volatile computer-readable storage medium operably coupled to the controller. The intelligent programmable logic controller updates the process image area with contents comprising signal data associated with a production unit and applies a compression algorithm to the contents to generate compressed signal data. The compressed signal data is stored on a non-volatile computer-readable storage medium operably coupled to the controller. The controller annotates the signal data with automation system context information to generate contextualized data. Next, the controller performs a statistical comparison of the signal data and the compressed signal data to generate statistical comparison results. Then, one or more compression parameters used by the compression algorithm may be automatically adjusted based on at least one of the statistical comparison and the contextualized data.

Automatic compression algorithm selection and parameter tuning based on contextual knowledge

A traffic control monitoring and abnormality determination system and associated methods are disclosed for receiving and analyzing traffic controller input/output data during a learning phase to determine a model indicative of normal or healthy operation of the traffic controller in regulating traffic flow at an intersection and receiving and evaluating additional traffic controller input/output data against the model during an evaluation phase to determine whether an abnormality exists in operation of the traffic controller. If an abnormality is detected during the evaluation phase, the system may initiate a corrective action to resolve the abnormality such as sending an alarm signal to a traffic controller to cause the traffic controller to alter an operating state to resolve the abnormality.

Growing community of inventors

Doylestown, PA, United States of America

Juan L Aparicio Ojea