Mitsubishi Electric Research Laboratories, Inc.

Mitsubishi Electric Corporation

Devesh Jha

Diego Romeres

Arvind Raghunathan

Daniel Nikolaev Nikovski

Yuki Shirai

Yebin Wang

Ankush Chakrabarty

William T Vetterling

William Yerazunis

Scott Bortoff

Bingnan Wang

Mouhacine Benosman

Rien Quirynen

Chungwei Lin

Emil Laftchiev

Radu Ioan Corcodel

Siddarth Jain

Alberto Dalla Libera

Seiji Shaw

Shivam Vats

Wenyu Zhang

Patrik Kolaric

A feedback controller for controlling a robotic manipulator to perform a task is provided. The feedback controller comprises a memory configured to store a hierarchical reinforcement learning (HRL) neural network including (i) nominal control policy, and (ii) recovery control policy. The recovery control policy is trained based on a recovery policy reward to select a switch-to-nominal action to transfer control to the nominal control policy to perform the task. The recovery policy reward is dependent on a nominal policy reward for training the nominal control policy. The feedback controller further comprises a processor configured to iteratively execute the HRL neural network to select at least one of a nominal action based on the nominal control policy, and a recovery action based on the recovery control policy and to control the robotic manipulator to perform the task based on the selected action.

System and method for controlling a robotic manipulator based on hierarchical reinforcement learning of control policies

A robotic controller is provided for generating sequences of movement primitives for sequential tasks of a robot having a manipulator. The controller includes at least one control processor, and a memory circuitry storing a dictionary including the movement primitives, a pretrained learning module, and a graph-search based planning module having instructions stored thereon. The controller to perform steps acquiring a planned task provided by an interface device operated by a user, wherein the planned task is represented by an initial state and a goal state with respect to an object, generating a planning graph by searching a feasible path of the object for the novel task using the graph-search based planning module and selecting movement primitives from the dictionary in the pretrained learning module, wherein the pretrained learning module has been trained based on demonstration tasks, parameterizing the feasible path represented by the movement primitives as dynamic movement primitives (DMPs) using the initial state and goal state, and implementing the parameterized feasible path as a trajectory according to the selected movement primitives using the manipulator of the robot by tracking and following the parameterized for the planned task.

System and method for learning sequences in robotic tasks for generalization to new tasks

The present disclosure discloses a system and a method for controlling an operation of a robotic arm holding a tool for manipulating object. The method comprises collecting measurements of tactile sensors associated with the robotic arm, estimating, based on the collected measurements and constraints enforced by a Model Predictive Controller (MPC), a feedback signal indicative of a pose of the object, and executing the MPC configured to produce, based on the pose of the object, control commands for actuators of the robotic arm by optimizing a cost function minimizing a deviation of the pose of the object from a target pose of the object. The optimization of the cost function is subject to the constraints that constrain one or more forces acting on the object at one or more point of contacts to be within corresponding friction regions. The method further comprises controlling the actuators according to the control commands.

System and method for controlling an operation of a robotic arm

A manipulation controller is provided for reorienting an object by a manipulator of a robotic system. The manipulation controller includes an interface controller configured to acquire measurement data from sensors arranged on the robotic system, at least one processor, and a memory configured to store a computer-implemented method. The instructions of the method include acquiring measurement data from vision sensors and force sensors arranged on the robotic system, determining an input-output relation for the object based on a nonlinear static model representing input-output relationships between contact forces and movements of the object on the workbench, representing interaction between the object and the manipulator using complementarity constraints to capture the contact state between the object and the manipulator, formulating a representation for frictional stability of the object based on the non-linear static model at the external contacts with the workbench; formulating a bilevel optimization problem so as to maximize the frictional stability over a position trajectory of the object being manipulated on the workbench, estimating uncertainty value in physical parameters to be compensated by performing the bilevel optimization problem, solving the bilevel optimization problem using the non-linear optimization solver and generating control data with respect to a sequence of the contact forces being applied to the object by using the manipulator.

System and method for robust pivoting for re-orienting parts during robotic assembly

A controller for controlling an operation of a robot to execute a task is provided. The controller comprises a memory configured to store a set of dynamic movement primitives (DMPs) associated with the task. The set of DMPs comprise a set of at least two dynamical systems: a function representing point attractor dynamics and a forcing function corresponding to a learned demonstration of the task. The controller comprises a processor configured to transform the set of DMPs to a set of constrained DMPs (CDMPs) by determining a perturbation function associated with the forcing function. The perturbation function is associated with a set of operational constraints. The processor is further configured to solve, a non-linear optimization problem for the set of CDMPs based on the set of operational constraints and generate, a control input for controlling the robot for executing the task, based on the solution.

System and method for controlling a robot using constrained dynamic movement primitives

The present disclosure provides a system and a method for controlling an operation of a manipulation system. The method comprises formulating an optimization problem based on a Stochastic Discrete-time Linear Complementarity Model (SDLCM) of a manipulation task, and a sample average approximation; solving the formulated optimization problem using an important-particle algorithm to compute an optimal state trajectory, an optimal feedforward control trajectory, an optimal complementarity variable trajectory, a state feedback gain, and a complementarity feedback gain; collecting, measurements indicative of a current state trajectory and a current complementarity variable trajectory; determining an online control input based on the optimal feedforward control trajectory, a deviation of the current state trajectory from the optimal state trajectory, a deviation of the current complementarity variable trajectory from the optimal complementarity variable trajectory, the state feedback gain, and the complementarity feedback gain; and controlling actuators of the manipulation system according to the online control input.

System and method for controlling an operation of a manipulation system

The present disclosure provides a system and a method for controlling a motion of a robot from a starting point to a target point within a bounded space with a floorplan including one or multiple obstacles. The method includes solving for an electric potential in a bounded virtual space formed by scaling the floorplan of the bounded space with the one or multiple obstacles and applying charge to at least one bound of the bounded virtual space while treating the scaled obstacles as metallic surfaces with a constant potential value, wherein the electric potential provides multiple equipotential curves within the bounded virtual space. The method further includes selecting an equipotential curve with a potential value different from a potential value of an obstacle equipotential curve, determining a motion path based on the selected equipotential curve, and controlling the motion of the robot based on the determined motion path.

System and method for controlling a motion of a robot

A controller is provided for generating a policy controlling a system by learning a dynamics of the system. The controller is configured to perform steps of acquiring measurement data from sensors arranged on the system, providing, to the memory, a non-linear system model represented by known part of the dynamics of the system and unknown part of the dynamics of the system, collecting states of the system by measuring the dynamics of the system using the sensors of the system based on a nominal policy and a noise term with respect to the states, estimating a sequence of sets of states of the system and sets of control inputs by collecting data of the system, wherein the data includes a collection of system states, applied control inputs and change in system states, wherein each of the control input is computed by the nominal policy and the additional noise term, learning a polytopic system by use of the collected data of the system for approximating the unknown part of the dynamics of the system using a linear probabilistic regression model, estimating an attractor basin of a terminal controller by sampling initial states in a neighborhood of a terminal state and estimating the attractor basin by supervised learning, and generating a polytopic policy using the estimated polytopic system to drive the system to the attractor basin of the terminal controller from an initial state.

System and method for polytopic policy optimization for robust feedback control during learning

A robotic system for manipulating an object with a robotic manipulator is provided. The robotic system is configured to collect a digital representation of a task for manipulating the object; solve a robust control problem to optimize a sequence of control forces to be applied by the robotic manipulator to change a state of the object, where an evolution of the state of the object is governed by a stochastic complementarity system modeling the task with a predefined probability. The robust control problem optimizes a cost function to generate the sequence of control forces performing the task subject to joint chance constraints including a first chance constraint on the state of the object being manipulated and a second chance constraint on stochastic complementarily constraints modeling manipulation of the object. The robotic system is further configured to control the manipulation of the object based on the sequence of control forces.

System and method for robust robotic manipulation using chance constrained optimization

A system for performing a task according to a reference trajectory is provided. The system includes at least one actuator configured to change a state of the system according to a control input, and a memory configured to store a model of dynamics of the system including a known part of the dynamics of the system as a function of the state of the system and the control input to the system and an unknown part of the dynamics of the system as a function of the state of the system, wherein the unknown part of the dynamics of the system is represented by parameters of a probabilistic distribution including a first-order moment and a second-order moment of the probabilistic distribution. The system also includes a control system configured to recursively determine and submit the control input to the actuator to change the state of the system.

System for performing a task according to a reference trajectory

A controller controls a motion of an object performing a task for changing a state of the object from a start state to an end state while avoiding collision of the object with an obstacle according to an optimal trajectory determined by solving an optimization problem of the dynamics of the object producing an optimal trajectory for performing the task subject to constraints on a solution of first-order stationary conditions modeling a minimum distance between the convex hull of the object and the convex hull of the obstacle using complementarity constraints.

Object manipulation with collision avoidance using complementarity constraints

A robot for performing an assembly operation is provided. The robot comprises a processor configured to determine a control law for controlling a plurality of motors of the robot to move a robotic arm according to an original trajectory, execute a self-exploration program to produce training data indicative of a space of the original trajectory, and learn, using the training data, a non-linear compliant control law including a non-linear mapping that maps measurements of a force sensor of the robot to a direction of corrections to the original trajectory defining the control law. The processor transforms the original trajectory according to a new goal pose to produce a transformed trajectory, update the control law according to the transformed trajectory to produce the updated control law, and command the plurality of motors to control the robotic arm according to the updated control law corrected with the compliance control law.

System and method for robotic assembly based on adaptive compliance

A computer-implemented learning method for optimizing a control policy controlling a system is provided. The method includes receiving states of the system being operated for a specific task, initializing the control policy as a function approximator including neural networks, collecting state transition and reward data using a current control policy, estimating an advantage function and a state visitation frequency based on the current control policy, updating the current control policy using the second-order approximation of the objective function, a second-order approximation of the KL-divergence constraint on the permissible change in the policy using a quasi-newton trust region policy optimization, and determining an optimal control policy, for controlling the system, based on the average reward accumulated using the updated current control policy.

System and method for policy optimization using quasi-Newton trust region method

A manipulator learning-control apparatus for controlling a manipulating system that includes an interface configured to receive manipulator state signals of the manipulating system and object state signals with respect to an object to be manipulated by the manipulating system in a workspace, wherein the object state signals are detected by at least one object detector, an output interface configured to transmit initial and updated policy programs to the manipulating system, a memory to store computer-executable programs including a data preprocess program, object state history data, manipulator state history data, a Derivative-Free Semi-parametric Gaussian Process (DF-SPGP) kernel learning program, a Derivative-Free Semi-parametric Gaussian Process (DF-SPGP) model learning program, an update-policy program and an initial policy program, and a processor, in connection with the memory, configured to transmit the initial policy program to the manipulating system for initiating a learning process that operates the manipulator system manipulating the object while a preset period of time.

System and design of derivative-free model learning for robotic systems

A system for controlling a robotic arm performing insertion of a component along an insertion line accepts measurements of force experienced by the wrist of robotic arm at current position along insertion line and determines probability of value of the force conditioned on the current value of the position according to a probabilistic relationship for the force experienced by the wrist of the robotic arm along the insertion line as a probabilistic function of the positions of the wrist of the robotic arm along the line of insertion. The probabilistic function is learned from measurements of the operation repeatedly performed by one or multiple robotic arms having the configuration of the robotic arm under the control. The system determines a result of anomaly detection based on the probability of the current value of the force and controls the robotic arm based on the result of anomaly detection.

Growing community of inventors

Cambridge, MA, United States of America

Devesh Jha