Mitsubishi Electric Research Laboratories, Inc.

Xiulong Liu

Sudipta Paul

Moitreya Chatterjee

A controller for controlling a robot is provided. The controller comprises a hierarchical multimodal reinforcement learning (RL) neural network including a first level controller and three second level controllers. The second level controllers comprise a first sub level controller to receive input data based on predefined questions, a second sub level controller to receive the input data by generating a validation question based on state of the RL neural network and a third sub level controller to determine the input data based on state of the RL neural network. The controller is configured to select one of the second level controllers using the first level controller to perform a first interaction relating to a task based on the state of the RL neural network; generate a control command using the selected second level controller based on the corresponding input data; and control operation of the robot by executing control command.

Growing community of inventors

Cambridge, MA, United States of America

Xiulong Liu