US 12,459,115 B2
System and method for controlling a robot
Anoop Cherian, Cambridge, MA (US); Xiulong Liu, Cambridge, MA (US); Sudipta Paul, Cambridge, AL (US); and Moitreya Chatterjee, Cambridge, AL (US)
Assigned to Mitsubishi Electric Research Laboratories, Inc., Cambridge, MA (US)
Filed by Mitsubishi Electric Research Laboratories, Inc.
Filed on Mar. 6, 2023, as Appl. No. 18/178,882.
Prior Publication US 2024/0300096 A1, Sep. 12, 2024
Int. Cl. B25J 9/16 (2006.01); B25J 13/00 (2006.01)
CPC B25J 9/163 (2013.01) [B25J 9/161 (2013.01); B25J 9/1664 (2013.01); B25J 9/1697 (2013.01); B25J 13/003 (2013.01)] 20 Claims
OG exemplary drawing
 
17. A method for controlling a robot, the method comprising:
selecting, using a first level controller, one of at least three second level controllers to perform a first interaction relating to a task based at least on a state of a hierarchical multimodal RL neural network, wherein
the hierarchical multimodal RL neural network includes the first level controller and at least three second level controllers, the at least three second level controllers comprising a first sub level controller, a second sub level controller and a third sub level controller, wherein the hierarchical multimodal RL neural network is configured to receive input data via an input interface of the robot such that
the first sub level controller is configured to receive the input data based on a set of predefined questions,
the second sub level controller is configured to receive the input data by generating a validation question based on the state of the hierarchical multimodal RL neural network, and
the third sub level controller is configured to obtain the input data based on the state of the hierarchical multimodal RL neural network;
generating, using the selected second level controller, a control command based on the corresponding input data; and
controlling an operation of the robot by executing the control command.