US 12,466,082 B2
Motion guidance and natural language commands based robotic systems
Thomas Kollar, San Jose, CA (US)
Assigned to TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US); and TOYOTA JIDOSHA KABUSHIKI KAISHA, Toyota (JP)
Filed by Toyota Research Institute, Inc., Los Altos, CA (US)
Filed on Jun. 14, 2022, as Appl. No. 17/840,070.
Prior Publication US 2023/0398696 A1, Dec. 14, 2023
Int. Cl. B25J 13/00 (2006.01); B25J 9/16 (2006.01); B25J 11/00 (2006.01); G06F 40/205 (2020.01)
CPC B25J 13/003 (2013.01) [B25J 9/163 (2013.01); B25J 9/1664 (2013.01); B25J 11/008 (2013.01); G06F 40/205 (2020.01)] 18 Claims
OG exemplary drawing
 
1. A method implemented by a computing device of a robot, the method comprising:
receiving a natural language command for performing an action within a real world environment, the natural language command including one or more terms;
parsing, by the computing device, the natural language command by analyzing the one or more terms of the natural language command to identify the action;
categorizing the action as being associated with guidance for performing the action;
receiving the guidance for performing the action, the guidance including a motion applied to at least one portion of the robot within the real world environment for performing the action;
storing the natural language command in correlation with the motion in memory of the computing device;
receiving a future natural language command at a time point subsequent to the natural language command and accessing the memory to determine a degree of similarity relative to the stored natural language command in correlation with the motion to perform the action associated with the future natural language command;
outputting a voice command indicating a request for clarification of the natural language command, wherein the voice command is outputted in response to determining an absence of a record of the action in the memory of the computing device to be performed in correlation with the motion;
entering, by the robot and in response to putting the voice command indication the request for clarification of the natural language command, a training mode including storing the motion applied to the at least one portion of the robot; and
performing, in response to the robot entering the training mode, the action specified in the natural language command stored in the memory of the computing device automatically and without user intervention.