US 12,304,081 B2
Deep compositional robotic planners that follow natural language commands
Yen-Ling Kuo, Cambridge, MA (US); Boris Katz, Cambridge, MA (US); and Andrei Barbu, Cambridge, MA (US)
Assigned to Massachusetts Institute of Technology, Cambridge, MA (US)
Filed by Massachusetts Institute of Technology, Cambridge, MA (US)
Filed on Dec. 4, 2020, as Appl. No. 17/112,699.
Claims priority of provisional application 62/944,924, filed on Dec. 6, 2019.
Claims priority of provisional application 62/944,932, filed on Dec. 6, 2019.
Prior Publication US 2021/0170594 A1, Jun. 10, 2021
Int. Cl. B25J 9/16 (2006.01); G06N 3/042 (2023.01); G06N 3/08 (2023.01)
CPC B25J 9/1664 (2013.01) [G06N 3/042 (2023.01); G06N 3/08 (2013.01)] 13 Claims
OG exemplary drawing
 
1. A method comprising:
determining, by a planner, a first point in a selected neighborhood of a configuration space;
receiving a natural language input, uttered by a human or generated by a computer;
parsing the received natural language input to determine (i) linguistic structure of the natural language input and (ii) a plurality of words in the natural language input;
modifying a first neural network (NN) to encode the determined linguistic structure of the natural language input by arranging a plurality of component neural networks of the first NN in an order corresponding to the determined linguistic structure of the natural language input, wherein each component neural network corresponds to one or more respective words of the determined plurality of words;
determining, by the modified first NN, a second point in the selected neighborhood of the configuration space;
choosing among the first point and second point to generate an additional node to add to a search tree; and
adding the additional node to the search tree by connecting the additional node to a node associated with the selected neighborhood;
wherein a second NN controls behavior of an agent corresponding to the search tree having the additional node.