US 12,434,380 B2
Robot systems, methods, control modules, and computer program products that leverage large language models
Geordie Rose, Vancouver (CA); and Suzanne Gildert, Vancouver (CA)
Assigned to Sanctuary Cognitive Systems Corporation, Vancouver (CA)
Filed by Sanctuary Cognitive Systems Corporation, Vancouver (CA)
Filed on Jun. 3, 2024, as Appl. No. 18/731,704.
Application 18/731,704 is a continuation of application No. 18/513,440, filed on Nov. 17, 2023, granted, now 11,999,063.
Claims priority of provisional application 63/441,897, filed on Jan. 30, 2023.
Prior Publication US 2024/0316761 A1, Sep. 26, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. B25J 9/16 (2006.01); B25J 19/02 (2006.01); G06F 40/279 (2020.01); G06F 40/40 (2020.01)
CPC B25J 9/1602 (2013.01) [B25J 9/161 (2013.01); B25J 9/163 (2013.01); B25J 9/1653 (2013.01); B25J 9/1658 (2013.01); B25J 9/1661 (2013.01); B25J 9/1671 (2013.01); B25J 9/1697 (2013.01); B25J 19/02 (2013.01); B25J 19/023 (2013.01); G06F 40/279 (2020.01); G06F 40/40 (2020.01)] 20 Claims
OG exemplary drawing
 
1. A robot system comprising:
a robot body;
at least one sensor to capture sensor data representing information about an environment of the robot body;
at least one processor; and
at least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions and/or data that, when executed by the at least one processor, cause the at least one processor to:
provide a natural language (NL) query to a large language model (LLM) module, the NL query including information about an environment of the robot body, an NL description of a work objective, an NL description of an instruction set which includes a plurality of instructions each selectively executable by the robot system, an NL request for a task plan, and an NL indication of a task plan step threshold corresponding to a maximum number of steps in the task plan;
receive the task plan from the LLM module, the task plan expressed in NL and including a number of steps which does not exceed the task plan step threshold;
for each step of the task plan which does not correspond to a single instruction in the instruction set:
provide an NL breakdown query for a respective component plan to the LLM module, the NL breakdown query including an NL request to break down the step of the task plan into a component plan including a plurality of component steps;
receive each component plan from the LLM module, each component plan expressed in NL; and
incorporate each component plan into the task plan;
and
cause the robot system to execute the task plan.