US 11,931,894 B1
Robot systems, methods, control modules, and computer program products that leverage large language models
Geordie Rose, Vancouver (CA); and Suzanne Gildert, Vancouver (CA)
Assigned to Sanctuary Cognitive Systems Corporation, Vancouver (CA)
Filed by Sanctuary Cognitive Systems Corporation, Vancouver (CA)
Filed on Oct. 2, 2023, as Appl. No. 18/375,943.
Claims priority of provisional application 63/441,897, filed on Jan. 30, 2023.
Int. Cl. B25J 9/16 (2006.01); G06F 40/279 (2020.01); G06F 40/40 (2020.01)
CPC B25J 9/1602 (2013.01) [G06F 40/279 (2020.01); G06F 40/40 (2020.01)] 20 Claims
OG exemplary drawing
 
1. A method of operating a robot system including a robot body, the method comprising:
capturing, by at least one sensor of the robot system, sensor data representing information about an environment of the robot body;
generating, by at least one processor of the robot system, a natural language (NL) description of at least one aspect of the environment based on the sensor data;
providing a NL query to a large language model (LLM) module, the NL query including the NL description of at least one aspect of the environment, an NL description of a work objective, an NL description of an instruction set executable by the robot system, and an NL request for a task plan;
receiving the task plan from the LLM module, the task plan expressed in NL; and
executing the task plan by the robot system.