| CPC B25J 9/1661 (2013.01) | 20 Claims |
|
1. A method of operating a robot system including a robot body, the method comprising:
capturing, by at least one sensor of the robot system, sensor data representing information about an environment of the robot body;
generating, by at least one processor of the robot system, a natural language (NL) description of at least one aspect of the environment based on the sensor data;
providing an NL query to a large language model (LLM) module, the NL query including the NL description of at least one aspect of the environment, an NL description of a work objective, an NL description of an instruction set which includes a plurality of instructions each selectively executable by the robot system, an NL request for a task plan, and an NL indication of a task plan step threshold corresponding to a maximum number of steps in the task plan;
receiving the task plan from the LLM module, the task plan expressed in NL and including a number of steps which does not exceed the task plan step threshold;
for each step of the task plan which does not correspond to a single instruction in the instruction set:
providing an NL breakdown query for a respective component plan to the LLM module, the NL breakdown query including an NL request to break down the step of the task plan into a component plan including a plurality of component steps;
receiving each component plan from the LLM module, each component plan expressed in NL; and
incorporating each component plan into the task plan; and
executing the task plan by the robot system.
|