CPC B25J 9/1697 (2013.01) [A47B 81/04 (2013.01); B25J 9/1664 (2013.01); B25J 11/0045 (2013.01); G06F 18/24 (2023.01); G06T 7/73 (2017.01); H04N 7/188 (2013.01); H04N 23/56 (2023.01); A47B 2220/0077 (2013.01); G06T 2207/10048 (2013.01); G06V 20/68 (2022.01)] | 16 Claims |
1. A system for perceiving a dish, planning its handling, and controlling its motion, comprising:
a. a cabinet holding said dish collocated with a plurality of other dishes;
b. a diffuse light source for illuminating dishes and a structured light source for illuminating dishes with a structured pattern;
c. a camera for capturing a diffuse image of said dishes when illuminated with said diffuse light source and a camera for capturing a structured image of said dishes when illuminated with said structured light source;
d. a robotic arm to pick up, hold, move or drop-off said dish;
e. a processor configured to:
i. classify said diffuse image using a dish detection model to determine the presence of a dish;
ii. classify said diffuse image using a dish identification model to determine the type of said dish;
iii. capture said structured image when at least one dish is placed, removed or relocated;
iv. estimate the position and orientation of said dish and the positions and orientations of dishes neighboring to said dish using a dish localization model with said diffuse image and said structured image;
v. estimate a collision-free trajectory for said robotic arm using a planning model with said positions and orientations of said dish and its neighbors,
whereby said dish is detected, identified, and localized to securely move it from one location to another.
|