US 11,938,640 B2
Dish perception, planning and control for robots
Sri Rama Prasanna Pavani, San Carlos, CA (US)
Assigned to Dishcare Inc., San Carlos, CA (US)
Filed by Sri Rama Prasanna Pavani, San Carlos, CA (US)
Filed on Mar. 15, 2021, as Appl. No. 17/202,266.
Prior Publication US 2022/0288789 A1, Sep. 15, 2022
Int. Cl. B25J 9/16 (2006.01); A47B 81/04 (2006.01); B25J 11/00 (2006.01); G06F 18/24 (2023.01); G06T 7/73 (2017.01); G06V 20/68 (2022.01); H04N 7/18 (2006.01); H04N 23/56 (2023.01)
CPC B25J 9/1697 (2013.01) [A47B 81/04 (2013.01); B25J 9/1664 (2013.01); B25J 11/0045 (2013.01); G06F 18/24 (2023.01); G06T 7/73 (2017.01); H04N 7/188 (2013.01); H04N 23/56 (2023.01); A47B 2220/0077 (2013.01); G06T 2207/10048 (2013.01); G06V 20/68 (2022.01)] 16 Claims
OG exemplary drawing
 
1. A system for perceiving a dish, planning its handling, and controlling its motion, comprising:
a. a cabinet holding said dish collocated with a plurality of other dishes;
b. a diffuse light source for illuminating dishes and a structured light source for illuminating dishes with a structured pattern;
c. a camera for capturing a diffuse image of said dishes when illuminated with said diffuse light source and a camera for capturing a structured image of said dishes when illuminated with said structured light source;
d. a robotic arm to pick up, hold, move or drop-off said dish;
e. a processor configured to:
i. classify said diffuse image using a dish detection model to determine the presence of a dish;
ii. classify said diffuse image using a dish identification model to determine the type of said dish;
iii. capture said structured image when at least one dish is placed, removed or relocated;
iv. estimate the position and orientation of said dish and the positions and orientations of dishes neighboring to said dish using a dish localization model with said diffuse image and said structured image;
v. estimate a collision-free trajectory for said robotic arm using a planning model with said positions and orientations of said dish and its neighbors,
whereby said dish is detected, identified, and localized to securely move it from one location to another.