US 12,070,860 B2
Machine learning based decision making for robotic item handling
Matthew R. Wicks, Saint Charles, MO (US); Dikshya Swain, St. Louis, MO (US); Abhijit Makhal, St. Louis, MO (US); and Dominic G. Clucas, Saint Peters, MO (US)
Assigned to INTELLIGRATED HEADQUARTERS, LLC, Mason, OH (US)
Filed by INTELLIGRATED HEADQUARTERS, LLC, Mason, OH (US)
Filed on Sep. 14, 2020, as Appl. No. 17/019,546.
Prior Publication US 2022/0080584 A1, Mar. 17, 2022
Int. Cl. B25J 9/00 (2006.01); B25J 9/16 (2006.01); B25J 15/06 (2006.01); G05B 13/02 (2006.01)
CPC B25J 9/163 (2013.01) [B25J 9/0093 (2013.01); B25J 9/1697 (2013.01); B25J 15/0616 (2013.01); G05B 13/027 (2013.01)] 11 Claims
OG exemplary drawing
 
1. A method for controlling a robotic item handler comprising, the method comprising:
obtaining first point cloud data related to a first three-dimensional image captured by a first sensor device of the robotic item handler;
obtaining second point cloud data related to a second three-dimensional image captured by a second sensor device of the robotic item handler;
transforming the first point cloud data and the second point cloud data to generate combined point cloud data;
constructing a machine learning model based on the combined point cloud data as an input to a convolution neural network;
outputting, via the machine learning model, a decision classification indicative of a first probability associated with a first operating mode and a second probability associated with a second operating mode, wherein:
(i) the first operating mode is associated with picking an item by grasping the item using an end effector of a robotic arm of the robotic item handler, and
(ii) the second operating mode is associated with sweeping a pile of items from an item docking station with a platform of the robotic item handler;
selecting one of the first operating mode and the second operating mode of the robotic item handler based on the decision classification outputted by the machine learning model, wherein:
(i) the first operating mode is selected in response to the first probability being higher than the second probability, and
(ii) the second operating mode is selected in response to the second probability being higher than the first probability;
operating the robotic item handler according to the selection of the one of the first operating mode and the second operating mode;
evaluating the selection of the one of: the first operating mode and the second operating mode of the robotic item handler based on a pre-defined heuristic associated with past operations of the robotic item handler;
adjusting a first weight associated with the decision classification and a second weight associated with the pre-defined heuristic used for the evaluation of the selection, based on a performance associated with the output of the machine learning model over a period of time, wherein during initial stages of learning by the machine learning model, the first weight is less than the second weight and after a substantial period of time is utilized to train the machine learning model, the second weight is less than the first weight; and
controlling the robotic item handler based on the evaluation of the selection.