US 11,839,983 B2
Systems and methods for robotic grasp verification
Ryan John Dick, West Toronto (CA); James Sterling Bergstra, West Toronto (CA); and Lavi Shpigelman, West Toronto (CA)
Assigned to Ocado Innovation Limited, Hatfield (GB)
Filed by Ocado Innovation Limited, Hatfield (GB)
Filed on Nov. 26, 2019, as Appl. No. 16/696,053.
Claims priority of provisional application 62/771,911, filed on Nov. 27, 2018.
Prior Publication US 2020/0164517 A1, May 28, 2020
Int. Cl. B25J 9/16 (2006.01); G06N 20/00 (2019.01); G06N 3/04 (2023.01)
CPC B25J 9/1682 (2013.01) [B25J 9/1697 (2013.01); G06N 3/04 (2013.01); G06N 20/00 (2019.01)] 19 Claims
OG exemplary drawing
 
1. A computer-implemented method of controlling a robotic apparatus for manipulating objects, comprising:
at a time window at least partially overlapping with a first point in time, obtaining first sensor data indicating at least a portion of an environment where the robotic apparatus resides;
feeding the first sensor data to at least one convolutional neural network (CNN) to generate a first output that feeds into at least one long short-term memory (LSTM) network;
at a time window at least partially overlapping with a second point in time that succeeds the first point in time, obtaining second sensor data indicating at least a portion of the environment;
feeding the second sensor data to the at least one CNN to generate a second output that feeds into the at least one LSTM network, the LSTM network further receiving feedback to itself based on a state of the LSTM achieved from the first point in time;
determining a robotic action based, at least in part, on an output of the at least one LSTM network; and
causing the robotic apparatus to perform the robotic action.