US 12,288,341 B2
System and methods for processing spatial data
Mark Buckler, Ithaca, NY (US); and Adrian Sampson, Ithaca, NY (US)
Assigned to Cornell University, Ithaca, NY (US)
Filed by Cornell University, Ithaca, NY (US)
Filed on Sep. 15, 2020, as Appl. No. 17/021,923.
Application 17/021,923 is a continuation of application No. PCT/US2018/029402, filed on Apr. 25, 2018.
Application 17/021,923 is a continuation of application No. PCT/US2018/029404, filed on Apr. 25, 2018.
Application 17/021,923 is a continuation of application No. PCT/US2018/029407, filed on Apr. 25, 2018.
Claims priority of provisional application 62/644,147, filed on Mar. 16, 2018.
Prior Publication US 2020/0410352 A1, Dec. 31, 2020
Int. Cl. G06F 9/50 (2006.01); G06F 18/21 (2023.01); G06N 3/063 (2023.01); G06N 3/08 (2023.01); G06T 7/20 (2017.01); H03M 7/30 (2006.01); H03M 7/46 (2006.01)
CPC G06T 7/20 (2013.01) [G06F 9/5027 (2013.01); G06F 18/2163 (2023.01); G06N 3/063 (2013.01); G06N 3/08 (2013.01); H03M 7/46 (2013.01); H03M 7/6005 (2013.01); H03M 7/6011 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A method for processing spatial data comprising:
configuring at least one processor to implement at least one neural network comprising at least a first neural network portion and a second neural network portion different than the first neural network portion;
processing a first spatial data set in the first neural network portion of the neural network to generate a first plurality of neural network outputs for the first spatial data set;
generating a plurality of predicted neural network outputs of the first neural network portion for a second spatial data set different than the first spatial data set, based at least in part on motion estimation performed in the at least one processor between at least portions of the first and second spatial data sets, the plurality of predicted neural network outputs being generated utilizing at least a portion of the first plurality of neural network outputs generated by the first neural network portion for the first spatial data set; and
processing in the second neural network portion at least respective subsets of both (i) the first plurality of neural network outputs generated by the first neural network portion for the first spatial data set and (ii) the plurality of predicted neural network outputs of the first neural network portion generated for the second spatial data set;
wherein the processing of the first spatial data set in the first neural network portion and the generating of the plurality of predicted neural network outputs further comprise:
receiving the first spatial data set and dividing the first spatial data set into a first plurality of receptive fields;
processing, by the first neural network portion, the first plurality of receptive fields to obtain the first plurality of neural network outputs, wherein each neural network output corresponds to a receptive field in the first plurality of receptive fields;
storing, in memory, the first plurality of neural network outputs;
receiving the second spatial data set and dividing the second spatial data set into a second plurality of receptive fields;
identifying, in the at least one processor, for each receptive field in the second plurality of receptive fields a prior location in the first spatial data set, wherein the prior location is between multiple receptive field locations in the first plurality of receptive fields;
obtaining, from memory, the neural network outputs corresponding to the receptive fields of the first plurality of receptive fields proximate to the identified prior location for each receptive field in the second plurality of receptive fields; and
calculating, in the at least one processor, the plurality of predicted neural network outputs for the second plurality of receptive fields by, for a given receptive field in the second plurality of receptive fields, interpolating between the neural network outputs corresponding to the receptive fields proximate to the prior location of the given receptive field in the first spatial data set;
wherein the method further comprises:
performing at least one computer vision task based at least in part on results of the processing, in the second neural network portion, of the at least respective subsets of both the first plurality of neural network outputs and the plurality of predicted neural network outputs.