US 11,836,970 B2
Tracking objects with changing appearances
Daniel Ribeiro Silva, San Jose, CA (US); and Jinmeng Rao, Madison, WI (US)
Assigned to MINERAL EARTH SCIENCES LLC, Mountain View, CA (US)
Filed by Mineral Earth Sciences LLC, Mountain View, CA (US)
Filed on Dec. 22, 2021, as Appl. No. 17/558,928.
Prior Publication US 2023/0196754 A1, Jun. 22, 2023
Int. Cl. G06V 10/84 (2022.01); G06V 10/75 (2022.01); G06V 10/82 (2022.01); G06V 20/68 (2022.01)
CPC G06V 10/84 (2022.01) [G06V 10/757 (2022.01); G06V 10/82 (2022.01); G06V 20/68 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A method implemented using one or more processors, comprising:
determining a first probability distribution over a plurality of classes of a first biological object depicted in a first image captured at a first point in time, wherein the plurality of classes represent stages of growth of biological objects;
determining one or more additional probability distributions over the plurality of classes of one or more candidate biological objects depicted in a second image, wherein the one or more candidate biological objects depicted in the second image potentially match the first biological object depicted in the first image, and wherein the second image is captured at a second point in time subsequent to the first point in time;
based on a time interval between the first and second points in time, comparing the first probability distribution of the first biological object to the one or more probability distributions of the one or more candidate biological objects depicted in the second image; and
based on the comparing, matching a given biological object of the one or more candidate biological objects depicted in the second image to the first biological object depicted in the first image;
wherein determining the probability distribution over the plurality of classes of the first biological object comprises applying at least a portion of the first image as input across a probability distribution classifier (PDC) machine learning model to generate the probability distribution over the plurality of classes; and
wherein the PDC machine learning model is trained based at least in part using a temporal sequence of images capturing a reference biological object, wherein a subset of keyframes selected from the temporal sequence are manually labeled with probability distributions over the plurality of classes, and images of the temporal sequence other than the keyframes in the subset are labeled automatically using interpolation based on timestamps of the keyframes and other images.