US 12,450,617 B1
Learning for individual detection in brick and mortar store based on sensor data and feedback
Marvin Balaoro, San Francisco, CA (US); Brett Andler, San Francisco, CA (US); and Nikolaj Leschly, Alameda, CA (US)
Assigned to Block, Inc., Oakland, CA (US)
Filed by Block, Inc., Oakland, CA (US)
Filed on Jun. 24, 2021, as Appl. No. 17/357,821.
Claims priority of provisional application 63/191,906, filed on May 21, 2021.
Int. Cl. G06Q 30/018 (2023.01); G06F 18/214 (2023.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01)
CPC G06Q 30/018 (2013.01) [G06F 18/214 (2023.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for individual identification, the system comprising:
one or more memory units storing instructions; and
one or more processors, wherein execution of the instructions by the one or more processors causes the one or more processors to:
receive sensor data captured by one or more sensors, wherein the sensor data includes at least one image of a brick-and-mortar (BAM) area captured by at least one camera that has a field of view covering at least part of the BAM area associated with a merchant;
automatically identify, in the sensor data, a first set of features of a representation of an individual that is in the BAM area;
automatically compare, using a trained machine learning model, the first set of features to a plurality of stored reference features to determine an identity of the individual;
generate, using the trained machine learning model, a confidence level associated with the determined identity, wherein the confidence level is based on a comparison between the first set of features and a second set of features associated with the determined identity, wherein the plurality of stored reference features include the second set of features;
determine that the confidence level is below a threshold;
automatically transmit, to a feedback device and in response to determining that the confidence level is below the threshold, a request for feedback regarding the determined identity of the individual;
receive, through a user interface of the feedback device, the feedback in response to the request, wherein the feedback includes a confirmation that the identity of the individual is correct as determined by the trained machine learning model;
automatically update the trained machine learning model to improve a confidence of the trained machine learning model in identifying the individual by further training the trained machine learning model based on at least the confirmation in the feedback, the first set of features, and the second set of features;
automatically track, using the one or more sensors and in response to the confirmation in the feedback, the individual in the BAM area to identify a condition associated with the individual; and
automatically activate a device in the BAM area in response to identifying the condition associated with the individual, wherein the device is within a threshold distance of the individual.