US 12,217,240 B1
Bar code detection and authentication for contactless transactions using wearable multimedia device
Imran A. Chaudhri, San Francisco, CA (US); Patrick Gates, San Francisco, CA (US); Bethany Bongiorno, San Francisco, CA (US); Yanir Nulman, San Francisco, CA (US); and Monique Relova, South San Francisco, CA (US)
Assigned to Humane, Inc., San Francisco, CA (US)
Filed by Humane, Inc., San Francisco, CA (US)
Filed on May 28, 2021, as Appl. No. 17/334,645.
Claims priority of provisional application 63/032,520, filed on May 29, 2020.
Int. Cl. G06Q 20/32 (2012.01); G06Q 20/40 (2012.01)
CPC G06Q 20/3276 (2013.01) [G06Q 20/321 (2020.05); G06Q 20/4014 (2013.01)] 20 Claims
OG exemplary drawing
 
17. A method comprising:
capturing, using a wide field-of-view camera of a wearable multimedia device worn by a user, an image of a scene that includes a plurality of unlabeled objects and a user gesture, wherein the wearable multimedia device comprises a housing, the wide field-of-view camera embedded in the housing, a depth sensor, one or more processors, and a memory storing instructions that are executed by the one or more processors;
capturing, by the one or more processors using the depth sensor, depth data of the scene;
performing, by the one or more processors, semantic segmentation on the image of the scene to predict an object mask for each unlabeled object;
performing, by the one or more processors, instance segmentation on pixel data within the each object mask;
labeling, by the one or more processors based on the instance segmentation, that one of the plurality of unlabeled objects is a contactless terminal device;
associating, by the one or more processors with sensor fusion, an intent of the user to engage with the contactless terminal device based at least in part on the user gesture, the depth data and the labeled contactless terminal device, where the user gesture is the user pointing in a direction of the contactless terminal device;
establishing, by the one or more processors, a communication channel between the wearable multimedia device and the contactless terminal device;
receiving, by the one or more processors using the communication channel, data from the contactless terminal device;
responsive to the received data, sending, using the communication channel, authentication credentials to the contactless terminal device;
receiving, by the one or more processors using the communication channel, access from the contactless terminal device based at least in part on the authentication credentials; and
responsive to the received access to the contactless terminal device, performing, by the one or more processors, an interaction with the contactless terminal device.