US 11,887,372 B2
Image-based self-serve beverage detection and assignment
Shahmeer Ali Mirza, Celina, TX (US); Sailesh Bharathwaaj Krishnamurthy, Irving, TX (US); Kyle Dalal, Coppell, TX (US); and Xinan Wang, Dallas, TX (US)
Assigned to 7-ELEVEN, INC., Irving, TX (US)
Filed by 7-Eleven, Inc., Irving, TX (US)
Filed on Nov. 25, 2020, as Appl. No. 17/104,841.
Application 17/104,841 is a continuation in part of application No. 16/663,710, filed on Oct. 25, 2019.
Application 16/663,710 is a continuation in part of application No. 16/663,766, filed on Oct. 25, 2019.
Application 16/663,766 is a continuation in part of application No. 16/663,451, filed on Oct. 25, 2019.
Application 16/663,451 is a continuation in part of application No. 16/663,794, filed on Oct. 25, 2019.
Application 16/663,794 is a continuation in part of application No. 16/663,822, filed on Oct. 25, 2019.
Application 16/663,822 is a continuation in part of application No. 16/941,415, filed on Jul. 28, 2020.
Application 16/941,415 is a continuation of application No. 16/794,057, filed on Feb. 18, 2020, granted, now 10,769,451, issued on Sep. 8, 2020.
Application 16/794,057 is a continuation of application No. 16/663,472, filed on Oct. 25, 2019, granted, now 10,614,318, issued on Apr. 7, 2020.
Application 16/794,057 is a continuation of application No. 17/104,841.
Application 17/104,841 is a continuation in part of application No. 16/663,856, filed on Oct. 25, 2019.
Application 16/663,856 is a continuation in part of application No. 16/664,160, filed on Oct. 25, 2019.
Application 16/664,160 is a continuation in part of application No. 17/071,262, filed on Oct. 15, 2020.
Application 17/071,262 is a continuation of application No. 16/857,990, filed on Apr. 24, 2020, granted, now 10,853,663.
Application 16/857,990 is a continuation of application No. 16/793,998, filed on Feb. 18, 2020, granted, now 10,685,237, issued on Jun. 16, 2020.
Application 16/793,998 is a continuation of application No. 16/663,500, filed on Oct. 25, 2019, granted, now 10,621,444, issued on Apr. 14, 2020.
Application 16/793,998 is a continuation of application No. 17/104,841.
Application 17/104,841 is a continuation in part of application No. 16/857,990, filed on Apr. 24, 2020, granted, now 10,853,663.
Application 16/857,990 is a continuation of application No. 16/793,998, filed on Feb. 18, 2020, granted, now 10,685,237, issued on Jun. 16, 2020.
Application 16/793,998 is a continuation of application No. 16/663,500, filed on Oct. 25, 2019, granted, now 10,621,444, issued on Apr. 14, 2020.
Application 16/793,998 is a continuation of application No. 17/104,841.
Application 17/104,841 is a continuation in part of application No. 16/664,219, filed on Oct. 25, 2019.
Application 16/664,219 is a continuation in part of application No. 16/664,269, filed on Oct. 25, 2019.
Application 16/664,269 is a continuation of application No. 16/664,332, filed on Oct. 25, 2019.
Application 16/664,332 is a continuation in part of application No. 16/664,363, filed on Oct. 25, 2019.
Application 16/664,363 is a continuation in part of application No. 16/664,391, filed on Oct. 25, 2019.
Application 16/664,391 is a continuation in part of application No. 16/664,426, filed on Oct. 25, 2019.
Application 16/664,426 is a continuation in part of application No. 16/884,434, filed on May 27, 2020.
Application 16/884,434 is a continuation of application No. 16/663,533, filed on Oct. 25, 2019, granted, now 10,789,720, issued on Sep. 29, 2020.
Application 16/884,434 is a continuation of application No. 17/104,841.
Application 17/104,841 is a continuation in part of application No. 16/663,901, filed on Oct. 25, 2019.
Application 16/663,901 is a continuation in part of application No. 16/663,948, filed on Oct. 25, 2019.
Prior Publication US 2021/0124946 A1, Apr. 29, 2021
Int. Cl. G06V 20/52 (2022.01); G06T 7/292 (2017.01); G06V 20/40 (2022.01); G06V 10/44 (2022.01); G06V 20/62 (2022.01)
CPC G06V 20/52 (2022.01) [G06T 7/292 (2017.01); G06V 10/44 (2022.01); G06V 20/41 (2022.01); G06V 20/62 (2022.01); G06T 2207/30208 (2013.01); G06V 20/44 (2022.01); G06V 2201/07 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A system, comprising:
a beverage machine operable to dispense a beverage;
an image sensor positioned such that a field-of-view of the image sensor encompasses at least a portion of the beverage machine, wherein the image sensor is configured to generate angled-view images of at least a portion of the beverage machine, wherein the portion includes a first zone associated with operating the beverage machine to dispense the beverage and a second zone in which a cup is placed to receive the beverage from the beverage machine; and
a beverage assignment subsystem coupled to the image sensor, the beverage assignment subsystem comprising at least one processor configured to:
receive an image feed comprising frames of the angled-view images generated by the image sensor;
detect an event associated with an object entering one or both of the first zone and the second zone;
determine, in a first one or more image frames associated with a start of the detected event, that both a hand of a person enters the first zone and the cup is placed in the second zone;
detect that the cup is removed from the second zone;
following detecting that the cup is removed from the second zone, determine a length of time during which the cup remained in the second zone;
if the cup remained in the second zone for at least a threshold time, assign the beverage to the person whose hand entered the first zone;
if the cup did not remain in in the second zone for at least the threshold time, do not assign the beverage to the person whose hand entered the first zone;
wherein the processor is further configured to, in response to detecting the event, identify frames of images from the image feed associated with the detected event, wherein the identified image frames include:
the first one or more image frames associated with the start of the detected event, wherein the first one or more image frames include images from the image feed from a first predefined time period before the detected event until a second predefined time period after the detected event; and
a second one or more image frames associated with removal of the cup from the second zone, wherein the second one or more image frames include images from the image feed from a third predefined time period before the cup is removed from the second zone until a fourth predefined time period after the cup is removed from the second zone.