US 12,236,628 B1
Detecting and confirming interactions by proximity and video
Saranya Karuppusamy, Seattle, WA (US); Ao Yan, Seattle, WA (US); Liem Dinh, Seattle, WA (US); Peiran Zhou, Seattle, WA (US); Sabarishkumar Iyer, Seattle, WA (US); Waqas Ahmed, Clyde Hill, WA (US); Helene Causse, Seattle, WA (US); and Danny Guan, Seattle, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Jun. 27, 2019, as Appl. No. 16/454,805.
Int. Cl. G06T 7/292 (2017.01); G06T 7/70 (2017.01)
CPC G06T 7/70 (2017.01) [G06T 7/292 (2017.01); G06T 2207/30196 (2013.01); G06T 2207/30232 (2013.01)] 22 Claims
OG exemplary drawing
 
1. A materials handling facility comprising:
a first digital camera having a first field of view;
a second digital camera having a second field of view;
a storage unit comprising a shelf, wherein at least a portion of the shelf is within each of the first field of view and the second field of view;
a first container of non-discretized items disposed on the portion of the shelf, and
a server in communication with the first digital camera and the second digital camera, wherein the server comprises at least one processor configured to at least:
establish a product space associated with the first container, wherein the product space is defined by a set of points in three-dimensional space including the first container;
receive a first plurality of images from the first digital camera, wherein each of the first plurality of images was captured during a first period of time;
receive a second plurality of images from the second digital camera, wherein each of the second plurality of images was captured during the first period of time;
determine a first trajectory of a first customer within the materials handling facility during at least a first portion of the first period of time;
determine a second trajectory of a second customer within the materials handling facility during at least a second portion of the first period of time;
determine, based at least in part on the first trajectory and the second trajectory, locations of the first customer and the second customer at each of a plurality of times during the first period of time, wherein each of the plurality of times is separated by a predetermined interval;
determine, at a first time in accordance with the predetermined interval, that at least one of the first customer or the second customer is within a predetermined range of the product space, wherein the first time is within the first period of time; and
in response to determining that at least one of the first customer or the second customer is within the predetermined range of the product space,
retrieve at least a first subset of the first plurality of images, wherein the first subset of the first plurality of images is captured over a second period of time including the first time;
retrieve at least a second subset of the second plurality of images, wherein the second subset of the second plurality of images is captured over the second period of time; and
cause a display of a user interface on a computer display, wherein the user interface is configured to display a video feed comprising at least one of the first subset of the first plurality of images or the second subset of the first plurality of images.