US 12,392,548 B2
System and method for tracking refrigerator inventory
John Anderson, Brooklyn, NY (US); Tyler Sanborn, Queens, NY (US); Boris Kontorovich, Brooklyn, NY (US); Charles Parks, Laramie, WY (US); Jennifer O'Neil, Boulder, CO (US); Bryce Copenhaver, Brooklyn, NY (US); Robert Marsh, Ponte Vedra Beach, FL (US); and Travis Ruddy, Dayton, OH (US)
Assigned to Springhouse Technologies Inc., McLean, VA (US)
Filed by Springhouse Technologies Inc., McLean, VA (US)
Filed on Sep. 29, 2023, as Appl. No. 18/477,942.
Prior Publication US 2025/0109904 A1, Apr. 3, 2025
Int. Cl. F25D 29/00 (2006.01); G06Q 10/087 (2023.01); G06T 7/292 (2017.01); G06V 10/764 (2022.01); G06V 10/80 (2022.01); G06V 20/68 (2022.01); H04N 7/18 (2006.01); H04N 23/90 (2023.01)
CPC F25D 29/005 (2013.01) [G06Q 10/087 (2013.01); G06T 7/292 (2017.01); G06V 10/764 (2022.01); G06V 10/809 (2022.01); G06V 20/68 (2022.01); H04N 7/181 (2013.01); H04N 7/188 (2013.01); H04N 23/90 (2023.01); F25D 2500/06 (2013.01); F25D 2700/06 (2013.01); G06T 2207/10016 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A refrigerator comprising:
a main body and a door pivotally mounted to and movable with respect to the main body between an open position and a closed position, the main body and the door defining a plurality of storage compartments within a refrigerated enclosure when the door is in the closed position;
a plurality of cameras mounted to at least one of the main body and the door, each camera positioned to have a field of view that includes an entrance opening leading to at least one storage compartment when the door is in the open position, the plurality of cameras being configured to capture images of an object passing through the entrance opening as the object is being loaded into or is being removed from the at least one storage compartment; and
at least one computing device in operable connection with the plurality of cameras, the at least one computing device including:
a session controller module configured to start a session and to instruct each camera to begin capturing images based on a trigger signal received by the session controller;
a synchronizer module configured to receive captured images from the plurality of cameras and to group the captured images into frame sets, each frame set made up of respective captured images that were captured at a same time from different cameras among the plurality of cameras;
an object identification module configured to determine whether each captured image in a respective frame set contains the object and to assign a class identification to the object;
an object tracking module configured to analyze sequential captured images from each camera and to assign track identifications to associate the object in one analyzed image to the object in other analyzed images per camera;
an object merge module configured to compare the track identifications among the analyzed images from different cameras and to assign a merge identification to the object when the object is detected by the object identification module in captured images captured by multiple different cameras among the plurality of cameras;
a subsequence delineation module configured to split the session into subsequences by corresponding merge identifications received from the object merge module, wherein each subsequence corresponds to an inventory change for the at least one storage compartment;
a direction module configured to assign a direction to each subsequence based on the analyzed images to determine if the object is moving into, out of or internally within the refrigerated enclosure; and
an inventory update module configured to communicate with a database to update an inventory status of the at least one storage compartment based on each inventory change and the direction assigned thereto.