US 11,942,208 B2
Food-recognition systems and methods
Dmitriy R. Starson, Palo Alto, CA (US); James Kelly, Lancashire (GB); and Zvi Ashkenazi, Prescott, AZ (US)
Assigned to PASSIO INC., Menlo Park, CA (US)
Filed by Passio, Inc., Menlo Park, CA (US)
Filed on May 20, 2021, as Appl. No. 17/325,475.
Claims priority of provisional application 63/113,688, filed on Nov. 13, 2020.
Claims priority of provisional application 63/027,501, filed on May 20, 2020.
Prior Publication US 2021/0365687 A1, Nov. 25, 2021
Int. Cl. G16H 20/60 (2018.01); G06F 18/243 (2023.01); G06N 3/08 (2023.01); G06V 10/82 (2022.01); G06V 20/40 (2022.01); G06V 20/68 (2022.01); G06V 30/19 (2022.01); G06V 30/224 (2022.01); G06V 30/412 (2022.01)
CPC G16H 20/60 (2018.01) [G06F 18/24323 (2023.01); G06N 3/08 (2013.01); G06V 10/82 (2022.01); G06V 20/46 (2022.01); G06V 20/68 (2022.01); G06V 30/19173 (2022.01); G06V 30/2247 (2022.01); G06V 30/412 (2022.01)] 26 Claims
OG exemplary drawing
 
1. A food-recognition method, comprising:
inputting each frame, of a plurality of frames of a video stream, into a multiple-object detector to obtain (i) a bounding box identifying where each food item, of one or more food items, appears within said each frame, and (ii) a predicted multiple-object class identifying said each food item with a multiple-object probability;
for each bounding box:
cropping said each frame into a cropped frame based on said each bounding box;
inputting the cropped frame into a classifier to obtain one or more predicted classifier classes that identify said each food item with corresponding one or more classifier probabilities; and
determining a food identity of said each food item based on the predicted multiple-object class, the multiple-object probability, the one or more predicted classifier classes, and the corresponding one or more classifier probabilities;
creating a frame list of the one or more food identities determined from said each frame;
enqueuing at least one of the one or more food identities in the frame list into a most-viewed queue of elements to create an updated most-viewed queue; and
outputting at least one of the one or more food identities in the updated most-viewed queue.