US 12,437,004 B2
Deep multi-modal pairwise ranking model for crowdsourced food data
Surender Reddy Yerva, Austin, TX (US); Iman Barjasteh, Austin, TX (US); Patrick Howell, Austin, TX (US); Chul Lee, Austin, TX (US); and Hesamoddin Salehian, Austin, TX (US)
Assigned to MyFitnessPal, Inc., Austin, TX (US)
Filed by MYFITNESSPAL, INC., Austin, TX (US)
Filed on Jan. 12, 2024, as Appl. No. 18/412,034.
Application 18/412,034 is a continuation of application No. 17/459,404, filed on Aug. 27, 2021, granted, now 11,874,879.
Application 17/459,404 is a continuation of application No. 16/354,863, filed on Mar. 15, 2019, granted, now 11,106,742, issued on Aug. 31, 2021.
Claims priority of provisional application 62/643,919, filed on Mar. 16, 2018.
Prior Publication US 2024/0232266 A1, Jul. 11, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 16/906 (2019.01); G06F 16/903 (2019.01); G06F 16/908 (2019.01); G06N 3/049 (2023.01); G06N 20/10 (2019.01); G16H 20/60 (2018.01)
CPC G06F 16/906 (2019.01) [G06F 16/90344 (2019.01); G06F 16/908 (2019.01); G06N 3/049 (2013.01); G06N 20/10 (2019.01); G16H 20/60 (2018.01)] 20 Claims
OG exemplary drawing
 
1. A method of operating a health tracking system having a processor and a database configured to store a plurality of data records, each of the plurality of data records comprising at least a descriptive string and nutritional data regarding a respective consumable item, the method comprising:
receiving, with the processor, a query string;
retrieving, with the processor, a first data record of the plurality of data records and a second data record of the plurality of data records from the database;
generating, with the processor, (i) a first nutrition information vector from the nutritional data of the first data record and (ii) a second nutrition information vector from the nutritional data of the second data record;
generating, with the processor, at least one feature vector using at least one first embedding function of a machine learning model, the at least one first embedding function being learned in a training process of the machine learning model;
generating, with the processor, a third nutrition information vector based on the query string, using a second embedding function of the machine learning model, the second embedding function being learned in the training process of the machine learning model, wherein the at least one first embedding function and the second embedding function each include a different Long Short Term Memory (LSTM); and
determining, with the processor, which of the first data record and the second data record is more relevant to the query string based at least in part on the first nutrition information vector, the second nutrition information vector, and the third nutrition information vector, and the at least one feature vector.