US 11,836,179 B1
Multimedia query system
Vincent Charles Cheung, San Carlos, CA (US); Tali Zvi, San Carlos, CA (US); Kent Austin White, Mountain View, CA (US); and Kevin Lin, Cupertino, CA (US)
Assigned to Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed by Meta Platforms Technologies, LLC, Menlo Park, CA (US)
Filed on Oct. 29, 2019, as Appl. No. 16/667,494.
Int. Cl. G06F 7/00 (2006.01); G06F 16/432 (2019.01); G06F 16/41 (2019.01); G06F 16/487 (2019.01); G06N 3/08 (2023.01); G06F 16/48 (2019.01); G06F 16/483 (2019.01)
CPC G06F 16/432 (2019.01) [G06F 16/41 (2019.01); G06F 16/483 (2019.01); G06F 16/487 (2019.01); G06F 16/489 (2019.01); G06N 3/08 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A multimedia query system comprising:
a multimedia capture system configured to capture raw multimedia data comprising at least one of raw video data or raw audio data;
a metadata engine configured to extract one or more anchor points of metadata from the raw multimedia data and to store the one or more anchor points of metadata, wherein the anchor points of metadata includes references to respective portions of the raw multimedia data;
a storage engine configured to store the raw multimedia data;
a recall engine configured to receive a query and to apply the query to the one or more anchor points of metadata to identify one or more raw multimedia data candidates from the portions of the raw multimedia data, wherein the one or more raw multimedia data candidates include information related to the query; and
a query engine comprising a multimedia artificial intelligence model trained to generate a response to the query by processing the one or more raw multimedia data candidates to infer the response, wherein the query engine is configured to output the response.