US 11,983,461 B2
Speech-based selection of augmented reality content for detected objects
Joseph Timothy Fortier, Los Angeles, CA (US); Celia Nicole Mourkogiannis, Los Angeles, CA (US); Evan Spiegel, Los Angeles, CA (US); and Kaveh Anvaripour, Santa Monica, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Mar. 24, 2021, as Appl. No. 17/211,321.
Claims priority of provisional application 63/000,071, filed on Mar. 26, 2020.
Prior Publication US 2021/0304451 A1, Sep. 30, 2021
Int. Cl. G06F 3/16 (2006.01); G06T 11/00 (2006.01); G06V 10/44 (2022.01); G06V 10/764 (2022.01); G06V 20/10 (2022.01); G06V 20/20 (2022.01); G06V 20/64 (2022.01); G06V 40/16 (2022.01); G10L 15/08 (2006.01); G10L 15/22 (2006.01); H04L 51/046 (2022.01); H04N 23/60 (2023.01)
CPC G06F 3/167 (2013.01) [G06T 11/00 (2013.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 20/10 (2022.01); G06V 20/20 (2022.01); G06V 20/64 (2022.01); G06V 40/161 (2022.01); G06V 40/168 (2022.01); G06V 40/174 (2022.01); G10L 15/08 (2013.01); G10L 15/22 (2013.01); H04L 51/046 (2013.01); H04N 23/60 (2023.01); G06T 2200/24 (2013.01); G10L 2015/088 (2013.01); G10L 2015/223 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A method, comprising:
causing, by a messaging application running on a device, a camera of the device to capture an image;
receiving by the messaging application, speech input to select augmented reality content for display with the image;
determining at least one keyword included in the speech input;
determining that the at least one keyword indicates an object depicted in the image and an action to perform with respect to the object;
determining first attributes of the object;
assigning weights to each of the first attributes of the object;
ranking plural augmented reality content items based on the assigned weights and on second attributes of the action;
selecting, based on the ranking, a highest-ranked augmented reality content item from among the plural augmented reality content items;
activating the highest-ranked augmented reality content item with respect to the image; and
displaying, in ranked order based on the ranking, an interface with user-selectable elements for activating remaining ones of the plural augmented reality content items.