US 12,175,999 B2
Audio response messages
Gurunandan Krishnan Gorumkonda, Kirkland, WA (US); and Shree K. Nayar, New York, NY (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Dec. 22, 2021, as Appl. No. 17/559,854.
Application 17/559,854 is a continuation of application No. 16/418,638, filed on May 21, 2019, granted, now 11,227,626.
Claims priority of provisional application 62/674,410, filed on May 21, 2018.
Prior Publication US 2022/0115034 A1, Apr. 14, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G10L 15/22 (2006.01); G06F 3/16 (2006.01); G06N 3/08 (2023.01); G10L 25/84 (2013.01)
CPC G10L 25/84 (2013.01) [G06F 3/167 (2013.01); G06N 3/08 (2013.01); G10L 15/22 (2013.01); G10L 2015/223 (2013.01); G10L 2015/226 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
receiving, at a server, a first message indicating an image and content data associated with a non-verbal audio interaction;
generating, at the server, a second message comprising the image, the content data associated with the non-verbal audio interaction, and an indication to perform the non-verbal audio interaction; and
sending the second message to a user device, the user device being configured to: display the image and the indication on a display of the user device, generate sound data from a microphone of the user device while the message is being displayed on the user device, generate a sound classification by applying a convolutional neural network that is trained to detect non-verbal sounds to the sound data, determine, using the sound classification, that the sound data corresponds to the non-verbal audio interaction, and display the content data on the display in response to determining that the sound data corresponds to the non-verbal audio interaction.