US 11,869,535 B1
Character-level emotion detection
Mohammad Taha Bahadori, Vancouver (CA); Viktor Rozgic, Belmont, MA (US); Alexander Jonathan Pinkus, Seattle, WA (US); Chao Wang, Newton, MA (US); and David Heckerman, Bellevue, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Dec. 12, 2019, as Appl. No. 16/711,883.
Int. Cl. G10L 25/63 (2013.01); G10L 15/22 (2006.01); G10L 15/16 (2006.01); G10L 15/06 (2013.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01); G10L 15/18 (2013.01); G06N 3/044 (2023.01)
CPC G10L 25/63 (2013.01) [G06N 3/044 (2023.01); G06N 3/08 (2013.01); G10L 15/063 (2013.01); G10L 15/16 (2013.01); G10L 15/1815 (2013.01); G10L 15/22 (2013.01); G10L 2015/223 (2013.01)] 23 Claims
OG exemplary drawing
 
1. A computing system, comprising:
one or more processors; and
a memory storing program instructions that when executed by the one or more processors cause the one or more processors to at least:
receive a goal for a first user;
generate, based at least in part on the goal, a threshold value for use in determining if the first user is achieving the goal;
receive audio data that includes speech;
process the audio data, without determining words included in the speech, to determine a first segment of the speech corresponding to the first user and a second segment of the speech corresponding to a second user;
process the first segment of the speech to determine character data that includes a sequence of characters representative of the speech corresponding to the first user;
process, with a neural network, the character data to produce sentiment data representative of a determined emotional state of the first user;
determine, based at least in part on the sentiment data, that the sentiment data of the first user does not exceed the threshold value corresponding to the goal;
in response to determination that the sentiment data does not exceed the threshold value, determine, based at least in part on the sentiment data and user status data corresponding to the first user, advisory data, wherein the advisory data includes at least:
an indication that the first user is not achieving the goal; and
a recommended action that the first user may follow in an effort to achieve the goal;
provide at least a portion of the advisory data for presentation to the first user.