US 12,443,822 B2
Context-based social agent interaction
Raymond J. Scanlon, Burbank, CA (US); Dawson Dill, Burbank, CA (US); Ashley N. Girdich, Glendale, CA (US); Robert P. Michel, Toluca Lake, CA (US); Komath Naveen Kumar, Los Angeles, CA (US); John J. Wiseman, Los Angeles, CA (US); and James R. Kennedy, Glendale, CA (US)
Assigned to Disney Enterprises, Inc., Burbank, CA (US)
Filed by Disney Enterprises, Inc., Burbank, CA (US)
Filed on Jun. 10, 2021, as Appl. No. 17/344,737.
Prior Publication US 2022/0398427 A1, Dec. 15, 2022
Prior Publication US 2025/0156675 A2, May 15, 2025
Int. Cl. G06N 3/006 (2023.01); G06T 13/40 (2011.01); G06T 19/00 (2011.01); G06T 19/20 (2011.01)
CPC G06N 3/006 (2013.01) [G06T 13/40 (2013.01); G06T 19/006 (2013.01); G06T 19/20 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for performing context-based social agent interactions with human beings, the system comprising:
a processing hardware;
a memory storing a software code;
a social agent instantiated as a robot, a virtual character, or a tabletop or wall-mounted device, the social agent comprising an output unit configured to effectuate an interactive expression of the social agent, the output unit comprising a display, a speaker, a mechanical actuator, or a haptic actuator;
at least one detector comprising at least one sensor or at least one microphone;
the processing hardware configured to execute the software code to:
detect, using the at least one detector, presence of a human being;
identify a present state of an interaction with the human being based on a first expression of the human being;
determine, based on scoring criteria and the present state, a first score for each of a plurality of interactive expressions for one of initiating or continuing the interaction to provide a plurality of first scores corresponding respectively to the plurality of interactive expressions, each of the plurality of interactive expressions being potential responses to be provided by the social agent in response to the first expression of the human being;
predict a state change of the interaction based on use of each of the plurality of interactive expressions in response to the first expression of the human being to determine a plurality of predicted state changes corresponding respectively to the plurality of interactive expressions;
determine, using the plurality of predicted state changes, a second score for each of the plurality of interactive expressions to determine a plurality of second scores corresponding respectively to the plurality of interactive expressions, wherein the second score is determined based on a desirability of a predicted state change resulting from use of each of the plurality of interactive expressions in response to the first expression of the human being; and
select, using the plurality of first scores and the plurality of second scores, at least one of the plurality of interactive expressions to initiate or continue the interaction; and
initiate or continue the interaction with the human being by providing, using the output unit of the social agent, the selected at least one of the plurality of interactive expressions to the human being.