US 12,009,941 B2
Assistive control of network-connected devices
Forest Johnson, Hazlet, NJ (US); Pamela Juhl Sokoler, Colts Neck, NJ (US); and Prakash Thiruvenkatam, Morganville, NJ (US)
Assigned to AT&T Intellectal P Property I, L.P., Atlanta, GA (US)
Filed by AT&T Intellectual Property I, L.P., Atlanta, GA (US)
Filed on Jan. 30, 2023, as Appl. No. 18/161,876.
Application 18/161,876 is a continuation of application No. 16/221,081, filed on Dec. 14, 2018, granted, now 11,570,016.
Prior Publication US 2023/0171122 A1, Jun. 1, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. H04L 12/28 (2006.01); G06F 3/01 (2006.01); G06V 40/16 (2022.01); H04L 41/0813 (2022.01)
CPC H04L 12/2816 (2013.01) [G06F 3/017 (2013.01); G06V 40/176 (2022.01); H04L 41/0813 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A device comprising:
a processing system in a communication network including at least one processor, wherein the processing system is remote from a home network that is connected to the communication network via a cellular access network; and
a non-transitory computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
obtaining, from a computing device located in the home network via the cellular access network, images of a face of a user;
detecting a first facial gesture of the user from the images;
obtaining, from the computing device via the cellular access network, a first non-speech utterance of the user;
determining an intention to change a state of a network-connected device in the home network, wherein the network-connected device is separate from the computing device, and wherein the intention is determined from the first facial gesture and the first non-speech utterance based upon a mapping of the user, and wherein the mapping of the user is generated via a machine learning process that learns that the first facial gesture in combination with the first non-speech utterance signifies the intention to change the state of the network-connected device;
generating an audio command to cause the state of the network-connected device to change in accordance with the intention, when the computing device confirms, in connection with a response of the user that includes a second facial gesture and a second non-speech utterance, the intention to change the state of the network-connected device; and
providing, via the cellular access network, the audio command to the computing device along with an instruction for the computing device to play the audio command via a speaker of the computing device for detection via a short range wireless communication by an interactive voice assistant device, separate from the computing device and located in the home network, to change the state of the network-connected device by the interactive voice assistant device.