US 12,081,830 B2
Video integration with home assistant
Jessica Yuan, Union City, CA (US); James Stewart, San Mateo, CA (US); Rajeev Nongpiur, Mountain View, CA (US); Patrick Lister, Mountain View, CA (US); and Chi Yeung Jonathan Ng, San Francisco, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Jul. 12, 2023, as Appl. No. 18/351,370.
Application 18/351,370 is a continuation of application No. 17/656,788, filed on Mar. 28, 2022, granted, now 11,736,760.
Application 17/656,788 is a continuation of application No. 16/877,847, filed on May 19, 2020, granted, now 11,303,955, issued on Apr. 12, 2022.
Application 16/877,847 is a continuation of application No. 16/230,349, filed on Dec. 21, 2018, granted, now 10,687,109, issued on Jun. 16, 2020.
Application 16/230,349 is a continuation of application No. 15/654,491, filed on Jul. 19, 2017, granted, now 10,200,746, issued on Feb. 5, 2019.
Prior Publication US 2023/0362434 A1, Nov. 9, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G10L 17/22 (2013.01); G06Q 10/109 (2023.01); H04L 12/28 (2006.01); H04L 67/62 (2022.01); H04N 21/234 (2011.01); H04N 21/258 (2011.01); H04N 21/4223 (2011.01); H04N 21/436 (2011.01)
CPC H04N 21/43615 (2013.01) [G06Q 10/109 (2013.01); G10L 17/22 (2013.01); H04L 12/2803 (2013.01); H04L 12/281 (2013.01); H04L 12/282 (2013.01); H04L 67/62 (2022.05); H04N 21/23418 (2013.01); H04N 21/25875 (2013.01); H04N 21/4223 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for handling spoken commands based on characteristics observed by an integrated video service, the method comprising:
receiving a video stream from a streaming video camera;
analyzing a field of view in the received video stream to determine a location for one or more identified or potential users;
beamforming audio from microphones of a home assistant device based on the location of the one or more identified or potential users;
adjusting an audio output based on the location of the one or more identified or potential users;
receiving a spoken command from the one or more identified or potential users; and
outputting a response to the spoken command.