US 11,921,988 B2
Identifying and controlling smart devices
Ibrahim Badr, Zurich (CH); Gokhan H. Bakir, Zurich (CH); Roland Peter Kehl, Gachnang (CH); and Nils Grimsmo, Gachnang (CH)
Assigned to GOOGLE LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Jul. 19, 2021, as Appl. No. 17/378,856.
Application 17/378,856 is a continuation of application No. 16/905,245, filed on Jun. 18, 2020, granted, now 11,086,493.
Application 16/905,245 is a continuation of application No. 16/017,394, filed on Jun. 25, 2018, granted, now 10,725,629, issued on Jul. 28, 2020.
Prior Publication US 2021/0342047 A1, Nov. 4, 2021
Int. Cl. G06F 3/048 (2013.01); G06F 3/0482 (2013.01); G06F 3/0484 (2022.01); G06F 3/16 (2006.01); G06F 9/451 (2018.01); G06V 20/20 (2022.01); G08C 17/02 (2006.01); G06T 19/00 (2011.01); H04W 88/02 (2009.01)
CPC G06F 3/0484 (2013.01) [G06F 3/0482 (2013.01); G06F 3/167 (2013.01); G06F 9/453 (2018.02); G06V 20/20 (2022.01); G08C 17/02 (2013.01); G06T 19/006 (2013.01); G08C 2201/30 (2013.01); H04W 88/02 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method, the method comprising:
receiving, by a computing system comprising one or more processors, image data for an image captured by a camera of a computing device of a user;
determining, by the computing system based on the image data and by processing the image data with an object recognizer to generate identified object data, that the image depicts a smart device, wherein the object recognizer comprises a coarse classifier that determines whether a pixel data set comprises an object in one or more particular smart device classes based on one or more image features;
in response to determining that the image depicts a smart device of the one or more particular smart device classes, registering, by the computing system, the computing device with the smart device, wherein registering the computing device comprises generating and transmitting a registration file associated with the computing device, wherein the registration file comprises device identification data and interface controls identification data, and wherein the interface controls identification data is generated based on a determined class of smart devices associated with the smart device;
receiving, by the computing system, a user interaction to control the smart device; and
controlling, by the computing system and based on the registration file, the smart device based on the user interaction to control the smart device.