US 11,875,022 B2
Camera interfaces to interact with IoT devices
Yu Jiang Tham, Seattle, WA (US); Rajan Vaish, Beverly Hills, CA (US); Andrés Monroy-Hernández, Seattle, WA (US); Sven Kratz, Mercer Island, WA (US); and Ana Maria Cardenas Gasca, Pasadena, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Yu Jiang Tham, Seattle, WA (US); Rajan Vaish, Beverly Hills, CA (US); Andrés Monroy-Hernández, Seattle, WA (US); Sven Kratz, Mercer Island, WA (US); and Ana Maria Cardenas Gasca, Pasadena, CA (US)
Filed on Aug. 29, 2021, as Appl. No. 17/460,228.
Prior Publication US 2023/0069042 A1, Mar. 2, 2023
Int. Cl. G06V 20/20 (2022.01); G06F 3/0484 (2022.01); H04N 23/63 (2023.01); G06F 3/01 (2006.01); G06F 3/0482 (2013.01); G16Y 40/35 (2020.01)
CPC G06F 3/0484 (2013.01) [G06F 3/017 (2013.01); G06F 3/0482 (2013.01); G06V 20/20 (2022.01); H04N 23/63 (2023.01); G16Y 40/35 (2020.01)] 20 Claims
OG exemplary drawing
 
1. A method of interfacing augmented reality (AR) camera devices to Internet of Things (I) devices, comprising:
pairing at least one AR camera device to at least one IoT device to establish which actions recognized by the at least one AR camera device may control the at least one IoT device;
receiving an action identifier from the at least one AR camera device, the action identifier including an indication that the at least one AR camera device has recognized a marker in an image obtained by the at least one AR camera device;
receiving context information relating to the action identifier, the context information including at least one of time of day, weather conditions, Internet download speed, or global positioning system (GPS) data locating the at least one AR camera device; and
sending a command to the at least one IoT device paired with the at least one AR camera device to perform an action based on the context information in response to the action identifier received from the at least one AR camera device when the at least one AR camera device recognizes the marker in the image,
wherein the context information relating to the action identifier from the at least one AR camera device is used to modify the command sent to the at least one IoT device, and the context information is evaluated to determine characteristics of operations of the at least one IoT device,
wherein when the at least one IoT device is an IoT enabled light bulb within a specified geographic radius from the marker in the image or in a room with the marker in the image, the action comprises instructing the IoT enabled light bulb to change to a specific color based on the context information indicating a user's location as determined from the GPS data, and the time of day, weather conditions, or Internet download speed, and when the at least one IoT device is an IoT enabled smart speaker within a specified geographic radius from the marker in the image or in a room with the marker in the image, the action further comprises instructing the IoT enabled smart speaker to play specific music based on the context information.