CPC G06F 3/011 (2013.01) [G06F 3/017 (2013.01); G06F 3/0334 (2013.01); G06V 10/255 (2022.01); G06V 40/20 (2022.01); G16Y 20/40 (2020.01); H04N 13/111 (2018.05)] | 20 Claims |
1. A method of controlling an Internet of Things (IoT) product using a portable electronic device comprising a camera and a display, the method comprising steps of:
capturing frames of video data with the camera;
detecting the IoT product at an IoT product location in a physical environment using the captured frames of video data;
determining a portable electronic device location relative to the IoT product location using the captured frames of video data;
presenting on the display a virtual control interface according to the IoT product location and the portable electronic device location, wherein the virtual control interface comprises a virtual surface virtually presented on the display at a surface position relative to the physical environment;
detecting a foot shape performing a foot action at a foot location relative to the physical environment using the captured frames of video data;
detecting an intersection between the foot location and the surface position using the captured frames of video data, the intersection and the foot action corresponding to an IoT action; and
sending a control signal to the IoT product in accordance with the detected intersection and the foot action, the control signal comprising instructions for use by the IoT product to perform the IoT action.
|