| CPC G06T 19/006 (2013.01) [G06F 3/011 (2013.01); G06F 3/04815 (2013.01); G06F 3/0482 (2013.01); G06T 3/40 (2013.01); G06T 7/20 (2013.01); G06T 7/70 (2017.01); G06T 2200/24 (2013.01)] | 20 Claims |
|
1. A computing device comprising:
a camera;
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the computing device to perform operations comprising:
capturing, by the camera, one or more images within a field of view of the camera;
activating an augmented reality (AR) content item that is executed within a client application;
in response to activating the augmented reality content item, providing a continuous stream of camera data that includes the one or more images to a camera service of a plurality of AR graphics services using one or more first application programming interface (API) calls;
generating, by the augmented reality content item, one or more second API calls for an object detection service to implement one or more object detection models to detect, in the camera data, one or more objects of interest;
obtaining, by the augmented reality content item and from the object detection service, first AR graphics display data indicating (i) an AR graphics display surface labeled as a first object of interest within the field of view, (ii) a graphics input tool labeled as a second object of interest, and (iii) a hand grasping the graphics input tool labeled as a third object of interest, wherein the AR graphics display surface includes a surface of one or more physical objects included in an environment in which the computing device is located;
generating, by the augmented reality content item, one or more third API calls for an object tracking service to implement one or more object tracking models to track, in the camera data, first motion of the graphics input tool and second motion of the hand grasping the graphics input tool;
obtaining, by the augmented reality content item and from the object tracking service, second AR graphics display data indicating a first path of motion of the graphics input tool based on differences in locations of the graphics input tool across a number of video frames;
obtaining, by the augmented reality content item and from the object tracking service, third AR graphics display data indicating a second path of motion of the hand grasping the graphics input tool based on differences in locations of the hand across the number of video frames;
obtaining, by the augmented reality content item and from the object tracking service, fourth AR graphics display data indicating an angle at which the graphics input tool is being held based on first images of the graphics input tool and second images of the hand grasping the graphics input tool within the number of video frames; and
causing, by the augmented reality content item and based on AR graphics display data obtained from the plurality of AR graphics services, display of a user interface overlaid on the AR graphics display surface, the user interface including an AR graphic having at least one of a shape or one or more contours that (i) follow the first path of motion of the graphics input tool and the second path of motion of the hand grasping the graphics input tool and (ii) have a width that is based on the angle at which the graphics input tool is being held.
|