US 12,284,324 B2
Augmented reality prop interactions
Tianying Chen, Pittsburgh, PA (US); Timothy Chong, Kirkland, WA (US); Sven Kratz, Mercer Island, WA (US); Fannie Liu, New York, NY (US); Andrés Monroy-Hernández, Seattle, WA (US); Olivia Seow, Cambridge, MA (US); Yu Jiang Tham, Seattle, WA (US); Rajan Vaish, Beverly Hills, CA (US); and Lei Zhang, Ann Arbor, MI (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Monica, CA (US)
Filed on Aug. 16, 2022, as Appl. No. 17/888,948.
Claims priority of provisional application 63/240,049, filed on Sep. 2, 2021.
Prior Publication US 2023/0066686 A1, Mar. 2, 2023
Int. Cl. H04N 13/111 (2018.01); G06T 3/60 (2006.01); G06V 10/44 (2022.01); G06V 10/764 (2022.01); G06V 20/20 (2022.01)
CPC H04N 13/111 (2018.05) [G06T 3/60 (2013.01); G06V 10/457 (2022.01); G06V 10/764 (2022.01); G06V 20/20 (2022.01); G06T 2200/24 (2013.01)] 12 Claims
OG exemplary drawing
 
1. An augmented reality (AR) device for generating an AR experience, the AR device comprising:
a camera system configured to capture images of a scene within a field of view;
a display system configured to present AR overlays on a display, the display having a viewing area corresponding to the camera system field of view;
a user input system configured to receive input from a user, wherein the input includes hand gestures captured by the camera system; and
a processor coupled to the camera system, the display system, and the user input system, the processor configured to:
capture, via the camera system, images of the scene;
identify an object receiving surface and corresponding surface coordinates within the scene;
identify, responsive to the input received from the user via the user input system, including the hand gestures captured by the camera system, an AR primary object and a prop object, wherein the AR primary object is associated with at least one set of primary object coordinates and one of the AR primary object or the prop object is identified by the user capturing an image of a physical object with the camera system and designating, according and in response to the input received from the user, the physical object captured in the image as the one of the AR primary object or the prop object;
establish a logical connection between the AR primary object and the prop object;
generate AR overlays including actions associated with the AR primary object responsive to commands received from the user via the user input system that position the AR primary object adjacent the object receiving surface responsive to the primary object coordinates and the surface coordinates within the scene and that position the AR primary object and the prop object with respect to one another in accordance with the logical connection; and
present, via the display system, the generated AR overlays;
wherein the prop object is a physical object within the scene and wherein to identify the physical object the processor is configured to receive a selection of the physical object via the user input system; and
wherein the processor is further configured to:
identify a category for the selected physical object; and
select logic for the physical object responsive to the identified category, the selected logic specifying predefined interactions between the AR primary object and the physical object;
wherein the established logical connection is based on the selected logic and the generated AR overlays are responsive to commands received via the user input system.