US 12,393,262 B2
Methods and systems of a handheld spatially aware mixed-reality projection platform
Arvind Sanjeev, Malmo (SE); Anoushka Garg, Copenhagen (DK); Can Yanardag, Copenhagen (DK); and Matthew Visco, San Francsico, CA (US)
Filed by Arvind Sanjeev, Malmo (SE); Anoushka Garg, Copenhagen (DK); Can Yanardag, Copenhagen (DK); and Matthew Visco, San Francsico, CA (US)
Filed on Feb. 3, 2021, as Appl. No. 17/165,939.
Application 17/165,939 is a continuation in part of application No. 16/984,124, filed on Aug. 3, 2020, abandoned.
Claims priority of provisional application 62/882,167, filed on Aug. 2, 2019.
Prior Publication US 2021/0255328 A1, Aug. 19, 2021
Int. Cl. G01S 17/894 (2020.01); A63F 13/211 (2014.01); A63F 13/213 (2014.01); G06F 3/01 (2006.01); G06F 3/16 (2006.01); G06T 7/521 (2017.01); G06T 19/00 (2011.01); G06V 10/10 (2022.01); G06V 20/64 (2022.01); A63F 13/216 (2014.01); A63F 13/219 (2014.01); A63F 13/327 (2014.01)
CPC G06F 3/011 (2013.01) [G01S 17/894 (2020.01); G06F 3/017 (2013.01); G06F 3/167 (2013.01); G06T 7/521 (2017.01); G06T 19/006 (2013.01); G06V 10/17 (2022.01); G06V 20/647 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A method for augmenting a physical environment with projected media content from a virtual environment, using a handheld spatially aware mixed reality projection platform, comprising the steps of:
providing a handheld spatially aware mixed-reality projection device;
with a mapping application in the handheld spatially aware mixed-reality projection device, relocalizing the handheld spatially aware mixed-reality projection device based on a virtual three-dimensional map of a physical environment, wherein the relocalizing generates pose information of the handheld spatially aware mixed-reality projection device in the physical environment with respect to the virtual three-dimensional map;
sharing the set of three-dimensional pose information with a game engine application of the handheld spatially aware mixed-reality projection device;
with the game engine application, placing the handheld spatially aware mixed-reality projection device in a set of equivalent virtual world coordinates and corresponding orientation in the virtual three-dimensional map of a space;
wherein the virtual three-dimensional map of the space is augmented with digital media content to create a virtual environment,
detecting that the handheld spatially aware mixed-reality projection device is pointing at a physical structure that corresponds to an image target; and
using a laser projector in the handheld spatially aware mixed-reality projection device, the handheld spatially aware mixed-reality projection device creates a projection field, wherein the spatially mapped digital media content is revealed from the virtual environment overlaid onto the physical environment.