US 11,989,339 B2
Visual tracking of peripheral devices
Zachary C. Nienstedt, Fort Lauderdale, FL (US); Samuel A. Miller, Hollywood, FL (US); Barak Freedman, Binyamina (IL); Lionel Ernest Edwin, Hollywood, FL (US); Eric C. Browy, Meridian, ID (US); William Hudson Welch, Fort Lauderdale, FL (US); and Ron Liraz Lidji, Haifa (IL)
Assigned to Magic Leap, Inc., Plantation, FL (US)
Filed by Magic Leap, Inc., Plantation, FL (US)
Filed on Feb. 22, 2023, as Appl. No. 18/112,573.
Application 18/112,573 is a continuation of application No. 17/497,782, filed on Oct. 8, 2021, granted, now 11,625,090.
Application 17/497,782 is a continuation of application No. 17/084,512, filed on Oct. 29, 2020, granted, now 11,181,974, issued on Nov. 23, 2021.
Application 17/084,512 is a continuation of application No. 16/588,830, filed on Sep. 30, 2019, granted, now 10,860,090, issued on Dec. 8, 2020.
Application 16/588,830 is a continuation of application No. PCT/US2019/021025, filed on Mar. 6, 2019.
Claims priority of provisional application 62/640,299, filed on Mar. 8, 2018.
Claims priority of provisional application 62/640,009, filed on Mar. 7, 2018.
Prior Publication US 2023/0195212 A1, Jun. 22, 2023
Int. Cl. G06F 3/01 (2006.01); G02B 27/01 (2006.01); G06F 1/16 (2006.01); G06F 3/16 (2006.01)
CPC G06F 3/011 (2013.01) [G02B 27/017 (2013.01); G06F 1/1686 (2013.01); G06F 1/1694 (2013.01); G02B 2027/0187 (2013.01)] 16 Claims
OG exemplary drawing
 
9. A system for performing localization of a handheld device with respect to a wearable device, the system comprising:
the wearable device;
the handheld device; and
one or more processors communicatively coupled to the wearable device and the handheld device, wherein the one or more processors are configured to perform operations including:
capturing, by a first imaging device mounted to the handheld device, a fiducial image containing a number of fiducials affixed to the wearable device;
capturing, by a second imaging device mounted to the handheld device, a world image containing one or more features surrounding the handheld device;
obtaining, by a sensor mounted to the handheld device, handheld data indicative of movement of the handheld device;
determining the number of fiducials contained in the fiducial image; and
updating a position and an orientation of the handheld device using at least one of the fiducial image or the world image and the handheld data.