US 11,758,100 B2
Portable projection mapping device and projection mapping system
Mehran Armand, Fulton, MD (US); Shuya Liu, Baltimore, MD (US); Wei-Lun Huang, Baltimore, MD (US); and Austin Shin, San Jose, CA (US)
Assigned to The Johns Hopkins University, Baltimore, MD (US)
Appl. No. 17/753,074
Filed by The Johns Hopkins University, Baltimore, MD (US)
PCT Filed Sep. 9, 2020, PCT No. PCT/US2020/070507
§ 371(c)(1), (2) Date Feb. 17, 2022,
PCT Pub. No. WO2021/051126, PCT Pub. Date Mar. 18, 2021.
Claims priority of provisional application 62/898,913, filed on Sep. 11, 2019.
Prior Publication US 2022/0321851 A1, Oct. 6, 2022
Int. Cl. G06T 7/80 (2017.01); H04N 9/31 (2006.01); G06V 10/143 (2022.01); G06V 20/64 (2022.01)
CPC H04N 9/3185 (2013.01) [G06T 7/85 (2017.01); G06V 10/143 (2022.01); H04N 9/3194 (2013.01); G06T 2207/10012 (2013.01); G06T 2207/30204 (2013.01); G06V 20/64 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
providing, by a device and to a camera and a projector of a portable projection mapping device, first instructions for calibrating the camera and the projector;
receiving, by the device, from the camera and the projector, and based on the first instructions, calibration parameters for the camera and the projector;
calculating, by the device, a stereo calibration between the camera and the projector based on the calibration parameters for the camera and the projector;
providing, by the device and to the camera, second instructions for recognizing a reference instrument and a digitizing probe associated with the portable projection mapping device;
receiving, by the device, from the camera, and based on the second instructions, binocular images;
determining, by the device, a marker extraction, a marker matching, a three-dimensional coordinate reconstruction of markers, a model fitting, and a three-dimensional registration, based on the binocular images;
determining, by the device, recognition parameters for recognizing the reference instrument and the digitizing probe, based on the binocular images, the marker extraction, the marker matching, the three-dimensional coordinate reconstruction of markers, the model fitting, and the three-dimensional registration;
processing, by the device, the recognition parameters for recognizing the reference instrument and the stereo calibration between the camera and the projector, with an optical tracking model, to generate overlay visualization data; and
providing, by the device and to the portable projection mapping device, the overlay visualization data.