US 12,220,104 B2
Systems and methods for changing the direction of view during video guided clinical procedures using real-time image processing
João Pedro De Almeida Barreto, Coimbra (PT); Carolina Dos Santos Raposo, Coimbra (PT); Michel Goncalves Almeida Antunes, Coimbra (PT); and Rui Jorge Melo Teixeira, Tondela (PT)
Assigned to S&N Orion Prime, S.A., Austin, TX (US)
Appl. No. 17/762,205
Filed by S&N Orion Prime, S.A., Austin, TX (US)
PCT Filed Oct. 7, 2020, PCT No. PCT/US2020/054640
§ 371(c)(1), (2) Date Mar. 21, 2022,
PCT Pub. No. WO2021/071991, PCT Pub. Date Apr. 15, 2021.
Claims priority of provisional application 62/911,986, filed on Oct. 7, 2019.
Claims priority of provisional application 62/911,950, filed on Oct. 7, 2019.
Prior Publication US 2022/0383588 A1, Dec. 1, 2022
Int. Cl. A61B 1/00 (2006.01); G06T 3/18 (2024.01); G06T 7/13 (2017.01); G06T 7/33 (2017.01); G06T 7/80 (2017.01); G06T 15/20 (2011.01)
CPC A61B 1/00179 (2013.01) [A61B 1/00009 (2013.01); A61B 1/000094 (2022.02); A61B 1/000096 (2022.02); G06T 3/18 (2024.01); G06T 7/13 (2017.01); G06T 7/33 (2017.01); G06T 7/80 (2017.01); G06T 15/205 (2013.01); G06T 2207/10 (2013.01); G06T 2207/10068 (2013.01); G06T 2210/41 (2013.01)] 30 Claims
OG exemplary drawing
 
1. A method for rendering an image Îi of a virtual target camera based on a source image Ii captured by a real source camera that is real, the source camera comprising a camera-head and a rigid endoscope with lens cut β that rotates in azimuth around a mechanical axis that intersects the image in a point Q, for which the focal length f, radial distortion ξ, and principal point O at a certain azimuth α0 are known, the method comprising:
finding a location of the principal point Oi in the source image Ii with azimuth αi by rotating O around Q by an angular displacement in azimuth δi=αi−α0;
updating a camera model cs of the source camera that maps points x in a canonical image into points u in a pixel image according to the focal length f, radial distortion ξ, and location of the principal point Oi;
determining a 3D location of a vertical plane Πi that contains, or passes close to, the mechanical axis and an optical axis of the source camera, by finding and back-projecting a line ni where Πi intersects the source image Ii;
defining a 3D motion m between the target and source cameras as a rotation by an angle γ=β−β around a direction ni that is normal to the vertical plane Πi i, such that x=m(x; γ, ni), with x being a point in the canonical image of the target camera;
computing a focal length f and a location of the principal point Ôi for the target camera and deriving a camera model ct that maps points x in a canonical image of the target image into points û in a pixel image of the target image according to the focal length f and location of the principal point Ôi; and
generating the target image Îi by mapping a plurality of pixels û in Îi into a point u in the source image Ii with a mapping function w that is a composition of the camera model cs of the source camera, the 3D motion m, and an inverse of a camera model ct of the target camera.