US 12,413,682 B2
Real time augmentation
Nils Frielinghaus, Heimstetten (DE); and Markus Neff, Ottobrunn (DE)
Assigned to Brainlab SE, Munich (DE)
Appl. No. 18/019,417
Filed by Brainlab SE, Munich (DE)
PCT Filed Aug. 7, 2020, PCT No. PCT/EP2020/072309
§ 371(c)(1), (2) Date Feb. 2, 2023,
PCT Pub. No. WO2022/028718, PCT Pub. Date Feb. 10, 2022.
Prior Publication US 2023/0276016 A1, Aug. 31, 2023
Int. Cl. H04N 5/265 (2006.01); G06T 7/20 (2017.01); G06V 10/74 (2022.01)
CPC H04N 5/265 (2013.01) [G06T 7/20 (2013.01); G06V 10/761 (2022.01); G06T 2207/10056 (2013.01); G06T 2207/10068 (2013.01); G06T 2207/10132 (2013.01); G06V 2201/07 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method of generating an overlay of medical video data and overlay data, comprising:
acquiring, from a medical video modality, the medical video data comprising at least a first video frame and a second video frame of different points in time, t1 and t2;
analysing the acquired medical video data comprising a comparison of the video data captured by the first and the second video frames;
providing initial overlay data;
generating modified overlay data by adapting the initial overlay data based on a result of the analysis of the medical video data; and
generating the overlay by generating a video output comprising at least medical video data originating from the medical video modality and including the generated modified overlay data
determining from the first video frame an augmentation model;
determining from the first video frame at least one landmark of said augmentation model;
automatically identifying the determined at least one landmark in the second video frame while reading out the second video frame;
adapting at least one parameter of the augmentation model based on a result of the landmark identification in the second video frame, thereby generating the modified overlay data;
wherein the step of generating the overlay includes:
generating the video output comprising the second video frame and the adapted augmentation model; and
wherein the landmark identification and the adaption of the at least one parameter of the augmentation model are carried out simultaneously, i.e. within the same video frame, with the generation of the overlay.