US 10,893,258 B1
Displacement-oriented view synthesis system and method
Yu-Seng Hsu, Taipei (TW); Sih-Sian Wu, Taipei (TW); and Liang-Gee Chen, Taipei (TW)
Assigned to National Taiwan University, Taipei (TW); and Himax Technologies Limited, Tainan (TW)
Filed by National Taiwan University, Taipei (TW); and Himax Technologies Limited, Tainan (TW)
Filed on May 12, 2020, as Appl. No. 15/930,238.
Int. Cl. H04N 13/268 (2018.01); H04N 13/128 (2018.01); G06T 3/00 (2006.01); G06T 5/50 (2006.01); H04N 13/122 (2018.01); H04N 13/00 (2018.01)
CPC H04N 13/268 (2018.05) [G06T 3/0093 (2013.01); G06T 5/50 (2013.01); H04N 13/122 (2018.05); H04N 13/128 (2018.05); H04N 2013/0081 (2013.01)] 12 Claims
OG exemplary drawing
 
1. A displacement-oriented view synthesis system, comprising:
a plurality of three-dimensional (3D) warping devices each coupled to receive at least one input image captured from a corresponding reference view, and each performing 3D warping on the input image to generate at least one corresponding warped image in a target view;
a view blending device coupled to receive the warped images, and performing view blending on the warped images to generate at least one blended image in the target view; and
an inpainting device coupled to receive the blended image, and performing inpainting on the blended image to generate a synthesized image in the target view;
wherein the inpainting is performed according to a difference displacement between frames of different views;
wherein the difference displacement is obtained by the following steps:
determining a foreground displacement associated with a foreground object, the foreground displacement being constructed by connecting a foreground-associated point in the target view with a corresponding point in the reference view;
determining a background displacement associated with a background object, the background displacement being constructed by connecting a background-associated point in the target view with the corresponding point in the reference view; and
obtaining the difference displacement by subtracting the background displacement from the foreground displacement.