US 12,382,005 B2
Image processing apparatus, image processing method, and storage medium
Tomohiro Yano, Kanagawa (JP)
Assigned to Canon Kabushiki Kaisha, Tokyo (JP)
Filed by CANON KABUSHIKI KAISHA, Tokyo (JP)
Filed on Aug. 24, 2023, as Appl. No. 18/454,991.
Application 18/454,991 is a continuation of application No. 17/522,508, filed on Nov. 9, 2021, granted, now 11,778,155.
Claims priority of application No. 2020-188220 (JP), filed on Nov. 11, 2020.
Prior Publication US 2023/0396748 A1, Dec. 7, 2023
Int. Cl. H04N 13/111 (2018.01); G06T 7/20 (2017.01); H04N 5/272 (2006.01)
CPC H04N 13/111 (2018.05) [G06T 7/20 (2013.01); H04N 5/272 (2013.01); G06T 2207/30196 (2013.01); G06T 2207/30221 (2013.01)] 8 Claims
OG exemplary drawing
 
1. An image processing apparatus comprising:
one or more memories; and
one or more processors in communication with the one or more memories, wherein the one or more processors and the one or more memories are configured to:
acquire information about a line of sight of a subject imaged by a plurality of image capturing apparatuses, based on a face of the subject in a predetermined time range;
determine a parameter including a position, orientation, and angle of view of a virtual camera; and
generate a virtual viewpoint image based on the acquired information and the determined parameter,
wherein the virtual viewpoint image is displayed with graphics indicating the line of sight or a line-of-sight range of the subject superimposed on the virtual viewpoint image,
wherein the one or more processors and the one or more memories are further configured to acquire the information based on a plurality of movement loci of lines of sight that is respectively acquired in a first predetermined time range and a second predetermined time range each of which is the predetermined time range,
wherein the one or more processors and the one or more memories are further configured to gradually change the parameter between a first parameter determined to correspond to the first predetermined time range and a second parameter determined to correspond to the second predetermined time range that is shifted by the predetermined time range from the first predetermined time range,
wherein the one or more processors and the one or more memories are further configured to determine the parameter and the parameter for key frames, and determine the parameters to be complemented for each time obtained by dividing the predetermined time range by a number of frames for displaying the virtual viewpoint image, as the parameters to be changed gradually.