US 12,294,685 B2
Image processing apparatus, image generating method, and storage medium
Shogo Mizuno, Tokyo (JP); and Atsushi Date, Tokyo (JP)
Assigned to Canon Kabushiki Kaisha, Tokyo (JP)
Filed by CANON KABUSHIKI KAISHA, Tokyo (JP)
Filed on Jul. 6, 2021, as Appl. No. 17/367,655.
Application 17/367,655 is a division of application No. 16/293,045, filed on Mar. 5, 2019, granted, now 11,089,284.
Application 16/293,045 is a continuation of application No. PCT/JP2017/030092, filed on Aug. 23, 2017.
Claims priority of application No. 2016-179529 (JP), filed on Sep. 14, 2016; and application No. 2017-124110 (JP), filed on Jun. 26, 2017.
Prior Publication US 2021/0337177 A1, Oct. 28, 2021
Int. Cl. G06T 19/00 (2011.01); G06T 3/40 (2024.01); G06T 7/20 (2017.01); G06T 7/50 (2017.01); G06T 7/70 (2017.01); H04N 13/111 (2018.01); H04N 13/183 (2018.01); H04N 23/90 (2023.01); G06T 3/20 (2006.01); H04N 5/272 (2006.01)
CPC H04N 13/183 (2018.05) [G06T 7/20 (2013.01); G06T 7/50 (2017.01); G06T 7/70 (2017.01); G06T 19/00 (2013.01); H04N 13/111 (2018.05); G06T 3/20 (2013.01); G06T 3/40 (2013.01); G06T 2207/30221 (2013.01); H04N 5/2723 (2013.01); H04N 23/90 (2023.01)] 18 Claims
OG exemplary drawing
 
1. An image processing apparatus comprising:
one or more hardware processors; and
one or more memories storing one or more programs configured to be executed by the one or more hardware processors, the one or more programs including instructions for:
(1) obtaining image data based on a plurality of captured images captured by a plurality of image capturing apparatuses;
(2) obtaining viewpoint information indicating (a) a position of a virtual viewpoint and (b) a direction of view from the virtual viewpoint in a three-dimensional space;
(3) determining a content of a virtual object which is to be included in a virtual viewpoint image corresponding to the viewpoint information and which does not exist in an area captured by the plurality of image capturing apparatuses, based on a relationship between (a) the position of the virtual viewpoint indicated by the viewpoint information, (b) a position of the virtual object in the three-dimensional space, and (c) a position and a size of a specific object based on the obtained image data, the specific object being captured in the obtained image data;
(4) reducing a size of an advertisement image corresponding to the content of the virtual object such that the advertisement image is displayed within a region corresponding to the virtual object in the virtual viewpoint image;
(5) generating the virtual viewpoint image including the virtual object having the determined content based on the image data and the viewpoint information; and
(6) changing, in a case where the relationship has changed, an amount of information of the content of a virtual object without changing an advertiser of the advertisement image based on the changed relationship so as to include in the virtual viewpoint image a whole area of the content of the virtual object.