| CPC G06T 3/4038 (2013.01) [G06T 3/18 (2024.01); G06T 7/11 (2017.01); G06T 7/174 (2017.01); G06T 7/248 (2017.01); G06T 7/30 (2017.01); G06T 17/205 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20021 (2013.01)] | 26 Claims |
|
1. A method executed by one or more computing devices for generating a novel view of a scene, the method comprising:
accessing one or more images corresponding to one or more viewpoints of the scene;
generating a novel image of the scene from a novel viewpoint based at least in part on one or more perceptual parameters corresponding to the one or more images, wherein the novel viewpoint corresponds to a viewpoint different than the one or more viewpoints;
coarsely aligning the one or more images with the novel image to generate one or more coarsely aligned images, wherein coarsely aligning the one or more images with the novel image comprises projecting each image in the one or more images onto an image plane of the novel image;
finely aligning the one or more coarsely aligned images with the novel image to generate one or more finely aligned images based at least in part on performing semantic segmentation on each coarsely aligned image in the one or more coarsely aligned images and identifying semantically matching features in the novel image;
generating a novel view of the scene based at least in part on the one or more finely aligned images; and
transmitting the novel view of the scene for display on a display device.
|