receiving a plurality of images captured from a reference viewpoint of a scene, each image of the plurality of images representing a view of the scene at a predefined depth from the reference viewpoint;

generating a two-dimensional (2D) pixel map of the scene based on the plurality of images;

determining one or more multi-plane image (MPI) layers from the 2D pixel map in real-time;

dividing the one or more MPI layers into a plurality of MPI regions,

wherein each MPI region of the plurality of MPI regions has a different set of depth planes than all other MPI regions of the plurality of MPI regions, and

wherein a number of depth planes of the one or more MPI layers is larger than a number of depth planes of each MPI region of the plurality of MPI regions;

constructing a textured three-dimensional mesh based on the plurality of MPI regions;

rendering the scene from a novel viewpoint based on the textured three-dimensional mesh.