US 11,677,923 B1
Method for image processing of image data for high-resolution images on a two-dimensional display wall
Kimball D. Thurston, III, Wellington (NZ); Peter M. Hillman, Wellington (NZ); Joseph W. Marks, Boston, MA (US); Luca Fascione, Wellington (NZ); Millicent Lillian Maier, Wellington (NZ); Kenneth Gimpelson, Wellington (NZ); Dejan Momcilovic, Wellington (NZ); and Keith F. Miller, Wellington (NZ)
Assigned to Unity Technologies SF, San Francisco, CA (US)
Filed by Unity Technologies SF, San Francisco, CA (US)
Filed on Dec. 10, 2021, as Appl. No. 17/548,103.
Claims priority of provisional application 63/288,108, filed on Dec. 10, 2021.
Int. Cl. H04N 13/133 (2018.01); H04N 13/15 (2018.01); G06F 3/14 (2006.01); H04N 13/204 (2018.01); H04N 13/189 (2018.01)
CPC H04N 13/133 (2018.05) [G06F 3/1446 (2013.01); H04N 13/15 (2018.05); H04N 13/189 (2018.05); H04N 13/204 (2018.05)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for processing, in an image processing system, a captured scene captured of a live action scene while a display wall is positioned to be part of the live action scene, wherein the display wall comprises one or more structures capable of displaying imagery, the method comprising:
receiving image data of the live action scene having a live actor and the display wall displaying a first rendering of a precursor image, wherein the image data is captured by at least one camera in at least one placement relative to the live actor and the display wall;
determining precursor metadata for the precursor image displayed on the display wall and display wall metadata for the display wall, wherein the precursor metadata comprises pixel display data for display wall pixels of the display wall, and wherein the display wall metadata comprises geometry data for a display wall position relative to the at least one camera;
accessing an image matte for the image data, wherein the image matte indicates a first portion associated with the live actor and a second portion associated with the precursor image on the display wall in the live action scene;
determining pixel display values for a replacement wall image of a higher resolution than the precursor image, wherein the pixel display values comprise a frequency adjustment to background pixels of the display wall in the image data; and
adjusting the image data of the captured scene using the pixel display values and the image matte, wherein adjusting the image data warps the background pixels to the replacement wall image in the image data.