US 12,474,769 B2
Image processing apparatus, image data transfer apparatus, image processing method, and image data transfer method
Katsushi Otsuka, Tokyo (JP)
Assigned to Sony Interactive Entertainment Inc., Tokyo (JP)
Appl. No. 17/638,535
Filed by Sony Interactive Entertainment Inc., Tokyo (JP)
PCT Filed Sep. 23, 2020, PCT No. PCT/JP2020/035831
§ 371(c)(1), (2) Date Feb. 25, 2022,
PCT Pub. No. WO2021/065628, PCT Pub. Date Apr. 8, 2021.
Claims priority of application No. 2019-179436 (JP), filed on Sep. 30, 2019.
Prior Publication US 2022/0291742 A1, Sep. 15, 2022
Int. Cl. G06F 3/01 (2006.01); G09G 5/00 (2006.01); G09G 5/36 (2006.01); H04N 13/344 (2018.01); H04N 13/383 (2018.01)
CPC G06F 3/012 (2013.01) [G09G 5/006 (2013.01); G09G 5/36 (2013.01); H04N 13/344 (2018.05); H04N 13/383 (2018.05)] 13 Claims
OG exemplary drawing
 
1. An image processing apparatus for causing a head mounted display to display a moving image, the image processing apparatus comprising:
a position-posture tracking section that acquires a position or a posture of a head of a user wearing the head mounted display;
an image data acquisition section that acquires data of a frame of the moving image and a prediction value of the position or posture used in drawing an image of the frame, in units of a partial image smaller than one frame, from an external apparatus not integrated with the image processing apparatus;
an image processing section that forms a display image corresponding to a latest position or posture by correcting the data of the frame in units of the partial image on a basis of a difference between the latest position or posture and the prediction value; and
a display control section that sequentially outputs the display image to a display panel of the head mounted display,
wherein the display control section changes a target to be corrected by the image processing section, at least on the basis of the difference, and at least one of:
(i) the image data acquisition section further acquires a formation time of the data of the frame formed by the external apparatus in units of the partial image, and the display control section reflects a delay time period from the formation time, in determination of the target to be corrected by the image processing section;
(ii) the display control section quantifies a degree of data lack by using a weight that increases as a distance between a gaze point of the user and a lacking position in a display screen becomes shorter, and reflects a result of the quantification in determination of the target to be corrected by the image processing section;
(iii) the image data acquisition section transmits, to the external apparatus at a predetermined timing, the latest position or posture, a history of the difference, and a history of a delay time period from a formation time of frame data formed by the external apparatus; and
(iv) the image data acquisition section further acquires data of a region outside the frame, as data additionally formed on a basis of a vector indicating deviation of the prediction value, the deviation being predicted in drawing the image of the frame, and the image processing section forms a display image by using the data of the region outside the frame.