US 12,452,481 B2
Video display system, information processing device, information processing method, and recording medium
Masayuki Kozuka, Tokyo (JP); Yoshihiro Mori, Osaka (JP); Junya Suzuki, Kyoto (JP); Yoshiichiro Kashiwagi, Kyoto (JP); Yasutoshi Yamamoto, Osaka (JP); and Toshiro Nishio, Osaka (JP)
Assigned to PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD., Osaka (JP)
Appl. No. 18/286,354
Filed by Panasonic Intellectual Property Management Co., Ltd., Osaka (JP)
PCT Filed Apr. 18, 2022, PCT No. PCT/JP2022/018087
§ 371(c)(1), (2) Date Oct. 10, 2023,
PCT Pub. No. WO2022/220306, PCT Pub. Date Oct. 20, 2022.
Claims priority of provisional application 63/176,004, filed on Apr. 16, 2021.
Prior Publication US 2024/0205513 A1, Jun. 20, 2024
Int. Cl. H04N 21/21 (2011.01); G02B 27/01 (2006.01); G06F 3/01 (2006.01); G06F 40/58 (2020.01); G06T 5/50 (2006.01); G06T 7/20 (2017.01); G06T 15/00 (2011.01); G06T 17/00 (2006.01); G09G 5/36 (2006.01); H04N 21/218 (2011.01); H04N 21/235 (2011.01); H04N 21/431 (2011.01); H04N 21/439 (2011.01); H04N 21/472 (2011.01); H04N 21/4788 (2011.01); H04N 21/81 (2011.01)
CPC H04N 21/4316 (2013.01) [G02B 27/0172 (2013.01); G06F 3/011 (2013.01); G06F 3/013 (2013.01); G06F 40/58 (2020.01); G06T 5/50 (2013.01); G06T 7/20 (2013.01); G06T 15/00 (2013.01); G06T 17/00 (2013.01); G09G 5/36 (2013.01); H04N 21/21805 (2013.01); H04N 21/2353 (2013.01); H04N 21/439 (2013.01); H04N 21/472 (2013.01); H04N 21/4788 (2013.01); H04N 21/8106 (2013.01); H04N 21/816 (2013.01); G06T 2207/20221 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A video display system for displaying a display video via a display device, the video display system comprising:
an observation device including:
an image capturing device configured to generate a wide-angle video; and
a first processor configured to:
obtain (i) data related to at least one of a position or a direction of a gazing target in the wide-angle video to be gazed at by a user of an occurrence of the display device and (ii) cue information for informing the user of a change in a state of the observation device,
compose metadata from the data and other information, and
transmit the wide-angle video together with the metadata; and
a VR device including:
a second processor configured to:
receive the wide-angle video, the data, and the cue information,
estimate at least one of a position or a direction of the display device in the wide-angle video,
calculate, based on a difference between (i) at least one of the position or the direction of the display device in the wide-angle video as estimated and (ii) at least one of the position or the direction of the gazing target in the metadata, at least one of a relative position or a relative direction of the gazing target, the relative position being relative to the position of the display device in the wide-angle video, the relative direction being relative to the direction of the display device in the wide-angle video,
present, to the user of the display device, (i) information on at least one of the relative position calculated or the relative direction calculated, and (ii) the change in the state that occurred in the observation system device according to the cue information,
generate, from the wide-angle video received, the display video including a portion corresponding to a viewing area corresponding to (i) information on at least one of the position or the direction of the display device in the wide-angle video as estimated from the wide-angle video, and (ii) the change in the state that occurred in the observation device according to the cue information; and
the display device that displays the display video.
 
16. An information processing device used in a video display system for displaying a display video on a display device, the display video being at least part of a wide-angle video, the information processing device comprising:
a first processor configured to:
receive metadata that is based on input data related to at least one of a position or a direction of a gazing target to be gazed at by a user of the display device in the wide-angle video,
calculate and output at least one of a relative position or a relative direction of the gazing target, based on a difference pertaining to at least one of the position or the direction of the gazing target in the metadata, the relative position being relative to a position of the display device in the wide-angle video, the relative direction being relative to a direction of the display device in the wide-angle video,
output information on at least one of the relative position calculated or the relative direction calculated,
obtain cue information for informing the user of a change in a state of an observation device, and
present, to the user of the display device, the information along with the change in the state that occurred in the observation device according to the cue information.
 
18. An information processing method for displaying a display video on a display device, the display video being at least part of a wide-angle video, the method comprising:
receiving metadata based on input data related to at least one of a position or a direction of a gazing target to be gazed at by a user of the display device in the wide-angle video;
calculating and outputting, based on a difference between (i) at least one of a position or a direction of the display device in the wide-angle video as estimated and (ii) at least one of the position or the direction of the gazing target in the metadata, at least one of a relative position or a relative direction of the gazing target relative to an orientation of the display device;
outputting information on at least one of the relative position calculated or the relative direction calculated;
obtaining cue information for informing the user of a change in a state of an observation device; and
presenting, to the user of the display device, the information along with a change in a state that occurred in an observation device according to cue information.