US 11,915,339 B2
Remote operation instructing system, and mount type device
Osamu Kawamae, Kyoto (JP); and Masuo Oku, Kyoto (JP)
Assigned to MAXELL, LTD., Kyoto (JP)
Appl. No. 17/436,400
Filed by MAXELL, LTD., Kyoto (JP)
PCT Filed Mar. 4, 2019, PCT No. PCT/JP2019/008456
§ 371(c)(1), (2) Date Sep. 3, 2021,
PCT Pub. No. WO2020/178960, PCT Pub. Date Sep. 10, 2020.
Prior Publication US 2022/0148230 A1, May 12, 2022
Int. Cl. G06T 11/00 (2006.01); G06F 3/01 (2006.01); G06F 3/0484 (2022.01); H04N 5/262 (2006.01); H04N 5/265 (2006.01)
CPC G06T 11/00 (2013.01) [G06F 3/011 (2013.01); G06F 3/0484 (2013.01); H04N 5/265 (2013.01); H04N 5/2628 (2013.01)] 6 Claims
OG exemplary drawing
 
1. A remote operation instructing system comprising:
a mount type device to be worn by a worker;
an operation instructing apparatus through which an instructor confirms a work site to issue an instruction; and
a network through which data are transmitted and received between the mount type device and the operation instructing apparatus,
wherein the mount type device includes:
a camera configured to photograph the work site in a wider range than a visual field of the worker;
a sensor configured to obtain a visual line of the worker as sensor information; and
a visual line movement detector configured to detect, as visual line movement information, movement of a direction of the visual line of the worker by using the sensor information,
wherein the mount type device is configured to transmit data on the image photographed by the camera and the visual line movement information to the operation instructing apparatus via the network,
wherein the operation instructing apparatus includes:
a display setter configured to extract a first image having a wider range than a visual field range of the worker from the image photographed by the camera, correct the first image using the visual line movement information, and cause a display to display the corrected first image; and
an instruction image generator configured to generate an instruction image for the worker,
a synthesizer configured to extract a second image having the visual field range of the worker from the photographed image, and superimpose the second image and the instruction image to cause a sub display to display the second image and the instruction image,
wherein, in a case where there is a possibility that a part of the second image protrudes outside the range of the first image, the display setter extracts an adjusted first image from the photographed image, wherein the center of the adjusted first image and the center of the second image coincide each other, and
wherein the operation instructing apparatus is configured to transmit the instruction image to the mount type device via the network.