US 12,276,085 B2
Work assist server, work assist method, and work assist system
Hitoshi Sasaki, Hiroshima (JP); and Yoichiro Yamazaki, Hiroshima (JP)
Assigned to Kobelco Construction Machinery Co., Ltd., Hiroshima (JP)
Appl. No. 17/780,795
Filed by KOBELCO CONSTRUCTION MACHINERY CO., LTD., Hiroshima (JP)
PCT Filed Oct. 8, 2020, PCT No. PCT/JP2020/038203
§ 371(c)(1), (2) Date May 27, 2022,
PCT Pub. No. WO2021/124654, PCT Pub. Date Jun. 24, 2021.
Claims priority of application No. 2019-228634 (JP), filed on Dec. 18, 2019.
Prior Publication US 2022/0412048 A1, Dec. 29, 2022
Int. Cl. E02F 9/20 (2006.01); G05D 1/00 (2024.01); G08C 17/02 (2006.01)
CPC E02F 9/205 (2013.01) [G05D 1/0033 (2013.01); G08C 17/02 (2013.01)] 8 Claims
OG exemplary drawing
 
1. A work assist system for assisting remote operation of a work machine by an operator using a remote operation device, including a plurality of the work machines existing in a worksite, a remote operation device having a function to remotely operate a connection-target work machine which is a work machine to be remotely operated among the plurality of the work machines, an image pickup device installed at the worksite, an actual machine image pickup device mounted on a work machine that is not the connection-target work machine among the plurality of the work machines, and a work assist server configured to mutually transmit data between the plurality of the work machines and the remote operation device and to acquire an work-site bird's-eye image representing a bird's-eye view of the worksite based on a captured image acquired through the image pickup device or the actual machine image pickup device, the work assist system comprising:
a database configured by the work assist server or a database server separate from the work assist server, and in a case where communication is established between the remote operation device and the connection-target work machine, the database registers identifiers of each of the remote operation device and the connection-target work machine and a real spatial position measured by a position determination device mounted on the connection-target work machine; and
an output interface provided in the remote operation device and configured by an image output device capable to output an image based on communication with the work assist server,
the work assist server comprises a first assist processing element and a second assist processing element, each composed of a processor,
the first assist processing element acquires the work-site bird's-eye image based on the captured image acquired through the image pickup device or the actual machine image pickup device, performs image analysis processing on the acquired work-site bird's-eye image to determine whether or not there is a work machine captured in the work-site bird's-eye image, and when a work machine is in the work-site bird's-eye image, determines whether or not a calculated real spatial position of the work machine obtained by transforming from the position of the connection-target work machine in an image coordinate system in the work-site bird's-eye image into a real spatial coordinate system, and matches a measured real spatial position of the connection-target work machine registered in the database; and
the second assist processing element, based on communication with the remote operation device, generates a second work environment image including a first sign image showing that the work machine in the work-site bird's-eye image is the connection-target work machine when the first assist processing element determines that the work machine is the connection-target work machine, by transmitting the second work environment image to the remote operation device, and causes the output interface to output the second work environment image,
wherein the connection-target work machine is defined as the work machine to be actually remotely operated through the remote operation device.