US 12,325,139 B2
Remote control system, remote control method, and control program
Yuka Iwanaga, Toyokawa (JP); Takashi Yamamoto, Seto (JP); Takemitsu Mori, Nisshin (JP); Masayoshi Tsuchinaga, Nisshin (JP); and Shintaro Yoshizawa, Nagoya (JP)
Assigned to TOYOTA JIDOSHA KABUSHIKI KAISHA, Toyota (JP)
Filed by TOYOTA JIDOSHA KABUSHIKI KAISHA, Toyota (JP)
Filed on Aug. 29, 2023, as Appl. No. 18/239,468.
Claims priority of application No. 2022-140230 (JP), filed on Sep. 2, 2022.
Prior Publication US 2024/0075623 A1, Mar. 7, 2024
Int. Cl. B25J 9/16 (2006.01)
CPC B25J 9/1689 (2013.01) [B25J 9/161 (2013.01); B25J 9/1612 (2013.01); B25J 9/163 (2013.01)] 11 Claims
OG exemplary drawing
 
1. A remote control system configured to remotely control a device to be operated comprising an end effector, the remote control system comprising:
an imaging unit configured to shoot an image of a first predetermined area of an environment in which the device to be operated is located;
a recognition unit configured to recognize, based on the shot image of the first predetermined area, objects that can be grasped by the end effector;
an operation terminal configured to display the shot image and receive handwritten information input to the shot image;
an image adjustment unit configured to cause the imaging unit to shoot, when a part of an area of the shot image designated by the handwritten information is a predetermined outer peripheral area of the shot image before the input of the handwritten information is completed, an image of a second predetermined area including the part of the area of the shot image and an area adjacent to the first predetermined area, and cause the operation terminal to display the shot image of the second predetermined area; and
an estimation unit configured to estimate, based on the objects that can be grasped and the handwritten information, an object to be grasped which has been requested to be grasped by the end effector from among the objects that can be grasped and estimate a content of a motion with regard to the object to be grasped which is requested to be performed by the device to be operated.