US 12,246,456 B2
Image generation device, robot training system, image generation method, and non-transitory computer readable storage medium
Kin Chung Denny Fu, Kyoto (JP); Yuki Yamaguchi, Nara (JP); Yohei Okawa, Koshigaya (JP); Kennosuke Hayashi, Tokyo (JP); Chisato Saito, Tokyo (JP); and Yoshiya Shibata, Tokyo (JP)
Assigned to OMRON Corporation, Kyoto (JP)
Appl. No. 17/269,992
Filed by OMRON Corporation, Kyoto (JP)
PCT Filed Oct. 18, 2019, PCT No. PCT/JP2019/041015
§ 371(c)(1), (2) Date Feb. 19, 2021,
PCT Pub. No. WO2020/110505, PCT Pub. Date Jun. 4, 2020.
Claims priority of application No. 2018-224285 (JP), filed on Nov. 29, 2018.
Prior Publication US 2021/0331311 A1, Oct. 28, 2021
Int. Cl. B25J 9/16 (2006.01); B25J 13/08 (2006.01); G06N 20/00 (2019.01); G06T 7/70 (2017.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/52 (2022.01)
CPC B25J 9/1697 (2013.01) [B25J 9/1605 (2013.01); B25J 9/163 (2013.01); B25J 13/08 (2013.01); G06N 20/00 (2019.01); G06T 7/70 (2017.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/52 (2022.01); G06T 2207/20081 (2013.01); G06T 2207/20212 (2013.01); G06T 2207/30164 (2013.01)] 9 Claims
OG exemplary drawing
 
1. An image generation device configured to generate a learning image for training an action of a robot which carries out a prescribed operation on a workpiece on the basis of an input image, comprising:
a hardware storage, storing a first image acquisition unit, a second image acquisition unit and a learner; and
a hardware processor, coupled to the hardware storage and configured to:
execute the first image acquisition unit to acquire a real image of a real operation space including the robot and not including the workpiece, wherein the real image includes an imaging of the robot and does not include an imaging of the workpiece; and
execute the second image acquisition unit to acquire a virtual image in which a virtual operation space including a virtual robot corresponding to the robot and a virtual workpiece corresponding to the workpiece is rendered;
wherein the learner is trained by machine learning, and the learner includes: a generator and a fourth image acquisition unit,
the hardware processor is configured to:
execute the fourth image acquisition unit to obtain the virtual workpiece based on the virtual image and generate a composite image by adding the virtual workpiece to the real image; and
execute the generator to generate the learning image by using the virtual image and the composite image, wherein the learning image is obtained by converting the virtual image such that at least the virtual robot included in the virtual image is made to approximate the robot included in the composite image is output.