US 11,951,622 B2
Domain adaptation using simulation to simulation transfer
Paul Wohlhart, Sunnyvale, CA (US); Stephen James, Santa Clara, CA (US); Mrinal Kalakrishnan, Palo Alto, CA (US); and Konstantinos Bousmalis, London (GB)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Mar. 23, 2022, as Appl. No. 17/656,137.
Application 17/656,137 is a continuation of application No. 16/692,509, filed on Nov. 22, 2019, granted, now 11,314,987.
Claims priority of application No. 20180100527 (GR), filed on Nov. 23, 2018.
Prior Publication US 2022/0215208 A1, Jul. 7, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 7/00 (2017.01); B25J 9/16 (2006.01); G05B 13/02 (2006.01); G06F 18/21 (2023.01); G06F 18/214 (2023.01); G06F 18/2431 (2023.01); G06N 3/045 (2023.01); G06N 3/08 (2023.01); G06T 7/50 (2017.01); G06V 10/764 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01); G06V 20/10 (2022.01)
CPC B25J 9/161 (2013.01) [B25J 9/163 (2013.01); B25J 9/1671 (2013.01); B25J 9/1697 (2013.01); G05B 13/027 (2013.01); G06F 18/2148 (2023.01); G06F 18/217 (2023.01); G06F 18/2431 (2023.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01); G06T 7/50 (2017.01); G06V 10/764 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01); G06V 20/10 (2022.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
receiving an input image of a real-word environment captured while a robotic agent is interacting with the real-world environment;
processing the input image of the real-world environment using a trained generator neural network to generate an adapted image from the input image, the trained generator neural network having been trained to adapt images of a randomized simulation of the real-world environment into images of a canonical simulation of the real-world environment;
providing the adapted image as input to a control policy for the robotic agent to select a subsequent action to be performed by the robotic agent; and
controlling the robotic agent to perform the selected subsequent action in the real-world environment.