US 11,853,895 B2
Mirror loss neural networks
Pierre Sermanet, Palo Alto, CA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Aug. 23, 2022, as Appl. No. 17/893,454.
Application 17/893,454 is a continuation of application No. 16/468,987, granted, now 11,453,121, previously published as PCT/US2018/023150, filed on Mar. 19, 2018.
Claims priority of provisional application 62/473,264, filed on Mar. 17, 2017.
Prior Publication US 2023/0020615 A1, Jan. 19, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G05B 19/04 (2006.01); G06N 3/084 (2023.01); B25J 9/16 (2006.01); G05B 13/02 (2006.01); G06V 10/70 (2022.01); G06V 10/82 (2022.01); G06V 20/52 (2022.01); H04N 7/18 (2006.01)
CPC G06N 3/084 (2013.01) [B25J 9/163 (2013.01); B25J 9/1697 (2013.01); G05B 13/027 (2013.01); G06V 10/70 (2022.01); G06V 10/82 (2022.01); G06V 20/52 (2022.01); H04N 7/181 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method of training a neural network having a plurality of network parameters, wherein the neural network is configured to receive an input observation characterizing a state of an environment and to process the input observation to generate a numeric embedding of the state of the environment, the method comprising:
obtaining a first observation captured by a first modality;
obtaining a second observation that is co-occurring with the first observation and that is captured by a second, different modality;
obtaining a third observation captured by the first modality that is not co-occurring with the first observation;
determining a gradient of a triplet loss that uses the first observation as an anchor example, the second observation as a positive example, and the third observation as a negative example; and
updating current values of the network parameters using the gradient of the triplet loss,
wherein the observations are images related to a same environment, wherein the first modality is a camera at a first viewpoint, and wherein the second modality is another camera at a second, different viewpoint.