US 11,055,989 B2
Viewpoint invariant object recognition by synthesization and domain adaptation
Kihyuk Sohn, Fremont, CA (US); Luan Tran, Haslett, MI (US); Xiang Yu, Mountain View, CA (US); and Manmohan Chandraker, Santa Clara, CA (US)
Assigned to NEC Corporation
Filed by NEC Laboratories America, Inc., Princeton, NJ (US)
Filed on Aug. 1, 2018, as Appl. No. 16/51,924.
Claims priority of provisional application 62/553,090, filed on Aug. 31, 2017.
Claims priority of provisional application 62/585,758, filed on Nov. 14, 2017.
Prior Publication US 2019/0066493 A1, Feb. 28, 2019
Int. Cl. G08G 1/017 (2006.01); G06K 9/62 (2006.01); G06K 9/00 (2006.01); G06N 3/08 (2006.01); G06N 3/04 (2006.01); G06N 5/04 (2006.01); G06T 17/00 (2006.01); G06N 20/00 (2019.01); G08G 1/01 (2006.01); G08G 1/04 (2006.01); G08G 1/048 (2006.01); G06K 9/46 (2006.01)
CPC G08G 1/0175 (2013.01) [G06K 9/00671 (2013.01); G06K 9/00771 (2013.01); G06K 9/4628 (2013.01); G06K 9/6256 (2013.01); G06K 9/6268 (2013.01); G06N 3/0454 (2013.01); G06N 3/0472 (2013.01); G06N 3/08 (2013.01); G06N 5/046 (2013.01); G06N 20/00 (2019.01); G06T 17/00 (2013.01); G08G 1/0116 (2013.01); G08G 1/0129 (2013.01); G08G 1/04 (2013.01); G08G 1/048 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for performing domain adaptation using a hardware processor, the method comprising:
collecting a labeled source image having a view of an object including a vehicle;
generating, using the processor, a plurality of transformed view augmented source images based on the source image by synthesizing a plurality of viewpoints of the object in the source image into the transformed view augmented source images;
generating a plurality of lighting and view augmented source images by adjusting photometrics of each of the plurality of viewpoints of the object in the source image;
extracting features from each of the plurality of lighting and view augmented source images with a first feature extractor and from each of a plurality of captured images captured by an image capture device with a second feature extractor;
classifying the extracted features using domain adaptation with domain adversarial learning between extracted features of the captured images and extracted features of the lighting and view augmented source images to generate labels corresponding to the object, wherein an entropy regularized version of the domain adaptation is determined by

OG Complex Work Unit Math
wherein θƒ are parameters, custom characterƒ is the cross entropy loss, λ is a hyper-parameter, i is an index from 1 to N+1, γ is a hyper-parameter, custom characterχs log C(ƒ(x), (y|custom character)) is a loss attributable to classification, χs is a set of source domain data, χt is a set of target domain data, custom characterχs is an expected value for the source domain, custom characterχt is an expected value for the target domain, C is a class score function, ƒ(x) is a feature representation function, y is a class label of a set of labels, and N is a number of categories of class labels;
generating a labeled target image corresponding to each of the captured images including labels corresponding to classifications of the extracted features of the captured images identifying vehicles in each of the captured images; and
automatically logging the objects in each of the captured images such that a user can identify a vehicle by viewing only the objects in each of the captured images using a display device.