US 12,272,110 B2
Lightweight real-time facial alignment network model selection process
Zihao Chen, Toronto (CA); Zhi Yu, Toronto (CA); and Parham Aarabi, Richmond Hill (CA)
Assigned to L'OREAL, Paris (FR)
Filed by L'OREAL, Paris (FR)
Filed on Mar. 3, 2022, as Appl. No. 17/685,691.
Claims priority of provisional application 63/155,839, filed on Mar. 3, 2021.
Claims priority of application No. 2201813 (FR), filed on Mar. 2, 2022.
Prior Publication US 2022/0284688 A1, Sep. 8, 2022
Int. Cl. G06V 10/20 (2022.01); G06Q 30/0601 (2023.01); G06V 10/24 (2022.01); G06V 10/70 (2022.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01); G06V 40/16 (2022.01)
CPC G06V 10/24 (2022.01) [G06Q 30/0631 (2013.01); G06V 10/776 (2022.01); G06V 10/82 (2022.01); G06V 10/87 (2022.01); G06V 40/16 (2022.01)] 15 Claims
OG exemplary drawing
 
14. A computing device comprising at least one or more processor coupled to a storage device storing instructions, that when executed by the at least one or more processor, cause the computing device to perform a method comprising:
defining a face alignment network (FAN) model for run-time execution by an edge device to process facial images, wherein the FAN model is a convolutional neural network that receives an image as input and identifies facial landmarks in the image as output, wherein defining comprises:
generating a set of candidate submodels from a network structure through training the network structure using i) a training dataset; and ii) expand and shrink training (EST) operations that define and retain candidate submodel instances of various structure parameters with which to define the set of candidate submodels;
wherein the EST operations initiate training from a baseline model previously defined in accordance with the network structure,
the EST operations define a search space for generation of candidate submodels in accordance with a plurality of search dimensions comprising depth size: kernel size; channel ratio; and expansion ratio;
wherein the EST operations initiate with a small depth size, a small kernel size, a large channel ratio, and a large expansion ratio and then operations, in phases and in the following order, gradually expand depth size and kernel size to respective maximums and, at ending phases, progressively shrink channel ratio and expansion ratio to respective minimums
and the method further includes
performing an evolutionary search of the candidate submodels using speed and accuracy evaluation criteria, and
selecting the FAN model from the candidate submodels based on the evolutionary search,
wherein the evolutionary search:
performs a plurality of cycles comprising:
sampling from a population of candidate submodels to select a quantity of candidate submodels;
selecting an optimal submodel from the quantity of candidate submodels according to at least some of the speed and accuracy evaluation criteria;
evolving the optimal submodel to determine a related submodel;
removing an oldest submodel from the population; and
if the related submodel satisfies a threshold test associated with the speed and accuracy evaluation criteria: adding the related submodel to the population for availability to subsequently sample and adding the related submodel to a set of final candidate submodels; and
following the performance of the plurality of cycles, selects one submodel from the set of final candidate submodels as the FAN model, in accordance with final selection criteria.