US 12,067,658 B1
Method, apparatus and device for automatically making up portrait lips, storage medium and program product
Xiao Ma, Shanghai (CN); Wangdu Chen, Shanghai (CN); Qi Wang, Shanghai (CN); Xinghao Pan, Shanghai (CN); and Kangjing Li, Shanghai (CN)
Assigned to MIGU VIDEO CO., LTD, Shanghai (CN); Migu Co., Ltd., Beijing (CN); and China Mobile Communications Group Co., Ltd., Beijing (CN)
Appl. No. 18/573,093
Filed by MIGU VIDEO CO., LTD, Shanghai (CN); Migu Co., Ltd, Bejing (CN); and China Mobile Communications Group Co., Ltd., Beijing (CN)
PCT Filed Aug. 25, 2022, PCT No. PCT/CN2022/114755
§ 371(c)(1), (2) Date Dec. 21, 2023,
PCT Pub. No. WO2023/025239, PCT Pub. Date Mar. 2, 2023.
Claims priority of application No. 202110982907.2 (CN), filed on Aug. 25, 2021.
Int. Cl. G06T 11/00 (2006.01); G01J 3/46 (2006.01); G01J 3/52 (2006.01); G06T 7/13 (2017.01); G06T 7/50 (2017.01); G06T 7/73 (2017.01); G06T 7/90 (2017.01); G06T 11/60 (2006.01); G06V 10/56 (2022.01); G06V 40/16 (2022.01)
CPC G06T 11/60 (2013.01) [G06T 7/13 (2017.01); G06T 7/50 (2017.01); G06T 7/73 (2017.01); G06T 7/90 (2017.01); G06T 11/001 (2013.01); G06V 10/56 (2022.01); G06V 40/171 (2022.01); G01J 3/463 (2013.01); G01J 3/526 (2013.01); G06V 40/161 (2022.01)] 5 Claims
OG exemplary drawing
 
1. A method for automatically making up portrait lips, comprising:
extracting lip key points from a portrait facial image and detecting a portrait facial orientation and a lip shape in the portrait facial image; adjusting positions of the lip key points based on the portrait facial orientation and the lip shape;
detecting a skin hue and a skin color number of a facial area in the portrait facial image;
and selecting a target lipstick color from a lipstick color sample library based on the skin hue and the skin color number; and,
performing fusion coloring by using the target lipstick color according to the positions of the lip key points;
wherein the adjusting positions of the lip key points based on the portrait facial orientation and the lip shape, includes:
for side lip corner key points in the lip key points, when the portrait facial orientation is front-facing and the lip shape is closed, finding two darkest pixel points in a front-facing adjustment area, and taking a middle position between the two pixel points as an adjusted position of the side lip corner key points; wherein the front-facing adjustment area is an area formed by extending a first preset number of pixel points around positions of the side lip corner key points;
when the portrait facial orientation is front-facing and the lip shape is open, performing edge detection on the front-facing adjustment area, and determining a first reference point according to a first edge detection result; by taking the first reference point as a center, finding a first skin color boundary point and a first lip color boundary point in a skin color area and a lip color area; and taking a middle position between the first skin color boundary point and the first lip color boundary point as an adjusted position of the side lip corner key points;
when the portrait facial orientation is left-leaning or right-leaning and the lip shape is closed, finding a darkest pixel point in a side-leaning adjustment area and adjusting positions of the side lip corner key points according to a relative position of the darkest pixel in the side-leaning adjustment area; wherein the side-leaning adjustment area is an area formed by extending a second preset number of pixel points in left-right directions and a third preset number of pixel points in up-down directions from the positions of the side lip corner key points;
when the portrait facial orientation is left-leaning or right-leaning and the lip shape is open, finding a darkest pixel point in the side-leaning adjustment area, performing edge detection on the side-leaning adjustment area, and determining a second reference point according to a second edge detection result; by taking the second reference point as a center, finding a second skin color boundary point and a second lip color boundary point in a skin color area and a lip color area, and adjusting positions of the side lip corner key points according to a relative position of the darkest pixel point in the side-leaning adjustment area, the second skin color boundary point and the second lip color boundary point.