US 11,722,727 B2
Special effect processing method and apparatus for live broadcasting, and server
Bingyuan Li, Beijing (CN)
Assigned to BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD., Beijing (CN)
Filed by BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD., Beijing (CN)
Filed on Jun. 24, 2021, as Appl. No. 17/357,968.
Claims priority of application No. 202010594821.8 (CN), filed on Jun. 28, 2020.
Prior Publication US 2021/0321157 A1, Oct. 14, 2021
Int. Cl. H04N 21/431 (2011.01); G06T 5/50 (2006.01); H04N 21/2187 (2011.01); G06V 40/10 (2022.01)
CPC H04N 21/4312 (2013.01) [G06T 5/50 (2013.01); G06V 40/103 (2022.01); H04N 21/2187 (2013.01); G06T 2207/20221 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A special effect processing method for live broadcasting, comprising:
acquiring a video stream collected during a live broadcasting by a live broadcasting client;
receiving positions of key points of a human body contained in each frame of the video stream sent by the live broadcasting client, wherein the positions are determined by extracting features from each frame of the video stream to obtain respective feature map, inputting the respective feature map into a trained human body key point detection model and determining positions of the key points based on an output from the trained human body key point detection model;
for each frame, adjusting positions of target key points contained in a virtual object template based on the positions of the key points of the human body in the frame to obtain a target virtual object corresponding to the frame;
acquiring a size of the human body and a center position of the human body recognized by performing the human body recognition on the frame through the live broadcasting client;
adjusting a size of the target virtual object based on the size of the human body to obtain an adjusted target virtual object and determining a fusion position in the frame for the target virtual object based on the center position of the human body in the frame;
fusing the adjusted target virtual object corresponding to the frame and the frame of the video stream based on the fusion position to generate a target video stream subjected to special effect processing; and
sending the target video stream to the live broadcasting client and a user client.