US 12,136,443 B1
System, method, and computer program for automatically generating an audio track for a video based on a current state of the video
Matthew Harney, Bangkok (TH); and Gary Alan Lipkowitz, Brisbane, CA (US)
Assigned to GoAnimate, Inc., San Mateo, CA (US)
Filed by GoAnimate, Inc., San Mateo, CA (US)
Filed on Jun. 15, 2023, as Appl. No. 18/210,430.
Claims priority of provisional application 63/445,975, filed on Feb. 15, 2023.
Int. Cl. G11B 27/036 (2006.01); G06F 16/68 (2019.01); G06F 16/75 (2019.01); G06F 16/78 (2019.01)
CPC G11B 27/036 (2013.01) [G06F 16/686 (2019.01); G06F 16/75 (2019.01); G06F 16/7867 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A method, performed by a computer system, for automatically generating an audio track for a video based on a current state of the video, the method comprising:
providing a multimedia video production workspace for creating videos;
enabling a user to add a plurality of multimedia assets for a video to the multimedia workspace;
providing an audio-track generation platform comprising a plurality of machine-learning models trained to generate audio clips in response to receiving attributes of assets in the video production workspace; and
automatically generating an audio track for a video based on a current state of the video by performing the following:
identifying a current state of a video, including identifying all the assets in the current state of the video;
identifying attributes of the assets in the current state;
identifying one or more machine learning models in the audio-track generation platform to use to generate the audio track;
applying the identified machine-learning model(s) to the attributes of the assets in the current state to obtain a plurality of audio clips corresponding to the assets in the current state;
mixing the plurality of audio clips to create an audio track for the video;
presenting the audio track to the user; and
enabling the user to add the audio track to the video.