| CPC G06T 13/80 (2013.01) [G06T 3/18 (2024.01); G06T 7/215 (2017.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)] | 20 Claims |
|
1. A system comprising:
a memory component; and
one or more processing devices coupled to the memory component, the one or more processing devices to perform operations comprising:
obtaining a still image, wherein the still image includes at least one fluid element;
obtaining a selection mask that includes the at least one fluid element;
obtaining one or more motion hints indicating motion directions of the at least one fluid element of the still image;
generating, using an optical flow prediction pipeline and the motion directions of the one or more motion hints, a sparse optical flow map;
generating, using the optical flow prediction pipeline and the sparse optical flow map, an intermediary dense optical flow for the still image;
generating, using a flow refinement machine learning model, a refined dense optical flow from the intermediary dense optical flow, the still image, and the selection mask, wherein the refined dense optical flow is generated by refining the intermediary dense optical flow by the flow refinement machine learning model using the selection mask, the still image, and the one or more motion hints, and wherein the refined dense optical flow indicates a pattern of apparent motion for the at least one fluid element; and
generating a plurality of video frames by projecting a plurality of pixels of the still image using the refined dense optical flow.
|