| CPC G06T 5/77 (2024.01) [G06T 5/50 (2013.01); G06T 7/215 (2017.01); G06T 7/251 (2017.01); G06T 7/75 (2017.01); G06T 11/40 (2013.01); G06V 10/26 (2022.01); G06V 20/40 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/20081 (2013.01)] | 20 Claims |
|
1. A computer-implemented method, comprising:
detecting, in a scene of a video, a first object to be tracked;
tracking, with a model, the first object as the first object moves through the scene;
determining, as part of the tracking of the first object, a first plurality of frames of the video that includes an unobscured representation of the first object;
segmenting, from each of the first plurality of frames, pixels representative of the first object;
storing, for each of the first plurality of frames, the pixels representative of the first object as visible object images;
determining, as part of the tracking of the first object, a second plurality of frames of the video that includes an obscured representation of the first object by an artifact;
segmenting, from each of the second plurality of frames, pixels representative of the first object and pixels representative of the obscured representation of the first object;
storing, for each of the second plurality of frames, the pixels representative of the first object and the pixels representative of the obscured representation of the first object as degraded object images;
refining, as the first object is tracked by the model and using the visible object images and the degraded object images, the model to at least learn an appearance of the artifact in the scene that obscures the first object in the second plurality of frames, wherein the appearance of the artifact includes at least a position, a size, and a shape of the artifact in the scene; and
subsequent to the refining:
detecting, in the scene of the video, a second object to be tracked;
tracking, with the refined model, the second object in the video as the second object moves within the scene;
determining a predicted tracked object appearance of the second object in a future frame of the video;
determining, with the model and based on the learned appearance of the artifact, that at least a portion of the predicted tracked object appearance of the second object in the future frame is at least partially obscured by the learned appearance of the artifact;
infilling artifact pixels of the learned appearance of the artifact in the scene with pixel values corresponding to the predicted tracked object appearance of the second object to generate an enhanced frame that illustrates the second object as at least partially visible through the learned appearance of the artifact in the scene; and
presenting the enhanced frame.
|