US 11,880,979 B2
Method and apparatus with video segmentation
Seungin Park, Yongin-si (KR); and Seokhwan Jang, Anyang-si (KR)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed on Mar. 29, 2022, as Appl. No. 17/707,023.
Application 17/707,023 is a continuation of application No. 16/900,649, filed on Jun. 12, 2020, granted, now 11,321,848.
Claims priority of application No. 10-2019-0148849 (KR), filed on Nov. 19, 2019.
Prior Publication US 2022/0222828 A1, Jul. 14, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 7/11 (2017.01); G06T 7/168 (2017.01); G06T 7/174 (2017.01); G06N 3/04 (2023.01); G06T 7/149 (2017.01); G06V 10/82 (2022.01); G06V 20/40 (2022.01)
CPC G06T 7/11 (2017.01) [G06N 3/04 (2013.01); G06T 7/149 (2017.01); G06T 7/168 (2017.01); G06T 7/174 (2017.01); G06V 10/82 (2022.01); G06V 20/46 (2022.01); G06T 2207/10016 (2013.01); G06T 2207/20084 (2013.01)] 12 Claims
OG exemplary drawing
 
1. A method with video segmentation, comprising: acquiring, overtime, a video sequence comprising a plurality of image frames, the plurality of image frames including a second image frame corresponding to a time t of the video sequence and a first image frame corresponding to a time t−1 before the time t;
extracting a second feature vector from the second image frame;
generating second hidden state information corresponding to the second image frame, by using information related to the second image frame based on a relation between a first feature vector corresponding to at least one object included in the second image frame stored in a memory and the second feature vector, wherein the information related to the second image frame is determined to have a predetermined relation with the second image frame from among hidden state information of a plurality of the image frames corresponding to a time before the time t;
generating a second segmentation mask corresponding to the second image frame, based on an output vector corresponding to the second hidden state information;
determining a dissimilarity based on an entropy-based correlation between the hidden state information stored in the memory and the second feature vector;
storing the second hidden state information in the memory, based on a result of comparing the dissimilarity to a preset reference value; and
outputting the second segmentation mask.