US 11,941,884 B2
Multi-source panoptic feature pyramid network
Jason Wen Yong Kuen, Santa Clara, CA (US); Bo Sun, San Jose, CA (US); Zhe Lin, Clyde Hill, WA (US); and Simon Su Chen, San Jose, CA (US)
Assigned to ADOBE INC., San Jose, CA (US)
Filed by ADOBE INC., San Jose, CA (US)
Filed on Nov. 12, 2021, as Appl. No. 17/454,740.
Prior Publication US 2023/0154185 A1, May 18, 2023
Int. Cl. G06K 9/00 (2022.01); G06F 18/21 (2023.01); G06N 3/08 (2023.01); G06T 9/00 (2006.01); G06V 10/75 (2022.01); G06V 20/40 (2022.01)
CPC G06V 20/41 (2022.01) [G06F 18/2163 (2023.01); G06N 3/08 (2013.01); G06T 3/4046 (2013.01); G06T 9/002 (2013.01); G06V 10/751 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A method for training a neural network, comprising:
encoding a first image from a first training set to obtain first image features, wherein the first training set includes ground truth object detection information corresponding to the first image;
decoding the first image features to obtain first object features using a shared decoder;
generating object detection information based on the first object features using an object detection branch;
comparing the object detection information with the ground truth object detection information to obtain an object detection loss;
updating parameters of the object detection branch based on the object detection loss;
encoding a second image from a second training set to obtain second image features, wherein the second training set includes ground truth semantic segmentation information corresponding to the second image;
decoding the second image features to obtain second object features using the shared decoder;
generating semantic segmentation information based on the second object features using a semantic segmentation branch;
comparing the semantic segmentation information with the ground truth semantic segmentation information to obtain a semantic segmentation loss; and
updating parameters of the semantic segmentation branch based on the semantic segmentation loss.