US 11,967,166 B2
Method and system for automated identification and classification of marine life
Tianye Wang, Halifax (CA); Shiwei Liu, Lehi, UT (US); and Xiaoge Cheng, Beijing (CN)
Filed by Tianye Wang, Halifax (CA); Shiwei Liu, Lehi, UT (US); and Xiaoge Cheng, Beijing (CN)
Filed on Nov. 23, 2021, as Appl. No. 17/533,224.
Claims priority of provisional application 63/116,939, filed on Nov. 23, 2020.
Prior Publication US 2022/0164570 A1, May 26, 2022
Int. Cl. G06V 40/10 (2022.01); G06F 18/213 (2023.01); G06F 18/241 (2023.01); G06N 3/04 (2023.01); G06T 7/11 (2017.01); G06V 10/94 (2022.01)
CPC G06V 40/10 (2022.01) [G06F 18/213 (2023.01); G06F 18/241 (2023.01); G06N 3/04 (2013.01); G06T 7/11 (2017.01); G06V 10/95 (2022.01); G06T 2207/20084 (2013.01)] 10 Claims
OG exemplary drawing
 
1. A computer-implemented method of identifying and classifying marine creatures, comprising:
receiving, by a Deep Neural Network (DNN) comprising a plurality of blocks of convolutional layers and activation layers, an image of an underwater environment;
generating, by the DNN, at least one feature map representing the image and passing the at least one feature map to a region proposal network and a classification network;
determining, by the region proposal network, a target object probability for each of the features of the at least one feature map and passing the probabilities for one or more of the target objects to the classification network;
calculating, by the region proposal network, the image coordinates for the one or more target objects and passing the image coordinates to a semantic segmentation network;
receiving, by the classification network, the at least one feature map and the probabilities for the one or more target objects and determining an object class and an object bounding box for each of the one or more target objects;
receiving, by the semantic segmentation network, the feature map and the coordinates for the one or more target objects and generating a segmentation mask for each respective target object;
receiving, by a post-processing module, the image and the object class, mask, and object bounding box for each target object and generating an overlaid image wherein each identified object is segmented and classified to aid with differentiation between target objects in post-processing and labelling operations.