US 12,216,737 B2
Systems, methods, and apparatuses for actively and continually fine-tuning convolutional neural networks to reduce annotation requirements
Zongwei Zhou, Tempe, AZ (US); Jae Shin, Chandler, AZ (US); and Jianming Liang, Scottsdale, AZ (US)
Assigned to Arizona Board of Regents on Behalf of Arizona State University, Scottsdale, AZ (US)
Filed by Arizona Board of Regents on behalf of Arizona State University, Scottsdale, AZ (US)
Filed on Mar. 18, 2022, as Appl. No. 17/698,805.
Claims priority of provisional application 63/163,656, filed on Mar. 19, 2021.
Prior Publication US 2022/0300769 A1, Sep. 22, 2022
Int. Cl. G06V 10/82 (2022.01); G06F 18/21 (2023.01); G06F 18/214 (2023.01); G06T 7/00 (2017.01); G06V 10/764 (2022.01); G16H 30/40 (2018.01)
CPC G06F 18/217 (2023.01) [G06F 18/2148 (2023.01); G06T 7/0012 (2013.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G16H 30/40 (2018.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30032 (2013.01); G06T 2207/30101 (2013.01); G06V 2201/032 (2022.01)] 15 Claims
OG exemplary drawing
 
1. A system comprising:
a memory to store instructions;
a processor to execute the instructions stored in the memory;
wherein the system is specially configured to actively and continually fine tune a convolutional neural network by performing the following operations:
generating a plurality of image candidates via a candidate generator, wherein the plurality of image candidates contain true positive samples of pathology and false positive samples of pathology;
determining a worthiness of each of the image candidates for annotation based on one or more of an entropy value and a diversity value calculated for each of a selected number of patches associated with each of the plurality of image candidates;
iteratively selecting for annotation, via an active, continual fine-tuning (ACFT) algorithm, a set of worthy image candidates from among the image candidates, based on the determined worthiness of each of the image candidates for annotation, and according to a sampling probability to inject randomization into the selecting;
annotating each of the image candidates in the selected set of worthy image candidates with a label; and
generating, via data augmentation, a plurality of patches for each labeled image candidate in the selected set of worthy image candidates, wherein the label for each labeled image candidate is applied to each of the plurality of patches associated with that labeled image candidate at an image candidate level.