US 11,915,823 B1
Systems and methods for frame-based validation
Harsha Vardhan Pokkalla, Allston, MA (US); Hunter L. Elliott, Boston, MA (US); Dayong Wang, Wellesley, MA (US); Benjamin P. Glass, Boston, MA (US); Ilan N. Wapinski, Brookline, MA (US); Jennifer K. Kerner, Brookline, MA (US); Andrew H. Beck, Brookline, MA (US); Aditya Khosla, Lexington, MA (US); Sai Chowdary Gullapally, Boston, MA (US); and Ramprakash Srinivasan, Brookline, MA (US)
Assigned to PathAI, Inc., Boston, MA (US)
Filed by PathAI, Inc., Boston, MA (US)
Filed on Nov. 10, 2022, as Appl. No. 17/984,866.
Application 17/984,866 is a continuation of application No. 17/019,142, filed on Sep. 11, 2020, granted, now 11,527,319.
Claims priority of provisional application 62/900,387, filed on Sep. 13, 2019.
This patent is subject to a terminal disclaimer.
Int. Cl. G06K 9/00 (2022.01); G16H 30/40 (2018.01); G16H 30/20 (2018.01); G06N 3/08 (2023.01); G16H 50/20 (2018.01); G06F 18/23 (2023.01)
CPC G16H 30/40 (2018.01) [G06F 18/23 (2023.01); G06N 3/08 (2013.01); G16H 30/20 (2018.01); G16H 50/20 (2018.01)] 20 Claims
OG exemplary drawing
 
17. A system for validating performance of a trained model configured to predict at least one of a plurality of tissue and/or cellular characteristics categories from a pathology image, the system comprising:
at least one computer hardware processor; and
at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:
receiving, from a plurality of users, reference annotations each describing at least one of a plurality of tissue and/or cellular characteristic categories, for one or more frames in a set of frames for one or more pathology images, wherein each frame in the set of frames includes a portion of a pathology image of the one or more pathology images;
processing, using the trained model, the set of frames to generate model predictions of annotations, each predicted annotations describing at least one of the plurality of tissue and/or cellular characteristic categories, for a processed frame of the set of frames; and
validating performance of the trained model based on determining a degree of association between the reference annotations and the model predicted annotations for the set of frames.