US 12,366,915 B2
Annotation data collection using gaze-based tracking
Elad Arbel, Santa Clara, CA (US); Itay Remer, Santa Clara, CA (US); and Amir Ben-Dor, Santa Clara, CA (US)
Assigned to Agilent Technologies, Inc., Santa Clara, CA (US)
Appl. No. 18/018,109
Filed by Agilent Technologies, Inc., Santa Clara, CA (US)
PCT Filed Jul. 20, 2021, PCT No. PCT/IB2021/056548
§ 371(c)(1), (2) Date Jan. 26, 2023,
PCT Pub. No. WO2022/023881, PCT Pub. Date Feb. 3, 2022.
Claims priority of provisional application 63/057,105, filed on Jul. 27, 2020.
Prior Publication US 2023/0266819 A1, Aug. 24, 2023
Int. Cl. G06T 11/00 (2006.01); G06F 3/01 (2006.01); G06F 3/04845 (2022.01); G06F 3/0485 (2022.01); G06T 3/40 (2006.01); G06T 5/50 (2006.01); G06T 7/30 (2017.01); G06V 10/70 (2022.01)
CPC G06F 3/013 (2013.01) [G06F 3/04845 (2013.01); G06F 3/0485 (2013.01); G06T 3/40 (2013.01); G06T 5/50 (2013.01); G06T 7/30 (2017.01); G06V 10/70 (2022.01); G06T 2207/10048 (2013.01); G06T 2207/10056 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20221 (2013.01); G06T 2207/30024 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method of automatically creating a training dataset comprising the step of:
(a) generating the training dataset, using a processor, wherein the training set comprises a plurality of records, and at least one a record includes:
an image of a sample of an object;
an indication of monitored manipulations by a user of a presentation of the sample; and
a ground truth indication of a monitored gaze of the user viewing the sample on a display or via an optical device mapped to pixels of the image of the sample, wherein the monitored gaze comprises at least one location of the sample the user has or is viewing and an amount of time spent viewing the at least one location;
wherein the monitored gaze is represented as a weak annotation, and the at least one record further comprises at least one of the following additional ground truth labels of the image of the sample:
when the sample comprises a sample of tissue of a subject: a pathology report created by the user viewing the sample, a pathological diagnosis created by the user viewing the sample, a sample score indicating a pathological evaluation for the sample created by the user viewing the sample, at least one clinical parameter of the subject whose sample is depicted in the sample, a history parameter of the subject, and/or an outcome of treatment administered to the subject,
when the sample comprises a manufactured microarray: a user provided indication of at least one manufacturing defect, an indication of pass/fail quality assurance test, and
when the sample comprises a live cell culture: a growth rate, a cell density, a homogeneity parameter, and/or a heterogeneity parameter, for the live cell culture.