| CPC G01Q 60/10 (2013.01) [G02B 21/10 (2013.01)] | 12 Claims |
|
1. A system for machine learning-driven operation of instrumentation with human in the loop, the system comprising:
a communication interface configured to
transmit instructions to a first instrument to acquire measurements of a physical characteristic of a sample, and receive from the first instrument corresponding physical-characteristic measurements, and
transmit instructions to a second instrument to scan a surface of the sample, and receive from the second instrument an image corresponding to the scanned sample surface;
a processor configured to
(a) access the image received through the communication interface;
(b) produce M×N patches of the image corresponding to non-overlapping locations of the sample surface, each image patch having m×n pixels and showing corresponding local features of the sample-surface structure;
(c) instruct the first instrument to acquire physical-characteristic measurements at sample-surface locations to which K of the M×N image patches correspond, where K<<M×N, and access the respective physical-characteristic measurements received through the communication interface;
(d) convert the physical-characteristic measurements corresponding to the K image patches into representations of the measurements;
(e) train a model to determine a relationship between the local features of the sample-surface structure shown in the K image patches and the measurement representations, wherein the processor is configured to perform K iterations of a training loop (c)-(e), each iteration associated with a pair of one of the K image patches and its corresponding measurement representation;
(f) predict, based on the trained model, representations of physical-characteristic measurements to be acquired at the sample-surface locations to which the remaining (M×N−K) image patches correspond, and estimate respective prediction uncertainties;
(g) select, based on an acquisition function associated with at least the prediction uncertainties, one of the remaining image patches corresponding to a sample-surface location for acquiring the next physical-characteristic measurement;
(h) instruct the first instrument to acquire the next physical-characteristic measurement at the sample-surface location to which the selected image patch corresponds, and access the acquired physical-characteristic measurement through the communication interface;
(i) convert the acquired physical-characteristic measurement into a representation thereof in association with the selected image patch; and
(j) retrain the model using the selected image patch and its associated measurement representation, wherein the processor is configured to iteratively perform a retraining loop (f)-(j) until reaching a training threshold; and
a user interface configured to receive one or more of
a respective selection input that, when provided by a user before the processor performs each of the K iterations of the training loop (c)-(e), identifies a respective image patch corresponding to the sample-surface location for acquiring the associated physical-characteristic measurement,
a reinforcement input that, when provided by a user after the processor performs one of the K iterations of the training loop (c)-(e), causes the processor to reinforce a result of the conversion of the associated physical-characteristic measurement into its corresponding measurement representation, or
an adjustment input that, when provided by the user after the processor performs one of the iterations of the retraining loop (f)-(j), causes the processor to change the basis of the selection of one of the remaining image patches corresponding to the sample-surface location for acquiring the next physical-characteristic measurement from a first acquisition function to a second acquisition function.
|