US 12,094,161 B2
Multi-perspective microscopic imaging guided microinjection of microscopic objects across large field of views
Suhasa B Kodandaramaiah, St Paul, MN (US); Daryl M. Gohl, Minneapolis, MN (US); Andrew Alegria, Minneapolis, MN (US); Amey S Joshi, Minneapolis, MN (US); and Benjamin Auch, Saint Paul, MN (US)
Assigned to Regents of the University of Minnesota, Minneapolis, MN (US)
Filed by Regents of the University of Minnesota, Minneapolis, MN (US)
Filed on Mar. 23, 2022, as Appl. No. 17/656,130.
Claims priority of provisional application 63/200,721, filed on Mar. 24, 2021.
Prior Publication US 2022/0309705 A1, Sep. 29, 2022
Int. Cl. G06T 7/73 (2017.01); G06T 3/40 (2006.01)
CPC G06T 7/74 (2017.01) [G06T 3/40 (2013.01); G06T 2207/10012 (2013.01); G06T 2207/20084 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for injecting a substance into a particular microscopic object of a plurality of microscopic objects, the system comprising:
a pressure controller configured to cause a micropipette to eject the substance from a tip of the micropipette;
a stereoscopic imaging apparatus configured to generate stereoscopic image data, wherein the stereoscopic image data comprises a stereoscopic image of the tip of the micropipette and the particular microscopic object of the plurality of microscopic objects;
a first camera configured to generate first image data, wherein the first image data comprises a first image of the plurality of microscopic objects at a first magnification;
a second camera configured to generate second image data, wherein the second image data comprises a second image of the particular microscopic object at a second magnification; and
one or more computing devices configured to:
determine, based on the stereoscopic image of the tip of the micropipette, a location of the tip of the micropipette on an x-axis, a y-axis, and a z-axis;
apply a machine-learned model to determine, based on the first image of the plurality of microscopic objects, a location of the particular microscopic object of the plurality of microscopic objects on the x-axis and the y-axis;
adjust a depth of focus of the second camera to focus, based on the second image of the particular microscopic object, on the particular microscopic object, wherein the depth of focus corresponds to a distance between the second camera and the particular microscopic object when the second camera focuses on the particular microscopic object;
determine, based on the depth of focus of the second camera, the location of the particular microscopic object on the z-axis; and
control the pressure controller to cause the micropipette to eject the substance from the tip of the micropipette and into the particular microscopic object.