US 11,861,805 B2
Eyeball positioning for 3D head modeling
Jie Hu, San Jose, CA (US); and Mohammad Gharavi-Alkhansari, San Jose, CA (US)
Assigned to SONY GROUP CORPORATION, Tokyo (JP)
Filed by SONY GROUP CORPORATION, Tokyo (JP)
Filed on Sep. 22, 2021, as Appl. No. 17/481,846.
Prior Publication US 2023/0101230 A1, Mar. 30, 2023
Int. Cl. G06T 19/20 (2011.01); G06T 17/00 (2006.01)
CPC G06T 19/20 (2013.01) [G06T 17/00 (2013.01)] 19 Claims
OG exemplary drawing
 
1. An electronic device, comprising:
circuitry configured to:
acquire a set of images comprising an eye of an object;
acquire a three-dimensional (3D) mesh of a head portion of the object;
acquire a 3D template mesh of an eyeball;
process the acquired set of images to extract 3D feature points associated with a plurality of regions of the eye;
fit a sphere to the extracted 3D feature points;
estimate an initial pose transformation between the 3D template mesh and the fitted sphere, wherein
the initial pose transformation includes estimation of a scale factor that corresponds to a ratio of a radius of the fitted sphere and a radius of the 3D template mesh, and
the scale factor represents a factor to scale and match a size of the 3D template mesh to a size of an eyeball socket of the 3D mesh of the object;
execute one or more operations by using the 3D template mesh, to interpolate a first set of points that correspond to the plurality of regions of the eye;
determine a second set of points which corresponds to the plurality of regions of the eye based on sampling parameters associated with the interpolated first set of points;
determine a final pose transformation based on a minimization of a difference between the first set of points and the second set of points; and
fit the 3D template mesh into the eyeball socket of the 3D mesh, based on the determined final pose transformation and the estimated scale factor.