| CPC H04N 7/157 (2013.01) [G06F 3/013 (2013.01); G06N 3/04 (2013.01); G06N 3/045 (2023.01); G06T 7/11 (2017.01); G06T 7/70 (2017.01); G06T 15/04 (2013.01); G06T 15/20 (2013.01); G06T 15/205 (2013.01); G06T 17/20 (2013.01); G06T 19/20 (2013.01); H04N 7/144 (2013.01); H04N 7/147 (2013.01); H04N 7/152 (2013.01); G06T 19/00 (2013.01); G06T 2200/08 (2013.01); G06T 2207/30201 (2013.01); G06T 2219/2004 (2013.01)] | 21 Claims |
|
1. A method for updating a current three dimensional (3D) model of a person, comprising:
(a) participating, using a computerized system of the person, in a virtual 3D video conference;
(b) calculating current locations, within a two-dimensional (2D) space, of current face landmark points of a face of a person within a first image; the calculating is based on the current 3D model, and one or more current acquisition parameters of a 2D camera; wherein the current 3D model of the person is located within a 3D space;
(c) calculating second locations, within the 2D space, of second face landmark points of the face of the person within a second image that follows the first image;
(d) calculating correspondences between the current locations and the second locations;
(e) calculating, based on the correspondences, locations of the second face landmark points within the 3D space; and
(f) modifying the current 3D model based on the locations of the second face landmark points within the 3D space;
wherein steps (b)-(f) are also executed by the computerized device of the person; wherein step (a) also comprises capturing the first image and the second image by a camera of the computerized device of the person;
wherein the method further comprises:
determining parameters of the current 3D model of the person; wherein the parameters differ from image pixels and there are less than three hundred parameters of the current 3D model;
transmitting the parameters of the current 3D model, without transmitting the current 3D model, from the computerized device of the person to computerized devices of other participants of the virtual 3D video conference; wherein the transmitting comprises reducing latency by directly transmitting the parameters to the computerized devices of other participants without sending the parameters via a central server; and
reconstructing an avatar of the person by the computerized devices of the other participants based on the parameters of the current 3D model;
wherein the transmitting the parameters of the current 3D model is preceded by selecting the parameters of the current 3D model, out of a larger group of parameters of the current 3D model, based on (i) an available bandwidth allocated for transmission and (ii) a contribution of the parameters of the current 3D model to a visual perception of the face of the person; where the available bandwidth is several hundred bits per second.
|