US 12,306,910 B2
System and method for markerless motion capture
Jordan Yallen, Los Angeles, CA (US); Walker Holmes, Indialantic, FL (US); and Joseph Poulose, Chatsworth, CA (US)
Assigned to Metatope LLC, Woodland Hills, CA (US)
Filed by Metatope LLC, Woodland Hills, CA (US)
Filed on Dec. 29, 2023, as Appl. No. 18/400,091.
Application 18/400,091 is a continuation of application No. 18/193,271, filed on Mar. 30, 2023, granted, now 11,860,981.
Claims priority of provisional application 63/362,179, filed on Mar. 30, 2022.
Claims priority of provisional application 63/369,734, filed on Jul. 28, 2022.
Claims priority of provisional application 63/362,860, filed on Apr. 12, 2022.
Claims priority of provisional application 63/369,735, filed on Jul. 28, 2022.
Claims priority of provisional application 63/382,314, filed on Nov. 4, 2022.
Prior Publication US 2024/0134941 A1, Apr. 25, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 21/10 (2013.01); G06T 13/40 (2011.01); G06T 17/00 (2006.01); G06V 40/16 (2022.01); H04N 21/8358 (2011.01); H04N 21/84 (2011.01)
CPC G06F 21/10 (2013.01) [G06T 13/40 (2013.01); G06T 17/00 (2013.01); G06V 40/171 (2022.01); H04N 21/8358 (2013.01); H04N 21/84 (2013.01); G06F 21/1014 (2023.08)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
capturing, by a camera associated with a computing system, markerless motion data of a user performing a movement, the capturing comprising:
capturing a first portion of the markerless motion data of the user, wherein the first portion of the markerless motion data of the user is captured within a threshold distance of the camera, the capturing comprising identifying a first plurality of points of the user to focus on, and
capturing a second portion of the markerless motion data of the user, wherein the second portion of the markerless motion data of the user is capture outside the threshold distance of the camera, the capturing comprising identifying a second plurality of points of the user to focus on, wherein the first plurality of points includes at least one point not included in the second plurality of points and wherein the second plurality of points includes at least one point not included in the first plurality of points;
retargeting, by the computing system, the first plurality of points and the second plurality of points to a three-dimensional model of an avatar associated with the user, and wherein retargeting the first plurality of points and the second plurality of points animates the three-dimensional model of the avatar to perform the movement performed by the user as the user performs the movement in real-time; and
in real-time, locally rendering, by the computing system, a video comprising the markerless motion data of the user retargeted to the three-dimensional model of the avatar causing hands, face, and body of the avatar to be animated in real-time.