US 11,918,910 B1
System and method for synthetic interaction of objects
Gregory S. Rabin, New York, NY (US); and Steven M. Hoffberg, West Harrison, NY (US)
Filed by Steven M. Hoffberg, West Harrison, NY (US)
Filed on Aug. 30, 2021, as Appl. No. 17/461,057.
Application 17/461,057 is a continuation of application No. 16/779,533, filed on Jan. 31, 2020, granted, now 11,103,787, issued on Aug. 31, 2021.
Application 16/779,533 is a continuation of application No. 16/153,627, filed on Oct. 5, 2018, granted, now 10,549,197, issued on Feb. 4, 2020.
Application 16/153,627 is a continuation of application No. 15/789,621, filed on Oct. 20, 2017, granted, now 10,092,843, issued on Oct. 9, 2018.
Application 15/789,621 is a continuation of application No. 14/851,860, filed on Sep. 11, 2015, granted, now 9,795,882, issued on Oct. 24, 2017.
Application 14/851,860 is a continuation of application No. 13/161,820, filed on Jun. 16, 2011, granted, now 9,132,352, issued on Sep. 15, 2015.
Claims priority of provisional application 61/358,232, filed on Jun. 24, 2010.
Int. Cl. A63F 13/65 (2014.01); A63F 13/30 (2014.01); A63F 13/57 (2014.01); A63F 13/812 (2014.01); G06T 7/20 (2017.01); G06T 9/00 (2006.01); G06T 13/40 (2011.01); G06T 15/04 (2011.01); G06T 15/40 (2011.01)
CPC A63F 13/65 (2014.09) [A63F 13/30 (2014.09); A63F 13/57 (2014.09); A63F 13/812 (2014.09); G06T 7/20 (2013.01); G06T 9/00 (2013.01); G06T 13/40 (2013.01); G06T 15/04 (2013.01); G06T 15/40 (2013.01); G06T 2200/04 (2013.01); G06T 2200/08 (2013.01); G06T 2207/30221 (2013.01); G06T 2215/16 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
defining a physical model describing a plurality of real objects within a real environment extracted from a live action video of the plurality of real objects, representing, for each real object, at least a surface, a mass, a motion, acoustic properties, and interaction with other real objects;
defining a dynamic motion vector state of the plurality of real objects in the real environment extracted from the live action video of the plurality of real objects;
receiving an input from a user comprising an influence on the defined dynamic motion vector of at least one real object in the real environment to the defined dynamic motion vector state; and
synthesizing with at least one automated processor, a virtual view and audio output representing the plurality of real objects in the real environment as modified by the received user input, representing an extrapolation of the defined dynamic motion vector state of the plurality of real objects in the real environment and audio output according to at least the surface, the mass, the motion, acoustic properties, and the interaction with other real objects according to the model modified by the received input from the user, the extrapolation comprising a modification of a movement of the plurality of real objects within the real environment, representing a synthetic interaction of the plurality of real objects in the real environment, different from an interaction of the plurality of real objects according to the defined motion vector state and the physical model in the real environment absent the received input from the user.