US 12,390,738 B2
Plotting behind the scenes with learnable game engines
Willi Menapace, Santa Monica, CA (US); Aliaksandr Siarohin, Los Angeles, CA (US); and Sergey Tulyakov, Santa Monica, CA (US)
Assigned to Snap Inc., Santa Monica, CA (US)
Filed by Snap Inc., Santa Montica, CA (US)
Filed on Mar. 14, 2023, as Appl. No. 18/121,268.
Prior Publication US 2024/0307783 A1, Sep. 19, 2024
Int. Cl. A63F 13/67 (2014.01); A63F 13/57 (2014.01)
CPC A63F 13/67 (2014.09) [A63F 13/57 (2014.09); A63F 2300/66 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A non-transitory machine readable medium comprising instructions stored thereon that are executable by one or more processors to cause a computing device to implement a learnable game engine, comprising:
a synthesis neural network trained to generate an image from a high-level representation of a game environment state including at least one of pose, location, object style or velocity of objects in the game environment, a background style, and a camera view of the game environment, wherein the game environment state comprises a combination of all individual object states in the game environment; and
a diffusion-based animation neural network trained to model logic of a game including player actions and interactions in the high-level representation of the game environment states, the diffusion-based animation neural network producing states of the game environment based on user-provided conditioning signals for states and actions, wherein the produced states are subsequently rendered by the synthesis neural network, the diffusion-based animation neural network including a text encoder that derives values for the user-provided conditioning signals from natural language scripts including actions that a user wants to impose on one or more object properties in a gaming sequence of the game environment.