US 12,354,294 B2
Positional synchronization of virtual and physical cameras
Joel N. Kerr, San Jose, CA (US); Maneli Noorkami, Menlo Park, CA (US); and Perry A. Caro, San Jose, CA (US)
Assigned to Apple Inc., Cupertino, CA (US)
Filed by Apple Inc., Cupertino, CA (US)
Filed on Sep. 9, 2020, as Appl. No. 17/016,210.
Claims priority of provisional application 62/897,910, filed on Sep. 9, 2019.
Prior Publication US 2021/0074014 A1, Mar. 11, 2021
Int. Cl. G06T 7/70 (2017.01); G06T 7/10 (2017.01); G06T 7/20 (2017.01); G06T 19/00 (2011.01); H04N 5/04 (2006.01)
CPC G06T 7/70 (2017.01) [G06T 7/10 (2017.01); G06T 19/006 (2013.01); H04N 5/04 (2013.01); G06T 7/20 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A device comprising:
a memory; and
at least one processor configured to:
determine a first position of a physical camera relative to the device in a physical environment, the physical camera being separate from the device;
initiate positioning of a virtual camera in a second position within a computer-generated environment, wherein the second position of the virtual camera relative to a virtual representation corresponding to a person in the computer-generated environment is synchronized with the first position of the physical camera relative to the device in the physical environment, the person being associated with the device;
receive a plurality of image frames captured using the physical camera of the physical environment and a plurality of virtual image frames generated using the virtual camera of the computer-generated environment, the plurality of image frames capturing movement of the person and the plurality of virtual image frames capturing movement of the virtual representation; and
generate a plurality of computer-generated reality image frames, each respective computer-generated reality frame comprising at least a portion of a respective image frame of the plurality of image frames composited with at least a portion of a respective virtual image frame of the plurality of virtual image frames, wherein the at least the portion of the respective image frame comprises a respective segmented image of the person that replaces the virtual representation corresponding to the person in the respective computer-generated reality image frame, the virtual representation being positionally aligned with the respective image of the person throughout the movement of the person across the plurality of image frames and the movement of the virtual representation across the plurality of virtual image frames based on the second position of the virtual camera being synchronized with the first position of the physical camera.