US 12,148,102 B2
Dynamic augmentation of stimuli based on profile of user
Jorge Arroyo Palacios, Livermore, CA (US)
Assigned to Sony Interactive Entertainment Inc., Tokyo (JP)
Filed by Sony Interactive Entertainment Inc., Tokyo (JP)
Filed on Aug. 25, 2021, as Appl. No. 17/411,989.
Prior Publication US 2023/0063681 A1, Mar. 2, 2023
Int. Cl. G06T 19/00 (2011.01); G02B 27/01 (2006.01); G06F 18/214 (2023.01); G06T 7/50 (2017.01); G06T 13/40 (2011.01); G06T 17/20 (2006.01); G06V 20/10 (2022.01); G06V 40/10 (2022.01); G06V 40/16 (2022.01); G06V 40/20 (2022.01)
CPC G06T 19/006 (2013.01) [G02B 27/0172 (2013.01); G06F 18/214 (2023.01); G06T 7/50 (2017.01); G06T 13/40 (2013.01); G06T 17/20 (2013.01); G06V 20/10 (2022.01); G06V 40/10 (2022.01); G06V 40/168 (2022.01); G06V 40/174 (2022.01); G06V 40/20 (2022.01); G02B 2027/0138 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/30196 (2013.01)] 6 Claims
OG exemplary drawing
 
1. A method for augmenting a real-world scene viewed through augmented reality (AR) glasses, comprising:
capturing a video stream of images of a real-world person using a camera of the AR glasses;
determining that an overlay should be processed for an arm or a leg of the real-world person being viewed via the augmented reality (AR) glasses, the determining using artificial intelligence (AI) to identify a trigger scenario associated with the arm or the leg of the real-world person from the video stream of images, the trigger scenario being associated with an intensity level of movement of the arm or leg of the real-world person;
identifying the overlay to replace the arm or the leg of the real-world person in the video stream of images, the arm or the leg being associated with a rig of joints of the real-world person, and the arm or the leg being further associated with a mesh that defines a three-dimensional outline of a shape of the arm or the leg, the overlay being configured to mask the portion of the arm or the leg and replace the portion of the arm or the portion of the leg, and the overlay that replaces the portion of the arm or the portion of the leg includes animation that changes a range of motion of the movement of the arm or leg of the real-world person when viewed via the AR glasses; and
generating an augmented video stream that includes the video stream composited with the overlay, the augmented video stream being presented via the AR glasses instead of the video stream of images captured by the camera, the augmented video stream presenting an adjusted intensity level based on said animation to avoid presenting the arm or the leg of the real-world person that is identified to be associated with the trigger scenario via the AR glasses.