US 11,900,672 B2
Integrated internal and external camera system in vehicles
Rocky Chau-Hsiung Lin, Cupertino, CA (US); Thomas Yamasaki, Anaheim Hills, CA (US); Koichiro Kanda, San Jose, CA (US); Diego Rodriguez Risco, Campbell, CA (US); Alexander Joseph Ryan, Mountain View, CA (US); Samah Najeeb, Menlo Park, CA (US); and Samir El Aouar, San Jose, CA (US)
Assigned to ALPINE ELECTRONICS OF SILICON VALLEY, INC., Santa Clara, CA (US)
Filed by Alpine Electronics of Silicon Valley, Inc., Santa Clara, CA (US)
Filed on Apr. 19, 2019, as Appl. No. 16/389,779.
Claims priority of provisional application 62/661,492, filed on Apr. 23, 2018.
Prior Publication US 2019/0325219 A1, Oct. 24, 2019
Int. Cl. G06V 20/20 (2022.01); G06V 20/56 (2022.01); G06V 20/59 (2022.01); G06V 40/20 (2022.01)
CPC G06V 20/20 (2022.01) [G06V 20/56 (2022.01); G06V 20/59 (2022.01); G06V 40/20 (2022.01)] 16 Claims
OG exemplary drawing
 
1. A camera system for use with a vehicle, comprising:
a processor;
an external camera system coupled to the processor, wherein the external camera system is configured to capture a first set of images of an area around the vehicle;
an internal camera system coupled to the processor, wherein the internal camera system is configured to capture a second set of images of one or more passengers inside the vehicle; and
a display system coupled to the processor,
wherein the processor is configured to:
recognize, based on the second set of images, at least one gesture made by the one or more passengers,
determine, based on the second set of images, a pose of at least one passenger of the one or more passengers,
determine, based on the second set of images and the determined pose of the at least one passenger of the one or more passengers, a field of view of the at least one passenger of the one or more passengers,
modify the first set of images based on the determined field of view of the at least one passenger of the one or more passengers to correspond to the actual field of view of the one or more passengers, wherein the processor is configured to modify the first set of images at least by overlaying adjacent images of the first set of images, wherein the modified first set of images captures a portion of the area around the vehicle that the at least one passenger was viewing at the time that the at least one passenger made the at least one gesture,
identify, based on the modified first set of images and the at least one gesture, an object or location located outside the vehicle that the user gestured at,
receive a point-of-interest stream,
display graphical representations of a plurality of points-of-interest from the point-of-interest stream in an augmented reality format on the window surface before the processor recognizes the at least one gesture, and
display, using the display system, information from the point-of-interest stream related to the object or location to the one or more passengers on a window surface of the vehicle,
wherein the identified object or location located outside the vehicle corresponds to one of the plurality of points-of-interest, and
wherein the display system displays the information related to the object or location in an augmented reality format on the window surface.