US 11,785,179 B1
Image and audio data processing to create mutual presence in a video conference
Jules C. Graybill, Seattle, WA (US); Katherine M. Ryan, Seattle, WA (US); Maren Marie Costa, Seattle, WA (US); Marshall Friend Tappen, Bainbridge Island, WA (US); and Kristina Lisa Klinkner, Vashon, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Aug. 9, 2022, as Appl. No. 17/884,047.
Application 17/884,047 is a continuation of application No. 17/204,145, filed on Mar. 17, 2021, granted, now 11,451,746.
Claims priority of provisional application 63/000,120, filed on Mar. 26, 2020.
This patent is subject to a terminal disclaimer.
Int. Cl. H04N 7/15 (2006.01); G06T 7/11 (2017.01); G06V 10/75 (2022.01); G06V 10/25 (2022.01); G06T 7/70 (2017.01); G06V 40/10 (2022.01); G06T 3/60 (2006.01); H04N 23/69 (2023.01); H04N 23/80 (2023.01); H04N 23/90 (2023.01)
CPC H04N 7/15 (2013.01) [G06T 3/60 (2013.01); G06T 7/11 (2017.01); G06T 7/70 (2017.01); G06V 10/25 (2022.01); G06V 10/751 (2022.01); G06V 40/10 (2022.01); H04N 23/69 (2023.01); H04N 23/80 (2023.01); H04N 23/90 (2023.01); G06T 2207/20132 (2013.01); G06T 2207/30196 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A video conference system to facilitate communications between a local environment and a remote environment, comprising:
a first presentation device, a first, wide angle, high resolution imaging device, and a first rearward facing imaging device associated with the local environment;
a second presentation device, a second, wide angle, high resolution imaging device, and a second rearward facing imaging device associated with the remote environment; and
a controller configured to at least:
receive, from the second high resolution imaging device associated with the remote environment, image data of a second user within the remote environment of the video conference system;
identify the second user within the image data;
identify a second background associated with the second user within the image data;
receive, from the first rearward facing imaging device associated with the local environment, image data of the local environment behind the first presentation device;
determine at least one visual characteristic associated with the image data of the local environment from the first rearward facing imaging device;
modify the second background associated with the second user based at least in part on the determined at least one visual characteristic associated with the image data of the local environment;
present, via the first presentation device to a first user within the local environment, the image data of the second user having the modified second background;
receive, from the first high resolution imaging device associated with the local environment, image data of the first user within the local environment of the video conference system;
identify the first user within the image data;
identify a first background associated with the first user within the image data;
receive, from the second rearward facing imaging device associated with the remote environment, image data of the remote environment behind the second presentation device;
determine at least one visual characteristic associated with the image data of the remote environment from the second rearward facing imaging device;
modify the first background associated with the first user based at least in part on the determined at least one visual characteristic associated with the image data of the remote environment; and
present, via the second presentation device to the second user within the remote environment, the image data of the first user having the modified first background.