US 11,056,012 B2
Systems and methods for generating improved environmental displays for vehicles
Jack Philip Cross, Reno, NV (US); William Edward Brandstetter, III, Sparks, NV (US); Igor Valigura, Sparks, NV (US); Justin Michael Bode, Reno, NV (US); Damien Bruno Jourdan, San Jose, CA (US); David McMartin Howard, Sparks, NV (US); Rodney Lee Pickens, Sparks, NV (US); Timothy L. Sweet, Sparks, NV (US); Joshua D. Gleason, Reno, NV (US); and Joseph H. Hansen, Jr., Kaysville, UT (US)
Assigned to SIERRA NEVADA CORPORATION, Sparks, NV (US)
Filed by SIERRA NEVADA CORPORATION, Sparks, NV (US)
Filed on Jul. 25, 2019, as Appl. No. 16/522,243.
Application 16/522,243 is a continuation of application No. 15/520,780, granted, now 10,410,531, previously published as PCT/US2015/059209, filed on Nov. 5, 2015.
Claims priority of provisional application 62/075,724, filed on Nov. 5, 2014.
Prior Publication US 2019/0355266 A1, Nov. 21, 2019
Int. Cl. G08G 5/04 (2006.01); G01C 23/00 (2006.01); B64D 45/08 (2006.01); B64D 47/08 (2006.01); G06T 17/00 (2006.01); G08G 5/00 (2006.01); G01S 13/95 (2006.01)
CPC G08G 5/045 (2013.01) [B64D 45/08 (2013.01); B64D 47/08 (2013.01); G01C 23/005 (2013.01); G01S 13/953 (2013.01); G06T 17/00 (2013.01); G08G 5/0021 (2013.01); G08G 5/0052 (2013.01); G08G 5/0086 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A machine implemented method for rendering a synthetic display of a scene including terrain and obstacles in a moving vehicle, comprising:
(a) loading from a database a priori scene data, including a priori terrain data and a priori obstacle data;
(b) accepting sensor data from at least one environmental sensor, wherein the sensor data include real-time terrain data and real-time obstacle data;
(c) transforming the sensor data into a 3-D data cloud comprising the real-time terrain data and the real-time obstacle data;
(d) fusing the a priori scene data with the 3-D data cloud in a scene fusion module so as to create a combined scene model comprising terrain data and obstacle data as a multi-resolution 3-D data structure with varying granularity;
(e) changing a resolution of at least one of the terrain data and the obstacle data in the multi-resolution 3-D data structure on the basis of motion of the vehicle;
(f) updating the combined scene model by receiving input from the scene fusion module and by feeding back to the scene fusion module data reflecting the updated combined scene model;
(g) rendering the combined scene model into an image signal; and
(h) displaying an image of a scene in response to the image signal.
 
9. A non-transitory, machine readable storage medium having stored thereon instructions for performing a method for rendering a display of a scene including terrain and obstacles in a moving vehicle, the instructions comprising machine executable code which, when executed by a machine, causes the machine to:
(a) load a priori scene data from database, wherein the a priori scene data include a priori terrain data and a priori obstacle data;
(b) accept sensor data from at least one environmental sensor, wherein the sensor data include real-time terrain data and real-time obstacle data;
(c) transform the sensor data from the at least one environmental sensor into a 3-D data cloud comprising the real-time terrain data and the real-time obstacle data;
(d) fuse the a priori scene data with the 3-D data cloud in a scene fusion module so as to create a combined scene model comprising terrain data and obstacle data as a multi-resolution 3-D data structure with varying granularity;
(e) change a resolution of at least one of the terrain data and the obstacle data in the multi-resolution 3-D data structure on the basis of motion of the vehicle;
(f) update the combined scene model by receiving input from the scene fusion module and by feeding back to the scene fusion module data reflecting the updated combined scene model; and
(g) render the combined scene model to fuse with 2-D imagery.
 
15. A system for rendering a synthetic display of a scene including terrain and obstacles in a moving vehicle, the system comprising:
a database comprising a priori scene data including a priori terrain data and a priori obstacle data;
an environmental sensor configured for providing real-time sensor data, including real-time terrain data and real-time obstacle data;
a memory having stored therein a set of machine-executable instructions that are executable to render a synthetic display of a scene including the a priori terrain data and the a priori obstacle data from the a priori scene data, and the real-time sensor data; and
a processor operatively coupled to the memory and configured to execute the machine executable instructions to:
(a) transform the sensor data from the environmental sensor into a 3-D data cloud comprising the real-time terrain data and the real-time obstacle data;
(b) fuse, in a scene fusion module, the a priori scene data with the real-time sensor data to create a combined scene model comprising terrain data and obstacle data as a multi-resolution 3-D data structure with varying degrees of granularity, wherein the combined scene model includes changes in resolution of at least one the terrain data and the obstacle data in the multi-resolution 3-D data structure on the basis of motion of the vehicle, the combined scene model being updated with input data from the scene fusion model, and by outputting to the scene fusion module data reflecting the updated scene model; and
(c) render the combined scene model into an image signal representing the synthetic display of the scene.