US 12,250,491 B2
Method for generating a bird's eye view image
Tobias Biester, Karlsruhe (DE); Boris Neubert, Karlsruhe (DE); and Dennis Keck, Karlsruhe (DE)
Assigned to DSPACE GMBH, Paderborn (DE)
Filed by dSPACE GmbH, Paderborn (DE)
Filed on Dec. 9, 2022, as Appl. No. 18/063,776.
Claims priority of application No. 102021133738.0 (DE), filed on Dec. 17, 2021; and application No. 21215590 (EP), filed on Dec. 17, 2021.
Prior Publication US 2023/0199135 A1, Jun. 22, 2023
Int. Cl. G06T 3/00 (2024.01); G06F 30/20 (2020.01); G06T 7/11 (2017.01); G06T 7/35 (2017.01); G06T 7/80 (2017.01); G06T 17/20 (2006.01); G06V 10/764 (2022.01); G07C 5/06 (2006.01); H04N 5/262 (2006.01); H04N 23/69 (2023.01)
CPC H04N 5/2628 (2013.01) [G06F 30/20 (2020.01); G06T 7/11 (2017.01); G06T 7/35 (2017.01); G06T 7/80 (2017.01); G06T 17/20 (2013.01); G06V 10/764 (2022.01); G07C 5/06 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20221 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A computer-implemented method for generating a bird's eye view image of a scene, the method comprising the steps:
(a) acquiring, by a lidar sensor, at least one lidar frame comprising points with inherent distance information and acquiring, by a camera, at least one camera image of the scene;
(b) generating a mesh representation of the scene based on the at least one lidar frame, wherein the mesh representation represents surfaces shown in the scene with inherent distance information;
(c) generating a mask image, wherein generating the mask image comprises:
generating a segmented lidar frame, wherein generating the segmented lidar frame comprises classifying points of the at least one lidar frame as representing ground points or non-ground points of the scene based on the mesh representation; and
mapping the segmented lidar frame to the at least one camera image to classify pixels of the at least one camera image as representing ground pixels or non-ground pixels of the at least one camera image; and
(d) generating the bird's eye view image based on the mask image and the mesh representation, wherein generating the bird's eye view image comprises performing an inverse perspective mapping on the at least one camera image using the inherent distance information from the mesh representation, and wherein in the bird's eye view image non-ground objects are masked out based on the classification of pixels of the at least one camera image as representing ground pixels or non-ground pixels.