US 12,229,671 B2
Method and device for optimized operation of a machine learning model with a data stack for a driving assistance function of a vehicle
Fabian Hüger, Wolfenbüttel (DE); and Peter Schlicht, Wolfsburg (DE)
Assigned to VOLKSWAGEN AKTIENGESELLSCHAFT, Wolfsburg (DE)
Appl. No. 17/414,675
Filed by Volkswagen Aktiengesellschaft, Wolfsburg (DE)
PCT Filed Nov. 25, 2019, PCT No. PCT/EP2019/082486
§ 371(c)(1), (2) Date Jun. 16, 2021,
PCT Pub. No. WO2020/126339, PCT Pub. Date Jun. 25, 2020.
Claims priority of application No. 10 2018 222 202.9 (DE), filed on Dec. 18, 2018.
Prior Publication US 2022/0044118 A1, Feb. 10, 2022
Int. Cl. G06V 20/58 (2022.01); G06F 17/16 (2006.01); G06F 18/214 (2023.01); G06N 3/08 (2023.01); G06V 10/44 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 40/10 (2022.01)
CPC G06N 3/08 (2013.01) [G06F 17/16 (2013.01); G06F 18/214 (2023.01); G06V 10/454 (2022.01); G06V 10/764 (2022.01); G06V 10/82 (2022.01); G06V 20/58 (2022.01); G06V 40/103 (2022.01)] 14 Claims
OG exemplary drawing
 
1. A method for optimized operation of a machine learning model in a vehicle using a driving assistance function, comprising, during a training phase:
receiving tagged multidimensional training data from one or more cameras, which training data comprise images from surroundings of one or more training vehicles;
selecting subsets from the received training data to compress input dimensions of the machine learning model, wherein the subsets are image sections of the images, wherein the selection of the subsets is carried out on based on a relevance of the respective subsets, wherein the subsets are selected depending on a security-relevant property of an object depicted in the subsets or the image section, wherein low-contrast subsets or image sections are selected;
generating a training data set, wherein the training data set includes data set elements which are generated on the basis of the selected subsets;
training the machine learning model for a driving assistance function of the vehicle using the training data set;
conducting, during an inference phase, conducted as a part of the driving assistance function of the vehicle during automated driving;
receiving sensor data of at least one image sensor, which sensor data comprises images from surroundings of the vehicle;
selecting subsets from the received sensor data, wherein the subsets are image sections, wherein the selection of the subsets is carried out based on a relevance of the respective subsets, wherein the subsets are selected depending on a safety-relevant property of an object depicted in the subset or the image section, wherein low-contrast subsets or image sections are selected;
generating a data stack, wherein the data stack comprises the respective selected subsets as stack elements, wherein a data resolution at an input of the machine learning model and a number of stack elements of the data stack is set depending on a computing power available during application of the trained machine learning model and a maximum possible latency time, wherein the computing power available and the maximum possible latency time are not fully used up;
applying the machine learning model trained according to the steps of the training phase to every stack element of the data stack, wherein the application occurs simultaneously; and deducing an inference result; and
outputting the inference result.