US 12,468,964 B2
Automatic onboard validation of a newly trained vehicle machine learning model
Sahib Singh, Ann Arbor, MI (US); Harsh Bhupendra Bhate, Dearborn, MI (US); Zaydoun Rawashdeh, Farmington, MI (US); Uttara Thakre, Dearborn, MI (US); Vyacheslav Zavadsky, Ottawa (CA); and Srujan Reddy Maram, Princeton, TX (US)
Assigned to Ford Global Technologies, LLC, Dearborn, MI (US)
Filed by FORD GLOBAL TECHNOLOGIES, LLC, Dearborn, MI (US)
Filed on Jun. 1, 2023, as Appl. No. 18/327,209.
Application 18/327,209 is a continuation of application No. 17/746,746, filed on May 17, 2022.
Prior Publication US 2023/0376804 A1, Nov. 23, 2023
Int. Cl. G06N 5/04 (2023.01); G05B 13/02 (2006.01); G05B 13/04 (2006.01); G06N 3/098 (2023.01); G06N 20/00 (2019.01); G06N 20/20 (2019.01)
CPC G06N 5/04 (2013.01) [G06N 20/00 (2019.01); G05B 13/027 (2013.01); G05B 13/04 (2013.01); G06N 3/098 (2023.01); G06N 20/20 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more processors of a vehicle configured to:
receive indication of a newly trained machine learning model designated for validation;
load a copy of the model into shadow execution hardware, capable of background execution of the model;
subscribe to one or more data topics to which input data for the model, gathered by a vehicle data gathering process, is published;
execute the model in the background as the vehicle travels, using data published to the data topics;
benchmark output from the model to determine whether the model outperforms a prior version of the model, that represents the model prior to the model being newly trained, based on relative performance of both models compared to performance expectations defined in a configuration file stored by the vehicle; and
responsive to the model outperforming the prior version of the model based on the performance expectations defined by the configuration file, validate the model as suitable for deployment.