US 12,462,581 B2
Systems and methods for training a neural network for generating a reasoning statement
Enna Sachdeva, San Jose, CA (US); Nakul Agarwal, San Francisco, CA (US); Sean F. Roelofs, Soquel, CA (US); Jiachen Li, Mountain View, CA (US); Behzad Dariush, San Ramon, CA (US); and Chiho Choi, San Jose, CA (US)
Assigned to Honda Motor Co., Ltd., Tokyo (JP); and The Board of Trustees of the Leland Stanford Junior University, Stanford, CA (US)
Filed by Honda Motor Co., Ltd., Tokyo (JP); and The Board of Trustees of the Leland Stanford Junior University, Stanford, CA (US)
Filed on Jun. 5, 2023, as Appl. No. 18/328,921.
Prior Publication US 2024/0404297 A1, Dec. 5, 2024
Int. Cl. G06V 20/58 (2022.01); G06V 10/778 (2022.01); G06V 10/82 (2022.01); G06V 20/56 (2022.01)
CPC G06V 20/58 (2022.01) [G06V 10/7788 (2022.01); G06V 10/82 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A system for training a neural network for generating a reasoning statement, comprising:
a memory storing instructions that when executed by a processor cause the processor to:
receive sensor data having a number of frames imaging at least one roadway environment from a perspective of an ego agent;
identify a plurality of captured objects in the at least one roadway environment from the number of frames;
receive a set of ranking classifications for a captured object, having an object type, of the plurality of captured objects, wherein a ranking classification of the set of ranking classifications includes an annotator reasoning statement and an applied attribute of a predetermined group of attributes including a plurality of importance attributes in an importance category and an unimportance attribute in an unimportance category, wherein the annotator reasoning statement is a natural language explanation for the applied attribute, and wherein each ranking classification is received from a different source of a plurality of sources;
generate a training dataset for the object type including the annotator reasoning statements of the set of ranking classifications that include the applied attribute from the plurality of importance attributes in the importance category; and
train the neural network to generate a generated reasoning statement based on the training dataset in response to a training agent detecting a detected object of the object type.