US 12,462,575 B2
Vision-based machine learning model for autonomous driving with adjustable virtual camera
John Emmons, Austin, TX (US); Danny Hung, Austin, TX (US); Ethan Knight, Austin, TX (US); and Lane McIntosh, Austin, TX (US)
Assigned to Tesla, Inc., Austin, TX (US)
Filed by Tesla, Inc., Austin, TX (US)
Filed on Aug. 18, 2022, as Appl. No. 17/820,859.
Claims priority of provisional application 63/365,078, filed on May 20, 2022.
Claims priority of provisional application 63/287,936, filed on Dec. 9, 2021.
Claims priority of provisional application 63/260,439, filed on Aug. 19, 2021.
Prior Publication US 2023/0057509 A1, Feb. 23, 2023
Int. Cl. G06V 20/58 (2022.01); G06N 20/00 (2019.01)
CPC G06V 20/58 (2022.01) [G06N 20/00 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A method implemented by a vehicle processor system, the method comprising:
obtaining images from a multitude of image sensors positioned about a vehicle;
determining features associated with the images, wherein the features are output based on a forward pass through a first portion of a machine learning model;
projecting, based on a second portion of the machine learning model, the features into a vector space associated with a virtual camera at a particular height;
aggregating, based on a plurality of video modules, the projected features with other projected features associated with prior images; and
determining, based on a plurality of heads of the machine learning model, a plurality of objects positioned according to the virtual camera,
wherein the machine learning model includes a first and a second branch, and wherein the first branch is associated with the virtual camera at the particular height and the second branch is associated with a different virtual camera at a different height.