| US 11,776,215 B1 | ||
| Pre-labeling data with cuboid annotations | ||
| Chiao-Lun Cheng, San Francisco, CA (US); Elliot Branson, San Francisco, CA (US); Leigh Marie Braswell, San Francisco, CA (US); Daniel Havíř, San Francisco, CA (US); and Jeffrey Alan Anders, Denver, CO (US) | ||
| Assigned to SCALE AI, INC., San Francisco, CA (US) | ||
| Filed by Scale AI, Inc., San Francisco, CA (US) | ||
| Filed on Dec. 16, 2019, as Appl. No. 16/716,396. | ||
| Int. Cl. G06T 19/00 (2011.01); G06F 3/0482 (2013.01); G06F 3/04847 (2022.01); G06T 7/60 (2017.01); G06N 5/04 (2023.01); G06N 20/00 (2019.01) | ||
| CPC G06T 19/00 (2013.01) [G06F 3/0482 (2013.01); G06F 3/04847 (2013.01); G06N 5/04 (2013.01); G06N 20/00 (2019.01); G06T 7/60 (2013.01); G06T 2200/24 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2219/004 (2013.01)] | 17 Claims |
|
1. A computer-implemented method for annotating point cloud data, the method comprising:
training a first machine learning (ML) model using a loss function evaluated by performing steps including:
predicting, using the first ML model, a first cuboid annotation;
determining a first distance between corners of a cuboid annotation included in training data and a second cuboid annotation that includes a size of the first cuboid annotation, a position of the cuboid annotation included in the training data, and a yaw of the cuboid annotation included in the training data;
determining a second distance between corners of the cuboid annotation included in the training data and a third cuboid annotation that includes a position of the first cuboid annotation, a size of the cuboid annotation included in the training data, and a yaw of the cuboid annotation included in the training data;
determining a third distance between corners of the cuboid annotation included in the training data and a fourth cuboid annotation that includes a yaw of the first cuboid annotation, a size of the cuboid annotation included in the training data, and a position of the cuboid annotation included in the training data; and
determining an average of the first distance, the second distance, and the third distance;
processing the point cloud data using the first ML model and a second ML model to generate cuboid annotations of objects in the point cloud data by performing steps including:
processing the point cloud data using the first ML model to determine cuboid tracks associated with the objects;
projecting regions of the point cloud data including the objects to a plurality of two-dimensional (2D) views; and
processing the plurality of 2D views and the cuboid tracks using the second ML model to determine updated cuboid tracks associated with the objects;
determining a confidence threshold based on a count of the cuboid annotations; and causing to be displayed at least one of the cuboid annotations based on the confidence threshold, and at least one user interface element that permits a user to select, confirm, or modify the at least one of the cuboid annotations that is displayed. |