US 12,482,243 B1
Leveraging environmental information to facilitate training and evaluating machine learning models and related technology
Shelby Cass, Corvallis, OR (US); Brent Griffin, Dexter, MI (US); and Robert DeBortoli, Corvallis, OR (US)
Assigned to Agility Robotics, Inc., Salem, OR (US)
Filed by Agility Robotics, Inc., Pittsburgh, PA (US)
Filed on Oct. 17, 2023, as Appl. No. 18/488,895.
Claims priority of provisional application 63/508,793, filed on Jun. 16, 2023.
Int. Cl. G06V 10/766 (2022.01); B25J 9/16 (2006.01); G06F 3/04842 (2022.01); G06T 7/00 (2017.01); G06T 7/73 (2017.01); G06V 10/774 (2022.01); G06V 10/776 (2022.01)
CPC G06V 10/776 (2022.01) [B25J 9/163 (2013.01); B25J 9/1689 (2013.01); B25J 9/1697 (2013.01); G06F 3/04842 (2013.01); G06T 7/0004 (2013.01); G06T 7/74 (2017.01); G06V 10/774 (2022.01); G06T 2200/24 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/30164 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer implemented method comprising:
providing, to a user via a graphical user interface, an image depicting an environment and corresponding to vision data collected by a vision sensor of a robot at a given timestep of a plurality of timesteps;
receiving first user input for a first expected object location within the environment, wherein the first user input indicates a presence of a first object at the first expected object location and at the plurality of timesteps and is based at least partially on the image;
receiving second user input for a second expected object location within the environment, wherein the second user input indicates a presence of a second object at the second expected object location and at the plurality of timesteps;
associating additional vision data with the first and second user inputs, wherein the additional vision data is collected by the vision sensor at an additional timestep of the plurality of timesteps, and wherein the vision sensor has different respective poses at the given timestep and at the additional timestep;
identifying an intersection of the first and second expected object locations in a plane corresponding to a perspective of the vision sensor at the additional timestep;
determining that the first expected object location overlaps the second expected object location at the intersection and at the additional timestep based at least partially on respective depth positions of the first and second expected object locations relative to the vision sensor at the additional timestep; and
modifying a portion of the additional vision data corresponding to the second expected object location based at least partially on identifying the intersection and determining that the first expected object location overlaps the second expected object location.