US 11,893,682 B1
Method for rendering 2D and 3D data within a 3D virtual environment
Kah Seng Tay, Mountain View, CA (US); Qing Sun, Mountain View, CA (US); and James Patrick Marion, Mountain View, CA (US)
Filed by Direct Current Capital LLC, Wilmington, DE (US)
Filed on Oct. 10, 2022, as Appl. No. 17/963,087.
Application 17/963,087 is a continuation of application No. 17/107,647, filed on Nov. 30, 2020, granted, now 11,468,646.
Application 17/107,647 is a continuation of application No. 16/379,728, filed on Apr. 9, 2019, granted, now 10,854,011, issued on Dec. 1, 2020.
Claims priority of provisional application 62/655,116, filed on Apr. 9, 2018.
Int. Cl. G06T 17/00 (2006.01); G06T 19/00 (2011.01); G01S 17/89 (2020.01); G01S 17/93 (2020.01); G05D 1/00 (2006.01); G06T 7/521 (2017.01)
CPC G06T 17/00 (2013.01) [G01S 17/89 (2013.01); G01S 17/93 (2013.01); G05D 1/0044 (2013.01); G05D 1/0088 (2013.01); G06T 7/521 (2017.01); G06T 19/006 (2013.01); G06T 2207/10028 (2013.01)] 15 Claims
OG exemplary drawing
 
1. One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:
accessing a two-dimensional (2D) image captured using a camera of a vehicle;
accessing a three-dimensional (3D) point cloud captured using a 3D sensor of the vehicle;
generating a 3D virtual environment representing a field external to the vehicle by projecting at least a portion of the 2D image onto a group of points in the 3D point cloud, the group of points representing a surface in the field external to the vehicle; and
rendering the 3D virtual environment for display on a remote operator display, the vehicle being remotely operated using the 3D virtual environment displayed on the remote operator display, wherein the vehicle is remotely operated following at least one of identification of at least one operational issue of the vehicle or receipt of a request for remote assistance.