US 12,012,189 B2
System and method of operation for remotely operated vehicles leveraging synthetic data to train machine learning models
Manuel Alberto Parente Da Silva, Maia (PT); and Pedro Miguel Vendas Da Costa, Oporto (PT)
Assigned to Ocean Infinity (Portugal), S.A., Matosinhos (PT)
Appl. No. 17/266,059
Filed by Abyssal S.A., Matosinhos (PT)
PCT Filed Aug. 8, 2018, PCT No. PCT/IB2018/055978
§ 371(c)(1), (2) Date Feb. 4, 2021,
PCT Pub. No. WO2020/030950, PCT Pub. Date Feb. 13, 2020.
Prior Publication US 2021/0309331 A1, Oct. 7, 2021
Int. Cl. B63G 8/00 (2006.01); G05B 13/02 (2006.01); G05D 1/00 (2006.01)
CPC B63G 8/001 (2013.01) [G05B 13/0265 (2013.01); G05D 1/0044 (2013.01); B63G 2008/005 (2013.01); G05D 1/0088 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for operating a remotely operated vehicle (ROV) leveraging synthetic data to train a machine learning model and to display classification labels on a display of a navigation interface, the system comprising:
a synthetic training engine comprising:
a video dataset including at least one of: video data or real images coming from the ROV;
a telemetry dataset including telemetry data coming from the ROV;
a 3D model dataset including 3D model of a scene where the ROV is configured to operate;
a synthetic dataset comprising synthetic images generated from different views of objects in the video data or different views of the 3D model of the scene, and associated training labels, the synthetic dataset providing additional data for the video dataset and the real images; and
a machine learning model configured to determine classification labels for the objects shown in the video data or the real images, the machine learning model trained using data comprising the synthetic dataset; and
a navigation interface configured to:
display an object within an environment of the ROV; and
annotate the displayed object using a corresponding classification label.