US 12,455,567 B2
Augmented reality for teleoperation
Mathew Alan Boesch, Plymouth, MI (US); Levasseur Tellis, Southfield, MI (US); and Anthony Mario D'Amato, Canton, MI (US)
Filed by Ford Global Technologies, LLC, Dearborn, MI (US)
Filed on Jun. 12, 2023, as Appl. No. 18/333,533.
Prior Publication US 2024/0411306 A1, Dec. 12, 2024
Int. Cl. G05D 1/00 (2024.01)
CPC G05D 1/0044 (2013.01) [G05D 1/0038 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method to manage a vehicle, the method comprising:
obtaining, by a processor of a vehicle teleoperation system, an image captured by a vehicle camera, a vehicle steering rack position, a steering motor pinion position/angle, a steering wheel position, a vehicle speed, and a vehicle rate of change of speed;
estimating, by the processor and based on the image, the vehicle steering rack position, the steering motion pinion position/angle, the steering wheel position, the vehicle speed, and the vehicle rate of change of speed, a time delay in teleoperation communication with the vehicle;
determining a stopping distance range in proximity to a vehicle front portion based on the vehicle speed, the vehicle rate of change of speed, and the time delay;
generating and displaying, by the processor, an augmented reality image based on the time delay and the vehicle steering rack position, the steering motion pinion position/angle, the steering wheel position, the vehicle speed, and the vehicle rate of change of speed;
determining a first predicted path of the vehicle based on the vehicle steering rack position, the steering motor pinion position/angle, and the steering wheel position;
overlaying, in the augmented reality image, the stopping distance range over the first predicted vehicle path of travel to generate a modified augment reality image; and
rendering, by the processor, the modified augmented reality image on a user interface of the vehicle teleoperation system, the modified augmented reality image including a first line corresponding to the first predicted path and a second line corresponding to the stopping distance range, wherein the vehicle is controlled by the processor.