US 12,353,207 B2
Digital panning of panoramic video to follow machine worktool movement
Aaron R. Shatters, Montgomery, IL (US); Eric W. Cler, Oswego, IL (US); Michael Anthony Spielman, Osseo, MI (US); and Jeffrey K. Berry, Yorkville, IL (US)
Assigned to Caterpillar Inc., Peoria, IL (US)
Filed by Caterpillar Inc., Peoria, IL (US)
Filed on Apr. 28, 2022, as Appl. No. 17/732,270.
Prior Publication US 2023/0350412 A1, Nov. 2, 2023
Int. Cl. G06T 3/4038 (2024.01); B60R 1/24 (2022.01); G05D 1/00 (2006.01); G06T 3/047 (2024.01)
CPC G05D 1/0094 (2013.01) [B60R 1/24 (2022.01); G05D 1/0038 (2013.01); G06T 3/047 (2024.01); G06T 3/4038 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A method comprising using at least one hardware processor to, for each frame of a plurality of input image frames in a video captured by a fisheye camera mounted on a mobile equipment with an articulated front portion:
(1) determine a projection surface by:
when no movement of the front portion of the mobile equipment is detected since a prior input image frame, using a prior projection surface determined for the prior input image frame, and,
when movement of the front portion of the mobile equipment is detected since the prior projection surface was determined for the prior input image frame, determining a new projection surface according to the movement of the front portion;
(2) generate a projected image frame based on a mapping for the determined projection surface, including for each of a plurality of coordinates in the projected image frame:
mapping the coordinate to a position in the input image frame based on the determined projection surface, and
generating a pixel value for the coordinate based on the position in the input image frame; and
(3) output the projected image frame to a real-time video stream.