US 12,234,627 B2
Object visualization in construction heavy equipment
Kye Morris-Takamoto, Akashi (JP); and Sirui Li, Akashi (JP)
Assigned to Caterpillar SARL, Geneva (CH)
Filed by Caterpillar SARL, Geneva (CH)
Filed on Dec. 22, 2022, as Appl. No. 18/086,713.
Prior Publication US 2024/0209595 A1, Jun. 27, 2024
Int. Cl. H04N 13/239 (2018.01); E02F 9/24 (2006.01); E02F 9/26 (2006.01); G09G 3/20 (2006.01); H04N 13/111 (2018.01); H04N 13/296 (2018.01); H04N 23/695 (2023.01); E02F 3/32 (2006.01)
CPC E02F 9/261 (2013.01) [E02F 9/24 (2013.01); E02F 9/264 (2013.01); G09G 3/2092 (2013.01); H04N 13/111 (2018.05); H04N 13/239 (2018.05); H04N 13/296 (2018.05); H04N 23/695 (2023.01); E02F 3/32 (2013.01); G09G 2340/12 (2013.01)] 20 Claims
OG exemplary drawing
 
1. An excavator having a working arm at a distal end of which a work tool is attached, the excavator comprising:
a stereo camera constructed to generate video data and distance data;
an operator cab constructed on a chassis and having an interior and an exterior;
a monitor in the interior of the operator cab and constructed to display visual data from the stereo camera based on the generated video data;
a camera mount mechanically coupled to the excavator and interposed between the exterior of the operator cab and the stereo camera, the camera mount constructed to extend to position the stereo camera at a camera position at a height above and spaced from the operator cab such that the stereo camera faces toward the work tool to capture images in a field of view of the stereo camera that includes a portion of the working arm including the work tool; and
processing circuitry constructed to:
compute a spatial location of the work tool based on collected machine sensor data from one or more sensors on the working arm;
determine a range to target from the stereo camera to a predetermined target position of the work tool based on said camera position of the stereo camera and based on the spatial location of the work tool;
analyze the distance data from the stereo camera and the determined range to target corresponding to the computed spatial location of the work tool to render a video overlay including;
an object indicator that visually emphasizes a first object image of a first physical object of a plurality of physical objects in the field of view of the stereo camera and external to the excavator at the spatial location of the work tool, and
deemphasis of a second object image of a second physical object of the plurality of physical objects in the field of view of the stereo camera and external to the excavator a set distance away from the spatial location of the work tool; and
display on the monitor a composite image comprised of:
imagery corresponding to the video data associated with the field of view of the stereo camera including the portion of the working arm including the work tool, the object image of the first physical object, and the second physical object, and
the rendered video overlay over the first object image of the first physical object and the second object image of the second physical object.