US 11,771,077 B2
Identifying and avoiding obstructions using depth information in a single image
Chia-Chun Fu, Sunnyvale, CA (US); Christopher Grant Padwick, Menlo Park, CA (US); and James Patrick Ostrowski, Mountain View, CA (US)
Assigned to BLUE RIVER TECHNOLOGY INC., Sunnyvale, CA (US)
Filed by Blue River Technology Inc., Sunnyvale, CA (US)
Filed on May 7, 2021, as Appl. No. 17/314,218.
Application 17/314,218 is a continuation of application No. 17/033,263, filed on Sep. 25, 2020, granted, now 11,367,207.
Claims priority of provisional application 62/905,935, filed on Sep. 25, 2019.
Prior Publication US 2021/0264624 A1, Aug. 26, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. A01M 7/00 (2006.01); G06T 7/50 (2017.01); G06V 20/10 (2022.01); G06V 10/762 (2022.01); G06V 10/82 (2022.01); A01M 21/04 (2006.01); A01G 25/16 (2006.01); A01C 23/00 (2006.01); A01G 25/09 (2006.01); A01C 23/02 (2006.01); G06V 30/24 (2022.01)
CPC A01M 7/0089 (2013.01) [A01C 23/007 (2013.01); A01C 23/02 (2013.01); A01G 25/09 (2013.01); A01G 25/16 (2013.01); A01M 7/0042 (2013.01); A01M 21/043 (2013.01); G06T 7/50 (2017.01); G06V 10/762 (2022.01); G06V 10/82 (2022.01); G06V 20/188 (2022.01); G06V 30/248 (2022.01); G06T 2207/10028 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01); G06T 2207/30188 (2013.01); G06V 30/2528 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A method for avoiding an obstruction by a machine that moves through an operational environment, the machine including a plurality of mechanisms for performing machine actions:
accessing a single image of the operational environment from an image sensor as the machine moves through the operational environment, the single image comprising one or more pixels representing at least a substrate and an obstruction;
applying a depth identification model to the single image, the depth identification model:
determining, for each pixel in the single image, a distance between the image sensor and the substrate or the obstruction represented by the pixel, a depth identification module including a plurality of layers in a convolutional neural network configured to identify distances between sensors and representative pixels in single images, wherein the single image is encoded onto a first neural network layer as an encoded image and transformed to a reduced image with latent features classified as distances corresponding to pixels on a second neural network layer,
classifying, based on the determined distance for each pixel, a first set of pixels in the single image as the substrate, and
classifying, based on the determined distance for each pixel, a second set of pixels in the single image as the obstruction; and
actuating a mechanism of the plurality of mechanisms to perform a machine action that changes a direction of the machine to avoid the classified obstruction, the machine action selected based on the determined distance for pixels in the first set of pixels representing the substrate and pixels in the second set of pixels representing the obstruction.