	US 12,205,356 B2
	Semantic image capture fault detection
Samuel Schulter, Long Island City, NY (US); Sparsh Garg, San Jose, CA (US); and Manmohan Chandraker, Santa Clara, CA (US)
Assigned to NEC Corporation, Tokyo (JP)
Filed by NEC Laboratories America, Inc., Princeton, NJ (US)
Filed on Mar. 23, 2023, as Appl. No. 18/188,766.
Application 18/188,766 is a continuation in part of application No. 18/178,821, filed on Mar. 6, 2023.
Claims priority of provisional application 63/343,202, filed on May 18, 2022.
Claims priority of provisional application 63/317,487, filed on Mar. 7, 2022.
Prior Publication US 2023/0281977 A1, Sep. 7, 2023
Int. Cl. G06V 10/776 (2022.01); G06T 7/00 (2017.01); G06T 7/11 (2017.01); G06V 10/74 (2022.01); G06V 10/774 (2022.01); G06V 20/70 (2022.01); H04N 17/00 (2006.01)

CPC G06V 10/776 (2022.01) [G06T 7/0002 (2013.01); G06T 7/11 (2017.01); G06V 10/761 (2022.01); G06V 10/774 (2022.01); G06V 20/70 (2022.01); H04N 17/002 (2013.01); G06T 2207/20021 (2013.01); G06T 2207/20081 (2013.01); G06T 2207/20084 (2013.01)]

20 Claims

1. A computer-implemented method for detecting faults, comprising:

capturing an image of a scene using a camera;

embedding the image using a segmentation model that includes an image branch having an image embedding layer that embeds images into a joint latent space and a text branch having a text embedding layer that embeds text into the joint latent space;

generating semantic information for a region of the image corresponding to a predetermined static object using the embedded image;

identifying a fault of the camera based on a discrepancy between the semantic information and semantic information of the predetermined static image; and

correcting the fault of the camera.