US 12,272,176 B2
Monitoring system that captures image stream of regions of interest based on identified facial cues
Amit Kumar Agrawal, Bangalore (IN); and Rahul Bharat Desai, Hoffman Estates, IL (US)
Assigned to Motorola Mobility LLC, Chicago, IL (US)
Filed by MOTOROLA MOBILITY LLC, Wilmington, DE (US)
Filed on Mar. 30, 2022, as Appl. No. 17/708,720.
Prior Publication US 2023/0316808 A1, Oct. 5, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06V 40/16 (2022.01); G06T 7/20 (2017.01); G06V 10/25 (2022.01); G06V 40/18 (2022.01); G10L 25/63 (2013.01)
CPC G06V 40/174 (2022.01) [G06T 7/20 (2013.01); G06V 10/25 (2022.01); G06V 40/18 (2022.01); G10L 25/63 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A monitoring system comprising:
a camera system comprising at least one image capturing device and which captures a first image stream that encompasses a face of a first person of interest and a second image stream that at least partially encompasses one or more surrounding objects and surfaces viewable by the first person;
a memory that stores (i) a visual object library; (ii) a facial expression recognition application; and (iii) an eye gaze detection (EGD) application; and
a controller communicatively coupled to the camera system and the memory, and which:
detects a facial expression of the face of the first person incorporated in the first image stream;
determines that the facial expression is a mode associated expression; and
in response to determining that the facial expression is a mood associated expression:
determines, from the first image stream, an eye gaze direction of the first person;
determines a first region of interest (ROI) that is aligned with the eye gaze direction, wherein to determine the first ROI, the controller: detects angles of eyes of the first person relative to a location of the first ICD; and extrapolates the angles of eyes in a 3-dimensional space to an area in direct line of sight of eye gaze direction;
focuses a lens of the second ICD on the area that is the ROI, using a result of the extrapolation;
captures the second image stream of the first ROI and identifies a first object contained within the first ROI; and
communicates a notification including the mood associated expression and at least the first object to an output device, which presents the mood associated expression and the first object to a second person.