US 12,327,419 B2
Sanitized surface monitoring and smart user alerts
Tiberiu Suto, Franklin, NY (US); Shikhar Kwatra, San Jose, CA (US); Jonathan Cottrell, Canandaigua, NY (US); Carolina G. Delgado, Zapopan (MX); and Kelly Camus, Durham, NC (US)
Assigned to Kyndryl, Inc., New York, NY (US)
Filed by Kyndryl, Inc., New York, NY (US)
Filed on Aug. 31, 2022, as Appl. No. 17/823,844.
Prior Publication US 2024/0071112 A1, Feb. 29, 2024
Int. Cl. G06V 10/00 (2022.01); G06N 20/00 (2019.01); G06V 10/82 (2022.01); G06V 20/70 (2022.01)
CPC G06V 20/70 (2022.01) [G06N 20/00 (2019.01); G06V 10/82 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
mapping, by one or more processors, a physical environment based on utilizing one or more image capture devices to scan aspects of the physical environment, wherein the mapping comprises identifying contamination levels and features associated with objects in the physical environment;
utilizing, by the one or more processors, unsupervised learning and supervised learning to identify activities engaged in by a user in the physical environment;
based on the mapping, determining, by the one or more processors, that a trigger event has occurred, wherein the trigger event comprises the one or more processors identifying a feature pre-configured to be the trigger event;
identifying, by the one or more processors, an activity engaged in by the user, wherein a convolutional neural network (CNN) automatically extracts high-level features by processing raw sensory data to extract discriminative features, wherein the CNN associates the discriminative features with a set of activity elements to predict the activity engaged in by the user, wherein the feature pre-configured to be the trigger event is switching from an initial activity in which the user is attentive to the physical environment to the activity predicted by the CNN, and wherein, based on the trigger event, the one or more processors determine whether the physical environment includes a contaminated or sanitized surface and select an alert mechanism to notify the user of the contamination based on the activity predicted;
based on the identifying, determining, by the one or more processors, if a user interface utilized by the program code to display results is visible to the user in light of the activity;
in response to determining that the interface utilized by the program code to display results is not visible to the user due to the activity, selecting, by the one or more processors, the alert mechanism to alert the user to the trigger event; and
alerting, by the one or more processors, the user to the trigger event, utilizing the alert mechanism.