US 12,304,090 B2
Systems and methods for anomaly detection and correction
Jayavardhana Rama Gubbi Lakshminarasimha, Bangalore (IN); Vartika Sengar, Bangalore (IN); Vighnesh Vatsal, Bangalore (IN); Balamuralidhar Purushothaman, Bangalore (IN); Arpan Pal, Kolkata (IN); Nijil George, Kolkata (IN); and Aditya Kapoor, Kolkata (IN)
Assigned to Tata Consultancy Services Limited, Mumbai (IN)
Filed by Tata Consultancy Services Limited, Mumbai (IN)
Filed on May 17, 2023, as Appl. No. 18/198,661.
Claims priority of application No. 202221029254 (IN), filed on May 20, 2022.
Prior Publication US 2023/0373096 A1, Nov. 23, 2023
Int. Cl. B25J 9/16 (2006.01)
CPC B25J 9/1697 (2013.01) [B25J 9/161 (2013.01); B25J 9/163 (2013.01); B25J 9/1653 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A processor implemented method comprising:
obtaining, via one or more hardware processors, an input stereo video captured by a video capturing device, wherein the input stereo video is associated with an environment;
identifying, via the one or more hardware processors, one or more red, green and blue (RGB) stereo image frames from the input stereo video;
detecting, via the one or more hardware processors, one or more representative image frames from the one or more RGB stereo image frames and identifying one or more bounding boxes associated with at least one of one or more groups and one or more objects comprised in the one or more groups present in the one or more representative image frames;
learning, via the one or more hardware processors, one or more concepts associated with each of the one or more objects and each of the one or more groups comprised in the one or more representative image frames using a concept learner;
detecting, via the one or more hardware processors, one or more attributes from each of the one or more objects and each of the one or more groups comprised in the one or more representative image frames;
generating, via the one or more hardware processors, a hierarchical scene graph from the one or more concepts and the one or more attributes, wherein the generated hierarchical scene graph comprises one or more interpretable sub-symbolic representations;
detecting, via the one or more hardware processors, one or more anomalies using at least one of the generated hierarchical scene graph and the one or more concepts;
identifying, via the one or more hardware processors, a goal task further comprising a sequence of actions based on (i) one or more constraints, and (ii) one or more interpretable symbolic representations obtained from the one or more interpretable sub-symbolic representations comprised in the generated hierarchical scene graph, wherein the one or more interpretable sub-symbolic representations of a current state of the environment are generated using a first neural network, and wherein the one or more interpretable sub-symbolic representations are mapped to one or more associated symbolic predicates using a second neural network;
generating, via the one or more hardware processors, a set of robotic actions associated with the identified goal task using the first neural network and a pre-defined symbolic knowledge base, wherein the pre-defined symbolic knowledge base is periodically updated using the second neural network; and
correcting, via the one or more hardware processors, the one or more detected anomalies based on a robot performing the generated set of robotic actions.