US 12,190,620 B2
Machined learning supporting document data extraction
Siddarth Sathi, San Jose, CA (US); Vibhas Gejji, San Jose, CA (US); Anish Hiranandani, San Jose, CA (US); Bruno Gomes Selva, San Jose, CA (US); and Anjana Prabhakar, Sunnyvale, CA (US)
Assigned to Automation Anywhere, Inc., San Jose, CA (US)
Filed by Automation Anywhere, Inc., San Jose, CA (US)
Filed on Jan. 27, 2021, as Appl. No. 17/160,082.
Claims priority of provisional application 63/087,851, filed on Oct. 5, 2020.
Claims priority of provisional application 63/087,847, filed on Oct. 5, 2020.
Claims priority of provisional application 63/087,844, filed on Oct. 5, 2020.
Prior Publication US 2022/0108106 A1, Apr. 7, 2022
Int. Cl. G06V 30/416 (2022.01); G06F 16/242 (2019.01); G06F 40/00 (2020.01); G06F 40/177 (2020.01); G06F 40/20 (2020.01); G06F 40/279 (2020.01); G06N 3/045 (2023.01); G06N 20/00 (2019.01); G06Q 10/10 (2023.01); G06Q 40/12 (2023.01); G06V 30/00 (2022.01); G06V 30/148 (2022.01); G06V 30/19 (2022.01); G06V 30/40 (2022.01); G06V 30/412 (2022.01); G06V 30/413 (2022.01); G06V 30/414 (2022.01)
CPC G06V 30/416 (2022.01) [G06F 16/243 (2019.01); G06F 40/00 (2020.01); G06F 40/177 (2020.01); G06F 40/20 (2020.01); G06F 40/279 (2020.01); G06N 3/045 (2023.01); G06N 20/00 (2019.01); G06Q 10/10 (2013.01); G06Q 40/12 (2013.12); G06V 30/00 (2022.01); G06V 30/153 (2022.01); G06V 30/19173 (2022.01); G06V 30/40 (2022.01); G06V 30/412 (2022.01); G06V 30/413 (2022.01); G06V 30/414 (2022.01)] 26 Claims
OG exemplary drawing
 
1. A computer-implemented method for extracting data from an image of a document, the computer-implemented method comprising:
retrieving object data pertaining to an object that has been detected in the image of the document, the object data denoting at least a portion of the document having the object;
acquiring text pertaining to the portion of the document having the object, the text having been recognized from the image of the document;
determining a key type for the object based on the text and a machine learned model, the machine learned model pertaining to at least a character level neural network model having at least a Character-level Convolutional Network and a Bidirectional Gated Recurrent Units Network;
determining a value for the object based on the determined key type for the object; and
storing the determined key type and the determined value for the object,
wherein the determining of the key type predicts the key type using the character level neural network model, and provides a confidence indication for key type predicted using the character level neural network model,
wherein the key type is predicted using the character level neural network model is accepted and recorded if the confidence indication is greater than a threshold level, and
wherein the determining of the key type predicts the key type using the pattern matching model is predicted and recorded if the confidence indication associated with prediction using the character level neural network model is not greater that the threshold level.