US 12,175,799 B2
Methods and systems for adaptive, template-independent handwriting extraction from images using machine learning models
Houssem Chatbri, New York, NY (US); and Bethany Kok, New York, NY (US)
Assigned to THE BANK OF NEW YORK MELLON, New York, NY (US)
Filed by THE BANK OF NEW YORK MELLON, New York, NY (US)
Filed on Jul. 9, 2021, as Appl. No. 17/371,951.
Prior Publication US 2023/0008766 A1, Jan. 12, 2023
Int. Cl. G06V 40/30 (2022.01); G06N 3/08 (2023.01); G06T 5/50 (2006.01); G06T 5/70 (2024.01)
CPC G06V 40/382 (2022.01) [G06N 3/08 (2013.01); G06T 5/50 (2013.01); G06T 5/70 (2024.01); G06V 40/394 (2022.01); G06T 2207/20024 (2013.01); G06T 2207/20221 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system for adaptive, template-independent handwriting extraction from images using machine learning models and without manual localization or review, the system comprising:
cloud-based storage circuitry configured to store:
a model, wherein the model:
filters out printed content in inputted images;
extracts units of handwritten content from the inputted images; and
adaptively merges the units of handwritten content in the inputted images;
a supervised machine learning model comprising a multi-layer perceptron that is integrated into the model, wherein the supervised machine learning model is trained to classify feature inputs as a native handwritten content type or a native typewritten content type;
cloud-based control circuitry configured to:
receive an input image, wherein the input image comprises native printed content and native handwritten content;
process the input image with a model to generate an output image, wherein the output image comprises extracted handwritten content based on the native handwritten content, wherein processing the input image with the model to generate the output image further comprises:
identifying the native printed content;
filtering out the native printed content;
identifying the native handwritten content;
extracting the native handwritten content;
identifying units in the native handwritten content;
adaptively merging the units using scale space filtering; and
process, using a Long Short-Term Memory (LSTM) network, the output image to digitally recognize the extracted handwritten content;
generate a digital representation of the input image, wherein the digital representation comprises the native printed content and the digitally recognized extracted handwritten content;
cloud-based input/output circuitry configured to:
generate for display, on a user interface, the digital representation.