US 12,340,319 B2
System and method for determining a structured representation of a form document utilizing multiple machine learning models
Stephanie Broyles, Plano, TX (US); Andrew Van Cao, Plano, TX (US); Stephen A. Eubanks, Plano, TX (US); William R. Georgen, Plano, TX (US); and Karpaga Ganesh Patchirajan, Plano, TX (US)
Assigned to Intuit Inc., Mountain View, CA (US)
Filed by INTUIT INC., Mountain View, CA (US)
Filed on Dec. 28, 2022, as Appl. No. 18/147,482.
Application 18/147,482 is a continuation of application No. 16/914,218, filed on Jun. 26, 2020, granted, now 11,568,284.
Prior Publication US 2023/0147276 A1, May 11, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 5/04 (2023.01); G06F 16/23 (2019.01); G06N 20/00 (2019.01)
CPC G06N 5/04 (2013.01) [G06F 16/2379 (2019.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computer implemented method for providing and using structured metadata, the method comprising:
acquiring, by a computer, from a document in a digital format, a plurality of document elements and a plurality of attributes describing the plurality of document elements, wherein at least one of the document elements is a field that receives data;
detecting, by the computer a plurality of form components using the plurality of attributes wherein each form component corresponds to a document element included in the plurality of document elements;
determining, by the computer a relevance to a particular context for the document element corresponding to each form component in the plurality of form components by applying a first machine learning model to the attributes of the plurality of document elements;
acquiring, by the computer, for the document element corresponding to each form component, an accuracy label for the relevance to the particular context determined by the first machine learning model, wherein the accuracy label indicates whether the document element is relevant to the particular context;
generating, by the computer, training data that includes the document element corresponding to each form component and the accuracy label for the document element, wherein the document element is represented as vectors indicating one or more location of the document element, type of the document element, type of data associated with the document element, and statistical data of how frequently a field associated with the document element was left blank;
retraining, by the computer, the first machine learning model using the training data;
re-determining, by the computer, the relevance to the particular context of the document element using the first machine learning model retrained on the training data;
clustering, by the computer the plurality of form components into a structured form representation based on the relevance to the particular context of the document element corresponding to each form component;
using, by the computer, the structured form representation to acquire, by the computer, from a reference document including the plurality of document elements, metadata input into the field and a plurality of attributes describing the metadata;
clustering, by the computer, a plurality of metadata components corresponding to the metadata into structured metadata by applying a second machine learning model to the plurality of attributes describing the metadata; and
distributing, by the computer at least one of the structured form representation and the structured metadata to a server.