US 11,657,306 B2
Form structure extraction by predicting associations
Milan Aggarwal, San Jose, CA (US); Mausoom Sarkar, San Jose, CA (US); and Balaji Krishnamurthy, San Jose, CA (US)
Assigned to Adobe Inc., San Jose, CA (US)
Filed by Adobe Inc., San Jose, CA (US)
Filed on Jun. 17, 2020, as Appl. No. 16/904,263.
Prior Publication US 2021/0397986 A1, Dec. 23, 2021
Int. Cl. G06N 5/04 (2023.01); G06N 20/00 (2019.01); G06N 3/08 (2023.01); G06V 10/82 (2022.01); G06N 20/10 (2019.01)
CPC G06N 5/04 (2013.01) [G06N 3/08 (2013.01); G06N 20/00 (2019.01); G06N 20/10 (2019.01); G06V 10/82 (2022.01)] 19 Claims
OG exemplary drawing
 
1. A method comprising:
accessing low-level form elements extracted from a static form;
identifying candidate low-level form elements, of the low-level form elements, that are co-located with a reference low-level form element in a portion of the static form;
generating, for the reference low-level form element, image patches comprising a respective image patch representing each candidate low-level form element corresponding to the reference low-level form element;
determining, using a first set of prediction models trained to determine associations between the low-level form elements, a second-level form element based on the generated image patches, wherein the second-level form element comprises the reference low-level form element and one or more of the candidate low-level form elements as a subset of the second-level form element;
determining, using a second set of prediction models trained to determine associations in the second-level form element and the low-level form elements, a high-level form element based on the second-level form element, wherein the high-level form element comprises the second-level form element and one or more other form elements as a subset of the high-level form element; and
generating a reflowable form using a hierarchy of form elements based on the static form by: i) linking the reference low-level form element and the one or more of the candidate low-level form elements that are a subset of the second-level form element; and (ii) linking together the second-level form element and the one or more other form elements that are a subset of the second-level form element.