US 11,874,899 B2
Automated multimodal adaptation of multimedia content
Sai Krishna Reddy Gudimetla, Jersey City, NJ (US); Aaron K Baughman, Cary, NC (US); Micah Forster, Round Rock, TX (US); and Craig M. Trim, Ventura, CA (US)
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Dec. 15, 2020, as Appl. No. 17/122,603.
Prior Publication US 2022/0188564 A1, Jun. 16, 2022
Int. Cl. G06F 18/214 (2023.01); G06F 18/21 (2023.01); G06F 18/22 (2023.01); G06N 3/08 (2023.01); G06V 20/40 (2022.01); G06N 3/045 (2023.01); G06N 3/047 (2023.01); G06V 10/82 (2022.01); G06V 20/70 (2022.01)
CPC G06F 18/214 (2023.01) [G06F 18/2178 (2023.01); G06F 18/22 (2023.01); G06N 3/08 (2013.01); G06V 20/47 (2022.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
transforming, using a first trained generative adversarial network, a non-text portion of a first multimedia content into a text description of the first multimedia content;
adjusting, using a trained attention layer, the text description, the adjusting creating an adjusted text description, the adjusting performed according to a presentation mode constraint, the presentation mode constraint specifying a presentation mode of a second multimedia content; and
transforming, using a trained model, the adjusted text description into a non-text portion of the second multimedia content, the second multimedia content presented in the presentation mode specified by the presentation mode constraint;
adjusting, according to feedback data, the presentation mode constraint;
transforming, using the first trained generative adversarial network, a second portion of the first multimedia content into a second text description of the second portion;
adjusting, using the trained attention layer, the second text description, the adjusting creating an adjusted second text description, the adjusting performed according to the adjusted presentation mode constraint; and
transforming, using the trained model, the adjusted second text description into a second portion of the second multimedia content, the second portion of the second multimedia content presented in the presentation mode specified by the presentation mode constraint.