US 12,455,911 B2
Cross-lingual meta-transfer learning adaptation to natural language understanding
Meryem M'hamdi, Los Angeles, CA (US); Doo Soon Kim, San Jose, CA (US); Franck Dernoncourt, San Jose, CA (US); and Trung Huu Bui, San Jose, CA (US)
Assigned to ADOBE INC., San Jose, CA (US)
Filed by ADOBE INC., San Jose, CA (US)
Filed on Mar. 18, 2022, as Appl. No. 17/655,395.
Prior Publication US 2023/0297603 A1, Sep. 21, 2023
Int. Cl. G06F 16/334 (2025.01); G06F 40/279 (2020.01); G06F 40/35 (2020.01); G06N 20/00 (2019.01)
CPC G06F 16/3344 (2019.01) [G06F 40/279 (2020.01); G06F 40/35 (2020.01); G06N 20/00 (2019.01)] 13 Claims
OG exemplary drawing
 
1. A method for training a machine learning model, comprising:
identifying a task set including a plurality of pseudo tasks, wherein each of the plurality of pseudo tasks includes a support set corresponding to a first natural language processing (NLP) task in a first language and a query set corresponding to a second NLP task in a second language, wherein the first language comprises a high-resource language and the second language comprises a low-resource language that is typologically different from the first language;
in each of a plurality of iterations of an inner loop, updating parameters of a machine learning model based on the support set to obtain an inner-loop parameter, wherein the updating comprises:
computing a support loss for the pseudo task based on the support set;
computing a gradient of the support loss; and
updating the parameters of the machine learning model based on the gradient of the support loss,
in each of a plurality of iterations of an outer loop, updating the parameters of the machine learning model based on the query set to obtain an outer-loop parameter, wherein the updating comprises:
computing a query loss for the pseudo task;
summing the query loss over the plurality of pseudo tasks to obtain a query loss sum;
computing a gradient of the query loss sum; and
updating the machine learning model based on the gradient of the query loss sum,
wherein the inner loop is nested within each of the plurality of iterations of the outer loop, and the outer-loop parameter corresponding to each iteration of the plurality of iterations of the outer loop is based on a different inner-loop parameter corresponding to an outcome of the inner loop nested within the iteration of the outer loop; and
performing the second NLP task in the second language using the machine learning model.