US 11,861,315 B2
Continuous learning for natural-language understanding models for assistant systems
Pooja Sethi, Kent, WA (US); Denis Savenkov, Redmond, WA (US); Yue Liu, Belmont, MA (US); Alexander Kolmykov-Zotov, Sammamish, WA (US); and Ahmed Aly, Kenmore, WA (US)
Assigned to Meta Platforms, Inc., Menlo Park, CA (US)
Filed by Meta Platforms, Inc., Menlo Park, CA (US)
Filed on Jun. 18, 2021, as Appl. No. 17/351,501.
Claims priority of provisional application 63/177,812, filed on Apr. 21, 2021.
Prior Publication US 2022/0374605 A1, Nov. 24, 2022
Int. Cl. G06F 17/00 (2019.01); G06F 40/30 (2020.01); G06F 1/3206 (2019.01); G06F 3/01 (2006.01); G06F 3/04815 (2022.01); G06N 5/02 (2023.01); G06N 5/046 (2023.01); G06T 19/00 (2011.01); G06T 19/20 (2011.01)
CPC G06F 40/30 (2020.01) [G06F 1/3206 (2013.01); G06F 3/011 (2013.01); G06F 3/04815 (2013.01); G06N 5/02 (2013.01); G06N 5/046 (2013.01); G06T 19/006 (2013.01); G06T 19/20 (2013.01); G06T 2219/2004 (2013.01)] 19 Claims
OG exemplary drawing
 
1. A method comprising, by one or more computing systems:
receiving a user request to automatically debug a natural-language understanding (NLU) model;
accessing a plurality of predicted semantic representations generated by the NLU model, wherein the plurality of predicted semantic representations are associated with a plurality of dialog sessions, respectively, wherein each dialog session is between a user from a plurality of users and an assistant xbot associated with the NLU model;
generating, based on an auto-correction model, a plurality of expected semantic representations associated with the plurality of dialog sessions, wherein the auto-correction model is learned from a plurality of dialog training samples generated based on active learning;
identifying, based on a comparison between the predicted semantic representations and the expected semantic representations, one or more incorrect semantic representations of the predicted semantic representations; and
automatically correcting the one or more incorrect semantic representations by replacing them with one or more respective expected semantic representations generated by the auto-correction model.