US 12,393,792 B2
Multilingual support for natural language processing applications
Tapan Bohra, Sunnyvale, CA (US); Ji Li, San Jose, CA (US); and Amit Srivastava, San Jose, CA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Feb. 25, 2022, as Appl. No. 17/681,250.
Prior Publication US 2023/0274096 A1, Aug. 31, 2023
Int. Cl. G06F 40/49 (2020.01); G06F 40/242 (2020.01); G06F 40/253 (2020.01); G06F 40/284 (2020.01); G06N 20/00 (2019.01)
CPC G06F 40/49 (2020.01) [G06F 40/242 (2020.01); G06F 40/253 (2020.01); G06F 40/284 (2020.01); G06N 20/00 (2019.01)] 14 Claims
OG exemplary drawing
 
1. A data processing system comprising:
a processor; and
a machine-readable storage medium storing executable instructions that, when executed, cause the processor to perform operations comprising:
training a first natural language processing model to analyze textual content in a target language and to output contextual information indicating one or more topics or subject matters of the textual content using training data that includes examples of non-standard grammar resulting from translating a plurality of source languages to the target language using a plurality of bilingual dictionaries;
obtaining the textual content from a first client device;
determining, based on analyzing the textual content, that is in a first language selected from among the plurality of source languages;
segmenting the textual content into a plurality of first tokens responsive to identifying the first language;
selecting a first bilingual dictionary from among the plurality of bilingual dictionaries based on the first language identified in the textual content, each bilingual dictionary of the plurality of bilingual dictionaries maps tokens in one of the plurality of source languages to tokens in the target language;
translating the plurality of first tokens to the target language using the first bilingual dictionary to create a plurality of second tokens;
analyzing the plurality of second tokens to extract features information from the plurality of second tokens to generate a feature vector;
providing the feature vector as an input to the first natural language processing model trained to analyze textual content in the target language to cause the first natural language processing model to analyze the feature vector and output the contextual information indicating one or more topics or subject matter of the textual content;
providing the contextual information obtained from the first natural language processing model as an input to a first machine learning model, the first machine learning model configured to analyze the contextual information and to output content information identifying one or more content items predicted to be relevant to the contextual information; and
providing the content information identifying the one or more content items to the first client device.