US 12,153,885 B2
Multi-feature balancing for natural language processors
Thanh Long Duong, Seabrook (AU); Vishal Vishnoi, Redwood City, CA (US); Mark Edward Johnson, Castle Cove (AU); Elias Luqman Jalaluddin, Seattle, WA (US); Tuyen Quang Pham, Springvale (AU); Cong Duy Vu Hoang, Wantirna South (AU); Poorya Zaremoodi, Melbourne (AU); Srinivasa Phani Kumar Gadde, Fremont, CA (US); Aashna Devang Kanuga, Foster City, CA (US); Zikai Li, Redwood City, CA (US); and Yuanxu Wu, Foster City, CA (US)
Assigned to Oracle International Corporation, Redwood Shores, CA (US)
Filed by Oracle International Corporation, Redwood Shores, CA (US)
Filed on Jan. 20, 2022, as Appl. No. 17/580,535.
Claims priority of provisional application 63/139,695, filed on Jan. 20, 2021.
Prior Publication US 2022/0229991 A1, Jul. 21, 2022
Int. Cl. G06F 40/289 (2020.01); G06F 40/166 (2020.01); G06F 40/205 (2020.01); G06F 40/263 (2020.01); G06F 40/279 (2020.01); G06F 40/295 (2020.01); G06N 3/08 (2023.01); H04L 51/02 (2022.01)
CPC G06F 40/289 (2020.01) [G06F 40/166 (2020.01); G06F 40/279 (2020.01); G06F 40/295 (2020.01); G06N 3/08 (2013.01); G06F 40/205 (2020.01); G06F 40/263 (2020.01); H04L 51/02 (2013.01)] 10 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
receiving a natural language query to be processed by a machine learning model, the machine learning model utilizing a dataset of natural language phrases for processing natural language queries;
determining, based on the natural language query, a feature dropout value for the machine learning model;
generating, based on the natural language query, a set of contextual features comprising a plurality of contextual features and a set of expressional features comprising one or more expressional features;
determining a feature correspondence between the plurality of contextual features and the one or more expressional features, to form a subset of contextual features comprising one or more contextual features corresponding to the one or more expressional features with which the feature correspondence is determined, among the plurality of contextual features;
removing, from the set of contextual features, at least a portion of the one or more contextual features of the subset of contextual features at a rate corresponding to the feature dropout value, to generate a set of modified contextual features;
generating a set of input features to include the set of modified contextual features and the one or more expressional features;
inputting, to the machine learning model, the set of input features; and
processing, by the machine learning model, the set of input features, to generate an output with respect to the natural language query,
wherein the feature dropout value comprises a first contextual feature dropout value corresponding to a first percentage of the plurality of contextual features of the set of contextual features.