US 12,073,178 B2
Systems and methods for refining pre-trained language models with improved gender fairness
Zahra Fatemi, Chicago, IL (US); Caiming Xiong, Menlo Park, CA (US); Wenhao Liu, Palo Alto, CA (US); and Chen Xing, Palo Alto, CA (US)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by salesforce.com, inc., San Francisco, CA (US)
Filed on Jan. 27, 2022, as Appl. No. 17/586,504.
Claims priority of provisional application 63/252,436, filed on Oct. 5, 2021.
Prior Publication US 2023/0104662 A1, Apr. 6, 2023
Int. Cl. G06F 40/279 (2020.01); G06F 40/35 (2020.01); G06F 40/56 (2020.01); G06N 3/045 (2023.01); G06N 3/08 (2023.01)
CPC G06F 40/279 (2020.01) [G06F 40/35 (2020.01); G06F 40/56 (2020.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for training a neural network language model to a generate gender-neutral output, the method comprising:
obtaining a gender-neutral dataset for training the neural network language model that has been previously trained to generate output for a text input;
freezing parameters of the neural network language model, wherein values of the parameters were determined during the previous training of the neural network language model and wherein the values of the parameters do not change after the parameters are frozen;
adding new parameters to the neural network language model, the new parameters associated with gender related terms; and
training the neural network language model using the gender-neutral dataset, wherein the training modifies values of the new parameters and not the values of the frozen parameters, and wherein the trained neural network language model generates the gender-neutral output for the text input.