US 12,260,028 B2
Data input system with online learning
Douglas Alexander Harper Orr, Brentwood (GB); Juha Iso-Sipila, London (GB); Marco Fiscato, London (GB); Matthew James Willson, London (GB); and Joseph Osborne, London (GB)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Mar. 30, 2017, as Appl. No. 15/475,038.
Claims priority of application No. 1620232 (GB), filed on Nov. 29, 2016.
Prior Publication US 2018/0150143 A1, May 31, 2018
Int. Cl. G06F 3/023 (2006.01); G06F 40/274 (2020.01); G06N 3/04 (2023.01); G06N 3/044 (2023.01); G06N 3/08 (2023.01); G06N 3/084 (2023.01)
CPC G06F 3/0237 (2013.01) [G06F 40/274 (2020.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06N 3/084 (2013.01); G06N 3/044 (2023.01)] 19 Claims
OG exemplary drawing
 
1. A data input system at an electronic device for inputting text items to the electronic device, comprising:
a store holding a vocabulary of embeddings of text items, each embedding being a numerical encoding of a text item that indicates weights for analysis of the text item in a neural network;
a processor which:
receives user input comprising one or more context text items followed by a new text item that is a user-entered word, the user-entered word including one or more rare words, proper nouns, idiosyncrasies, or combinations thereof, the user-entered word being entered via individual letters and not initially being in the vocabulary of the neural network, and wherein the user-entered word does not exist in the store;
determines, based on the user-entered word, whether to carry out an online training;
computes, when determined to carry out the online training, one of:
an embedding to be added to the vocabulary as a sequence of the one or more context text items followed by the new text item, or
an update to an embedding already in the vocabulary for the sequence of the one or more context text items followed by the new text item;
implements the neural network, the neural network trained to produce a prediction of a next text item in the sequence given the context text items and the vocabulary;
implements online training to change the vocabulary by comparing the new text item and the predicted next text item; and
propagates results of the comparison to a final layer of the neural network.