US 12,308,022 B2
Apparatus, method, and program for utilizing language model
Marc Delcroix, Tokyo (JP); Atsunori Ogawa, Tokyo (JP); Tomohiro Nakatani, Tokyo (JP); and Michael Hentschel, Nara (JP)
Assigned to NIPPON TELEGRAPH AND TELEPHONE CORPORATION, Tokyo (JP)
Appl. No. 16/976,692
Filed by NIPPON TELEGRAPH AND TELEPHONE CORPORATION, Tokyo (JP)
PCT Filed Feb. 18, 2019, PCT No. PCT/JP2019/005871
§ 371(c)(1), (2) Date Aug. 28, 2020,
PCT Pub. No. WO2019/171925, PCT Pub. Date Sep. 12, 2019.
Claims priority of application No. 2018-041781 (JP), filed on Mar. 8, 2018.
Prior Publication US 2021/0049324 A1, Feb. 18, 2021
Int. Cl. G10L 15/18 (2013.01); G06F 40/279 (2020.01); G06N 3/045 (2023.01); G06N 3/08 (2023.01); G10L 15/183 (2013.01)
CPC G10L 15/183 (2013.01) [G06F 40/279 (2020.01); G06N 3/045 (2023.01); G06N 3/08 (2013.01)] 20 Claims
OG exemplary drawing
 
8. A system for predicting an object of a language based on adapting aspects of a language model, the system comprises:
a processor, and
a memory storing computer-executable instructions that when executed by the processor cause the system to:
receive an input symbol of a plurality of input symbols of the language in series;
receive a set of auxiliary data, wherein the auxiliary data is distinct from the input symbol, and the set of auxiliary data represents a latent topic of the plurality of input symbols of the language according to a bag-of-words model;
generate, based on the received input symbol, an intermediate state symbol using a first neural network;
generate, based on the generated intermediate state symbol and the received set of auxiliary data, an output symbol following the input symbol as a result of predicting the object using a second neural network, thereby adapting the output symbol to the latent topic of the plurality of input symbols of the language according to the bag-of-words model, wherein the second neural network comprises a plurality of hidden layers, and each hidden layer of the plurality of hidden layers receives at least a part of the generated intermediate state symbol and a distinct auxiliary data of the set of auxiliary data as input.