US 12,271,801 B2
Memory network method based on automatic addressing and recursive information integration
Ge Li, Guangdong Province (CN); Zhangheng Li, Guangdong Province (CN); Jiaxing Zhong, Guangdong Province (CN); Jingjia Huang, Guangdong Province (CN); and Tao Zhang, Guangdong Province (CN)
Assigned to PEKING UNIVERSITY SHENZHEN GRADUATE SCHOOL, Guangdong Province (CN)
Appl. No. 17/423,223
Filed by Peking University Shenzhen Graduate School, Guangdong Province (CN)
PCT Filed Aug. 21, 2019, PCT No. PCT/CN2019/101806
§ 371(c)(1), (2) Date Jul. 15, 2021,
PCT Pub. No. WO2021/007919, PCT Pub. Date Jan. 21, 2021.
Claims priority of application No. 201910635623.9 (CN), filed on Jul. 15, 2019.
Prior Publication US 2022/0138525 A1, May 5, 2022
Int. Cl. G06N 3/04 (2023.01); G06F 12/06 (2006.01)
CPC G06N 3/04 (2013.01) [G06F 12/06 (2013.01); G06F 2212/1044 (2013.01)] 5 Claims
OG exemplary drawing
 
1. A memory network method based on automatic addressing and recursive information integration, the method comprising performing, by a computer processor, operations of:
1) Using automatic addressing to conduct reading operation on a memory matrix, coding historical memory addressing information by directly utilizing a hidden state ht−1 transferred between different time steps of a recurrent neural network (RNN) and addressing a memory in combination with current input xt,
2) Comprehensively computing the hidden state ht−1, memory information rt and input xt by integrating recursive information:

OG Complex Work Unit Math
wherein in formula (1), two gates (gth, gtr) for computing are used for controlling information inflow of ht−1 and rt element by element respectively, that is meanings of formula (2) and formula (3); thereafter, an information processing mode of a long-short term memory network (LSTM) is reused for comprehensive computing, as shown in formula (4) and formula (5), and finally, newly generated hidden state ht and memory information rt use two output gates computed in the formula (4) to conduct information control and connection and then are output;
3) Conducting writing operation on the memory:
writing the new hidden state ht generated at this moment as information needing to be remembered; and
4) Entering a next time step:
transferring ht to the next time step, receiving input xt+1, and returning to the operation 1) for cyclically executing the above operation.