US 11,704,538 B2
Data processing method and device
Huanxin Zheng, Beijing (CN); and Guibin Wang, Beijing (CN)
Assigned to Baidu Online Network Technology (Beijing) Co., Ltd., Beijing (CN)
Filed by Baidu Online Network Technology (Beijing) Co., Ltd., Beijing (CN)
Filed on Nov. 21, 2019, as Appl. No. 16/691,130.
Claims priority of application No. 201910492996.5 (CN), filed on Jun. 6, 2019.
Prior Publication US 2020/0387772 A1, Dec. 10, 2020
Int. Cl. G06N 3/04 (2023.01); G06N 3/10 (2006.01)
CPC G06N 3/04 (2013.01) [G06N 3/10 (2013.01)] 9 Claims
OG exemplary drawing
 
1. A data processing method, implemented by circuits for implementing functions, comprising:
performing, by the circuits for implementing functions, a forward calculation of a neural network on global data to obtain output data of the forward calculation;
storing, by the circuits for implementing functions, the output data of the forward calculation in a global memory unit, wherein intermediate data is not stored in the global memory unit;
performing, by the circuits for implementing functions, the forward calculation of the neural network on the global data again, to obtain the intermediate data for a reverse calculation of the neural network;
storing, by the circuits for implementing functions, the intermediate data in a buffer unit;
reading, by the circuits for implementing functions, the intermediate data from the buffer unit; and
performing, by the circuits for implementing functions, the reverse calculation of the neural network on the intermediate data to obtain a result of the reverse calculation,
wherein the neural network is a recurrent neural network (RNN):
following formulas are used in the forward calculation in the RNN:
xt=atw1+ft-1w2;
yt=xtft-1;
zt=xtyt;
ft=ytzt;
following formulas are used in the reverse calculation in the RNN:

OG Complex Work Unit Math
wherein w1, w2 represent weights of the RNN, at represents an input, f1 represents an output, xt, yt, zt are the intermediate data, t is at a time t, and L is a cost function.