US 12,153,898 B1
Method and system for weight memory mapping for streaming operation of giant generative artifical intelligence hardware
Junsoo Kim, Hwaseong-si (KR); Jung-Hoon Kim, Hwaseong-si (KR); and Junseo Cha, Hwaseong-si (KR)
Assigned to HyperAccel Co., Ltd., Hwaseong-si (KR)
Filed by HyperAccel Co., Ltd., Hwaseong-si (KR)
Filed on Jun. 14, 2024, as Appl. No. 18/744,211.
Claims priority of application No. 10-2023-0077569 (KR), filed on Jun. 16, 2023.
Int. Cl. G06F 7/544 (2006.01); G06F 17/14 (2006.01); G06F 17/16 (2006.01)
CPC G06F 7/5443 (2013.01) [G06F 17/14 (2013.01); G06F 17/16 (2013.01)] 16 Claims
OG exemplary drawing
 
1. A weight memory mapping system comprising:
a weight memory configured to store a weight matrix for a pretrained artificial intelligence model;
an input register configured to store a plurality of input data;
a first hardware operator configured to process a matrix multiplication operation between the plurality of input data and the weight matrix and to compute a lane-level final sum during the progress of the matrix multiplication operation by reusing a partial sum of the matrix multiplication operation; and
a second hardware operator configured to preprocess a next matrix multiplication operation during the progress of the matrix multiplication operation using the final sum.