US 12,412,079 B2
Z-first reference neural processing unit for mapping winograd convolution and a method thereof
Gopinath Vasanth Mahale, Bangalore (IN); Pramod Parameshwara Udupa, Bangalore (IN); Kiran Kolar Chandrasek Haran, Bangalore (IN); and Sehwan Lee, Suwon-si (KR)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed on Apr. 26, 2021, as Appl. No. 17/239,892.
Claims priority of application No. 201941039524 (IN), filed on May 13, 2020; and application No. 10-2020-0138655 (KR), filed on Oct. 23, 2020.
Prior Publication US 2021/0357734 A1, Nov. 18, 2021
Int. Cl. G06N 3/063 (2023.01)
CPC G06N 3/063 (2013.01) 18 Claims
OG exemplary drawing
 
1. A z-first reference neural processing circuitry (NPU) for mapping Winograd Convolution supporting both direct convolution (DConv) mode and Winograd convolution (WgConv) mode, the NPU comprising:
memory banks configured to store input feature maps (IFMs) in a z-first data storage layout, each of the memory banks being configured to store the IFMs in one of the DConv mode or the WgConv mode;
a reconfigurable IFM distributor circuitry configured to receive the IFMs from the memory banks;
a parallel reconfigurable Winograd forward transform circuitry configured to receive the IFMs from the reconfigurable IFM distributor and to transform the IFMs in a Winograd domain to transformed IFMs in the WgConv mode;
multiply and accumulate (MAC) circuitries configured to perform dot product operations on one of IFMs in the DConv mode and the transformed IFMs in the WgConv mode to obtain intermediate output feature maps (OFMs); and
a reconfigurable OFM adder and Winograd inverse transform circuitry configured to generate one of an OFM from the intermediate OFMs in the DConv mode and OFMs from the intermediate OFMs in the WgConv mode,
wherein each of the memory banks are further configured to store a batch of IFMs from the IFMs in the DConv mode and store channels of each of coordinates of each of the IFMs in the WgConv mode as IFM blocks.