US 12,131,182 B2
Convolutional neural network operation grouping based on memory unit capacity
Zhenjiang Wang, Beijing (CN); Jianjun Li, Beijing (CN); Liang Chen, Beijing (CN); Kun Ling, Beijing (CN); Delin Li, Beijing (CN); and Chen Sun, Beijing (CN)
Assigned to Nanjing Horizon Robotics Technology Co., Ltd., Nanjing (CN)
Filed by Nanjing Horizon Robotics Technology Co., Ltd., Nanjing (CN)
Filed on Mar. 22, 2019, as Appl. No. 16/362,281.
Claims priority of application No. PCT/CN2018/080010 (WO), filed on Mar. 22, 2018.
Prior Publication US 2019/0294438 A1, Sep. 26, 2019
Int. Cl. G06F 9/48 (2006.01); G06F 9/30 (2018.01); G06F 9/345 (2018.01); G06F 9/50 (2006.01); G06N 3/063 (2023.01)
CPC G06F 9/4881 (2013.01) [G06F 9/30007 (2013.01); G06F 9/3004 (2013.01); G06F 9/30076 (2013.01); G06F 9/345 (2013.01); G06F 9/5016 (2013.01); G06N 3/063 (2013.01)] 6 Claims
OG exemplary drawing
 
1. A method of data processing, said method comprising:
(a) receiving, by a processor, an input data to be processed by a series of sequentially performed operations;
(b) identifying, by the processor, a first operation from the series of operations, wherein the first operation has a first amount of an input data and an output data exceeding a capacity of a memory unit;
(c) selecting, by the processor, at least one second operation from the series of operations to be grouped with the first operation based at least in part on a second amount of an input data and an output data of the grouped operations and the capacity of the memory unit, wherein the at least one second operation comprises an operation from the series of operations which is immediately preceding the first operation; and
(d) processing, by the processor, a portion of the input data of the grouped operations, wherein the portion of the input data of the grouped operations is determined based at least in part on an amount of an intermediate data result of the grouped operations,
wherein (c) comprises determining whether the second amount of the input data and the output data of the grouped operations exceeds the capacity of the memory unit, and incrementally increasing a number of operations in the at least one second operation until the second amount of the input data and the output data of the grouped operations does not exceed the capacity of the memory unit or until a number of operations in the grouped operations equals a number of operations in the series of operations and the second amount of input data and the output data of the grouped operations exceeds the capacity of the memory unit; and
wherein the portion of the input data of the grouped operations comprises one of N equal parts of the input data of the grouped operations and a marginal data, N being an integer of 2 or larger, and wherein (c) comprises determining the at least one second operation such that the value of N is minimal,
wherein (c) further comprises storing the output data of the grouped operations in an external memory when (1) a number of operations in the grouped operations is equal to a number of operations in the series of operations, and (2) the second amount of the input data and the output data of the grouped operations exceeds the capacity of the memory unit, or
(c) further comprises storing the input data of the grouped operations in an external memory when (1) a number of operations in the grouped operations is equal to a number of operations in the series of operations, and (2) the second amount of the input data and the output data of the grouped operations exceeds the capacity of the memory unit, or
(c) further comprises storing the input data and the output data of the grouped operations in an external memory when (1) a number of operations in the grouped operations is equal to a number of operations in the series of operations, and (2) the second amount of the input data and the output data of the grouped operations exceeds the capacity of the memory unit.