US 12,229,589 B2
Method and apparatus for scheduling matrix operations in digital processing systems
Shang-Tse Chuang, Lost Altos, CA (US); Sharad Vasantrao Chole, San Jose, CA (US); and Siyad Chih-Hua Ma, Palo Alto, CA (US)
Assigned to Expedera, Inc., Santa Clara, CA (US)
Filed by Expedera, Inc., Santa Clara, CA (US)
Filed on May 7, 2020, as Appl. No. 16/869,520.
Claims priority of provisional application 62/844,499, filed on May 7, 2019.
Prior Publication US 2020/0371835 A1, Nov. 26, 2020
Int. Cl. G06F 9/48 (2006.01); G06F 17/16 (2006.01); G06N 3/00 (2023.01); G06N 3/04 (2023.01); G06N 3/08 (2023.01); G06N 3/10 (2006.01)
CPC G06F 9/4881 (2013.01) [G06F 17/16 (2013.01); G06N 3/04 (2013.01); G06N 3/08 (2013.01); G06N 3/10 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method of scheduling computational matrix operations for execution, the method comprising:
selecting a data sample batch for a particular neural network model;
creating a set of work queues for the data sample batch;
determining a number of active queues within the set of work queues to be allowed to be processed simultaneously;
setting a priority level for each computational matrix operation in the set of work queues;
determining if preemption will be allowed during the execution, the preemption allowing a first computational matrix operation to be paused for a second computational matrix operation in the set of work queues; and
executing a computational matrix operation on the data sample batch in the set of work queues.