US 12,321,849 B1
Performing hardware operator fusion
Animesh Jain, Sunnyvale, CA (US); Tobias Joseph Kastulus Edler von Koch, Austin, TX (US); Yizhi Liu, Fremont, CA (US); Taemin Kim, Portland, OR (US); Jindrich Zejda, Saratoga, CA (US); Yida Wang, Palo Alto, CA (US); Vinod Sharma, Menlo Park, CA (US); Richard John Heaton, San Jose, CA (US); and Randy Renfu Huang, Morgan Hill, CA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Aug. 28, 2023, as Appl. No. 18/238,722.
Application 18/238,722 is a continuation of application No. 16/698,753, filed on Nov. 27, 2019, granted, now 11,809,981.
Int. Cl. G06N 3/063 (2023.01); G06F 9/30 (2018.01); G06F 9/54 (2006.01)
CPC G06N 3/063 (2013.01) [G06F 9/30007 (2013.01); G06F 9/545 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A compiler-implemented method of generating executable instructions for a computing system, comprising:
receiving a first set of instructions including a kernel of a first operator and a kernel of a second operator, the kernel of the first operator including instructions of the first operator and write instructions to a virtual data node, the kernel of the second operator including instructions of the second operator and read instructions to the virtual data node;
determining a mapping between the write instructions and read instructions based on common tensor addresses shared by the write instructions and the read instructions;
determining, based on a mapping between the write instructions and read instructions, instructions of data transfer operations between the first operator and the second operator, wherein the instructions of the data transfer operations are determined such that a duplication of a same write instruction from the write instructions is reduced; and
generating a second set of instructions representing a fused operator of the first operator and the second operator, the second set of instructions being executable by the computing system and including the instructions of the first operator, the instructions of the second operator, and the instructions of the data transfer operations.