| CPC G06F 17/16 (2013.01) | 22 Claims |
|
1. A method of performing tensor operations, the method comprising:
loading a first tensor into a plurality of processing devices, the first tensor split into a plurality of first tensor tiles that are distributed among the plurality of processing devices, the plurality of processing devices further including portions of a second tensor split into a plurality of second tensor tiles that are distributed among the plurality of processing devices;
performing a tensor operation with the first tensor and the second tensor using the plurality of processing devices to generate an intermediate tensor that is split in a plurality of intermediate tensor tiles distributed among the plurality of processing devices;
after performing the tensor operation with the first tensor and the second tensor, transferring one or more of the plurality of intermediate tensor tiles amongst one or more of the plurality of processing devices without any of the plurality of processing devices including the entire intermediate tensor; and
after transferring the one or more of the plurality of intermediate tensor tiles, performing, using the plurality of processing devices, a tensor operation with the intermediate tensor and a third tensor, which is split into a plurality of third tensor tiles that are distributed among the plurality of processing devices, to generate a fourth tensor.
|