US 12,406,189 B2
Multi-thread distributed training of a recommender model
Xuejun Liao, Cary, NC (US); and Patrick Nathan Koch, Morrisville, NC (US)
Assigned to SAS INSTITUTE INC., Cary, NC (US)
Filed by SAS INSTITUTE INC., Cary, NC (US)
Filed on Feb. 21, 2024, as Appl. No. 18/583,837.
Claims priority of provisional application 63/540,315, filed on Sep. 25, 2023.
Claims priority of provisional application 63/534,561, filed on Aug. 24, 2023.
Prior Publication US 2025/0068927 A1, Feb. 27, 2025
Int. Cl. G06N 3/098 (2023.01); G06F 16/9536 (2019.01); G06F 17/16 (2006.01); G06F 30/27 (2020.01); G06N 20/00 (2019.01)
CPC G06N 3/098 (2023.01) [G06F 16/9536 (2019.01); G06F 17/16 (2013.01); G06F 30/27 (2020.01); G06N 20/00 (2019.01)] 30 Claims
OG exemplary drawing
 
1. A computer-program product embodied in a non-transitory machine-readable storage medium storing computer instructions that, when executed by one or more processors, perform operations comprising:
receiving an input comprising a plurality of pre-defined factor matrices and an implicit feedback dataset partitioned into a plurality of implicit feedback data subsets;
distributing the input across a controller node and a plurality of worker nodes implemented in a distributed computing environment; and
training a model using the controller node and the plurality of worker nodes, wherein training the model includes:
initializing, by the controller node, a controller-specific user parameters matrix and a controller-specific item parameters matrix,
broadcasting, by the controller node, the controller-specific user parameters matrix and the controller-specific item parameters matrix to each worker node of the plurality of worker nodes, and
concurrently executing an aggregation model training algorithm at the controller node, a first localized model training algorithm at a first worker node of the plurality of worker nodes, and a plurality of additional localized model training algorithms across a remainder of the plurality of worker nodes until a training termination condition is satisfied, wherein:
the plurality of additional localized model training algorithms compute a plurality of worker-specific direction matrices,
the first localized model training algorithm aggregates the plurality of worker-specific direction matrices into an aggregate worker-specific direction matrix using a binary tree,
the aggregation model training algorithm iteratively updates the controller-specific user parameters matrix and the controller-specific item parameters matrix for at least a first iteration and a second iteration,
the first iteration of the aggregation model training algorithm updates the controller-specific user parameters matrix and the controller-specific item parameters matrix based on a first plurality of messages received from the first localized model training algorithm and each of the plurality of additional localized model training algorithms, wherein the first plurality of messages at least include the aggregate worker-specific direction matrix, and
the second iteration of the aggregation model training algorithm updates the controller-specific user parameters matrix and the controller-specific item parameters matrix based on a second plurality of messages received from the first localized model training algorithm and each of the plurality of additional localized model training algorithms, different from the first plurality of messages.