US 12,298,982 B2
Diversity and explainability parameters for recommendation accuracy in machine learning recommendation systems
Wenzhuo Yang, Singapore (SG); Jia Li, Mountain View, CA (US); Chenxi Li, Belmont, CA (US); Latrice Barnett, Oakland, CA (US); Markus Anderle, Moraga, CA (US); Simo Arajarvi, Dublin (IE); Harshavardhan Utharavalli, Hyderabad (IN); Caiming Xiong, Menlo Park, CA (US); Richard Socher, Menlo Park, CA (US); and Chu Hong Hoi, Singapore (SG)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by Salesforce, Inc., San Francisco, CA (US)
Filed on Nov. 10, 2020, as Appl. No. 17/093,885.
Claims priority of application No. 202021022981 (IN), filed on Jun. 1, 2020.
Prior Publication US 2021/0374132 A1, Dec. 2, 2021
Int. Cl. G06F 16/2457 (2019.01); G06N 20/20 (2019.01)
CPC G06F 16/2457 (2019.01) [G06N 20/20 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
a non-transitory memory; and
one or more processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
receiving a user query for generating a recommendation for one or more items and one or more explanations associated with the one or more items;
for each item in a plurality of items:
obtaining input features comprising at least one user feature and at least one item feature;
determining, using one or more relevancy models in one or more first machine learning networks and the input features, a predicted score of the item;
determining, using a first diversity model in second machine learning networks, first diversity scores and a first explainability score for the item from at least a first portion of the input features;
determining, using a second diversity model in the second machine learning networks, second diversity scores and a second explainability score for the item from at least a second portion of the input features, the second portion of the input features different from the first portion of the input features, wherein the second machine learning networks are associated with a weight parameter indicative of the degree of diversity in recommendations;
determining an explanation narrative for the item using the first and second explainability scores; and
combining the predicted score, the first diversity scores of the first diversity model and the second diversity scores of the second diversity model into a combined score for the item;
ranking the plurality of items according to combined scores;
providing the recommendation that includes the one or more items from the plurality of items, wherein the one or more items correspond to top combined scores;
receiving a user input indicative of an updated value for the weight parameter;
generating an updated recommendation for the one or more items and the one or more explanations associated with the one or more items based on the updated value for the weight parameter; and
providing the updated recommendation that includes the one or more items from the plurality of items.