US 11,669,767 B2
Automatic generation of an explanation, based on data lineage and user feedback, of a prediction produced by a machine learning system
Mayukh Bhaowal, San Francisco, CA (US); Leah McGuire, Redwood City, CA (US); Kin Fai Kan, Sunnyvale, CA (US); Christopher Rupley, San Francisco, CA (US); Xiaodan Sun, San Francisco, CA (US); Michael Weil, San Francisco, CA (US); and Subha Nabar, Sunnyvale, CA (US)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by salesforce.com, inc., San Francisco, CA (US)
Filed on Aug. 15, 2019, as Appl. No. 16/542,228.
Prior Publication US 2021/0049419 A1, Feb. 18, 2021
Int. Cl. G06Q 10/0637 (2023.01); G06F 18/21 (2023.01); G06N 20/00 (2019.01); G06N 5/045 (2023.01); G06F 18/214 (2023.01)
CPC G06F 18/2185 (2023.01) [G06F 18/2148 (2023.01); G06N 5/045 (2013.01); G06N 20/00 (2019.01)] 23 Claims
OG exemplary drawing
 
1. A method for modifying a set of data for training a machine learning system, the method comprising:
causing, by a processor, the machine learning system to be trained using a first set of data:
determining by the machine learning system to transmit to a user device, sales lead information from among a plurality of sales leads;
transmitting, by the processor and to the user device, a message, wherein the message includes an explanation of the determination, by the machine learning system, to transmit the sales lead information from among the plurality of sales leads;
receiving, by the processor in response to the message and from the user device, a signal, wherein the signal includes an indication of a usefulness of the message to a user of the user device, in making a decision to perform an action based on a knowledge associated with the sales lead information;
modifying, by the processor and in response to a receipt of the signal, the first set of data to produce a second set of data; and
causing, by the processor, the machine learning system to be trained using the second set of data.