US 11,874,861 B2
Retraining a conversation system based on negative feedback
Jason M. Brown, Cary, NC (US); Andrew R. Freed, Cary, NC (US); Aaron T. Smith, Raleigh, NC (US); Joshua S. Allen, Durham, NC (US); and Ryan S. Brink, Raleigh, NC (US)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on May 17, 2019, as Appl. No. 16/415,545.
Prior Publication US 2020/0364511 A1, Nov. 19, 2020
Int. Cl. G06F 40/35 (2020.01); G06F 16/332 (2019.01); G06N 3/08 (2023.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06N 3/006 (2023.01); G06N 3/084 (2023.01); G06V 10/82 (2022.01); G06F 18/21 (2023.01); G06F 18/40 (2023.01); G06F 18/24 (2023.01)
CPC G06F 16/3329 (2019.01) [G06F 18/2178 (2023.01); G06F 18/24765 (2023.01); G06F 18/41 (2023.01); G06F 40/216 (2020.01); G06F 40/30 (2020.01); G06F 40/35 (2020.01); G06N 3/006 (2013.01); G06N 3/08 (2013.01); G06N 3/084 (2013.01); G06V 10/82 (2022.01)] 17 Claims
OG exemplary drawing
 
1. A method comprising:
identifying an answer returned by a cognitive interactive information system to a question asked by a user, wherein the answer returned by the cognitive information system is selected based, at least in part, on matching an intent classification determined by the cognitive interactive information system to be associated with the question to one or more processing rules;
determining that the answer returned by the cognitive interactive information system to the question asked by the user is incorrect;
reevaluating the question utilizing one or more alternative intent classifications associated with the question to determine one or more alternative answers to the question, wherein the one or more alternative answers are determined based, at least in part, on:
providing a user interface that enables the user to adjust the determined intent classification associated with the question by selecting one or more alternative intent classifications from the user interface and submitting a respective confidence score indicating how confident the user is of a selected alternative intent classification being an accurate intent of the question asked by the user; and
responsive to the user utilizing the user interface to select the one or more alternative intent classifications and submit the confidence score indicating how confident the user is of the selected alternative intent classification being an accurate intent of the question asked by the user, determining one or more alternative answers based on matching the alternative intent classifications, starting in order of highest confidence score, to the one or more processing rules;
providing the one or more alternative answers to the question to the user;
receiving a selection of an alternative answer from the one or more alternative answers from the user;
updating processing rules for analyzing a similar question to the question asked by the user based on the processing rules that were used by the cognitive interactive information system to determine the alternative answer selected by the user; and
retraining the cognitive interaction information system based on the updated processing rules.