| CPC G06F 16/3329 (2019.01) [G06F 16/3326 (2019.01); G06F 16/3334 (2019.01); G06N 3/08 (2013.01)] | 19 Claims |
|
1. A computer-implemented method comprising:
receiving, by one or more processors, an input question originating from a user computing entity;
generating, by the one or more processors and using a deep question answering (QA) application, a deep learning answer and a deep learning confidence score for the deep learning answer, wherein (a) the deep QA application comprises an artificial neural network, and (b) the deep learning answer is generated by the artificial neural network for the input question;
determining, by the one or more processors, that the deep learning confidence score for the deep learning answer does not satisfy a configurable deep QA threshold associated with a likelihood that the deep learning answer generated using the deep QA application is a correct answer to the input question:
querying, by the one or more processors, a retrieval QA application for a retrieved answer, wherein (a) the retrieval QA application identifies the retrieved answer at least in by ranking a plurality of documents responsive to the querying, and generates a plurality of retrieval confidence scores for the plurality of documents respectively, and (b) the retrieved answer comprises a highest retrieval confidence score of the plurality of retrieval confidence scores;
responsive to determining, by the one or more processors, that the highest retrieval confidence score for the retrieved answer satisfies a retrieval confidence score threshold, providing, by the one or more processors, the retrieved answer from the retrieval QA application as an answer from a hybrid QA platform to the input question, wherein the hybrid QA platform comprises the retrieval QA application and the deep QA application; and
responsive to determining, by the one or more processors, that the highest retrieval confidence score for the retrieved answer does not satisfy the retrieval confidence score threshold, requesting, by the one or more processors, an expert user answer by transmitting the input question to a user computing entity associated with an expert user;
identifying, by the one or more processors, a plurality of retrieved answers, wherein (a) one or more of the plurality of retrieved answers satisfies a training threshold associated with a quality of the plurality of retrieved answers being suitable for generating training data for the deep QA application and (b) the plurality of retrieved answers comprises the retrieved answer;
generating, by the one or more processors, training data for the artificial neural network, wherein the training data comprises one or more training question-answer pairs, each comprising a respective question and a corresponding respective answer, for respective one or more retrieved answers in the plurality of retrieved answers, wherein one of the one or more training question-answer pairs comprises the input question and the retrieved answer; and
initiating, by the one or more processors, the performance of training the artificial neural network based at least in part on the training data, wherein initiating the performance of training comprises, for the one or more training question-answer pairs in the training data: (i) generating a comparison measure between the corresponding respective answer and the deep learning answer, and (ii) updating trainable weights of the artificial neural network based at least in part on the comparison measure.
|