| CPC G06Q 30/0254 (2013.01) [G06F 16/24578 (2019.01); G06F 40/279 (2020.01); G06F 40/30 (2020.01); G06N 20/00 (2019.01); G06Q 30/0203 (2013.01); G06Q 30/0244 (2013.01); G06Q 30/0277 (2013.01)] | 20 Claims |
|
1. A computer-implemented method of processing campaign datasets wherein a given campaign dataset comprises data representing selections and instructions to be submitted to an online platform to execute an online campaign represented by the given campaign dataset, wherein the given campaign dataset can be machine-generated, human-generated, or a combination of both, the method comprising:
under control of one or more processors of a computer system executing executable instructions stored in a non-transitory computer-readable storage medium that, when executed by the one or more processors of the computer system, performing the steps of:
(a) training a neural network to form a trained neural network, at least in part, with a plurality of input human-generated campaign datasets and a plurality of ground truths for the plurality of input human-generated campaign datasets, wherein a given ground truth of the plurality of ground truths represents a quality metric indicating a quality of a given input human-generated campaign dataset of the plurality of input human-generated campaign datasets;
(b) generating a machine-generated candidate campaign dataset using the trained neural network;
(c) identifying at least one first similar prior campaign dataset from feedback records comprising a plurality of prior campaign datasets by computing a respective degree of similarity score between the machine-generated candidate campaign dataset and each of the plurality of prior campaign datasets, wherein a given degree of similarity score is computed using one or more campaign dataset elements comprising at least one of language, location, demographics, or product/service type, and wherein the respective degree of similarity scores between the machine-generated candidate campaign dataset and the at least one first similar prior campaign dataset exceeds a similarity score threshold;
(d) computing a first evaluation scam of the machine-generated candidate campaign dataset for a first online campaign using the machine-generated candidate campaign dataset, the first evaluation score comprising a first targeting value and respective confidence value, wherein the first evaluation score is computed based on campaign results associated with the at least one first similar prior campaign dataset;
(e) when the first evaluation score satisfies a first criteria, generating a request for a human-generated candidate campaign dataset;
(f) identifying at least one second similar prior campaign dataset from the feedback records comprising the plurality of prior campaign datasets by computing a respective degree of similarity score between the human-generated candidate campaign dataset and each of the plurality of prior campaign datasets, wherein the given degree of similarity score is computed using one or more campaign dataset elements comprising at least one of language, location, demographics, or product/service type, and wherein the respective degree of similarity scores between the human-generated candidate campaign dataset and the at least one second similar prior campaign dataset exceeds the similarity score threshold,
(g) computing a second evaluation score of the human-generated candidate campaign dataset for a second online campaign using the human-generated candidate campaign dataset, the second exaltation score comprising a second targeting value and respective confidence value, wherein the second evaluation score is computed based on campaign results associated with the at least one second similar prior campaign dataset;
(h) based on the first evaluation score and the second evaluation score, selecting the machine-generated candidate campaign dataset or the human-generated candidate campaign dataset to be a submitted campaign dataset, wherein in at least one instance, the submitted campaign dataset is the machine-generated candidate campaign dataset and the first evaluation score reflects an improvement over the second evaluation score;
(i) submitting the submitted campaign dataset to the online platform;
(j) obtaining a campaign result resulting from the submitted campaign dataset; and
(k) storing the submitted campaign dataset and the campaign result as the feedback records comprising the plurality of prior campaign datasets for future campaigns.
|