| CPC G06F 16/9535 (2019.01) [G06F 16/954 (2019.01)] | 20 Claims |
|
1. A method, comprising:
receiving a request for content associated with a client device;
in response to receiving the request for content, identifying a plurality of content items comprising a first content item and a second content item;
accessing one or more machine learning models stored in a computer system as part of a ranking system;
determining, using a content scoring model of the ranking system, a content item score for each of the plurality of content items comprising a first content item score for the first content item and a second content item score for the second content item, wherein the content scoring model is one of the one or more machine learning models stored in the computer system;
generating, using an exploration model of the ranking system, a first ranked list of the plurality of content items indicative of the first content item being ranked higher than the second content item;
generating, using the exploration model, a second ranked list of the plurality of content items indicative of the first content item being ranked lower than the second content item;
generating a first list score for the first ranked list based upon a first function of (i) first rankings indicative of the first content item being ranked higher than the second content item and (ii) the first content item score of the first content item and the second content item score of the second content item;
generating a second list score for the second ranked list based upon a second function of (i) second rankings indicative of the first content item being ranked lower than the second content item and (ii) the first content item score of the first content item and the second content item score of the second content item;
in response to the request for content associated with the client device, selecting, by the computer system implementing the ranking system and from among a plurality of ranked lists comprising the first ranked list and the second ranked list, the first ranked list based upon the first list score for the first ranked list being higher than the second list score for the second ranked list;
arranging, by the computer system implementing the ranking system, a plurality of selectable objects on a content item selection interface according to rankings indicated by the first ranked list; and
implementing, using processing circuitry, a closed-loop control process to continuously update the one or more machine learning models stored as part of the ranking system in the computer system based upon (i) feedback reflective of an impact of one or more content item selection interfaces on short-term behavior associated with one or more client devices and (ii) feedback reflective of an impact of the one or more content item selection interfaces on long-term behavior associated with the one or more client devices, wherein the closed-loop control process is associated with a reduction of errors and an improvement of operation of hardware of the computer system.
|