US 12,455,887 B2
Online post-processing in rankings for constrained utility maximization
Swetasudha Panda, Burlington, MA (US); Ariel Kobren, Cambridge, MA (US); Jean-Baptiste Frederic George Tristan, Burlington, MA (US); and Michael Louis Wick, Lexington, MA (US)
Assigned to Oracle International Corporation, Redwood City, CA (US)
Filed by Oracle International Corporation, Redwood City, CA (US)
Filed on Jul. 6, 2021, as Appl. No. 17/368,544.
Claims priority of provisional application 63/066,044, filed on Aug. 14, 2020.
Prior Publication US 2022/0050848 A1, Feb. 17, 2022
Int. Cl. G06F 16/2457 (2019.01); G06F 16/28 (2019.01); G06N 20/00 (2019.01)
CPC G06F 16/24578 (2019.01) [G06F 16/285 (2019.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system, comprising:
at least one processor;
a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to implement a ranking system providing fairness across a plurality of rankings, the ranking system configured to:
receive a stream of data items over time;
provide, as part of continuous monitoring of the ranking system, respective fair rankings of respective pluralities of data items for a plurality of time steps including a current time step and one or more time steps earlier than the current time step, wherein to provide a fair ranking for the current time step the ranking system is configured to:
apply a ranking model to generate a ranking of a plurality of data items in a batch of data items obtained from the stream of data items, wherein the batch of data items are associated with the current time step, and wherein the ranking model is a machine learning model trained to generate the ranking of the plurality of data items in the batch in a descending order of relevance for an application;
apply a re-ranking model to generate a fair ranking of the plurality of data items in the batch, wherein an order of the fair ranking is different from the descending order of relevance for the application; wherein the re-ranking model is applied according to a re-ranking policy that satisfies a fairness constraint applicable, in aggregate, to the generated ranking for the batch of data items and respective generated fair rankings for one or more other batches of data items from the stream of data items associated with different respective time steps earlier than the current time step, wherein the one or more other batches of data items were previously ranked using the ranking model and re-ranked, using the re-ranking model, with respective fair rankings of one or more further batches with respective time steps earlier than the different respective time steps of the one or more other batches before being sent to the application, wherein the fairness constraint is determined based on parity between different groups associated with individual ones of the plurality of data items; and
send the fair ranking of the plurality of data items to the application.