US 12,111,837 B1
Dense retrieval employing progressive distillation training
Jian Jiao, Bellevue, WA (US); Yeyun Gong, Beijing (CN); Xingwei He, Beijing (CN); Nan Duan, Beijing (CN); and Eren Manavoglu, Menlo Park, CA (US)
Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Apr. 25, 2023, as Appl. No. 18/306,869.
Int. Cl. G06F 16/2457 (2019.01); G06F 16/242 (2019.01); G06F 16/248 (2019.01); G06N 20/00 (2019.01)
CPC G06F 16/24578 (2019.01) [G06F 16/2438 (2019.01); G06F 16/248 (2019.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computing system comprising:
a processor; and
memory that stores instructions that, when executed by the processor, cause the processor to perform acts comprising:
responsive to receiving a query from a client computing device, computing an embedding of the query by way of a first encoder;
selecting candidate search results from a pool of potential results based upon the embedding of the query and embeddings of the potential results, the candidate search results being a subset of the potential results;
computing a similarity score for a first candidate search result in the candidate search results by way of a second encoder, wherein the second encoder receives the query and the first candidate search result as input, wherein the second encoder is trained based upon an order metric that is indicative of an ordering of the potential results, and wherein the first encoder is trained based upon output of the second encoder;
ranking the candidate search results based upon the similarity score;
identifying a search result pertaining to the query based upon the ranking of the candidate search results; and
outputting the identified search result to the client computing device.