US 11,909,829 B1
Online testing efficiency through early termination
Yu Liu, Bothell, MA (US); Runzhe Wan, Secaucus, NJ (US); James McQueen, Seattle, WA (US); Doug Hains, Shoreline, MA (US); Rui Song, Seattle, WA (US); and Roger Henry Castillo, Seattle, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Jun. 16, 2023, as Appl. No. 18/336,802.
Int. Cl. H04L 67/143 (2022.01)
CPC H04L 67/143 (2013.01) 20 Claims
OG exemplary drawing
 
1. A computer-implemented method comprising:
obtaining first data during a first time period of execution of an experiment;
determining that a predicted duration of the experiment includes the first time period and subsequent time periods that end in time after the first time period;
determining a predicted sample size of each of the first time period and the subsequent time periods;
for a second time period of the subsequent time periods that ends in time after the first time period:
obtaining subsequent data during the second time period;
generating a second probability value of the experiment that is valid as of the second time period based on at least one of (i) the subsequent data obtained during the second time period, (ii) the predicted sample size of each of the first time period and the subsequent time periods, or (iii) individual second probability values valid as of each of the subsequent time periods that precede in time the second time period;
determining that the second probability value of the experiment that is valid as of the second time period satisfies a threshold value; and
in response to determining that the second probability value of the experiment that is valid as of the second time period satisfies the threshold value, causing a user interface to display a termination notification, wherein the termination notification, when selected, causes the experiment to terminate prior to completion of the predicted duration.