US 12,333,450 B1
Sample size and duration prediction for online activity
Yu Liu, Bothell, WA (US); Thomas Richardson, Seattle, WA (US); James McQueen, Seattle, WA (US); Doug Hains, Shoreline, WA (US); Will Poff, Des Moines, WA (US); and Tridiv Sardesai, Seattle, WA (US)
Assigned to AMAZON TECHNOLOGIES, INC., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Aug. 20, 2021, as Appl. No. 17/407,968.
Int. Cl. G06F 17/00 (2019.01); G06F 17/18 (2006.01); G06N 5/04 (2023.01)
CPC G06N 5/04 (2013.01) [G06F 17/18 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method, comprising:
receiving a request via a graphical user interface to modify a website from a previous version of the website to a modified version of the website for a first duration, the modified version of the website including at least one new feature relative to a previous version of a website;
implementing the at least one new feature and subjecting the modified version of the website to live traffic;
receiving first data describing a first sample of users that navigated to the modified version of the website during a first time period, wherein the first sample of users has a first sample size;
receiving first key performance indicator (KPI) data related to the first sample of users;
determining a predicted sample size of a predicted sample of users navigating to the modified version of the website during a second time period after the first time period, wherein the predicted sample size is determined based on the first data;
determining a predicted mean KPI and a predicted variance of KPI for the predicted sample of users based on the first KPI data;
calculating a statistical power using the predicted sample size of users, the predicted mean KPI, and the predicted variance of KPI, the statistical power being related to an ability to predict a predefined effect on the KPI for a given time period;
determining a second duration for which to subject the modified version of the website to traffic using the statistical power;
providing, using the graphical user interface, a recommendation to subject the modified version of the website to traffic for the second duration;
receiving input accepting the recommendation; and
removing the modified version of the website from live traffic after the second duration of time has elapsed.