US 12,273,255 B1
Adaptive testing service that generates test cases from observed behaviors
Abhijit Prakash Bhatnagar, Bellevue, WA (US); Yusof Ganji, Bellevue, WA (US); Mohsen Azimi, Seattle, WA (US); Jason Adonis Timmons, Seattle, WA (US); Jacob Shannan Carr, Seattle, WA (US); Tristan Niles Cecil, Los Altos, CA (US); Evan Corriere, Sunnyvale, CA (US); Sahil Sharma, San Francisco, CA (US); Xinrui Li, Seattle, WA (US); and Huaqing Fang, New Jersey, NJ (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Oct. 2, 2023, as Appl. No. 18/479,712.
Int. Cl. H04L 43/55 (2022.01); H04L 43/024 (2022.01); H04L 43/106 (2022.01); H04L 41/147 (2022.01)
CPC H04L 43/55 (2022.05) [H04L 43/024 (2013.01); H04L 43/106 (2013.01); H04L 41/147 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
one or more computer devices that implement an adaptive testing service (ATS), configured to:
execute a test case generation process, including to:
receive telemetry data of a production version of a network service in a production environment;
derive, from the telemetry data, a plurality of behaviors of the network service and frequency or recency metrics of individual ones of the behaviors;
generate a plurality of test cases executable to test individual ones of the behaviors, including weights for individual ones of the test cases determined based at least in part on the frequency or recency metrics; and
store the test cases and the weights in a test case repository; and
execute a code change testing process, including to:
receive a code change to a development version of the network service in a development environment;
execute the test cases in the test case repository to identify one or more changed behaviors of the development version of the network service as a result of the code change;
determine to accept or reject the code change based at least in part on the one or more changed behaviors and one or more weights of one or more test cases that identified the one or more changed behaviors; and
after the code change is applied into the development version, update at least one of the one or more weights to cause at least one of the one or more test cases to be deprecated in the test case repository.