US 11,687,380 B2
Optimizing resource allocation for distributed stream processing systems
Bradley William Fawcett, Byron, MN (US); Paul Gerver, Rochester, MN (US); Jingdong Sun, Rochester, MN (US); Jason A. Nikolai, Rochester, MN (US); and Henry Chiu, San Jose, CA (US)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Sep. 10, 2020, as Appl. No. 17/16,508.
Prior Publication US 2022/0075664 A1, Mar. 10, 2022
Int. Cl. G06F 9/50 (2006.01); G06N 20/00 (2019.01); G06F 9/48 (2006.01); G06F 18/214 (2023.01)
CPC G06F 9/5077 (2013.01) [G06F 9/4881 (2013.01); G06F 9/50 (2013.01); G06F 9/5061 (2013.01); G06F 18/214 (2023.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
15. A computer system comprising:
a processor set; and
a computer readable storage medium;
wherein:
the processor set is structured, located, connected and/or programmed to run program instructions stored on the computer readable storage medium; and
the stored program instructions comprise:
program instructions to train a machine learning model to determine allocations of computing resources to processing elements of a stream processing job according to a specified objective, and
program instructions to allocate a set of computing resources to the processing elements by:
allocating to the processing elements a first subset of the set of computing resources based, at least in part, on a minimum resource requirement for the processing elements, and
allocating to the processing elements a second subset of the set of computing resources based, at least in part, on an allocation determined using the trained machine learning model;
wherein the allocating of the second subset is based on scoring of historical resource allocation information associated with the trained machine learning model and selecting CPU core adjustments of processing elements that result in a best score of the scoring.