US 12,112,263 B2
Reversal-point-based detection and ranking
Bo Yang, Brighton, MA (US); Chaofan Huang, Atlanta, GA (US); Songtao Guo, Cupertino, CA (US); Robert Perrin Reeves, Castro Valley, CA (US); Wan Qi Gao, San Francisco, CA (US); Patrick Ryan Driscoll, Oakland, CA (US); Kristina Caroline Ryan, San Francisco, CA (US); Michael Mario Jennings, San Francisco, CA (US); Jeremy Lwanga, San Francisco, CA (US); and Manzarul Azad Kazi, Walnut Creek, CA (US)
Assigned to Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed by Microsoft Technology Licensing, LLC, Redmond, WA (US)
Filed on Dec. 9, 2020, as Appl. No. 17/116,184.
Prior Publication US 2022/0180181 A1, Jun. 9, 2022
Int. Cl. G06N 3/08 (2023.01); G06F 18/20 (2023.01); G06F 18/21 (2023.01); G06F 18/2113 (2023.01); G06F 18/25 (2023.01)
CPC G06N 3/08 (2013.01) [G06F 18/2113 (2023.01); G06F 18/2163 (2023.01); G06F 18/217 (2023.01); G06F 18/25 (2023.01); G06F 18/29 (2023.01)] 18 Claims
OG exemplary drawing
 
1. A system for training and using a machine learned model, comprising:
a non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, cause the system to perform operations comprising:
obtaining time series data, the time series data including a value for a first metric at each of a plurality of time points separated by time intervals;
identifying, in the time series data, a plurality of reversal points, a reversal point being a time point at which the value for the first metric changed from positive to negative or vice-versa;
obtaining training data from one or more databases;
training a machine learned model, using the training data as input to a machine learning algorithm, to generate a ranking score for an input reversal point, the ranking score being based on:
calculating an abnormality score by:
determining a difference between a length of a trend, prior to the input reversal point, in a segment of the time series containing the input reversal point and a total number of reversal points in a plurality of reversal points in the segment, wherein a trend is a sequence of time points at which the value for the first metric maintained a same sign;
dividing the difference by a value computed based on a total number of time points in the segment;
calculating a significance score by:
determining a maximum absolute value of differences between a magnitude change at the input reversal point and those magnitude changes in the trend in the segment prior to the input reversal point, applying a constant β to the maximum absolute value, β learned via the training of the machine learning model;
applying a first weight to the abnormality score and a value related to the first weight to the significance score; and
combining the weighted abnormality score and weighted significance score into the ranking score;
evaluating one or more of the plurality of reversal points by passing each of the one or more reversal points to the machine learned model, thus outputting a ranking score for each of the one or more of the plurality of reversal points;
ranking the one or more of the plurality of reversal points based on their corresponding ranking scores; and
highlighting one or more of the reversal points in a graphical user interface based on the ranking.