| CPC G06F 11/3495 (2013.01) [G06F 11/327 (2013.01); G06N 20/00 (2019.01)] | 20 Claims |

|
1. A method comprising:
receiving a plurality of messages destined for a first application over a period of time, the first application comprising a production application executing on a first computing device;
duplicating each message of the plurality of messages to create a corresponding plurality of duplicate messages;
successively sending each message of the plurality of messages to the first application and each duplicate message to a second application, the second application comprising a pre-production application corresponding to the production application and executing on a second computing device, wherein the pre-production application is a different version of the production application and implements new functionality not implemented in the production application;
determining, based on behavior information that identifies behaviors exhibited by the first application in response to receiving the plurality of messages and behaviors exhibited by the second application in response to receiving the plurality of duplicate messages over the period of time, that a behavior of the second application differs from a behavior of the first application beyond an alert criterion; and
in response to determining that the behavior of the second application differs from the behavior of the first application beyond the alert criterion, sending, to a destination, a message indicating that the behavior of the second application differs from the behavior of the first application, the message identifying the behavior of the second application that differs from the behavior of the first application.
|