CPC G06F 16/2471 (2019.01) [G06F 16/2452 (2019.01); G06F 16/24535 (2019.01); G06F 16/24568 (2019.01); G06F 16/9535 (2019.01)] | 20 Claims |
1. A method, comprising:
receiving, by a computer system, a query to be executed against an event stream, the query comprising a query operator;
determining, by the computer system, a state type of the query operator, wherein the state type is one of a plurality of state types comprising a fully stateful state type, a stateless state type, or a semi-stateful state type, wherein the fully stateful state type comprises a first query with operators in a query plan that maintain state and requires that the event stream be present on each node of the plurality of distributed nodes, wherein the stateless state type comprises a second query with operators in the query plan that do not maintain any state, and wherein the semi-stateful state type comprises a third query with operators in the query plan that maintain state and does not require that the event stream be present on each node of the plurality of distributed nodes; and
in accordance with a determination that the query operator is of the fully stateful state type:
dividing the query, using a query rewrite algorithm, into i) a plurality of partial aggregators for parallel processing on a plurality of distributed nodes and ii) a global aggregator for processing on a single node;
distributing, by the computer system, each partial aggregator of the plurality of partial aggregators of the query to each node, respectively, of the plurality of distributed nodes of a cluster of virtual machines;
enabling, by the computer system, parallel execution of each partial aggregator at each node of the plurality of distributed nodes;
enabling, by the computer system, execution results of each partial aggregator to be joined at a global node; and
enabling, by the computer system, the global aggregator of the query to compute a result at the global node that includes each of the execution results from each node of the plurality of distributed nodes.
|