CPC G06F 9/3885 (2013.01) [G06F 9/505 (2013.01); G06F 9/5072 (2013.01)] | 18 Claims |
1. A method for distributed and parallel processing of data within a data processing platform, the method comprising:
receiving, at the data processing platform, the data to be processed by the data processing platform, wherein the data processing platform comprises a plurality of processing components;
ingesting, at the plurality of processing components, the data, wherein the ingesting comprises the data being located at more than one of the plurality of processing components simultaneously;
processing, simultaneously at the more than one of the plurality of processing components, the data, wherein the processing comprises incorporating, by each of the more than one of the plurality of processing components, an identifier into an output of the processing of the data, the identifier corresponding to the data and indicating the output was generated from the data;
and
receiving, at a downstream component of the data processing platform, the output from each of the plurality of processing components that processed the data, wherein the receiving comprises the downstream component identifying the outputs corresponding to the data based upon the identifier within the outputs and aggregating all the outputs corresponding to the data.
|