CPC G06F 16/254 (2019.01) [G06F 16/2228 (2019.01); G06F 16/25 (2019.01); G06Q 10/06 (2013.01); G06Q 10/067 (2013.01); G06Q 10/10 (2013.01)] | 16 Claims |
1. A method for persisting and reconciling one or more data sources on a computer system, the method comprising:
periodically extracting batch data from at least one primary data source,
transforming the batch data into a first format specified by a data schema for a database;
loading the transformed batch data into the database
continuously receiving real-time data from at least one secondary data source;
indexing the received real-time data to generate an index;
transforming the real-time data into the first format, wherein transforming the real-time data comprises discarding any real-time data that is available in the batch data;
loading the transformed real-time data into the database;
generating business intelligence reports comprising the real-time data in the index, the transformed real-time data in the database, the extracted batch data and the transformed batch data in the database; and
storing the real-time data by key-value in a distributed key-value store for subsequent integration into the database, wherein transforming the real-time data comprises transforming the real-time data stored in the distributed key-value store.
|