| CPC G06F 16/254 (2019.01) [G06F 16/2379 (2019.01)] | 19 Claims |
|
1. A method comprising:
starting an extract, transform, load (ETL) process for a federated database system, the ETL process used for migrating changes to records in a source database to a target database located remotely from the source database;
parsing, with a Change Data Capture (CDC) reader, a batch of audit log entries in an audit log, the audit log entries identifying a plurality of changes to a plurality of records in the source database, and the batch being defined by a start of batch (SOB) field and an end of batch (EOB) field;
aggregating updates to the plurality of database records in an aggregation table that have occurred since a last ETL process, the aggregation table having a single entry for each record of the plurality of database records that is changed by the plurality of changes, and representing less than the entirety of a set of records in a source table of the source database,
wherein the aggregation table includes, in a same entry of the single entry for a record that is changed by the plurality of changes, an aggregation of two or more updates to a corresponding record in the source database as reflected in two or more audit log entries in the audit log that are consolidated in a single record in the aggregation table, and wherein each audit log entry causes a modification of the single entry for the corresponding record in the a relation table such that the two or more updates appear once in the aggregation table as the single entry including the aggregation of the two or more updates for the record that represents only a last change of the plurality of changes for the record and eliminates previous and intermediary changes; and
sending, from the aggregation table to the target database, an update of each of the plurality of records without transmitting each of the audit log entries to the target database in the federated database system.
|