CPC G06F 16/215 (2019.01) [G06F 16/24556 (2019.01)] | 20 Claims |
1. A method of data unification, the method comprising:
receiving a conflation plan;
building an index for each partition of a plurality of indexed partitions;
receiving a first plurality of data records;
generating an updated first plurality of data records by performing a data unification process based on the conflation plan for the first plurality of data records, wherein the data unification process comprises performing an in-memory clustering for each partition of the plurality of indexed partitions in parallel;
receiving a second plurality of data records;
generating a plurality of merged data records by merging the second plurality of data records and the updated first plurality of data records;
generating a stable identifier (stableID) for each data record of the plurality of merged data records; and
updating each index of each partition of the plurality of indexed partitions with the plurality of merged data records.
|