US 12,235,869 B2
Identifying changes between source and destination databases using snapshots of source database and invertible bloom filters
Jason Nochlin, Denver, CO (US)
Assigned to FIVETRAN INC., Oakland, CA (US)
Filed by Fivetran Inc., Oakland, CA (US)
Filed on Nov. 15, 2023, as Appl. No. 18/510,451.
Application 18/510,451 is a continuation of application No. 17/529,740, filed on Nov. 18, 2021, granted, now 11,860,891.
Claims priority of provisional application 63/115,904, filed on Nov. 19, 2020.
Prior Publication US 2024/0086418 A1, Mar. 14, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 16/27 (2019.01); G06F 11/14 (2006.01); G06F 16/22 (2019.01); G06F 16/23 (2019.01); G06F 16/24 (2019.01); G06F 16/2453 (2019.01)
CPC G06F 16/27 (2019.01) [G06F 11/1446 (2013.01); G06F 16/2255 (2019.01); G06F 16/2282 (2019.01); G06F 16/2379 (2019.01); G06F 16/24534 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
retrieving a source data table and a destination data table;
determining a size for both a first invertible bloom filter and a second invertible bloom filter based on an estimated number of changes to the source data table, wherein an invertible bloom filter of a larger size occupies a larger space in a computer memory, and wherein determining the size for both the first invertible bloom filter and the second invertible bloom filter comprises:
determining a difference between the estimated number of changes and an actual number of changes to the source data table, and
adjusting the size responsive to the difference meeting a threshold;
obtaining the first invertible bloom filter for the source data table, the first invertible bloom filter having the determined size;
obtaining the second invertible bloom filter for the destination data table, the second invertible bloom filter having the determined size;
generating a third invertible bloom filter by subtracting the second invertible bloom filter from the first invertible bloom filter, the third invertible bloom filter comprising information associated with a change between the source data table and the destination data table;
identifying the change by decoding the third invertible bloom filter; and
transmitting information associated with the identified change to the destination data table to synchronize the destination data table with the source data table based on the identified change.