US 11,983,165 B1
Load history calculation in internal stage replication
Abdullah Al Mahmood, Bothell, WA (US); Chong Han, Bellevue, WA (US); Ganeshan Ramachandran Iyer, Bellevue, WA (US); Jiaxing Liang, Bellevue, WA (US); Nithin Mahesh, Kirkland, WA (US); and Yanrui Zhang, Mercer Island, WA (US)
Assigned to Snowflake Inc., Bozeman, MT (US)
Filed by Snowflake Inc., Bozeman, MT (US)
Filed on Mar. 29, 2023, as Appl. No. 18/128,212.
Int. Cl. G06F 16/23 (2019.01); G06F 16/174 (2019.01); G06F 16/27 (2019.01)
CPC G06F 16/2365 (2019.01) [G06F 16/1748 (2019.01); G06F 16/27 (2019.01)] 24 Claims
OG exemplary drawing
 
1. A method comprising:
as a set of files are replicated from a first deployment to an internal stage of a second deployment, embedding user-defined file-level metadata in each of the set of files, the user-defined file-level metadata of each file corresponding to original file-level metadata of the file;
storing, in a dedicated column of a directory table corresponding to the internal stage, the original file-level metadata for one or more of the set of files that are registered with the directory table;
generating, from the set of files, an initial list of files to be loaded into a table of the second deployment based on the internal stage and the directory table;
determining, by a processing device, whether the directory table and the internal stage are consistent; and
in response to determining that the directory table and the internal stage are consistent, deduplicating the initial list of files to be loaded using a load history of the second deployment to generate a list of files to be loaded into the table of the second deployment.