US 11,893,029 B2
Real-time streaming data ingestion into database tables
Tyler Arthur Akidau, Seattle, WA (US); Istvan Cseri, Seattle, WA (US); Tyler Jones, Redwood City, CA (US); Daniel E. Sotolongo, Seattle, WA (US); and Zhuo Zhang, Kirkland, WA (US)
Assigned to Snowflake Inc., Bozeman, MT (US)
Filed by Snowflake Inc., Bozeman, MT (US)
Filed on Oct. 25, 2022, as Appl. No. 18/049,325.
Application 18/049,325 is a continuation of application No. 17/647,500, filed on Jan. 10, 2022, granted, now 11,526,517.
Application 17/647,500 is a continuation of application No. 17/386,258, filed on Jul. 27, 2021, granted, now 11,250,006.
Application 17/386,258 is a continuation of application No. 17/226,423, filed on Apr. 9, 2021, granted, now 11,100,111.
Prior Publication US 2023/0070152 A1, Mar. 9, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 16/245 (2019.01); G06F 16/25 (2019.01); G06F 16/22 (2019.01); G06F 16/2455 (2019.01); G06F 16/2453 (2019.01)
CPC G06F 16/24568 (2019.01) [G06F 16/2219 (2019.01); G06F 16/2456 (2019.01); G06F 16/24544 (2019.01); G06F 16/258 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A method comprising:
receiving a registration request for a per-account group of files, the registration request including identification information for the per-account group of files and a storage location of the per-account group of files;
accessing the per-account group of files based on the registration request, the per-account group of files being stored in a first format;
deduping and validating the per-account group of files using sequencing information included in the per-account group of files;
committing the per-account group of files stored in the storage location and making data in the per-account group of files in the first format accessible for query processing before the data is ingested into one or more source tables;
generating a hybrid table for query processing, the hybrid table including the committed data in the first format and data from the one or more source tables;
receiving a query prior to the data being ingested;
executing the query using the hybrid table to generate results for the query including:
converting the committed data from the first format into a common format;
converting the data from the one or more source tables into the common format;
joining the committed data in the common format and the data from the one or more source tables in the common format to generate joined data; and
executing the query based on the joined data; and
ingesting the data into the one or more source tables in a second format.