US 12,216,656 B2
Scalable query processing
Thierry Cruanes, San Mateo, CA (US); Igor Demura, Mountain View, CA (US); Varun Ganesh, San Bruno, CA (US); Prasanna Rajaperumal, Bangalore (IN); Libo Wang, Foster City, CA (US); and Jiaqi Yan, Menlo Park, CA (US)
Assigned to Snowflake Inc., Bozeman, MT (US)
Filed by Snowflake Inc., Bozeman, MT (US)
Filed on Sep. 29, 2023, as Appl. No. 18/477,808.
Application 18/477,808 is a continuation of application No. 17/823,572, filed on Aug. 31, 2022, granted, now 11,809,428.
Application 17/823,572 is a continuation of application No. 17/657,257, filed on Mar. 30, 2022, granted, now 11,461,326.
Application 17/657,257 is a continuation of application No. 16/889,033, filed on Jun. 1, 2020, granted, now 11,347,735.
Prior Publication US 2024/0028592 A1, Jan. 25, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 16/2453 (2019.01)
CPC G06F 16/24542 (2019.01) [G06F 16/24537 (2019.01); G06F 16/24539 (2019.01)] 21 Claims
OG exemplary drawing
 
1. A method comprising:
receiving, by one or more processors, a query directed at a data set stored in a network-based database system;
generating a query plan to execute the query;
identifying a portion of the query plan that is eligible for fragment processing based on an output-to-input ratio where input represents size of one or more input files for fragment processing and output represents size of an output file based on the one or more input file;
executing, by a parent query coordinator using one or more computing resources of a first set of computing resources assigned to the parent query coordinator, the identified portion of the query plan on a first batch of files of the data set to generate a first batch result;
transmitting instructions to a fragment query coordinator for the fragment query coordinator to execute the identified portion of the query on a second batch of files of the data set to generate the output file using a second set of computing resources assigned to the fragment query coordinator, the output file being stored in a storage location;
scanning, by the parent query coordinator, the output file from the storage location to generate a second batch result;
combining the first batch result and the second batch result to generate combined results; and
executing remaining portion of the query plan on the combined results to generate a response to the query.