| CPC G06F 16/27 (2019.01) [A61F 5/566 (2013.01); G06F 9/4881 (2013.01); G06F 9/5016 (2013.01); G06F 9/5044 (2013.01); G06F 9/5083 (2013.01); G06F 9/5088 (2013.01); G06F 16/148 (2019.01); G06F 16/1827 (2019.01); G06F 16/211 (2019.01); G06F 16/221 (2019.01); G06F 16/2365 (2019.01); G06F 16/2456 (2019.01); G06F 16/2471 (2019.01); G06F 16/24532 (2019.01); G06F 16/24545 (2019.01); G06F 16/24552 (2019.01); G06F 16/254 (2019.01); G06F 16/283 (2019.01); G06F 16/951 (2019.01); G06F 16/9535 (2019.01); H04L 67/1095 (2013.01); H04L 67/1097 (2013.01); H04L 67/568 (2022.05)] | 30 Claims |
|
1. A non-transitory computer-readable medium storing instructions which, when executed by a first plurality of processors, cause the first plurality of processors to:
receive a plurality of queries with a database system to be processed by a plurality of virtual warehouses that includes a first plurality of processors, wherein each of the plurality of virtual warehouses includes multiple ones of the first plurality of processors and corresponding local storage;
identify a plurality of query tasks associated with the plurality of queries;
identify one or more resources associated with the plurality of query tasks;
identify a local cache storing a particular resource of the plurality of resources using metadata accessible to the first plurality of processors, wherein the local cache is associated with one or more processors of a second plurality of processors, the metadata is stored separately from the local storage of each of the second plurality of processors, and each of second plurality of processors is stateless with respect to the metadata;
distribute one or more query tasks of the plurality of query tasks to the plurality of virtual warehouses to be executed by the one or more associated processors of the second plurality of processors in response to identifying that the particular resource is stored in the cache associated with the one or more processors;
change, during the processing of the plurality of the query tasks by the plurality of virtual warehouses, a total number of virtual warehouses in the plurality of virtual warehouses using a load of the plurality of virtual warehouses due to the processing of the plurality of the query tasks by the plurality of virtual warehouses, wherein the change is at least one of creating a new virtual warehouse or deleting an existing virtual warehouse and the change in the total number of virtual warehouses are made in a unit of the multiple ones of the first plurality of processors and the corresponding local storage, wherein cache resources associated with the new virtual warehouse are populated with data files associated with the processing of the plurality of query tasks at the time the virtual warehouse is created and wherein the cache resources vary among the plurality of processors, wherein a first subset of the plurality of processors comprises minimal cache resources and a second subset of the plurality of processors comprises cache resources providing faster input-output operations;
as a result of the change in the total number of virtual warehouses, redistribute the one or more query tasks of the plurality of query tasks to the plurality of virtual warehouses, the one or more query tasks to be executed by the one or more associated processors of the second plurality of processors; and
update the metadata in response to the one or more associated processors of the second plurality of processors executing at least a portion of the plurality of query tasks.
|