US 12,216,653 B1
Thrashing detection and avoidance for a tiered data processing service
Induja Sreekanthan, Mountain View, CA (US); Sriram Subramanian, Mountain View, CA (US); Athanasios Papathanasiou, Los Gatos, CA (US); and Vijayan Prabhakaran, San Francisco, CA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Mar. 31, 2022, as Appl. No. 17/710,476.
Int. Cl. G06F 15/16 (2006.01); G06F 16/2453 (2019.01)
CPC G06F 16/24539 (2019.01) [G06F 16/24545 (2019.01)] 19 Claims
OG exemplary drawing
 
1. A system, comprising:
one or more computing devices configured to implement a data processing service, wherein the data processing service uses a plurality of storage tiers to store data blocks, the plurality of storage tiers comprising at least:
a warm storage tier implemented using storage included in nodes of a node cluster of the data processing service, wherein the node cluster has a first storage capacity to store a first quantity of data blocks; and
a cold storage tier implemented using storage of a data storage service; and
wherein the data processing service is further configured to:
maintain, in one or more buffers of the warm storage tier, metadata comprising block identifiers for an additional number of data blocks that are stored in the cold storage tier that are in addition to the first quantity of data blocks stored in the warm storage tier, wherein the additional number of data blocks that are stored in the cold storage tier, and for which metadata comprising block identifiers are maintained in the one or more buffers, are data blocks which have previously been evicted from the warm storage tier with a number of query hits below a threshold number;
responsive to a query targeting a data block stored in the cold storage tier with a respective block identifier included in the one or more buffers of the warm storage tier, increment a counter corresponding to the block identifier for the data block targeted by the query; and
promote the data block, targeted by the query, from the cold storage tier to the warm storage tier, in response to the counter corresponding to the block identifier exceeding a threshold count, wherein the threshold count is two or more.