CPC G06F 12/0261 (2013.01) [G06F 3/0604 (2013.01); G06F 3/0608 (2013.01); G06F 16/122 (2019.01); G06F 16/1734 (2019.01); G06F 16/182 (2019.01); G06F 16/2219 (2019.01); G06F 3/0649 (2013.01)] | 20 Claims |
1. A computer-implemented method executed by data processing hardware that causes the data processing hardware to perform operations comprising:
receiving data objects, each received data object held by a respective data bucket of a plurality of data buckets;
for each received data object:
determining a predicted lifespan of the respective data object based on a historical object lifespan table associated with the respective data bucket holding the respective data object, the historical object lifespan table indicating a creation time for each past data object uploaded into the respective data bucket and a deletion timestamp indicating a time for each past data object that is deleted from the respective data bucket;
selecting, based on the predicted lifespan of the respective data object, a particular shard file of a plurality of shard files that has an associated predicted lifespan range at instantiation of the particular shard file that includes the predicted lifespan of the respective data object; and
writing the respective data object into the selected particular shard file; and
storing the plurality of shard files in a distributed storage system.
|