US 12,298,900 B2
Methods and systems for garbage collection and compaction for key-value engines
Jiaxin Ou, Beijing (CN); Yi Wang, Beijing (CN); Jingwei Zhang, Beijing (CN); and Zhengyu Yang, Los Angeles, CA (US)
Assigned to Lemon Inc., (KY)
Filed by Lemon Inc., Grand Cayman (KY); and Beijing Volcano Engine Technology Co., Ltd., Beijing (CN)
Filed on Sep. 27, 2023, as Appl. No. 18/475,664.
Prior Publication US 2024/0020231 A1, Jan. 18, 2024
Int. Cl. G06F 12/02 (2006.01)
CPC G06F 12/0253 (2013.01) [G06F 2212/7205 (2013.01)] 17 Claims
OG exemplary drawing
 
1. A method of garbage collection for a key-value engine in a data storage and communication system, the method comprising:
determining disk capacity usage of the key-value engine;
adjusting a garbage collection percentage threshold and a number of garbage collection threads based on whether the disk capacity usage of the key-value engine meets and/or exceeds predetermined disk capacity usage thresholds; and
determining a garbage volume for one or more blob files in the key-value engine,
wherein the determining the garbage volume includes:
storing a value size of the one or more blob files in a delta block in a sorted string table (SST) of the key-value engine;
summing sizes of all SST entries that are dependent on the one or more blob files to determine an effective data size of the one or more blob files; and
determining any remaining size of the one or more blob files as the garbage volume; and executing an occurrence of a garbage collection process on selected blob files.