US 12,405,943 B2
Generating minor compactions to capture aggregated actions for commit ranges to data files
Frederick Ryan Johnson, Orem, UT (US); and Prakhar Jain, Sunnyvale, CA (US)
Assigned to Databricks, Inc., San Francisco, CA (US)
Filed by Databricks, Inc., San Francisco, CA (US)
Filed on Jan. 17, 2024, as Appl. No. 18/415,396.
Prior Publication US 2025/0231930 A1, Jul. 17, 2025
Int. Cl. G06F 16/23 (2019.01); G06F 16/174 (2019.01)
CPC G06F 16/2379 (2019.01) [G06F 16/1744 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
receiving, from one or more client devices, requests to commit one or more transactions to a data table and write metadata for the one or more transactions to one or more log files, wherein the data table is stored in a cloud storage as a set of data files;
generating a checkpoint file aggregating the one or more transactions described in the one or more log files to compute a state of the data table at a first time;
receiving requests to commit a set of transactions to the data table and write metadata for the set of transactions to a set of log files, wherein the set of transactions are transactions that occur after the one or more transactions;
responsive to determining that a number of log files in the set of log files reaches a threshold commit number, generating a minor compaction file aggregating the set of transactions in the set of log files; and
generating a second checkpoint file aggregating the state of the data table at the first time with information on the set of transactions obtained from the minor compaction file to compute a state of the data table at a second time.