US 12,079,172 B2
Separation of storage and compute in database management systems
Joseph Victor, Oakland, CA (US); Adam Prout, Mississauga (CA); Rodrigo Toste Gomes, San Francisco, CA (US); and Kai Yuan Lung, San Francisco, CA (US)
Assigned to Singlestore, Inc., San Francisco, CA (US)
Filed by Singlestore, Inc., San Francisco, CA (US)
Filed on Dec. 22, 2021, as Appl. No. 17/559,491.
Prior Publication US 2023/0195697 A1, Jun. 22, 2023
Int. Cl. G06F 16/178 (2019.01)
CPC G06F 16/178 (2019.01) 18 Claims
OG exemplary drawing
 
1. A method for managing data in a database management system, the database management system comprising a first node, the first node being a master storage node for storing user data from a database, and a second node, the second node performing operations on user data stored in the database, wherein the operations include reading or writing the user data, wherein the method comprises configuring the second node to:
write log data indicative of one or more operations that have been performed on user data in the database to a set of log files, wherein a said operation is made permanent in the database once the set of log files have been updated in the non-volatile storage of the second node to include the log data indicative of the operation and without requiring user data associated with the operation to be transmitted to the first node;
segment the set of log files into a set of log file chunks, wherein each log file chunk is a portion of a log file in the set of log files; and
for a said log file chunk, transmit user data from the database corresponding to the log file chunk to the first node, the transmitted user data comprising a binary large object, and thereafter transmit the log file chunk to the first node,
wherein the second node segments the set of log files and transmits the log file chunks and associated user data to the first node asynchronously to its updating of the set of log files;
apply a selection algorithm to identify user data to be deleted from the second node; and
delete the identified user data,
wherein user data in the database which is identified for deletion includes the binary large object which has been transmitted to the first node.