US 11,657,025 B2
Parallel processing of filtered transaction logs
Thomas Demoor, Ghent (BE); Satish Kumar Chakka, Bangalore (IN); and Viswanath Chandrasekara Bharathi, Karnataka (IN)
Assigned to Western Digital Technologies, Inc., San Jose, CA (US)
Filed by Western Digital Technologies, Inc., San Jose, CA (US)
Filed on Mar. 24, 2020, as Appl. No. 16/828,953.
Claims priority of provisional application 62/865,880, filed on Jun. 24, 2019.
Prior Publication US 2020/0401562 A1, Dec. 24, 2020
Int. Cl. G06F 7/00 (2006.01); G06F 16/18 (2019.01); G06F 16/16 (2019.01); G06F 16/17 (2019.01)
CPC G06F 16/1865 (2019.01) [G06F 16/164 (2019.01); G06F 16/1734 (2019.01); G06F 16/1858 (2019.01)] 19 Claims
OG exemplary drawing
 
1. A storage system, comprising:
at least one storage node configured to execute storage operations;
a transaction log configured to store a plurality of log entries corresponding to storage operations for the at least one storage node;
a first metadata streamer configured to sequentially retrieve a set of log entries from the transaction log; and
a plurality of transaction log filters configured to selectively provide filtered subsets of the set of log entries from the first metadata streamer to a plurality of metadata operations, wherein:
each transaction log filter of the plurality of transaction log filters corresponds to a different metadata operation of the plurality of metadata operations;
each different metadata operation of the plurality of metadata operations is configured to process a different filtered subset of the set of log entries;
the different metadata operations of the plurality of metadata operations are configured to operate in parallel on the different filtered subsets of the set of log entries;
a first transaction log filter of the plurality of transaction log filters is configured with a first metadata operation type selected from a group of metadata operation types, the group including:
replication;
notification;
metadata indexing;
garbage collection;
data scrub; and
parameter calculation;
a second transaction log filter of the plurality of transaction log filters is configured with a second metadata operation type selected from the group of metadata operation types;
the first metadata operation type is different than the second metadata operation type;
each metadata operation of the plurality of metadata operations is configured with an operation group;
a selected operation group includes a plurality of synchronized metadata operations; and
filtered subsets of the set of log entries are configured for release when each metadata operation for the selected operation group are all complete.