CPC G06F 16/1865 (2019.01) [G06F 16/164 (2019.01); G06F 16/1734 (2019.01); G06F 16/1858 (2019.01)] | 19 Claims |
1. A storage system, comprising:
at least one storage node configured to execute storage operations;
a transaction log configured to store a plurality of log entries corresponding to storage operations for the at least one storage node;
a first metadata streamer configured to sequentially retrieve a set of log entries from the transaction log; and
a plurality of transaction log filters configured to selectively provide filtered subsets of the set of log entries from the first metadata streamer to a plurality of metadata operations, wherein:
each transaction log filter of the plurality of transaction log filters corresponds to a different metadata operation of the plurality of metadata operations;
each different metadata operation of the plurality of metadata operations is configured to process a different filtered subset of the set of log entries;
the different metadata operations of the plurality of metadata operations are configured to operate in parallel on the different filtered subsets of the set of log entries;
a first transaction log filter of the plurality of transaction log filters is configured with a first metadata operation type selected from a group of metadata operation types, the group including:
replication;
notification;
metadata indexing;
garbage collection;
data scrub; and
parameter calculation;
a second transaction log filter of the plurality of transaction log filters is configured with a second metadata operation type selected from the group of metadata operation types;
the first metadata operation type is different than the second metadata operation type;
each metadata operation of the plurality of metadata operations is configured with an operation group;
a selected operation group includes a plurality of synchronized metadata operations; and
filtered subsets of the set of log entries are configured for release when each metadata operation for the selected operation group are all complete.
|