US 11,960,464 B2
Customer-related partitioning of journal-based storage systems
Timothy Daniel Cole, Seattle, WA (US); John Michael Morkel, Seattle, WA (US); Tate Andrew Certain, Seattle, WA (US); Christopher Richard Jacques de Kadt, Seattle, WA (US); Artem Danilov, Kirkland, WA (US); Andrew Wayne Ross, Seattle, WA (US); and Allan Henry Vermeulen, Corvallis, OR (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Jul. 23, 2018, as Appl. No. 16/042,884.
Application 16/042,884 is a continuation of application No. 14/833,003, filed on Aug. 21, 2015, granted, now 10,031,935.
Prior Publication US 2018/0329936 A1, Nov. 15, 2018
Int. Cl. G06F 16/23 (2019.01); G06F 16/27 (2019.01); G06F 16/28 (2019.01)
CPC G06F 16/2315 (2019.01) [G06F 16/2372 (2019.01); G06F 16/2393 (2019.01); G06F 16/27 (2019.01); G06F 16/278 (2019.01); G06F 16/284 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system, comprising:
one or more computing devices including respective processors and memory to implement a control plane component for a distributed storage system, the control plane component to:
receive, from a client via a programmatic interface provided by the control plane component, a request indicating partitioning instructions for materialization of writes indicated in committed transaction entries of a journal and directed to a first data object of the distributed storage system;
configure, responsive to the request indicating the partitioning instructions for the materialization of the writes indicated in the committed transaction entries of the journal and directed to the first data object, a plurality of materialization nodes and write appliers for respective partitions of the first data object in accordance with the partitioning instructions, including a first materialization node and a first write applier configured by the control plane to apply writes from the journal to a first partition of the first data object at the first materialization node and a second materialization node and a second write applier configured by the control plane to apply writes from the journal to a second partition of the first data object at the second materialization node,
wherein the first write applier for the first partition of the first data object is configured by the control plane to apply writes from the journal at the first materialization node and is configured to identify and select, according to the partitioning instructions received from the client, a first set of writes corresponding to the first partition of the first data object from among a plurality of sets of writes of the committed transaction entries of the journal that correspond to different partitions of the first data object, and apply, by the first write applier, the first set of writes that were identified and selected from the journal by the first write applier to the first partition of the first data object at the first materialization node, and
wherein the second write applier for the second partition of the first data object is configured by the control plane to apply writes from the journal at the second materialization node and is configured to identify and select, according to the partitioning instructions received from the client, a second set of writes corresponding to the second partition of the first data object from among the plurality of sets of writes of the committed transaction entries of the journal that correspond to the different partitions of the first data object, and apply, by the second write applier, the second set of writes that were identified and selected from the journal by the second write applier to the second partition of the first data object at the second materialization node,
wherein the first set of writes and the second set of writes indicated in the committed transaction entries of the journal are to be respectively applied by different write appliers to different partitions of the first data object at different materialization nodes based on the request received from the client; and
send an acknowledgement of the request, indicating the partitioning instructions for the materialization of the writes indicated in the committed transaction entries of the journal and directed to the first data object, to the client.