US 12,086,445 B1
Maintaining partition-level parity data for improved volume durability
Kun Tang, Bellevue, WA (US); Hon Ping Shea, Seattle, WA (US); and Michael Scott Ryan, Woodinville, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Mar. 23, 2020, as Appl. No. 16/827,565.
Int. Cl. G06F 3/06 (2006.01); G06F 11/10 (2006.01); G06F 11/20 (2006.01)
CPC G06F 3/0644 (2013.01) [G06F 3/0619 (2013.01); G06F 3/0652 (2013.01); G06F 3/067 (2013.01); G06F 3/0683 (2013.01); G06F 11/1076 (2013.01); G06F 11/2074 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A data storage system, comprising:
a plurality of nodes, each node comprising a computing device and a storage device;
one or more computing devices configured to implement a control plane for the data storage system, wherein the control plane is configured to:
assign respective ones of a plurality of partitions of a volume to respective ones of the nodes, wherein the respective ones of the nodes are configured to host the assigned respective partitions of the volume;
assign a volume-level or volume-partition-level parity partition for the volume to another one of the nodes, wherein the other one of the nodes is configured to host the assigned volume-level or volume-partition-level parity partition;
generate, for the volume-level or volume-partition-level parity partition, volume-level or volume-partition-level parity data by applying an erasure code to volume data of one or more of the plurality of partitions of the volume, wherein the generated volume-level or volume-partition-level parity data is stored to the volume-level or volume-partition-level parity partition hosted by the other one of the nodes;
cause the partitions of the volume and the volume-level or volume-partition-level parity partition for the volume to be stored on the respective ones of the nodes and the respective ones of the other nodes using a node level erasure coding or a node level replication, wherein the volume-level or volume-partition-level parity data is used as an input to the node level erasure coding or the node level replication; and
reconstruct, in response to a failure of at least one of the partitions, a reconstructed version of the at least one partition using the volume-level or volume-partition-level parity data stored in the volume-level or volume-partition-level parity partition hosted by the other node of the data storage system.