US 12,259,800 B2
Scalable exactly-once data processing using transactional streaming writes
Pavan Edara, Sammamish, WA (US); Reuven Lax, Mountain View, CA (US); Ji Yang, San Mateo, CA (US); and Gurpreet Singh Nanda, Seattle, WA (US)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Dec. 20, 2023, as Appl. No. 18/391,229.
Application 18/391,229 is a continuation of application No. 18/165,257, filed on Feb. 6, 2023, granted, now 11,880,290.
Application 18/165,257 is a continuation of application No. 17/085,576, filed on Oct. 30, 2020, granted, now 11,573,876, issued on Feb. 7, 2023.
Prior Publication US 2024/0143469 A1, May 2, 2024
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 11/14 (2006.01); G06F 9/30 (2018.01); G06F 9/46 (2006.01); G06F 11/07 (2006.01); G06F 11/30 (2006.01); G06F 12/02 (2006.01)
CPC G06F 11/3034 (2013.01) [G06F 9/30047 (2013.01); G06F 9/467 (2013.01); G06F 11/0757 (2013.01); G06F 11/0772 (2013.01); G06F 11/1402 (2013.01); G06F 12/0246 (2013.01); G06F 12/0253 (2013.01); G06F 2201/84 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising:
receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware;
partitioning the batch of data blocks into a plurality of sub-batches of data blocks;
assigning each sub-batch of data blocks of the plurality of sub-batches of data blocks to a respective buffered stream, each respective buffered stream configured to write the assigned sub batches to the memory hardware;
writing, using the assigned respective buffered stream, the sub-batch of data blocks to the memory hardware;
determining that a particular sub-batch of data blocks of the plurality of sub-batches of data blocks failed to be written to the memory hardware; and
based on determining that the particular sub-batch of data blocks of the plurality of sub-batches of data blocks failed to be written to the memory hardware, retrying to write the particular sub-batch of data blocks to the memory hardware using the assigned respective buffered stream.