US 12,477,030 B1
Client-configurable in-transit segmentation of streaming input at managed data transfer services
Tanvi Mohandas Bandodkar, San Jose, CA (US); Manjunath Tumkur Maheshchandra, Santa Clara, CA (US); Chao Deng, Menlo Park, CA (US); and Ayaskant Pani, Fremont, CA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Sep. 30, 2021, as Appl. No. 17/491,077.
Int. Cl. G06F 16/28 (2019.01); H04L 67/06 (2022.01)
CPC H04L 67/06 (2013.01) [G06F 16/284 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system, comprising:
one or more computing devices;
wherein the one or more computing devices include instructions that upon execution on or across the one or more computing devices:
provide one or more programmatic interfaces for receiving client-specified in-transit segmentation descriptors for a data transfer service of a provider network,
wherein the client-specified in-transit segmentation descriptors indicate data record properties of records to be received via messages, the data record properties to be used to dynamically identify respective storage object groups to which at least a portion of content of individual ones of a plurality of data records in messages from a plurality of streaming data sources is to be directed, and
wherein a storage object group to which a data record, of the plurality of data records, is to be directed is not specified within the corresponding one of the messages, received at the data transfer service, comprising the data record;
obtain, via one or more of the programmatic interfaces, a first in-transit segmentation descriptor of the client-specified in-transit segmentation descriptors;
receive, at the data transfer service, a particular message from a streaming data source of the plurality of streaming data sources, wherein the particular message comprises a particular data record having one or more data record properties;
obtain results of one or more computations performed on the one or more data record properties of the particular data record, the one or more data record properties indicated in the first in-transit segmentation descriptor obtained via the one or more programmatic interfaces, wherein the one or more computations include extracting values of one or more content attributes of the particular data record;
create, responsive to said obtain results of the one or more computations, a particular storage object group, of the storage object groups; and
cause a version of at least a portion of contents of the particular data record to be stored at a destination, within a storage service of the provider network, comprising the particular storage object group created responsive to said obtain the results of the one or more computations.