| CPC G06F 16/27 (2019.01) | 18 Claims |
|
1. A method for providing cross-account data distribution management, the method being implemented by at least one processor, the method comprising:
receiving, by the at least one processor, an indication of a change in an object repository of a producer account, the change including an addition of at least one data set into the object repository;
assigning, by the at least one processor via a scheduler service, at least one computing resource to perform at least one task, based on the indication, a predetermined guideline, and an operation efficiency level for each of the least one computing resource;
retrieving, by the at least one processor via an application programming interface, at least one parameter that corresponds to a data warehouse of a consumer account, wherein the application programming interface is triggered by a serverless compute instance of the producer account to perform the retrieving;
initiating, by the at least one processor, the at least one computing resource, based on the assigning, to copy the at least one data set from the object repository to a producer data lake;
transferring, by the at least one processor via a managed data lake and based on the retrieved at least one parameter, the at least one data set from the producer data lake to a consumer data lake without duplication of the at least one data set in a storage layer; and
persisting, by the at least one processor, the at least one data set from the consumer data lake in the data warehouse,
wherein each of the producer account and the consumer account is linked via a cross-account role that enables access between a plurality of accounts, and
wherein the cross-account role is leveraged for the transferring of the at least one data set from the producer data lake to a consumer data lake without duplication of the at least one data set from the producer data lake.
|