US 11,709,836 B2
Data fetch engine
Jason Prentice, Lexington, MA (US); Ian James, Stoneham, MA (US); and Leo Shapiro, Acton, MA (US)
Assigned to S&P Global Inc., New York, NY (US)
Filed by S&P Global Inc., New York, NY (US)
Filed on Oct. 21, 2021, as Appl. No. 17/451,704.
Prior Publication US 2023/0127966 A1, Apr. 27, 2023
Int. Cl. G06F 16/2453 (2019.01); G06F 16/23 (2019.01); G06F 11/34 (2006.01)
CPC G06F 16/24544 (2019.01) [G06F 11/3428 (2013.01); G06F 16/2393 (2019.01); G06F 16/24535 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A method for retrieving data records, comprising:
providing a set of static configuration objects including:
a set of resources that describe available data items; and
a set of views that express a serialized transformation of resources objects of the set of resources into a response;
in response to receiving a data request, generating a data fetch execution plan from the set of resources and the set of views, comprising:
deriving, by executor from a view stored in the set of views, a dependency structure that captures data provenance relationships and describes data dependencies among interlinked backing tables that need to be queried to generate a response;
determining, by the executor from the dependency structure, a data fetch order; and
generating the data fetch execution plan based on data fetch order, wherein the set of performance optimizations are selected from:
a first optimization that fetches data from a SQL database, generates a single SQL query for each table that the view depends on, and coordinates execution of these queries via the dependency structure;
a second optimization that generates a single SQL query which includes joins across all required tables; and
a third optimization that returns data from a data store in a stream instead of a single batch;
wherein the data fetch execution plan is generated by an executor from a set of executors, wherein each executor of the set of executors is adapted to a particular data store and set of performance requirements, and wherein views and resources of different executors of the set of executors can be interchangeably combined; and
retrieving the data records according to the data fetch execution plan.