US 12,204,523 B2
Retrieval and caching of object metadata across data sources and storage systems
Zhaoxing Li, Mountain View, CA (US); Rayman Preet Singh, Livermore, CA (US); Fuat Can Efeoglu, San Francisco, CA (US); Daniel Tenedorio, Belmont, CA (US); and Sarah Cai, Cupertino, CA (US)
Assigned to Databricks, Inc., San Francisco, CA (US)
Filed by Databricks, Inc., San Francisco, CA (US)
Filed on Apr. 14, 2023, as Appl. No. 18/135,078.
Prior Publication US 2024/0346007 A1, Oct. 17, 2024
Int. Cl. G06F 16/00 (2019.01); G06F 16/23 (2019.01); G06F 16/2455 (2019.01)
CPC G06F 16/2365 (2019.01) [G06F 16/24552 (2019.01)] 17 Claims
OG exemplary drawing
 
1. A method, implemented by a computing system, comprising:
receiving, from a client device, a request to perform a query operation on a set of data objects stored in a remote data source, wherein the remote data source is remote from the computing system;
prior to accessing the set of data objects in the remote data source,
accessing a metadata cache storing metadata of one or more data objects, the metadata cache located at the computing system, and the metadata describing information of the one or more data objects stored in the remote data source;
identifying metadata corresponding to the set of data objects for the query operation in the metadata cache;
determining whether the identified metadata for the set of data objects meets an update condition, the update condition including a refresh interval indicating a time limit between two succussive updates of the metadata;
fetching, responsive to the identified metadata meeting the update condition, updated metadata for at least the set of data objects from the remote data source; and
storing the updated metadata in the metadata cache; and
performing, using the updated metadata, the query operation on the set of data objects.