US 12,298,980 B1
Optimized storage of metadata separate from time series data
Rajesh Raman, Palo Alto, CA (US); Jennifer Williamson, Menlo Park, CA (US); Edward Crossman, Redwood City, CA (US); Uday Sagar Shiramshetty, Fremont, CA (US); Arijit Mukherji, Fremont, CA (US); Phillip Liu, Palo Alto, CA (US); and Tianyu Wang, San Francisco, CA (US)
Assigned to SPLUNK Inc., San Francisco, CA (US)
Filed by Splunk Inc., San Francisco, CA (US)
Filed on Jul. 24, 2020, as Appl. No. 16/938,807.
Claims priority of provisional application 62/968,948, filed on Jan. 31, 2020.
Int. Cl. G06F 16/2455 (2019.01); G06F 16/2457 (2019.01); G06F 16/248 (2019.01)
CPC G06F 16/24568 (2019.01) [G06F 16/2456 (2019.01); G06F 16/24573 (2019.01); G06F 16/248 (2019.01)] 28 Claims
OG exemplary drawing
 
1. A computer-implemented method for storing metadata in a distributed system, the method comprising:
receiving, from a source independent of a data source of an input data stream, metadata comprising information regarding data included in the input data stream, wherein introduction of the metadata does not increase an amount of data in the input data stream;
associating the metadata with a metric time series (MTS) object, the MTS object comprising a plurality of metadata objects, wherein at least one metadata object of the plurality of metadata objects comprises a metric;
storing, in a first in-memory data structure of a logical database, the MTS object;
storing, in a second in-memory data structure of the logical database, the plurality of metadata objects, the second in-memory data structure different than the first in-memory data structure;
replicating the MTS object to generate at least three replicas of the MTS object, the at least three replicas distributed across multiple clusters;
jointly querying the plurality of metadata objects and the MTS object based on the first in-memory data structure and the second in-memory data structure, wherein local replicas of the at least three replicas are prioritized over cross-region replicas;
associating, based on the querying, dimensions with the MTS object according to the first in-memory data structure and the second in-memory data structure, wherein the first in-memory data structure and the second in-memory data structure include information correlating the dimensions with MTS objects, each dimension comprising a subset of the plurality of metadata objects;
storing, in a third in-memory data structure, a set of queries associated with the MTS object, wherein updates to the MTS object relating to one or more queries in the set of queries are automatically output to a user interface; and
partitioning the at least three replicas into partitions shared across at least three nodes in a cluster of the multiple clusters, wherein each cluster of the multiple clusters are isolated from each other and the partitions of the at least three replicas across nodes are different in each of the clusters.