US 12,079,175 B2
Streaming synthesis of distributed traces from machine logs
Poornima Devaraj, Fremont, CA (US); Joseph Gabriel Echeverria, San Francisco, CA (US); Venkata SreeKrishna Koganti, Milpitas, CA (US); Shyam Mundhra, Pleasanton, CA (US); Hardik Shah, Vancouver (CA); Xiangyu Wu, Vancouver (CA); and Ryan Konrad Yee, Burnaby (CA)
Assigned to Splunk Inc., San Francisco, CA (US)
Filed by Splunk Inc., San Francisco, CA (US)
Filed on Oct. 19, 2020, as Appl. No. 17/074,426.
Prior Publication US 2022/0121628 A1, Apr. 21, 2022
Int. Cl. G06F 16/18 (2019.01); G06F 11/30 (2006.01); G06F 11/32 (2006.01); G06F 16/17 (2019.01); G06F 16/182 (2019.01)
CPC G06F 16/1865 (2019.01) [G06F 11/3034 (2013.01); G06F 11/323 (2013.01); G06F 16/1734 (2019.01); G06F 16/1824 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A method for using a streaming data processing system to generate operational traces from log data, the method comprising:
ingesting into the streaming data processing system a data stream comprising log entries generated by one or more network services;
identifying at least two log entries from the data stream that are associated with an individual transaction implemented on the one or more network services;
generating a trace for the individual transaction from the at least two log entries, the trace including information of one or more operations that make up the individual transaction, each operation represented as a span within the trace, wherein generating the trace includes implementing a stack, adding a span to the stack for individual log entries of the at least two log entries that indicates an operation start, and removing a span from the stack for individual log entries of the at least two log entries that represent an operation end, wherein the trace includes at least a first span corresponding to a first network service generated from the at least two log entries by identifying, by use of the stack, that a first log entry of the at least two log entries, demarking a start of a first operation of the first network service represented by the first span, corresponds to a second log entry of the at least two log entries demarking an end of the first operation of the first network service represented by the first span, wherein at least two log entries indicate an interaction between the first network service and a second network service, and wherein the first span indicates the interaction between the first network service and the second network service using an identifier for the second network service deterministically derived from the at least two log entries;
transmitting the trace to a services monitoring application;
identifying one or more additional log entries from the data stream generated by the second network service that are associated with the individual transaction;
generating an amendment for the trace from the one or more additional log entries, the amendment including a second span corresponding to the second network service, wherein the second span indicates the interaction between the first network service and the second network service using an identifier for the first network service deterministically derived from the one or more additional log entries; and
transmitting the trace to the services monitoring application, wherein the services monitoring application is configured to amend the trace using the amendment for the trace to generate an amended trace including the first span and the second span.