US 12,450,087 B2
Data processing system that performs stream processing on received data and performs batch processing on the received data after storage, a first processing apparatus, and a non-transitory computer-readable storage medium
Kazuya Matsuo, Musashino (JP); Masaru Takagi, Musashino (JP); Ryota Nakada, Musashino (JP); and Koya Mori, Musashino (JP)
Assigned to NTT, Inc., Tokyo (JP)
Appl. No. 17/776,082
Filed by NTT, Inc., Tokyo (JP)
PCT Filed Nov. 18, 2019, PCT No. PCT/JP2019/045085
§ 371(c)(1), (2) Date May 11, 2022,
PCT Pub. No. WO2021/100087, PCT Pub. Date May 27, 2021.
Prior Publication US 2022/0382582 A1, Dec. 1, 2022
Int. Cl. G06F 9/48 (2006.01); G06F 9/46 (2006.01); G06F 16/245 (2019.01); G06F 16/2455 (2019.01); H04L 65/401 (2022.01); G06V 20/58 (2022.01)
CPC G06F 9/4843 (2013.01) [G06F 16/245 (2019.01); G06F 16/24568 (2019.01); H04L 65/4015 (2013.01); G06F 9/466 (2013.01); G06F 2201/805 (2013.01); G06V 20/58 (2022.01)] 6 Claims
OG exemplary drawing
 
1. A data processing system that performs stream processing on received data and performs batch processing on the received data after storage, the data processing system comprising:
a fast response processing apparatus disposed at an edge and comprising one or more processor configured to perform a part of the stream processing on the received data;
a first result providing apparatus comprising one or more processor configured to store a first processing result of the fast response processing apparatus and return the first processing result in response to a query from one or more client terminal;
a real-time processing apparatus comprising one or more processor configured to perform a remaining part of the stream processing on the first processing result received from the fast response processing apparatus;
a second result providing apparatus comprising one or more processor configured to store a second processing result of the real-time processing apparatus and return the second processing result in response to the query;
a storage configured to store the received data;
a batch processing apparatus comprising one or more processor configured to perform the batch processing on the received data read from the storage at a predetermined timing; and
a third result providing apparatus comprising one or more processor configured to store a third processing result of the batch processing apparatus and return the third processing result in response to the query,
wherein the data processing system is a lambda architecture having a real-time processing layer and a batch processing layer, the fast response processing apparatus and the real-time processing apparatus belonging to the real-time processing layer, and the batch processing apparatus belonging to the batch processing layer,
wherein the same received data is transmitted to both the real-time processing layer and the batch processing layer, and
wherein the part of the stream processing is a group of processes extracted in order of execution from multiple processes in the stream processing such that a total processing time in the fast response processing apparatus does not exceed a predetermined threshold value.