US 12,141,628 B2
Distributed threaded streaming platform reader
Charles Patrick Nugent, Antrim (GB); Gareth Albert McCarter, Belfast (GB); and Michael Patrick King, Naperville, IL (US)
Assigned to Chicago Mercantile Exchange Inc., Chicago, IL (US)
Filed by Chicago Mercantile Exchange Inc., Chicago, IL (US)
Filed on May 3, 2023, as Appl. No. 18/142,797.
Application 18/142,797 is a continuation of application No. 17/213,369, filed on Mar. 26, 2021, granted, now 11,675,639.
Application 17/213,369 is a continuation of application No. 16/557,248, filed on Aug. 30, 2019, granted, now 10,990,459, issued on Apr. 27, 2021.
Prior Publication US 2023/0273844 A1, Aug. 31, 2023
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 9/54 (2006.01); G06F 9/48 (2006.01); G06Q 40/04 (2012.01)
CPC G06F 9/546 (2013.01) [G06F 9/4881 (2013.01); G06Q 40/04 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A streaming platform system comprising:
a memory;
a processor;
a streaming platform implemented by the processor, the streaming platform configured to:
receive a plurality of messages transmitted by a plurality of data producer systems, wherein each message of the plurality of messages includes a message content;
augment the message content with data associated with a unique sequential identifier based on when each message of the plurality of messages was transmitted by the plurality of data producer systems relative to when other messages of the plurality of messages were transmitted by the plurality of data producer systems;
store the plurality of messages in a plurality of partitions; and
continuously stream, to a multithreaded streaming reader implemented by the processor, each message of the plurality of messages from each partition of the plurality of partitions or an end of signal if the partition is empty; and
the multithreaded streaming reader including:
a plurality of first processing threads configured to be executed by the processor to:
continuously receive the plurality of messages or the end of partition signal from each partition of the plurality of partitions of the streaming platform; and
store the plurality of received messages or the end of partition signal in a vector of queues in the memory based on the unique sequential identifier, wherein the plurality of received messages are stored in a time ordered sequence from an earliest message to a most recent message based on the unique sequential identifier, the earliest message of which being stored in a first position of a respective queue of the vector of queues, wherein each queue of the vector of queues is associated with one of the plurality of partitions; and
a second processing thread independent of the plurality of first processing threads, configured to be executed by the processor to continuously:
compare the unique sequential identifier of all of the plurality of messages in the first position of each queue of the vector of queues;
extract the message content from the earliest message associated with the unique sequential identifier of the plurality of messages stored in the first position of each queue of the vector of queues; and
forward the extracted message content to a receiver in an order in accordance with the time ordered sequence defined by the unique sequential identifier associated therewith.