CPC G06Q 10/087 (2013.01) [G06Q 10/063114 (2013.01)] | 18 Claims |
1. A method of managing requests and coordinating fulfilment of the requests by one or more processors in communication with a distributed database and a key-value store, the method comprising:
receiving, at the one or more processors, a first request, the first request having a first plurality of events to take place to fulfill the first request;
writing, by the one or more processors, a first plurality of messages corresponding to the first plurality of events in a first queue of a plurality of independent queues in a distributed database, comprising:
assigning, by a leader server, a first shard of the distributed database to a first worker server for the first request, wherein the first shard is exclusively accessed by the first worker server;
creating, by the first worker server, a first identifier for the first queue in the first shard of the distributed database; and
providing a first signal to a distributed key value store in communication with the distributed database, the first signal indicating the first identifier for the first queue;
receiving, at the one or more processors, a second request, the second request having a second plurality of events to take place to fulfill the second request;
writing, by the one or more processors, a second plurality of messages corresponding to the second plurality of events in a second queue of the plurality of independent queues in the distributed database, the second queue separate from the first queue, the writing comprising:
assigning, by the leader server, a second shard of the distributed database to a second worker server for the second request wherein the second shard is exclusively accessed by the second worker server;
creating, by the second worker server, a second identifier for the second queue in the second shard of the distributed database; and
providing a second signal to the distributed key value store in communication with the distributed database, the second signal indicating the second identifier for the second queue; and
processing, by the one or more processors, the first plurality of events in the first queue in sequence and the enqueued second plurality of events in the second queue in sequence independent of a status of processing of the first plurality of events,
wherein the processing for each queue comprises:
receiving, at a respective worker server, a notification that a message is available in a respective queue;
reading, by the respective worker server, a message payload from the distributed database;
sending, by the respective worker server, the message payload to a consumer for performing an action based on the message payload;
receiving, by the respective worker server, an acknowledgement from the consumer that the message was processed by the consumer;
updating a cursor in the distributed database, the cursor recording progress of processing the message; and
activating, by the respective worker server, a next payload in the respective queue.
|