US 11,789,936 B2
Storage engine for hybrid data processing
Jianjun Chen, Los Angeles, CA (US); Yonghua Ding, Los Angeles, CA (US); Ye Liu, Los Angeles, CA (US); Fangshi Li, Los Angeles, CA (US); Li Zhang, Los Angeles, CA (US); Mingyi Zhang, Los Angeles, CA (US); Kui Wei, Los Angeles, CA (US); Wei Ding, Los Angeles, CA (US); Kai Wu, Los Angeles, CA (US); and Jason Yang Sun, Los Angeles, CA (US)
Assigned to LEMON INC., Grand Cayman (KY)
Filed by LEMON INC., Grand Cayman (KY)
Filed on Aug. 31, 2021, as Appl. No. 17/462,853.
Prior Publication US 2023/0063730 A1, Mar. 2, 2023
Int. Cl. G06F 16/00 (2019.01); G06F 16/23 (2019.01); G06F 16/28 (2019.01)
CPC G06F 16/2379 (2019.01) [G06F 16/28 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A method, comprising:
receiving data captured by a first processing engine, wherein the first processing engine is configured to perform online transactional processing;
distributing multiple replicas of logical logs generated based on the data to a Delta Store in a storage system by applying a quorum protocol on the multiple replicas, wherein data in the Delta Store are stored in a row format and visible to a query for online analytical processing performed by a second processing engine;
flushing data from the Delta Store to a Base Store in the storage system based on one or more predetermined rules, wherein data in the Base Store are stored in a columnar format and accessible by the second processing engine, the data in the Base Store are partitioned into a plurality of partitions based on partition keys, and each partition in the Base Store comprises a plurality of data blocks;
wherein the storage system is configured to persist a same data in different formats to be consumed by the first processing engine and the second processing engine, respectively; and
wherein the storage system, the first processing engine, and the second processing engine are configured to be decoupled from each other.