US 12,443,440 B2
Method for executing data processing task in cluster mixed deployment scenario, electronic device and storage medium
Zhaowei Huang, Beijing (CN); Kejian Ju, Beijing (CN); and Leilei Hu, Beijing (CN)
Assigned to Beijing Volcano Engine Technology Co., Ltd., Beijing (CN)
Filed by Beijing Volcano Engine Technology Co., Ltd., Beijing (CN)
Filed on Nov. 1, 2024, as Appl. No. 18/934,437.
Claims priority of application No. 202311452596.4 (CN), filed on Nov. 2, 2023.
Prior Publication US 2025/0147801 A1, May 8, 2025
Int. Cl. G06F 9/48 (2006.01); G06F 9/50 (2006.01)
CPC G06F 9/4881 (2013.01) [G06F 9/5016 (2013.01)] 14 Claims
OG exemplary drawing
 
1. A method for executing a data processing task in a cluster mixed deployment scenario, wherein a first electronic device is provided with computing nodes, and the computing nodes belong to at least two data engines, the first electronic device comprises a memory, and the method comprises:
in response to a first data engine node receiving the data processing task, determining, by the first electronic device, a first occupancy of the memory corresponding to the data processing task, wherein the first data engine node is deployed on the first electronic device and belongs to a first data engine, and the first data engine node is one of the computing nodes; and
determining, by the first electronic device, whether to execute the data processing task based on a second occupancy of the memory corresponding to another data engine node, wherein the another data engine node is deployed on the first electronic device and belongs to another data engine other than the first data engine in the at least two data engines,
wherein determining, by the first electronic device, whether to execute the data processing task based on the second occupancy of the memory corresponding to the another data engine node comprises:
determining the second occupancy of the memory and determining a third occupancy of the memory corresponding to the first data engine node, wherein the second occupancy of the memory comprises a memory amount occupied by a running process in processes created by the another data engine node, and the third occupancy of the memory comprises a memory amount occupied by a running process in processes created by the first data engine node; and
determining whether to execute the data processing task based on the second occupancy of the memory and the third occupancy of the memory,
wherein determining whether to execute the data processing task based on the second occupancy of the memory and the third occupancy of the memory comprises:
determining a first sum based on the first occupancy of the memory, the second occupancy of the memory, and the third occupancy of the memory;
in response to the first sum being less than a preset electronic device threshold, determining a second sum based on the first occupancy of the memory and the third occupancy of the memory; and
executing the data processing task in response to the second sum being less than a preset first data engine threshold,
or,
determining a second sum based on the first occupancy of the memory and the third occupancy of the memory;
in response to the second sum being less than a preset first data engine threshold, determining a first sum based on the first occupancy of the memory, the second occupancy of the memory, and the third occupancy of the memory; and
executing the data processing task in response to the first sum being less than a preset electronic device threshold.