US 11,900,155 B2
Method, device, and computer program product for job processing
YuHong Nie, Shanghai (CN); Pengfei Wu, Shanghai (CN); Jinpeng Liu, Shanghai (CN); and Zhen Jia, Shanghai (CN)
Assigned to EMC IP Holding Company LLC, Hopkinton, MA (US)
Filed by EMC IP Holding Company LLC, Hopkinton, MA (US)
Filed on Feb. 26, 2020, as Appl. No. 16/801,473.
Claims priority of application No. 201911187922.7 (CN), filed on Nov. 28, 2019.
Prior Publication US 2021/0165685 A1, Jun. 3, 2021
Int. Cl. G06F 9/48 (2006.01); G06F 9/50 (2006.01); G06Q 10/0631 (2023.01)
CPC G06F 9/4881 (2013.01) [G06F 9/5083 (2013.01); G06F 2209/486 (2013.01); G06Q 10/06315 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for processing a job using a plurality of processing devices in a distributed processing system, each processing device having a processing resource comprising at least one of a computing resource and a storage resource, the method comprising:
obtaining a first group of tasks in a first portion of the job, the job having a time limitation for indicating a time when a job result is desired to be completed, the first group of tasks being executable in parallel;
setting a respective priority to each processing device of the plurality of processing devices based on a state of a processing resource of the processing device, wherein the priority of each processing device is set based on a time duration for which the processing devices can process one or more tasks associated with the job within the time limitation;
selecting a first group of processing devices from the plurality of processing devices based on the respective priorities, the first group of processing devices including one or more processing devices in each of at least first and second distinct processing systems, the first processing system comprising a cloud-based processing system and the second processing system comprising a blockchain-based processing system; and
allocating the first group of tasks to the first group of processing devices, respectively, so that the first group of processing devices utilize their respective processing resources to process the first group of tasks for generating a first group of task results;
wherein allocating the first group of tasks to the first group of processing devices, respectively comprises:
generating a first group of data partitions associated with the first group of tasks, respectively, based on raw data associated with the job;
storing the first group of data partitions to storage resources in the first group of processing devices; and
instructing the first group of processing devices to obtain the first group of data partitions from the storage resources in the first group of processing devices;
wherein setting a respective priority to each processing device of the plurality of processing devices based on a state of a processing resource of the processing device comprises:
setting a long-term priority to at least one of the processing devices in accordance with determining that a processing resource of the processing device is usable to process a plurality of tasks associated with the job, the long-term priority being based on the processing device not refusing to execute any task associated with the job;
wherein the first group of task results comprises one or more task results generated by one or more processing devices of the blockchain-based processing system; and
wherein the one or more task results generated by the one or more processing devices of the blockchain-based processing system are subject to a multi-level verification process comprising at least a first level in which a verification determination is made as to whether or not the one or more task results are trusted and a second level in which, responsive to the verification determination indicating that the one or more task results are not trusted, voting of multiple processing devices of the blockchain-based processing system is utilized to determine if at least a threshold number of the processing devices vote in a particular manner.