US 12,468,443 B2
Method and apparatus for processing data, reduction server, and mapping server
Hua Xu, Hangzhou (CN); Xiaoming Bao, Beijing (CN); Hongwei Sun, Beijing (CN); Yihai Zheng, Shenzhen (CN); and Chun Liu, Hangzhou (CN)
Assigned to Huawei Technologies Co., Ltd., Shenzhen (CN)
Filed by HUAWEI TECHNOLOGIES CO., LTD., Guangdong (CN)
Filed on Oct. 12, 2023, as Appl. No. 18/485,847.
Application 18/485,847 is a continuation of application No. PCT/CN2022/085771, filed on Apr. 8, 2022.
Claims priority of application No. 202110401463.9 (CN), filed on Apr. 14, 2021; application No. 202110638812.9 (CN), filed on Jun. 8, 2021; and application No. 202110812926.0 (CN), filed on Jul. 19, 2021.
Prior Publication US 2024/0036728 A1, Feb. 1, 2024
Int. Cl. G06F 3/06 (2006.01); G06F 12/02 (2006.01)
CPC G06F 3/0608 (2013.01) [G06F 3/064 (2013.01); G06F 3/0644 (2013.01); G06F 3/067 (2013.01); G06F 12/0223 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for processing data, comprising:
obtaining, by a reduction server of a plurality of reduction servers in a distributed processing system and from a first storage area, metadata of first data to be read, wherein the first data comprises a target data block in a plurality of data blocks of second data, and wherein the second data comprises a processing result of a mapping server of a plurality of mapping servers for input data, and the plurality of data blocks of the second data are generated based on operations comprising:
partitioning, by the mapping server in response to determining that data in the plurality of data blocks needs to be sorted, a second storage area in a global memory into a plurality of sub-areas based on a preset second size; and
storing, by the mapping server, the plurality of data blocks into the plurality of sub-areas according to a sequence of the plurality of sub-areas;
determining, by the reduction server, a first address of the first data in the global memory based on the metadata, wherein the global memory comprises memories of the plurality of mapping servers and the plurality of reduction servers; and
reading, by the reduction server, the first data from the global memory based on the first address.