The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
May. 09, 2017
Filed:
Nov. 30, 2016
International Business Machines Corporation, Armonk, NY (US);
Liang Liu, Beijing, CN;
Junmei Qu, Beijing, CN;
Chao Qiang Zhu, Beijing, CN;
Wei Zhuang, Beijing, CN;
International Business Machines Corporation, Armonk, NY (US);
Abstract
A computer implemented method provides shuffle optimization in map-reduce processing. The computer implemented method obtains intermediate results from a plurality of mappers for an application on a computing device. The intermediate results are combined from the plurality of mappers and are then partitioned into intermediate results for respective reducers for the application. The intermediate results for the respective reducers are stored into a buffer storage, which is partitioned into buffer sections for the respective reducers. Based on a rate of increase in the intermediate results, the method predicts a time at which a data capture notification shall be sent. The method sends the data capture notification based on a predicted time being reached, to indicate that the intermediate results for respective reducers stored in the buffer storage can be transmitted to the respective reducers, and transmits the intermediate results for the respective reducers to the respective reducers.