Towards systematic parallel programming over mapreduce

摘要: MapReduce is a useful and popular programming model for data-intensive distributed parallel computing. But it still challenge to develop programs with systematically, since usually not easy derive proper divide-and-conquer algorithm that matches MapReduce. In this paper, we propose homomorphism-based framework named Screwdriver systematic MapReduce, making use of the program calculation theory list homomorphisms. implemented as Java library on top Hadoop. For any problem which can be resolved by two sequential functions satisfy requirements third homomorphism theorem, automatically transform initial an efficient program. Users need neither care about parallelism nor have deep knowledge addition simplicity our framework, such calculational approach enables us resolve many problems would nontrivial directly