Coded Computing for Distributed Graph Analytics

作者: Saurav Prakash , Amirhossein Reisizadeh , Ramtin Pedarsani , Salman Avestimehr

DOI: 10.1109/ISIT.2018.8437860

关键词: Graph analyticsRandom graphServerTheoretical computer scienceVertex (graph theory)GraphComputer scienceBottleneckNumerical analysisComputation

摘要: Many distributed graph computing systems have been developed recently for efficient processing of massive graphs. These require many messages to be exchanged among machines at each step the computation, making communication bandwidth a major performance bottleneck. We present coded framework that systematically injects redundancy in computation phase enable coding opportunities thus reducing load substantially. Specifically, we propose schemes an inverse-linear trade-off (asymptotically) between and average Erdos-Renyi (ER) random graph. The proposed scheme ER is shown optimal asymptotically as size $n\rightarrow\infty$ . For finite $n$ , demonstrate via numerical analysis given $r$ i.e. when vertex carefully stored servers, slashes by (nearly)

参考文章(21)
Jimmy Lin, Michael Schatz, Design patterns for efficient graph algorithms in MapReduce mining and learning with graphs. pp. 78- 85 ,(2010) , 10.1145/1830252.1830263
ANDREW LUMSDAINE, DOUGLAS GREGOR, BRUCE HENDRICKSON, JONATHAN BERRY, CHALLENGES IN PARALLEL GRAPH PROCESSING Parallel Processing Letters. ,vol. 17, pp. 5- 20 ,(2007) , 10.1142/S0129626407002843
E. Dahlhaus, D. S. Johnson, C. H. Papadimitriou, P. D. Seymour, M. Yannakakis, The complexity of multiway cuts (extended abstract) symposium on the theory of computing. pp. 241- 251 ,(1992) , 10.1145/129712.129736
Robert Ryan McCune, Tim Weninger, Greg Madey, Thinking Like a Vertex: A Survey of Vertex-Centric Frameworks for Large-Scale Distributed Graph Processing ACM Computing Surveys. ,vol. 48, pp. 25- ,(2015) , 10.1145/2818185
W. Xing, A. Ghorbani, Weighted PageRank algorithm conference on communication networks and services research. pp. 305- 314 ,(2004) , 10.1109/DNSR.2004.1344743
Yucheng Low, Danny Bickson, Joseph Gonzalez, Carlos Guestrin, Aapo Kyrola, Joseph M. Hellerstein, Distributed GraphLab Proceedings of the VLDB Endowment. ,vol. 5, pp. 716- 727 ,(2012) , 10.14778/2212351.2212354
Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, Dennis Fetterly, Dryad: distributed data-parallel programs from sequential building blocks european conference on computer systems. ,vol. 41, pp. 59- 72 ,(2007) , 10.1145/1272996.1273005
Rong Chen, Xin Ding, Peng Wang, Haibo Chen, Binyu Zang, Haibing Guan, Computation and communication efficient graph processing with distributed immutable view high performance distributed computing. pp. 215- 226 ,(2014) , 10.1145/2600212.2600233
Grzegorz Malewicz, Matthew H Austern, AJ Bik, James C Dehnert, Ilan Horn, Naty Leiser, Grzegorz Czajkowski, Pregel Proceedings of the 2010 international conference on Management of data - SIGMOD '10. pp. 135- 146 ,(2010) , 10.1145/1807167.1807184
Jeffrey Dean, Sanjay Ghemawat, MapReduce Communications of the ACM. ,vol. 51, pp. 107- 113 ,(2008) , 10.1145/1327452.1327492