A tale of two erasure codes in HDFS
作者： Mario Blaum , David A. Pease , Mingyuan Xia , Mohit Saxena
关键词: Distributed data store 、 Workload 、 Computer science 、 Erasure code 、 Distributed File System 、 Parallel computing 、 Online codes 、 Recovery performance 、 Computer data storage 、 Latency (engineering)
摘要: … The HDFS-RAID module is implemented by Facebook to support a single erasure code for distributed storage in an HDFS cluster. Our implementation of HACFS spans nearly 2 K lines …
acm.org
sci-hub.st 参考文章(20)
Cheng Huang, Yikang Xu, Brad Calder, Huseyin Simitci, Sergey Yekhanin, Aaron Ogus, Parikshit Gopalan, Jin Li, Erasure coding in windows azure storage usenix annual technical conference. pp. 2- 2 ,(2012)
James Plank, Randal Burns, Cheng Huang, Osama Khan, William Pierce, Rethinking erasure codes for cloud file systems: minimizing I/O for recovery and degraded reads file and storage technologies. pp. 20- 20 ,(2012) , 10.5555/2208461.2208481
Kevin M. Greenan, James S. Plank, Ethan L. Miller, Screaming fast Galois field arithmetic using intel SIMD instructions file and storage technologies. pp. 299- 306 ,(2013) , 10.5555/2591272.2591303
Cristina L. Abad, Nathan Roberts, Yi Lu, Roy H. Campbell, A storage-centric analysis of MapReduce workloads: File popularity, temporal locality and arrival patterns ieee international symposium on workload characterization. pp. 100- 109 ,(2012) , 10.1109/IISWC.2012.6402909
Cheng Huang, Minghua Chen, Jin Li, Pyramid Codes ACM Transactions on Storage. ,vol. 9, pp. 1- 28 ,(2013) , 10.1145/2435204.2435207
John Wilkes, Richard Golding, Carl Staelin, Tim Sullivan, The HP AutoRAID hierarchical storage system ACM Transactions on Computer Systems. ,vol. 14, pp. 108- 136 ,(1996) , 10.1145/225535.225539
Kai Ren, YongChul Kwon, Magdalena Balazinska, Bill Howe, Hadoop's adolescence Proceedings of the VLDB Endowment. ,vol. 6, pp. 853- 864 ,(2013) , 10.14778/2536206.2536213
Yanpei Chen, Sara Alspaugh, Randy Katz, Interactive analytical processing in big data systems Proceedings of the VLDB Endowment. ,vol. 5, pp. 1802- 1813 ,(2012) , 10.14778/2367502.2367519
Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung, The Google file system symposium on operating systems principles. ,vol. 37, pp. 29- 43 ,(2003) , 10.1145/1165389.945450
M. Blaum, J. L. Hafner, S. Hetzler, Partial-MDS Codes and Their Application to RAID Type of Architectures IEEE Transactions on Information Theory. ,vol. 59, pp. 4510- 4519 ,(2013) , 10.1109/TIT.2013.2252395