External sorting on a parallel interleaved file system

摘要: Parallel computers with non-parallel file systems find limited by the performance of the processor running the file system. We have designed and implemented a parallel file system called Bridge that eliminates this problem by spreading both data and file system computation over a large number of processors and disks. To assess the effectiveness of Bridge we have used it as the basis of a parallel external merge sort, an application re-quiring significant amounts of interprocessor communica-tion and data movement. A detailed analysis of this application indicates that Bridge can profitably be used on configurations in excess of one hundred processors with disks. Empirical results on a 32-processor implementation agree closely with the analysis, providing us with a high degree of confidence in this prediction. Based on our experience, we argue that file systems such as Bridge will satisfy the 1/0 needs of a wide range of parallel architectures and applications.