Efficient data restructuring and aggregation for I/O acceleration in PIDX

摘要: Hierarchical, multiresolution data representations enable interactive analysis and visualization of large-scale simulations. One promising application these techniques is to store high performance computing simulation output in a hierarchical Z (HZ) ordering that translates from Cartesian coordinate scheme one-dimensional array ordered by locality at different resolution levels. However, when the dimensions are not an even power 2, parallel HZ produces sparse memory network access patterns inhibit I/O performance. This work presents new technique for datasets restructures into large (power 2) blocks facilitate efficient aggregation. We perform both weak strong scaling experiments using S3D combustion on Cray-XE6 (65,536 cores) IBM Blue Gene/P (131,072 platforms. demonstrate can be written hierarchical, format with competitive native data-ordering methods.