A paper is accepted in IPDPS 2019: C-GDR: High-Performance Container-aware GPUDirect MPI Communication Schemes on RDMA Networks. Congratulations to all the authors.

Paper Info

[IPDPS'19] C-GDR: High-Performance Container-aware GPUDirect MPI Communication Schemes on RDMA Networks

Jie Zhang, Xiaoyi Lu, Ching-Hsiang Chu, and Dhabaleswar K. Panda.

In Proceedings of the 33rd IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2019.

Abstract

In recent years, GPU-based platforms have received significant success for parallel applications. In addition to highly optimized computation kernels on GPUs, the cost of data movement on GPU clusters plays critical roles in delivering high performance for end applications. Many recent studies have been proposed to optimize the performance of GPU- or CUDA-aware communication runtimes and these designs have been widely adopted in the emerging GPU-based applications. These studies mainly focus on improving the communication performance on native environments, i.e., physical machines, however GPU-based communication schemes on cloud environments are not well studied yet. This paper first investigates the performance characteristics of state-of-the-art GPU-based communication schemes on both native and container-based environments, which show a significant demand to design high-performance container-aware communication schemes in GPU-enabled runtimes to deliver near-native performance for end applications on clouds. Next, we propose the C-GDR approach to design high-performance Container-aware GPUDirect communication schemes on RDMA networks. C-GDR allows communication runtimes to successfully detect process locality, GPU residency, NUMA, architecture information, and communication pattern to enable intelligent and dynamic selection of the best communication and data movement schemes on GPU-enabled clouds. We have integrated C-GDR with the MVAPICH2 library. Our evaluations show that MVAPICH2 with C-GDR has clear performance benefits on container-based cloud environments, compared to default MVAPICH2-GDR and Open MPI. For instance, our proposed C- GDR can outperform default MVAPICH2-GDR schemes by up to 66% on micro-benchmarks and up to 26% on HPC applications over a container-based environment.

Xiaoyi will serve as a General Co-Chair for Bench 2019. Please submit your papers to Bench 2019. Call for Papers

Xiaoyi will serve as a TPC Co-Chair for The 5th IEEE International Workshop on High-Performance Big Data and Cloud Computing (HPBDC 2019). Please submit your papers to HPBDC 2019. Call for Papers

Xiaoyi will serve as TPCs for the following conferences in 2019!

• SC 2019
• HiPC 2019
• ICCCN 2019
• Others

A collaborative grant for research on large-scale hybrid memory systems is funded by NSF. I'm the PI from the OSU side.

Congratulations to our great team!

Thanks a lot for NSF's support!!!

Grant Information: SPX: Collaborative Research: Memory Fabric: Data Management for Large-scale Hybrid Memory Systems

I start my new faculty position from this month! I'm looking forward to working with self-motivated students who are interested in doing system research.

More importantly, we are happy to announce that PADSYS Lab Starts the great journey from The Ohio State University (OSU)!

(Courtesy: Celebration by Nick Youngson CC BY-SA 3.0 Alpha Stock Images)