INEC: Fast and Coherent in-Network Erasure Coding

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2020

Haiyang Shi, Xiaoyi Lu

Abstract

Erasure coding (EC) is a promising fault tolerance scheme that has been applied to many well-known distributed storage systems. The capability of Coherent EC Calculation and Networking on modern SmartNICs has demonstrated that EC will be an essential feature of in-network computing. In this paper, we propose a set of coherent in-network EC primitives, named INEC. Our analyses based on the proposed α-β performance model demonstrate that INEC primitives can enable different kinds of EC schemes to fully leverage the EC offload capability on modern SmartNICs. We implement INEC on commodity RDMA NICs and integrate it into five state-of-the-art EC schemes. Our experiments show that INEC primitives significantly reduce 50th, 95th, and 99th percentile latencies, and accelerate the end-to-end throughput, write, and degraded read performance of the key-value store co-designed with INEC by up to 99.57

Conference Proceedings

Isbn
9781728199986
Publisher
IEEE Press
Booktitle
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
Articleno
66
Numpages
17
Location
Atlanta, Georgia
Series
SC '20

Cite

Plain text

BibTeX