DPU-KV: On the Benefits of DPU Offloading for In-Memory Key-Value Stores at the Edge

Proceedings of the 34th International Symposium on High-Performance Parallel and Distributed Computing, 2025

Arjun Kashyap, Yuke Li, Xiaoyi Lu

Abstract

In-memory key-value stores (KVS) are widely used for edge data storage, where low latency and high throughput are essential. Data Processing Units (DPUs), with their low power use and offloading capabilities, suit resource-constrained edge computing. While DPUs offer a new design point for KVS, their offloading in edge environments remains underexplored and challenging. In this paper, we unveil the potential of offloading in-memory CPU-based KVS to SoC-based DPUs, specifically NVIDIA's BlueField-2 (BF-2) and BlueField-3 (BF-3), with the aim of enhancing KVS performance. We propose a principled exploration methodology of dividing a KVS (i.e., MICA) into its logical components and identifying the CPU-intensive KVS component (i.e., communication engine). Next, we perform fine-grained offloading analysis and explorations on DPUs. To maximize benefits in terms of latency and throughput from fine-grained KVS offloading on DPUs, we propose a series of significant performance optimizations, including a key-value-based queue-pair model, overlapped KV request/response processing, reduced DMA operations per KV batch, dual-communication engine, and a sharding-based design. Our key finding is that our proposed fine-grained KVS offloading designs on modern DPU architectures (i.e., BF-2 and BF-3) can provide much lower latency (up to 68%) and higher throughput (up to 36%) than MICA (CPU-only) and coarse-grained DPU offloading schemes at the edge. To our knowledge, this paper is the first to explore the performance benefits of fine-grained KVS offloading to DPUs at the edge.

Full text links

External link

Conference Proceedings

Isbn
9798400718694
Publisher
Association for Computing Machinery
Address
New York, NY, USA
Doi
10.1145/3731545.3731571
Booktitle
Proceedings of the 34th International Symposium on High-Performance Parallel and Distributed Computing
Articleno
11
Numpages
14
Location
University of Notre Dame Conference Facilities, Notre Dame, IN, USA
Series
HPDC '25

Cite

Plain text

BibTeX