Technical Program
13:30-13:35 Opening Remarks
13:35-14:25 Keynote Speech
Rethinking the Data Path: Distributed In-Memory Key-Value Stores with DPUs
Speaker: André Brinkmann (Saarland University)
Abstract: Remote in-memory key-value stores are a cornerstone of modern distributed systems, powering caching layers, real-time analytics, and coordination services at scale. As application demands grow, so do the requirements on these systems, spanning throughput, range query support, fault tolerance, and operational simplicity. Achieving all of these simultaneously remains an open challenge, and the diversity of existing approaches reflects how differently these tradeoffs can be resolved. This talk surveys the landscape of distributed, high-performance KV store designs, from classical host-based architectures limited by the kernel network stack, to kernel-bypass techniques and RDMA-based distributed designs. We then focus on Data Processing Units (DPUs) and SmartNICs, hardware that sits directly on the network data path and processes requests without involving the host CPU or OS. We base this discussion on DPA-Store, our DPU-based KV store, illustrating how on-path processing enables low-latency request handling, stateless clients, and range query support, while avoiding the complications of RDMA-based approaches. We close with open challenges, including consistency under failure, storage hierarchy integration, and learned data structures in network-bound environments.
Biography: Prof. Dr.-Ing. André Brinkmann has been a Full Professor in the Department of Computer Science at Saarland University in Saarbrücken since 2026. Prior to this, he held the position of Full Professor of Computer Science at Johannes Gutenberg University (JGU) Mainz and served as director of the JGU Data Center. He earned his Ph.D. in Electrical Engineering from Paderborn University in 2004 and subsequently served as Assistant Professor in the Department of Computer Science at Paderborn University from 2008 to 2011. His research focuses on applying algorithm engineering to storage systems, data center management, and high-performance computing. He has authored over 150 publications in leading conferences and journals and serves as Senior Associate Editor of ACM Transactions on Storage.
14:25-15:00 Invited Talk
LLM Inference at Wafer Scale: When the Model Lives in On-Chip Memory
Speaker: Luo Mai (University of Edinburgh)
Abstract: LLM inference systems are built on GPU-centric assumptions: large off-chip memory, coarse-grained kernels, and communication among a few devices. Wafer-scale accelerators break all of them, offering hundreds of thousands of cores and tens of gigabytes of distributed on-chip memory with far more bandwidth than HBM. This talk presents WaferLLM, an LLM inference system that treats inference as a distributed on-chip memory management problem: it partitions weights and the KV-cache across the wafer, parallelizes prefill and decode at wafer scale, and introduces two near-memory kernels, MeshGEMM and MeshGEMV. On a Cerebras WSE-2, WaferLLM serves a single user at 2,700 tokens/s — sub-millisecond per-token latency. I will close with ongoing work spanning on-chip and external memory tiers for interactive reasoning and test-time compute.
Biography: PLuo Mai is an Associate Professor at the University of Edinburgh, where he leads the Large-Scale Machine Learning Systems Group. He builds systems for LLM training and inference, including WaferLLM (OSDI '25), ServerlessLLM (OSDI '24), and Tenplex (SOSP '24), with wafer-scale compilation and runtime work appearing at SOSP '26. He co-directs the EPSRC Centre for Doctoral Training in Machine Learning Systems and leads projects in the UK ARIA Scaling AI Compute programme. He received his PhD from Imperial College London and previously worked at Microsoft Research.
15:00-15:30 Coffee and Tea Break
15:30-16:20 Keynote Speech
Two Peas in a Pod: Storage and Memory Heterogeneity in the Age of AI
Speaker: Sudarsun Kannan (Rutgers University)
Abstract: Modern servers no longer have a simple split between memory and storage. They have a continuum of unequal tiers, from HBM and DRAM to CXL-attached memory and flash, differing 10 to 100 times in latency and bandwidth, with compute increasingly embedded network, memory, and storage devices. Software abstractions for memory and storage, however, have barely changed, and today's AI challenges echo classic ones: KV-cache eviction recalls buffer-cache management; model paging, virtual memory; checkpointing, logging; pipeline stalls, prefetching. The solutions transfer too, because the principles were never device specific. In this talk, using cross-layered design as a running example, I will explore what carries over from the past decade of storage and memory systems research and where it falls short, from datacenter and HPC AI to edge cyberinfrastructure for natural-hazard monitoring. I will conclude the talk with open challenges and ongoing work toward treating storage and memory heterogeneity as two peas in a pod.
Biography: Sudarsun Kannan is an Associate Professor in the Computer Science Department at Rutgers University. His research focuses on operating system design and its intersection with computer architecture, distributed systems, and high-performance computing. His group also builds edge cyberinfrastructure for hazard-monitoring systems, including wildfire detection and mitigation. He is a recipient of the NSF CAREER Award, the Google Research Scholar Award, and the 2025 Rutgers Board of Trustees Fellowship for outstanding early-career faculty, and his group's research has received best paper and distinguished paper awards at SOSP, ASPLOS, and SPAA. He co-chaired the HotStorage '22 workshop and serves as an Associate Editor for ACM Transactions on Storage. Before joining Rutgers, he was a postdoctoral research associate at the University of Wisconsin-Madison, and he received his M.S. and Ph.D. from Georgia Tech.
16:20-16:55 Invited Talk
TBD
16:55-17:30 BigMem Infrastructure
CXL-SDK: Building a Software Development Kit for CXL Shared Memory
Authors: Fangnuo Wu, Mingkai Dong, Haibo Chen (Shanghai Jiao Tong University)
Fresco: Enabling ACK-Time Searchable Durability for Fresh Vector Index Updates with Tiered Persistent Memory
Authors: Sen Jiang, Yinjin Fu (Sun Yat-sen University)