Inspiration
We were inspired by the performance of modern GPUs. They are massively parallel, possess immense memory bandwidth, and excel at processing large blocks of structured data simultaneously. We asked ourselves a simple engineering question: What if the GPU, not the CPU, handled the critical path of network packet processing?
What it does
Modern data centers are overwhelmed by an engineering paradox: while network interface cards (NICs) have reached scorching speeds of 100, 200, and 400 Gbps, the CPU-centric architecture designed to process that data is struggling to keep up.
In a traditional router, every single packet arriving at the network interface must trigger an interrupt on the CPU. The CPU must then pause its application work, context-switch to the networking stack, and often copy the packet data multiple times through host memory. At high loads (e.g., 10 Million packets per second), this creates an inescapable "interrupt storm" and massive CPU bottleneck, particularly in environments running distributed AI training.
How we built it
This project is a detailed Technical Proposal and Performance Model Evaluation, built upon three pillars: System Architecture Design: We designed a robust, two-plane system that separates the Data Plane from the Control Plane. Data Plane: Network Interface DPU . GPU Memory GPU Packet Kernels Transmission. Control Plane: Supervisory CPU managing routing tables, security policies, and telemetry. Performance Model Simulation: We developed a mathematical model to estimate the latency and CPU load reduction of the proposed path compared to a traditional CPU-centric router, based on assumed costs for interrupts, context switching, memory transfers, and GPU kernel launch overhead .Validation Roadmap: We created a rigorous, 25-point engineering structure and a comprehensive testbed validation plan. This plan defines the exact methodology needed to measure Median, p95, and p99 latency distributions, throughput (Mpps), packet loss, and power/thermal costs on physical hardware.## Challenges we ran into
Challenges we ran into
The most difficult aspect of this engineering challenge was designing the optimal path for GPU Software and Kernel Design. While GPUs are massively parallel and thrive on large batches of data ($>1,000$ packets), networking is sensitive to tail latency. Waiting to collect enough packets to saturate the GPU maximizes throughput but can destroy the application's response time (latency). Balancing this trade-off—finding the precise batch size and launch interval that satisfies strict latency constraints without starving the GPU—was our primary technical hurdle. We had to carefully model the overhead associated with GPU kernel launches, batching delays, and memory registration contention to ensure our proposed solution remained viable under various packet workload scenarios.
Accomplishments that we're proud of
We are incredibly proud that our mathematical model provides strong evidence motivating the shift toward GPU-centric networking.
Our initial simulation results, based on the assumed parameters in our project document, are compelling:
Modeled Latency Reduction: The simulation predicted a decrease from a 37 µs baseline in the traditional path to just 8.5 µs in the GPU-centric path—a 77.0% reduction in simulated latency.
Modeled CPU Offload: At a workload of 10 Million packets per second (Mpps), the modeled CPU demand dropped from 4.0 busy cores in the conventional approach to an estimated 0.15 cores—a significant offload of system resources for actual application work.
What we learned
The development of this project was a masterclass in advanced heterogeneous data center design. We learned three vital principles: The OS Kernel is the Bottleneck: For the future of extreme networking (e.g., distributed AI training collectives like NCCL), kernel processing is no longer scalable. "Zero-copy" paths that bypass host memory and the operating system are mandatory .GPU-Centric Automatically Superior: GPU-accessible memory does not inherently guarantee zero-copy efficiency, low latency, or low total system power. Correctness, packet ordering, and congestion handling must all be validated on the physical hardware/software combination. The "Three-Plane" Architecture: A robust data center architecture must manage not just the Data Plane and the Control Plane, but also a robust Telemetry Plane to monitor latency, power budgets, and thermal feedback to prevent failure.
What's next for GPU-Centric Router and Data Center Architecture
The development of this project was a masterclass in advanced heterogeneous data center design. We learned three vital principles: The OS Kernel is the Bottleneck: For the future of extreme networking (e.g., distributed AI training collectives like NCCL), kernel processing is no longer scalable. "Zero-copy" paths that bypass host memory and the operating system are mandatory. GPU-Centric Automatically Superior: GPU-accessible memory does not inherently guarantee zero-copy efficiency, low latency, or low total system power. Correctness, packet ordering, and congestion handling must all be validated on the physical hardware/software combination .The "Three-Plane" Architecture: A robust data center architecture must manage not just the Data Plane and the Control Plane, but also a robust Telemetry Plane to monitor latency, power budgets, and thermal feedback to prevent failure.
Log in or sign up for Devpost to join the conversation.