RL Learning Track LeetGPU

Inspiration

In a recent interview, Vlad Fienberg noted that top-tier frontier research engineers must deeply read and understand the relevant literature on Reinforcement Learning. I took that advice to heart and began reading the foundational papers behind modern LLM alignment (PPO, DPO, etc.).

However, I quickly realized a massive gap in the industry: the math in the papers looks clean, but implementing it in production is a nightmare of systems engineering. Standard PyTorch math causes massive memory spikes that crash GPUs. I realized that the best way for developers to truly understand frontier RL isn't just to read the equations—it's to implement them under real hardware constraints. I built this track because I wanted a hands-on way for the community to learn these concepts by doing.

What it does

RL Learning Track LeetGPU is "LeetCode for AI Systems Engineers." It takes the core algorithms that power modern frontier models and turns them into bite-sized, interactive coding challenges.

• Currently featuring a curriculum of PPO, DPO, and GRPO (the algorithm behind DeepSeek-R1), the platform allows developers to:

• Learn the distinct differences between token-level and sequence-level alignment.

• Write custom, memory-optimized GPU kernels.

• Use a web UI to submit their code and have it instantly evaluated against rigorous test suites with free, remote access to GPUs for performance benchmarking.

How we built it

We integrated this curriculum directly into the open-source leetgpu-challenges testing framework. For each algorithm, we engineered a complete sandbox:

• Reference Implementations: We built mathematically pure baselines in both PyTorch and JAX to serve as the ground truth.

• Robust Test Harnesses: We generated edge-case functional tests (like checking exact PPO clipping boundaries or negative advantages).

• Hardware-Native Starters: We provided the scaffolding for users to write low-level hardware kernels, abstracting away the boilerplate so they can focus on the memory constraints and the math.

Challenges we ran into

Building a universal test harness that seamlessly handles PyTorch's in-place memory mutation (output[0] = ...) alongside JAX's purely functional execution model required careful state management during validation.

Accomplishments that we're proud of

Being incredibly up-to-date with frontier research by including GRPO (Group Relative Policy Optimization) and giving developers hands-on experience with the exact algorithm powering today's most efficient reasoning models.

What we learned

We gained a deep understanding of how token-level algorithms (like PPO) surgically penalize step-by-step decisions, while sequence-level algorithms (like DPO) simplify the math by judging the response as a single cohesive unit.

What's next for RL Learning Track LeetGPU

Expanding the Sequence Family: Adding challenges for KTO (Kahneman-Tversky Optimization) and IPO to show how tweaking a single mathematical penalty changes the entire behavior of the policy.

Built With

• deep-learning
• dpo
• grpo
• jax
• modal
• ppo
• pytorch
• rl