Achieved geometric-mean latency improvement of 13.90x over the naive eager baseline, across the 13 dense-baseline shapes of the canonical matrix, at 38.3% mean MFU (baseline: 10.1%).
Inspiration Explore how far agent-guided, hardware-aware optimization can push transformer inference on an accessible RTX 5060 Ti 16 GB.
What it does Runs a correctness-preserving transformer forward pass using CUDA graphs, Triton kernels, fused operations, and capacity-aware streaming to reduce latency and improve MFU.
How we built it We paired PyTorch and Triton with a recursive profile-hypothesize-measure-verify workflow, tuning each accepted optimization against eager correctness and compiled-baseline speed.
Challenges we ran into Limited VRAM, only 36 SMs, Windows tooling constraints, numerical precision, noisy GPU timing, and a sequence-length-100,000 case that cannot use dense attention.
Accomplishments that we're proud of All 14 canonical cases are executable, including the streamed extreme case, while preserving strict accuracy checks and artifact-backed, reproducible measurements.
Log in or sign up for Devpost to join the conversation.