Inspiration
Inference engineering is one of the parts of AI systems that excites me most, but most learning resources are papers, diagrams, and benchmark charts. I wanted a place where I could change one variable and inspect what actually happens.
What P99 does
P99 is a hands-on playground for students and engineers learning how LLM serving systems behave. Six foundation labs introduce tail latency, continuous batching, KV cache, quantization, concurrency, and speculative decoding. The experiment builder turns serving choices into a reproducible JSON spec. With the optional runner connected, the incident lab executes an allow-listed llama.cpp workload on a GPU and returns a measured trace. Without a runtime, P99 clearly shows that there is no measured result instead of inventing a benchmark.
How I built it
The product uses Next.js, React, and TypeScript. The optional Modal runner can provision a T4, L4, or A10G, run a Qwen2.5 7B GGUF workload, and return llama.cpp plus nvidia-smi telemetry. Codex was my main engineering collaborator throughout the build, including the product pivot, implementation, tests, documentation, deployment, and demo. I used GPT-5.6 through Codex for core implementation and review decisions.
Challenges and decisions
P99 began as a broad interactive education platform. The hard part was narrowing it to one problem I genuinely care about and keeping every benchmark claim honest. I removed the earlier neural predictor and hand-built simulator because the available traces were not enough to support a credible world model. Measured results and future work are now clearly separated.
What's next
The next step is bring your own environment: connect local llama.cpp, a workstation GPU, a lab cluster, or cloud compute through an authenticated trace adapter. A broad corpus of real transitions could eventually support a learned next-state model, but that is future work.
Live demo: https://learnscape-education.syedmujahedalih.chatgpt.site
Built With
- cloudflare-workers
- codex
- gpt-5.6
- llama.cpp
- modal
- next.js
- qwen2.5
- react
- typescript
Log in or sign up for Devpost to join the conversation.