PerfPilot

• 

  The main entry point: repository analysis plus a detected Arm performance regression.

• 

  Measured runtime strategies ranked by latency, speedup, memory, and quality tradeoffs.

• 

  A PR-ready recommendation explaining the root cause, expected recovery, and files to change.

About the project

I built PerfPilot because performance problems in AI projects are easy to miss until they become expensive.

A team can have tests, linting, code review, and security checks in place, then still merge a change that makes inference slower, uses more memory, or runs poorly on Arm. By the time anyone notices, the service may already be costing more or missing latency targets.

PerfPilot is my attempt to make performance review feel like a normal part of shipping AI software. It runs benchmarks, compares a change against a baseline, points out what likely caused the regression, and suggests practical ways to improve it.

How I built it

I built the product with a Next.js and TypeScript frontend, plus a FastAPI service for running repeatable benchmarks.

The dashboard shows a full pull request workflow. You can see a regression report, inspect the likely bottleneck, compare optimization options, and read an AI-generated performance review with suggested files to change.

I also added a GitHub Actions workflow for Arm64 runners. It runs the benchmark and saves the raw result as an artifact, so the performance evidence can be checked later instead of disappearing into a dashboard.

One thing I cared about from the start was being honest about the data. If PerfPilot actually runs a benchmark, the result is labeled Measured. If it is a projected tradeoff, like an energy estimate, it is labeled Estimated.

What I learned

The biggest thing I learned is that a performance tool has to earn trust.

It is easy to show charts and big speedup claims. It is harder to show where a number came from, how it was measured, what machine it ran on, and whether another engineer can reproduce it. That is the direction I wanted PerfPilot to take.

I also learned that Arm performance is affected by many small decisions. Tokenizer behavior, quantization, runtime choice, thread counts, memory copies, and batch size can all have a real impact. A tool that keeps those decisions visible during code review can save a lot of debugging later.

Challenges

The hardest part was balancing a polished developer experience with accurate claims.

The dashboard uses a realistic example so people can understand the workflow, but I did not want to pretend that demo values were real measurements from a random repository. The benchmark API only calls a result measured when it actually produced that result.

Another challenge was moving beyond “latency got worse.” A useful tool should help answer what changed, why it matters, and what to try next. That is why PerfPilot includes root-cause explanations, optimization comparisons, confidence, tradeoffs, and a clear recommendation.

Why this matters

I think AI performance should be reviewed before production, not after an incident.

PerfPilot makes that workflow easier for teams building AI inference services on Arm. The goal is simple: help developers catch regressions early, make optimization decisions with evidence, and ship faster inference with more confidence.

Built With

• ai-inference
• arm64
• aws-graviton
• css
• fastapi
• github-actions
• llama.cpp
• next.js
• node.js
• onnx-runtime
• performance-testing
• python
• pytorch
• react
• sqlite
• tailwind
• typescript