AI Model Analysis Dashboard

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for AI Model Analysis Dashboard

Inspiration

Most model leaderboards give you somebody else's definition of “best.” That breaks down quickly. A coding agent, a cheap production model, and a fast interactive assistant need very different tradeoffs, but they are usually compressed into one opaque score.

I built this because I wanted the underlying evidence without being locked into someone else's weighting.

What it does

The AI Model Analysis Dashboard combines model, provider, benchmark, pricing, throughput, and release data in one workspace. You can inspect individual models, compare their shared benchmarks on normalized radar charts, and explore cost-versus-performance frontiers.

The core feature is Personal Index. You choose the benchmarks that matter, give each one a weight, optionally require specific benchmark coverage, and immediately get your own model ranking. The index saves automatically, and every result remains traceable to the component scores instead of becoming a black box.

How I built it

The application uses a Flask backend and a lightweight browser UI. Data collectors pull public artifacts and rendered benchmark data, preserve raw source material, normalize model identities and score scales, and build query-ready payloads for the dashboard.

Codex was the primary development environment. I used it for repository analysis, implementation, browser QA, scraper repair, normalization work, test creation, and deployment verification. GPT-5.6 handled the hardest parts through Codex: reconciling inconsistent model identities, designing Personal Index scoring and coverage behavior, normalizing comparison charts without making them misleading, debugging stale live-data paths, and repeatedly exercising the deployed user flows.

Challenges

The hardest problem was not drawing charts. It was making data from unrelated benchmark sources comparable without pretending the sources were cleaner than they are. Model names, effort levels, providers, score directions, cost definitions, and versions all vary. The warehouse keeps source detail and variant metadata so the UI can group obvious aliases without erasing meaningful differences.

Another challenge was visual normalization. A fixed zero-to-one-hundred radar makes close frontier models look identical, while naïve min-max scaling makes every difference look extreme. The comparison view uses bounded, per-axis expansion with the exact raw scores directly below it.

Accomplishments

• A Personal Index builder with weighted benchmarks, required-coverage gates, autosave, and shareable configuration.
• A live model explorer covering intelligence, coding, pricing, latency, throughput, and provider data.
• Normalized multi-model radar comparisons with exact score tables.
• Horizontal leaderboards and labeled Pareto frontiers for performance, cost, and speed.
• Automated backend, browser, responsive, dark-mode, chart-semantics, and persistence checks.

What I learned

The useful product is not another universal leaderboard. It is a way to interrogate the assumptions behind a leaderboard and replace them with the assumptions that match the actual job.

What's next

The next step is making the entire application agent-native. Every user action—creating an index, changing a weight, pinning a model, exporting a comparison—should be exposed as a reusable action for the built-in agent, a CLI, and MCP clients. I also want to continue expanding scraper-first benchmark coverage while showing provenance and freshness directly in the interface.

Built With

• chartjs
• flask
• javascript
• openai
• openrouter
• python