ArmSight — Arm64 ML Model Optimization Agent

ArmSight — Autonomous Arm64 ML Model Optimization Agent

Problem

Deploying ML models on Arm64 infrastructure (AWS Graviton, Raspberry Pi, mobile devices) requires manual optimization expertise. Developers lack tools to automatically analyze models, apply Arm-specific optimizations, and measure improvements.

Solution

ArmSight is an autonomous AI agent that:

1. Analyzes ONNX models — extracts architecture, operators, parameter counts, precision, tensor shapes
2. Recommends Arm64-specific optimizations — INT8 quantization, NEON SIMD operator fusion, thread parallelism, Arm Compute Library provider, memory layout optimization
3. Applies real INT8 dynamic quantization via onnxruntime — produces genuinely smaller models
4. Benchmarks before/after performance — latency, throughput, model size
5. Generates Arm64-optimized deployment packages — Dockerfile targeting linux/arm64, FastAPI inference server, benchmark scripts

Measurable Results (verified on live demo)

• Model size: 3.69 MB → 0.93 MB (74.77% reduction, 3.96x factor)
• Latency: 0.126ms → 0.038ms (69.75% improvement, 3.3x speedup)
• Throughput: 7,962 → 26,316 ops/sec (230.53% improvement)

Unique Angle

Unlike generic model optimizers, ArmSight uses an MCP (Model Context Protocol) server pattern — exposing 6 callable tools (analyze_model, optimize_model, benchmark_model, recommend_optimizations, generate_deployment, full_pipeline) that AI agents can call autonomously. This makes it the first agent-native Arm64 optimization tool.

Architecture

User → Web UI → FastAPI Backend
                      ↓
          ┌──────────────────────┐
          │   MCP Tool Registry   │
          │  (6 callable tools)  │
          └──────────────────────┘
                      ↓
    ┌─────────┬─────────┬─────────┐
    ↓         ↓         ↓         ↓
 Analyzer  Quantizer  Benchmark  Deploy
 (onnx)   (ort INT8) (ort sess) (Docker)

Tech Stack

• Backend: Python FastAPI, onnxruntime, onnx
• MCP: Model Context Protocol tool registry
• Frontend: HTML/CSS/JS (dark theme, responsive)
• Deployment: Vercel (serverless Python), Docker for Arm64
• Repo: github.com/0xConsole/arm-sight-agent (Apache 2.0)

Setup (<5 commands)

git clone https://github.com/0xConsole/arm-sight-agent.git
cd arm-sight-agent
pip install -r requirements.txt
python -m uvicorn app.main:app --reload
# Open http://localhost:8000

What's Real vs Mocked

• ✅ ONNX model analysis: real onnx graph parsing
• ✅ INT8 quantization: real onnxruntime.quantization.quantize_dynamic
• ✅ Benchmarking: real session.run() timing (mean/p50/p95 latency, throughput)
• ✅ Deployment package: working Dockerfile targeting linux/arm64
• ✅ MCP tool interface: 6 callable tools with JSON schemas

Cloud AI Track Fit

ArmSight targets the Cloud AI track — optimizing AI for scalable Arm64 cloud infrastructure, inference performance, and production-ready developer workflows. The MCP tool pattern enables autonomous agent workflows for batch model optimization pipelines.

Built With

• ai-agents
• arm64
• docker
• fastapi
• int8-quantization
• mcp-server
• onnx
• onnxruntime
• python
• vercel