ArmFinAI

Execution Flow
Autonomous Investment Intelligence
Market View and Optimisation Analytics

Inspiration

AI is becoming increasingly important for financial document analysis, but making an AI system intelligent is only half the problem. The other half is making it fast, memory-efficient, cost-efficient, and reliable on real infrastructure.

That led us to a simple question:

Can we take a real AI workload, profile where the computation is going, optimize it for Arm64, and prove the improvement with actual benchmarks?

We chose financial document intelligence as the workload because it combines several expensive AI operations—document extraction, embeddings, retrieval, and inference—into one realistic pipeline.

This became ArmFinAI, built around the principle:

Measure → Profile → Optimize → Benchmark → Verify

The goal was not to build another generic financial chatbot, but to make AI optimization itself a measurable part of the product.

What it does

ArmFinAI is an Arm64-optimized agentic financial intelligence platform with an integrated AI Optimization Laboratory.

Users can upload a financial document and ask questions such as:

What was the company's revenue? What was the year-over-year growth? What was the EBITDA margin? What are the major financial risks? Are there inconsistencies in the reported numbers? What evidence supports the analysis?

The document is processed through:

PDF → Text Extraction → Chunking → Embeddings → Retrieval → Financial AI Agent → Answer + Evidence

The same AI workload can then be executed in two modes:

Baseline

Standard inference Standard model configuration Standard batching

Arm Optimized

Quantization where supported Arm-optimized inference paths Dynamic batching Asynchronous processing Model routing Caching Efficient memory handling

The benchmark engine compares the two configurations using metrics including:

Model size Memory usage Load time Average latency P50/P95 latency Throughput Time to first token Output quality

ArmFinAI also includes an Optimization Autopilot that analyzes profiling data, identifies potential bottlenecks, and recommends optimization actions.

Most importantly, performance numbers are generated from actual benchmark runs rather than being hardcoded.

How we built it

We built ArmFinAI as a modular full-stack system.

The backend uses Python, FastAPI, Pydantic, and asyncio. The AI layer uses lightweight CPU-friendly models with PyTorch, Transformers, embeddings, and optimized inference runtimes where appropriate.

For document intelligence, we implemented a RAG pipeline consisting of:

Document ingestion → extraction → chunking → embeddings → vector retrieval → AI analysis

The optimization layer provides separate baseline and optimized inference paths so that both can execute the same workload under controlled conditions.

We built a custom benchmark engine using Python performance measurements and system-level monitoring to capture latency, throughput, memory, and other performance metrics.

The frontend uses React, Vite, Tailwind CSS, and Recharts, providing dedicated views for:

Financial analysis Documents Benchmark comparison Optimization Lab System performance Optimization Autopilot

The entire application is containerized with Docker and designed for reproducible ARM64 cloud deployment.

A key design decision was keeping optimization modular. Each optimization can be independently enabled, benchmarked, and compared rather than combining multiple changes and losing track of what actually improved performance.

Challenges we ran into Choosing the right model

One of our biggest challenges was resisting the temptation to use a large language model simply because it looked more impressive.

For this project, the model had to be practical for CPU-based Arm64 inference, quantizable, reproducible, and capable of demonstrating meaningful optimization opportunities.

We learned that a smaller model that can actually be measured and optimized is more valuable than a massive model that cannot run reliably in the target environment.

Making performance claims trustworthy

Another major challenge was benchmarking.

It is easy to say:

"Our optimized model is faster."

It is much harder to prove it fairly.

We therefore designed the benchmark around the same workload, same input data, same evaluation questions, warm-up runs, multiple iterations, and consistent measurement methodology.

This prevents us from accidentally comparing two different workloads.

Balancing speed and quality

Financial intelligence cannot be optimized for speed alone.

If latency improves but numerical accuracy or retrieval quality deteriorates, the optimization isn't necessarily useful.

We therefore included quality evaluation alongside performance metrics so that optimization decisions consider both efficiency and answer quality.

Arm-specific optimization

We also had to distinguish between generic CPU optimization and genuine Arm optimization.

We chose not to label every performance improvement as "Arm optimized." Arm-specific technologies and optimized runtimes are only considered when they actually participate in the workload and can be reproduced in the target environment.

Accomplishments that we're proud of

The biggest accomplishment is that ArmFinAI makes AI optimization observable instead of theoretical.

We built a system where developers can follow:

Workload → Baseline → Profile → Optimize → Benchmark → Compare → Verify

We are particularly proud of:

Building a complete financial-document RAG workload around a real optimization problem. Creating separate baseline and optimized inference paths. Building a reproducible benchmark engine instead of manually entering performance numbers. Tracking latency, throughput, memory, model size, TTFT, and quality together. Designing the system for ARM64 deployment. Creating the Optimization Autopilot concept to turn profiling data into actionable recommendations. Keeping the architecture modular so individual optimizations can be tested independently. Prioritizing reproducibility and technical honesty over unsupported performance claims.

The most important accomplishment is the philosophy behind the system:

Every optimization should produce evidence.

What we learned

We learned that AI optimization is a multi-dimensional engineering problem.

Reducing latency alone does not necessarily mean the system is better. A meaningful optimization needs to consider:

Latency + Throughput + Memory + Model Size + Quality + Hardware

We also learned how important controlled benchmarking is. Without a consistent workload and methodology, performance comparisons can easily become misleading.

Another important lesson was that hardware-aware AI optimization starts with understanding the workload. Profiling tells us where the bottleneck actually exists; only then should we decide whether quantization, batching, caching, model routing, or a different runtime is appropriate.

Finally, we learned that a transparent optimization that shows no improvement is more valuable than a fabricated improvement. ArmFinAI is designed so that the benchmark decides whether an optimization worked.

What's next for ArmFinAI

The next step is to expand ArmFinAI from a hackathon prototype into a broader AI performance engineering platform.

We want to:

Benchmark additional AI workloads on Arm64. Expand support for Arm-optimized inference technologies. Improve profiling and bottleneck detection. Make the Optimization Autopilot more intelligent and data-driven. Explore automated optimization selection. Add larger and more diverse financial-document evaluation datasets. Compare additional model architectures and quantization strategies. Improve reproducibility across different Arm64 cloud environments. Build historical benchmark tracking to understand performance regressions over time.

Our long-term vision is to make ArmFinAI a tool where developers don't simply ask:

"Does my AI application work?"

but also ask:

"Where is it spending compute, what can I optimize, and can I prove that the optimization actually worked?"

ArmFinAI — Measure. Profile. Optimize. Verify.