Hullspeed

Inspiration

Wanting to run larger LLM's on my OrangePi 6.

What it does

Any developer running AI workloads on Arm silicon might not be able to fully unlock all capabilities of the very heterogenous platform of Arm. Instead of assembling a multitude of (dev)tools Hullspeed aspires to give quick insights on how to unlock current and future silicon based possibilities. Hullspeed should tell them in seconds whether they're compute-bound or memory-bound, what their hardware can actually do, and the single next change worth making."

How we built it

Iteratively using multiple frontier models and old fashion software engineering, and 64-bit Arm-assembly books. Test machines: OrangePi 6 32GB RAM, 2TB storage. And Graviton4-based AWS EC2 instance.

Challenges we ran into

The Graviton4-based AWS EC2 instance route lead to nothing. The instance doesn't expose enough hardware info.

Accomplishments that we're proud of

We built a beta =/

What we learned

Developing for production is WAY harder than working on something that will only run on a homelab.

What's next for Hullspeed

Sprints to v1. A beautiful and working TUI in which you can test any combination of Arm ISA and any model (to reasonable extent)

Built With

• bash
• rust