Home
Performance
AI Codes
AI Chat
AI Models
icon

Inspiration

My mother, my family, and I started using chatbots for daily questions, from what to cook to things that are private. I did not feel at peace sending those conversations to a company's server. Offline model apps existed, but few of them used the Neural chip in a phone, and fewer of them worked on the older, cheaper phones my grandparents in India actually have.

What it does

Chakuli downloads a model once and runs it on the phone. It can run GGUF models, which is the format most people share, and LiteRT models built for Google's on-device runtime. Some LiteRT models use the Neural chip when the phone has one. The two engines let a person pick a current model instead of being stuck with one format. There is no account. Chats stay on the device.

The same work is aimed at low-end phones: ship a build that matches the processor, and refuse a model only when the phone cannot hold it.

How we built it

I started from the LiteRT docs, so the first version was Google's runtime only. I later added GGUF. I also wanted it to feel like a consumer app, so I designed the welcome, the privacy page, and the first chat screen for someone who is not an engineer. The tip jar uses RevenueCat. It is there for people who want to support the work. Paying does not unlock anything.

Challenges we ran into

A multi-gigabyte model will crash an ordinary phone if you just load it. Chakuli ships several ARM processor builds and checks the phone's memory before a load. Gemma 4 E2B is one model that already runs on an 8 GB phone. Getting a useful speed out of a phone that is several years old took the longest.

Accomplishments that we're proud of

Chakuli is a small assistant that runs on the phones already in a house. The name is from Marathi. Chakuli means a little girl, and here she is a little assistant for the questions you have.

What we learned

I learned how much of on-device AI is processor support, memory, and not assuming a new phone. The lesson I want to share is that this kind of app does not only come out of a large company. RevenueCat made the tip jar as simple as adding the Android SDK, and I now use RevenueCat in my other apps too.

What's next for Chakuli - Local LLMs

I am training my own assistant model aimed at low-end hardware. The app will keep adding public models, and I want it to grow the tools a local model needs, including skills and a way to call them.