MemoRoo – Your On-Device AI Memory Layer

Inspiration

I’ve always wanted a personal AI that feels more like a companion than a tool. Something that can remember my ideas, understand my life, and help me think better without sending private data to the cloud. I wanted a futuristic assistant that feels like a tiny Jarvis but runs locally on mobile hardware, so I built MemoRoo.

What it does

MemoRoo turns messy, unstructured inputs like text, images, voice notes, and PDFs into a structured, living memory graph.

It lets you: • Spread ideas across an infinite spatial canvas • Explore your memories as glowing planets in a 3D stellar map • Chat with an on-device LLM that retrieves your personal context • Track moods, events, habits, and life moments in a Life OS timeline

All AI processing happens fully on-device for speed, privacy, and offline use.

How we built it

The frontend is built in React and TypeScript with a neon cyber-glass aesthetic powered by Tailwind. The backend uses FastAPI, PostgreSQL, and a clean architecture structure that keeps AI components modular and maintainable.

I integrated several on-device AI pipelines: • Quantized LLM via ExecuTorch • TFLite int8 embedding models • OCR and Whisper-tiny transcription models • Faiss or Annoy vector search for the RAG pipeline • Custom layout and inference logic for the 3D memory graph

Everything is tuned for Arm CPUs and Mali GPUs to keep it fast and lightweight.

Challenges we ran into

• Running an LLM smoothly on-device with low memory • Keeping vector search fast while storing hundreds of embeddings • Making the 3D memory engine performant without relying on heavy libraries • Designing storage models that support messy real-world data • Getting spatial canvas interactions to feel natural and fluid • Balancing aesthetics with device performance limits

Accomplishments that we're proud of

• A full on-device RAG pipeline running privately • A custom 3D graph engine that feels smooth and cinematic • A Life OS that auto-links memories, moods, and events • A clean full-stack architecture that can scale • UI animations that feel futuristic without sacrificing performance • MemoRoo genuinely feels like a small personal operating system

What we learned

I learned a lot about on-device AI optimization, model quantization, Arm hardware acceleration, and structuring large AI features inside a clean backend architecture. I also improved my understanding of building smooth 3D visualizations and handling multi-modal data pipelines.

What's next for MemoRoo – Your On-Device AI Memory Layer

• Add local-first syncing across devices • Support more AI models with dynamic model switching • Expand the 3D universe with richer physics and layouts • Build downloadable iOS and Android versions • Add deeper personal analytics and long-term memory insights • Explore a plugin system for custom AI workflows

Built With

• api
• fast
• postgresql
• react
• typescript
• vite