Rasta-AI

Inspiration: In emerging economies, over 70% of elderly and rural citizens suffer from severe digital banking anxiety, frequently handing their unlocked smartphones and PINs to strangers or kiosk operators just to transfer money or pay bills. We built Rasta AI to restore their financial dignity, independence, and safety through an empathetic voice assistant that speaks their native language and guides them directly through the app.

What it does: Rasta AI transforms mobile banking into a bilingual voice conversation in colloquial Urdu and English. It features a Guidance Mode that highlights on-screen buttons step-by-step so hesitant users keep full physical control, a Smart Fill Mode that automatically pre-fills transfer and utility bill forms for rapid review, an autonomous Scam Shield that flags unknown payees and high-value transfer anomalies over 25,000 PKR, and a voice screen reader that speaks balances and transaction receipts aloud.

How we built it: We built Rasta AI using an NVIDIA Nemotron-3 Super 120B model deployed on Nebius Token Factory via OpenAI-compatible endpoints to handle complex multilingual intent classification and slot extraction. The frontend is developed in Next.js 15 and Tailwind CSS with a live AI Brain Inspector telemetry drawer, connected over full-duplex WebSockets to an asynchronous Python agent orchestrator, Whisper speech-to-text, and neural text-to-speech engines.

Challenges we ran into: Our biggest challenge was overcoming the agency paradox: early tests showed elderly users panicked when an AI autonomously submitted transactions on their behalf, so we invented Guidance Mode to act as a visual mentor rather than an automated pilot. We also had to solve code-switching challenges with Roman Urdu and slang, and reduce round-trip audio and inference latency below 350 milliseconds to prevent awkward conversational interruptions.

What we learned: We learned that true financial accessibility is not about simplifying visual text, but about multimodal redundancy that combines voice, dynamic screen highlighting, and tactile confirmation. We also discovered that open-weight architectures like NVIDIA Nemotron on Nebius Token Factory provide the reasoning capacity and data sovereignty required for production-grade, safety-critical financial workflows.

What's next: Next, we plan to fine-tune regional provincial language models for Punjabi, Pashto, and Sindhi on Nebius AI Cloud compute using NVIDIA NeMo, integrate on-device biometric voiceprint authentication for hands-free transaction signing, and package Rasta AI as an embeddable SDK for microfinance banks and digital wallets across the Global South.

Built With

• accessibility
• ai-agents
• fintech
• llm
• multimodalpython
• nebius
• nebius-token-factory
• nemotron
• nvidia
• nvidia-nemotron
• speech-to-text
• text-to-speech
• websockets
• whisper