Taweeli

Command detection
Wake word detection

Inspiration

I personally faced a potential accident while traveling with my friend. He was driving and got distracted by his phone to check a message. At that moment, another vehicle approached from the opposite direction, and our vehicle veered slightly off track. I had to quickly pull the steering to the left to avoid a crash.

This near-miss experience inspired me to create Taweeli an AI assistant that helps drivers stay focused on the road by automating WhatsApp, email, and navigation tasks using voice commands, reducing the risk of distracted driving accidents.

What it does

Taweeli is an AI powered mobile app that automates messaging, email, and navigation through voice commands. It enables drivers to send and receive WhatsApp messages, read emails, search for locations, and control other apps all without touching their phone. The app runs 100% locally for privacy and speed, and it’s designed to keep users safe and productive while driving.

How we built it

We built Taweeli using a AI driven UI automation engine that processes voice commands and executes actions within mobile in real time.

The app is using on-device speech recognition, voice activity detection for power efficiency, and adaptive learning to personalize user experience.

Our current tech stack includes android built-in speech to text and text to speech module, Android Accessibility Services for UI automation, and local storage for privacy.

We will be including Google ML kit and small language model for personalized experience

Challenges we ran into

We are facing challenges with voice recognition accuracy in noisy car environments and ensuring seamless app integration across WhatsApp, Gmail, and Maps through UI automation which needs to be verified and approved by each app.

We are also balancing privacy and security, making sure all data stays on-device and never gets transmitted to the cloud. Testing in real driving conditions and optimizing for battery efficiency are ongoing hurdles.

Accomplishments that we're proud of

We have built MVP mobile app with UI automation and validated the problem with potential users from Linkedin and Reddit users by DMs and survey. The intent classification is completed, Our voice text to speech and speech text model is tested and it is working. The app completes the messaging task within 4 seconds

What we learned

We learned that real-world testing is crucial for refining voice assistants, especially in challenging environments like moving vehicles.

We also discovered the importance of user-centric design, privacy first approaches, and the power of adaptive AI in creating seamless experiences.

Building Taweeli taught us how to balance innovation with safety and accessibility.

What's next for Taweeli

We will be getting approval from WhatsApp and plan to expand Taweeli’s capabilities to include more apps (social media, CRM, workspace tools), add support for 10+ regional languages, and develop enterprise API integrations.

We aim to partner with automotive companies for in-vehicle integration and continue improving our AI engine for better accuracy and personalization.

Our ultimate goal is to make Taweeli the go-to hands-free assistant for drivers worldwide.