Vani.ai

Transform any business into a 24/7 Multilingual AI Call Center in seconds.

Comment

Vani (वाणी)

Meaning: "Voice" or "Speech" in Sanskrit/Hindi. Chosen to signify that we are giving a powerful voice to small businesses, truly representing them.

❤️ The Inspiration (Our "Why")

Our mother runs Rainbow Driving School Kalaburagi, a small 2-wheeler driving school in India for women, which grew through word-of-mouth, generating 20-40 daily calls. Juggling the business and household was exhausting, as she couldn't miss a lead. The constant interruption cost her sleep and peace of mind. Realizing 80% of questions were repetitive, we created Vani.ai to handle these inquiries, giving her back time to focus on customers and rest.

🚀 The Problem

Small businesses lose 60% of leads because they can't answer the phone 24/7. Hiring support staff is expensive, and language barriers (English vs. Hindi, local languages) further alienate customers.

💡 The Solution: Vani.ai

Vani.ai is a white-label AI Voice Agent platform designed to instantly learn your business logic and efficiently manage your communication.

✨ Key Features (Demo Highlights)

  • Instant Logic Ingestion: Upload call recordings, PDFs, or other relevant business data, and the AI rapidly learns your business operations.
  • Dedicated Business Number: A new business phone number is created and assigned to the AI agent.
  • Fluent Multilingual Communication: The agent handles calls in both English and Hindi with a natural, emotive, and native-like accent.
  • Telephony: Users can call a real phone number (e.g., +1408...) to talk to the agent.
  • Turn-Taking: Intelligent silence detection to handle interruptions naturally.
  • Automatic Lead Capture: It automatically extracts essential lead information (Name, Phone, and Intent) and pushes it directly to a live dashboard.

🛠️ Tech Stack

  • Frontend: React, Vite, Tailwind CSS (Custom Dashboard & Admin Panel)
  • Backend: Python, Flask, SQLite
  • AI Orchestration: Gemini 2.0 Flash (Reasoning & Extraction)
  • Voice Pipeline: ElevenLabs Scribe (STT) + ElevenLabs Turbo (TTS) + Vapi.ai (Telephony)
  • Engagement: Twilio (Automated SMS Follow-ups)
  • Open Source: Released under MIT License. ## 🧠 Learnings During the development of this project, we encountered and solved several critical challenges in building Voice AI for India:

1. Latency and Responsiveness

  • Issue: Standard large language models (LLMs) like Gemini 1.5 Pro introduced noticeable delays (2–3 seconds) on phone calls, making the interaction feel awkward.
  • Solution: Switched to the gemini-2.0-flash model. This change dramatically reduced inference time, resulting in a more natural, "interruptible" conversation flow.

2. The Hindi TTS (Text-to-Speech) Problem

  • Issue: The agent would successfully generate a correct Hindi text response, but the user would hear silence instead of speech.
  • Root Cause: The default ElevenLabs voice model (eleven_monolingual_v1) was not Hindi-capable and simply skipped the foreign language text.
  • Solution: Explicitly configured the system to use the multilingual model, eleven_turbo_v2_5, which is capable of authentic Hindi pronunciation.

3. Preventing Language "Stickiness" (Multi-lingual Context Management)

  • Issue: The agent had a tendency to get "stuck" in a language (e.g., Hindi) and would refuse to switch back to the user's current language (e.g., English), or vice versa, even when the user changed languages.
  • Solution: Implemented a Strict 4-Step Execution Protocol within the System Prompt to override the LLM's natural contextual inertia:
    1. Detect the language of the current user query.
    2. Extract facts (without regard to the source language).
    3. Translate the extracted facts to the target language.
    4. Generate the response.
    5. This algorithmic approach forces the LLM to dynamically adapt its output language based on the user's latest input.

🧪 Vani.ai Testing Instructions

I. Create a New Business Agent

  1. Access the Vani.ai Directory: Navigate to the main landing page.
  2. Start Registration: Click the "Register Business" button.
  3. Complete the Form: Fill in the required details, including:
    • Business information.
    • Owner information.
    • Agent preference settings.
    • All relevant business knowledge to be used by the agent. Added
  4. Register and Deploy: Click "Register." This action will generate a new business phone number and deploy the corresponding agent.
  5. Initial Agent Test: Utilize the provided web interfaces to test the agent's functionality.
  6. Live Customer Test: For real customer interactions, call the newly generated business phone number.
  7. Dashboard Verification: Click the "Dashboard" button on the business page:
    • Identify and highlight the Hot Leads card.
    • Verify the Recent Conversations table and ensure "Ankita" is listed.
  8. Multi-Tenancy Check: Return to the directory, select a different business (e.g., "Joe’s Pizza Chatbot"), and perform a test on that new business agent.

II. Test Existing Business Agents

Select a Business: Click on either the "Rainbow Driving School" or "Happy Hearts" business page. Initiate Contact: Test the agent through either the on-page chatbot interface or by calling the agent's dedicated phone number.

Query the Agents:

A. Rainbow Driving School Tests

  • Greeting: "Hello"
  • Pricing: "How much is your fees"
  • Location (in Hindi): "Aapka address kahan ka hai"
  • Course Duration: "How many days of course?"
  • Verification: Confirm that the agent provides a good greeting and accurately responds to all queries.

B. Happy Hearts Daycare Tests

  • Greeting: "Say hello"
  • Location: "Address?"
  • Pricing: "Fees?"
  • Discounts: "Do you offer sibling discount?"
  • Services: "What do your provide?"
  • Out-of-Scope Query: Ask any unrelated question.
  • Verification: The agent should attempt to handle the query and, for an unrelated question, provide the owner's contact details for further information.

🔮 Future Roadmap

Key Features and Capabilities of Vani-AI:

  • Post-Call Text Message: Send an automated text message to the customer following the call (pending Twilio Business Verification).
  • Two-Way WhatsApp Chat: Enable full, bidirectional communication through WhatsApp.
  • Automated Calendar Booking: Integrate with the business calendar to automatically schedule appointments upon customer confirmation.
  • Instant Knowledge Management: Provide a simple text interface for business owners to quickly update the agent's knowledge base (e.g., "We will be closed this Friday").
  • Intelligent Call Handoff: Automatically transfer the call to a human business owner when the AI agent detects it is unable to provide an effective answer.

👥 Team

  • Ankita Jhawar
  • Venkatesh Bhattad

Built With

Share this project:

Updates