CallAI

CallAI bridges telecommunications and AI by allowing users to access intelligent assistants through simple phone calls or text messages, no internet required.

Comment

Inspiration

AI assistants are becoming more powerful, but most of them are still limited to apps and websites. We wanted to explore how artificial intelligence could be made accessible through something as simple and universal as a phone call or a text message.

The idea behind CallAI was to remove barriers to AI access by allowing anyone with a phone number to interact with an intelligent assistant, without needing internet connection.

What it does

CallAI enables users to interact with an AI assistant through SMS and voice calls.

Users can:

  • Send a text and receive an AI-generated reply
  • Speak during a phone call and hear an AI-generated voice response

In short, CallAI turns any phone into an AI-powered assistant.

How we built it

CallAI was built using a modular backend architecture:

  • A Flask server to handle incoming webhooks for SMS and voice calls
  • The Gemini API for natural language understanding and response generation
  • Twillio for handling inbound and outbound sms and call

Each user request is processed by receiving the input, sending it to the AI, generating a response, and returning it to the user either as text or audio.

Challenges we ran into

One of the main challenges we faced was implementing conversational context across multiple messages and calls. Due to time and technical constraints during the hackathon, each AI request was treated independently, meaning the assistant did not yet maintain memory between user interactions.

Designing a system that preserves context across stateless AI calls and asynchronous webhooks proved more complex than expected, especially when coordinating multiple APIs in real time.

This highlighted the importance of session management and state handling in conversational AI systems, and shaped our plans for future improvements.

Accomplishments that we're proud of

  • Successfully integrated AI with real-time phone and SMS interactions
  • Built a working conversational AI system over traditional telecom channels
  • Designed a clean and scalable backend architecture
  • Delivered a fully functional prototype during the hackathon

What we learned

Through CallAI, we learned how to design AI systems that go beyond web interfaces and interact with real-world communication infrastructure.

We gained hands-on experience with API orchestration, webhook-based system design, and integrating AI with voice and messaging platforms.

What's next for CallAI

Future improvements include:

  • Adding persistent memory using a database
  • Supporting multiple languages
  • Deploying the system to the cloud
  • Improving speech recognition accuracy
  • Expanding into business use cases such as customer support or scheduling
Share this project:

Updates