RoleCall Voice AI

Most voice AI is built for 1:1 conversations. RoleCall brings Voice AI into group meetings, taking on roles like Scrum Master, instructor, debate judge etc.. to guide, engage, and deliver outcomes.

Comment

Inspiration

Most voice AI agents are designed for one-to-one conversations. I wanted to explore a different possibility: an AI agent that could actively participate in group meetings and take on meaningful roles rather than being an assistant.

Meetings often need structure, guidance, moderation, teaching, or evaluation. A Scrum Master keeps discussions focused, a course instructor guides learners, and a debate judge evaluates arguments. This inspired me to build RoleCall Voice AI—a flexible meeting participant whose behavior can be defined by its role.

What it does

RoleCall Voice AI allows users to create or join a virtual meeting with an AI participant.

Before the meeting, the host/admin selects or defines the AI’s role, responsibilities, tone, and objectives. The AI then joins the conversation and interacts with multiple participants through voice.

The platform can support different meeting scenarios, such as:

  • Scrum Master for daily stand-ups
  • Virtual Class Teacher for interactive learning
  • Debate Judge for evaluating arguments
  • Interviewer for conducting mock interviews
  • Fun Friday Organiser for team engagement
  • Project Manager for tracking progress and blockers

The AI listens to the discussion, identifies relevant participants, responds in context, asks useful follow-up questions, and helps keep the meeting focused. It remembers past meetings context. At the end, it can produce a structured summary containing key discussion points, decisions, blockers, and action items.

The core idea is simple: instead of using a separate AI tool for every use case, users can assign a role to one intelligent meeting agent.

How I built it

I designed RoleCall as a role-driven voice meeting platform.

The system consists of:

  • A meeting interface for participants to join and communicate
  • A configurable role system for defining the AI’s behavior
  • RAG system to provide larger context about the team and their ongoing project
  • A real-time voice interaction layer
  • Conversation context and session memory
  • Participant-aware response handling
  • Meeting summaries and action-item generation
  • Lyria to generate music during lobby waiting

Each role is represented through a structured prompt containing the AI’s responsibilities, communication style, goals, and interaction rules. This allows the same AI system to behave differently depending on the selected role.

For example, a Scrum Master is instructed to keep updates concise, identify blockers, ask focused follow-up questions, and ensure that every participant gets an opportunity to speak. A virtual class teacher follows a different approach by explaining concepts, asking questions, and adapting to the learner’s responses.

I focused on creating a modular architecture so that new roles can be added without changing the core meeting experience.

Challenges I ran into

The biggest challenge was moving beyond simple question-and-answer interactions. In a group meeting, the AI must understand who is speaking, follow the flow of the conversation, avoid interrupting unnecessarily, and respond at the right moment. Handling media in a cloud environment had its own challenges like creating a dedicated media server , configuring load balancers , WebRTC, understanding on TURN protocols , VPC, NAT, apply networking profiles for audio streams and fine tuning it.

I also had to consider:

  • Maintaining context throughout the meeting
  • Designing prompts that produce consistent role-specific behavior
  • Balancing natural conversation with structured meeting outcomes
  • Avoiding excessive interruptions from the AI
  • Handling ambiguous statements and incomplete updates
  • Generating useful summaries from unstructured discussions
  • Reducing the cost as the solution involves huge setup.
  • Authentication and security

Another challenge was designing the system to be flexible. Different roles require completely different interaction styles, but they must all work through the same underlying platform.

Accomplishments that I'm proud of

I am proud to have created a working prototype that demonstrates how an AI agent can become an active participant in a meeting instead of remaining a passive chatbot.

My key accomplishments include:

  • Building a voice-enabled AI meeting participant
  • Supporting configurable AI roles
  • Demonstrating multi-participant interaction
  • Creating role-specific meeting behavior
  • Supporting real-time questions and follow-ups
  • Generating structured meeting outcomes
  • Turning one meeting agent into a reusable framework for multiple use cases

RoleCall shows that the value of voice AI is not limited to personal assistants or customer support. AI can also facilitate collaboration, learning, evaluation, and team communication.

What I learned

I learned that building a useful voice AI agent requires more than connecting speech recognition, an LLM, and text-to-speech / speech to speech.

The quality of the experience depends heavily on role design, conversation timing, context management, and clear instructions. A good role prompt must define not only what the AI should say, but also when it should speak, what it should remember, and how it should guide the conversation.

I also learned that meeting AI should support people rather than dominate the discussion. Short, relevant responses and well-timed follow-up questions are more effective than lengthy explanations.

Most importantly, I learned that a single role-aware agent can support many real-world scenarios when it is designed with a modular architecture.

What's next for RoleCall Voice AI

I plan to expand RoleCall into a complete AI meeting platform with:

  • MCP tools integration like Jira , calendar, email , drive etc..
  • meeting-platform integrations Google meet, webex, zoom
  • Jira, task-management, and collaboration integrations
  • Automatic decisions, risks, blockers, and action-item tracking
  • Multilingual voice support
  • Custom voices and communication styles
  • Post-meeting analytics and improvement insights
  • Enterprise controls for privacy, security, and access management

My long-term vision is for RoleCall to become a flexible AI teammate that can join any meeting, understand its purpose, take on the appropriate role, and help participants achieve better outcomes.

Built With

  • acme
  • adk
  • agent
  • cloud-run
  • fastapi
  • firestore
  • gemini
  • gemini-live-api
  • gke
  • google-adk
  • google-gen-ai-sdk
  • kubernetes
  • livekit
  • lyria
  • memory-bank
  • opentelemetry
  • pub/sub
  • python
  • rag
  • stun
  • terrraform
  • turn
  • vertex
  • vertex-ai-memory-bank-api
  • webrtc
Share this project:

Updates

Submission history