VitalVoice

VitalVoice: The Call for Life. We replace manual calling with instant AI voice agents that proactively reach donors during critical hospital shortages. Speed saves lives.

Comment

Inspiration

In the medical field, the need for blood donors is ever increasing (Source: RedCross). In situations where blood is required, finding blood donor using traditional methods is slow, inefficient, and often fails during emergencies. Also, when the patient’s family is already dealing with the psychological burden, calling and confirming donors adds to their woes.

Our attempt is to ease the painstaking process of obtaining a matching blood donor using AI for Social Good - AI4SG (With Generative AI and Voice AI) which not only helps the patient but also takes some weight off the shoulders of the patient’s family.

What it does

VitalVoice is an AI based solution which facilitates the process of acquiring the blood for medical procedures. It has been possible to implement VitalVoice only because STT (Speech to Text) can now seamlessly understand human speech in multi-language; and TTS (Text to Speech) has evolved to sound ultra-realistic.

When the blood need arises, the blood bank can submit a form with all required details (like blood group required, location where the donor needs to donate, reason for blood requirement, how many donors to be reached out to). This request is received by the campaign manager app which gets the list of potential/matching donors from a master donor list maintained in Supabase. The campaign manager makes call to these potential donors one by one using Elevenlabs voice agent and Twilio voice platform for donor lookup. During the call, donor's consent and availability is obtained. After that, the agent also presents the standard blood donation eligibility questionnaire to the donor. Based on the donor's consent, availability and eligibility, the donor is finalised. We also collect donor's current location to calculate distance, time, ETA and driving directions so that they can be guided to the donation centre. Details collected during the call are then relayed back to the blood-bank. At the end, a WhatsApp message is sent to the donor with donation time, location and driving directions.

VitalVoice supports multiple languages (English, Hindi and Tamil) so that language shouldn't become a barrier in communication in this modern era of technology.

We also have added capability for the blood-bank admin to monitor the campaign in real-time. We have done this for web calls using a custom solution as Elevenlabs's real-time monitoring feature is available only for enterprise tier.

For deep dive into each component of VitalVoice, please refer to VitalVoice Agent - Deep Dive.

How we built it

We have used the following tech stack for implementing the VitalVoice:

  1. Elevenlabs Agents Platform
  2. Twilio Voice Services
  3. GCP Cloud Run
  4. Google Maps API
  5. Twilio Messaging Services
  6. NodeJS
  7. ReactJS
  8. TypeScript
  9. Supabase
  10. Supabase Realtime

Challenges we ran into

We wanted our AI agent to do more than just talk; we wanted it to provide utility. Specifically, we needed to calculate travel distance and time in real-time and deliver a clickable map link to the user’s WhatsApp. We had the following challenges:

  1. Asynchronous State Management: ElevenLabs tool calls happen mid-conversation, but WhatsApp messages (via Twilio) are best sent after the call or as a summary. Capturing data during the call and holding it for after the call was non-trivial.
  2. Latency & Context: We had to ensure the LLM didn't "hallucinate" the travel time while waiting for the Google Maps API response, requiring optimisation of prompt and the ElevenLabs tool-calling schema.
  3. Real-time Synchronisation: Updating donor status across multiple concurrent calls without conflicts.

Accomplishments that we're proud of

Completing the project on/before time which involves multiple channels (Web, Telephone, WhatsApp) and a complex yet intelligent campaign automation solution which supports a very time-sensitive cause.

What we learned

We learnt how to use Supabase Realtime, building rich GUI using Antigravity IDE, integrating google maps API with Elevenlabs agent for real-time processing.

What's next for VitalVoice

  1. Build real-time monitoring for phone calls using Elevenlabs enterprise tier.
  2. Add a rule based pre-processing engine for filtering the donors before the outbound campaign is kicked off. This aids in cost saving by checking for Do Not Disturb registry and other mandatory compliance.
  3. We can add more filters in the blood request form like if the whole blood, red cells, platelets, or plasma is required and the donor lookup can be changed accordingly.
  4. If multiple units of blood are required, then multiple donors will need to be secured as each donor can donate only upto 450ml.
  5. We can add implementation at a scale where a hospital has a tie-up with multiple blood banks. We can use the pub-sub model where the hospital will be the publisher, posting about blood requirements and all the blood-banks which are in the network of the hospital would subscribe to it. So that as soon as a request comes in, multiple blood-banks will get into action providing swift response to the blood need.
  6. If during the screening questionnaire, it's found that the donor is no longer eligible for blood donation due to chronic illness or permanent relocation, then they can be either marked as ineligible or can be removed from the master donor list.

Note

Our main demo video link is https://youtu.be/xuvV-vycn-g.

Please check our another video (https://youtu.be/40M9cdYpBMo) which contains the demo with even more use-cases, deeper explanation and features like:

  1. Outbound call campaign where donor is willing to donate blood, but is ineligible.
  2. Realtime conversation monitoring using web calls to Elevenlabs agents using their Websockets API.

Built With

Share this project:

Updates