AskLepius

Lepius Voice Assistant
Lepius Home Page
Lepius Intelligence

Inspiration

We were inspired by the idea of Impericus Ascend, the Impericus agent that matches doctors with professional representatives who can answer their questions. We thought this was a very novel idea, and thought it could help doctors who needed help with dosage and medication, and wanted to add on to this feature. Doctors have limited time to go through huge databases and sources to find relevant information, and general chatbots can’t be trusted with medical content as they may hallucinate information or use untrustworthy data.

What it does

We realized that most doctors might prefer having an agentic AI to talk to and get their information in real time instead of waiting for a call. This way, they can get all the information in real time and prompt for whatever they need whenever.

How we built it

We built our project using ElevenLabs speech-to-text & text-to-speech, Gemini API for RAG-based responses, Tiger Data for chat history and context memory, TypeScript for the frontend, and Python for the backend. We deployed our frontend to Vercel and self-hosted the backend.

Challenges we ran into

One challenge we faced was implementing the automatic voice activation upon saying “Hey Lepius,” as initially it wouldn’t pick up on the user’s voice. Additionally, we ran into rate-limiting issues with Gemini, resulting in consecutive prompts responding slowly or not at all, however this was resolved by switching to a lighter-weight model.

Accomplishments that we're proud of

We're proud of building a full-stack application that can be extended to real-world medical contexts to allow healthcare professionals to better obtain relevant information and make correct decisions. Additionally, we are proud of the various new technologies that we employed, from using ElevenLabs, to learning about Tiger Data.

What we learned

We learned a lot about specifically using ElevenLabs and speech dictation. None of us had used any sort of speech dictation and speech-to-text API, so we were able to learn this through our project.

What's next for AskLepius

We hope to fully connect with the Impiricus database to unleash the full potential of the AI assistant. Additionally, we intend to gather real feedback from physicians and healthcare professionals to improve, iterate, and fine tune the application interface and the model.