Action AI UNT club

• 

  This is our functional, context-aware clinical assistant- **NaviDoc**

Inspiration

In modern hospitals, doctors and administrators are overwhelmed with patient data — most of which is buried in long, unstructured clinical reports.

We saw two major gaps: Doctors spend too much time during initial visits gathering patient history and repeating diagnostic steps. Hospital management struggles to analyze trends or extract insights from historical records, as they are often in free-text form.

We wanted to build an AI assistant that could read and understand past medical records, assist with initial diagnosis, and surface meaningful insights — all while saving time for clinicians and improving operational efficiency.

What it does

Clinical AI Assistant is an intelligent tool that helps doctors and hospital administrators make faster, smarter decisions by analyzing past medical records.

For Doctors : Provides an initial diagnosis based on patient symptoms before the first visit. Helps plan tests or consultations in advance, reducing unnecessary visits. Saves time during appointments by surfacing relevant medical history instantly.

For Hospital Management : Extracts patterns and insights from unstructured clinical records. Enables analysis of treatment trends, outcomes, and case histories across departments. Supports better planning and decision-making with AI-generated summaries.

How we built it

Used MongoDB to store anonymized medical transcriptions and also retrieve them using vector searches. Each document includes fields like age, gender, diagnosis, medication and a precomputed embedding for fast search.

Used SentenceTransformers (all-MiniLM-L6-v2) for creating semantic embeddings of clinical text. Integrated a CrossEncoder model for reranking based on query relevance. Leveraged Google Vertex AI Gemini Pro to generate concise, clinically accurate answers from relevant documents.

Built the frontend using Gradio for quick and responsive UI. Hosted and deployed using GCP Cloud Run for public accessibility.

Challenges we ran into

Handling unstructured text: Clinical notes are often messy, full of shorthand, and inconsistent. Balancing speed and relevance using hybrid search + reranking was complex. We initially built with Streamlit but faced runtime issues with torch.classes and file watching — so we switched to Gradio for smoother deployment.

Accomplishments that we're proud of

Built a fully working AI assistant capable of analyzing rich clinical notes and returning relevant, medically responsible answers. Integrated multiple AI technologies — combining vector search, reranking, and large language models into one seamless workflow. Migrated to a scalable cloud-based deployment using GCP, setting the foundation for real-world integration. Designed with privacy and clinical safety in mind, ensuring the assistant respects data boundaries and never guesses.

What we learned

How unstructured medical data poses significant challenges for both diagnosis and hospital analytics. How to integrate semantic search, language models, and generative AI to solve real-world healthcare problems. The importance of prompt engineering in generating medically responsible AI outputs. How to combine backend intelligence (vector search + LLMs) with a user-friendly frontend interface.

What's next for Action AI UNT club

We plan to expand the Clinical AI Assistant by integrating with actual EHR systems through secure, standards-compliant APIs. This would allow: Seamless, real-time access to patient histories. Smart triage suggestions directly within hospital workflows. Support for structured + unstructured clinical data.

Built With

• cloudrun
• crossencoder(msmarco)
• googlevertexai
• gradio
• javascript/html(gradio)
• mongodbatlas
• python
• sentencetransformers