WoundCare Copilot

A privacy-first, local AI assistant that uses Moondream & Qwen to analyze wound severity, provide instant clinical first-aid steps, and log visual analytics over time.

Comment

🔍 What it does

WoundCare Copilot is an intelligent, offline medical triage companion and healing progress tracker.

AI Visual Triage: Users simply upload an image of a wound, burn, or skin condition and provide a brief description of how it occurred.

Dual-Model Local Pipeline: The application uses a multi-model approach. A dedicated vision model analyzes visual markers like color, swelling, and superficial traits. This analysis is then processed by a secondary reasoning model to determine an urgency level (Low, Medium, or High) along with structured first-aid steps.

Healing Progress Analytics: Instead of just treating each interaction as an isolated chat, the application automatically extracts a calculated metrics log and saves it locally. This populates a dynamic Healing Progress Tracker chart, allowing patients to visually monitor their wound recovery and structural changes over consecutive scans.

🛠️ How we built it

The project is built on a decoupled, modern full-stack architecture optimized for low-latency local execution:

Frontend: Developed with React and styled using Tailwind CSS to construct a highly responsive, dark-themed medical dashboard tailored for clinical scannability.

Backend: Powered by a lightweight Flask (Python) server that acts as the orchestrator for our local AI ecosystem.

Local AI Orchestration: We utilized Ollama to host and run our open-source models locally. The visual data is first processed by the Moondream vision model. The raw spatial description is then piped directly into Qwen2.5-Coder (7B), utilizing a strict JSON format structure to guarantee clean API schemas.

Data Persistence: Implemented a lightweight, localized file-based JSON NoSQL database system (database.json) to safely append user histories, dynamic date markers, and estimated severity metrics chronologically.

⚠️ Challenges we ran into

Building a multi-model pipeline that runs strictly on consumer-grade hardware came with distinct technical hurdles:

Enforcing Deterministic JSON Structures: Raw LLM responses are notoriously unstructured. Early iterations of the Qwen output included conversational filler or markdown code blocks which crashed our frontend components. We resolved this by explicitly activating Ollama's native JSON schema constraints and reinforcing strict, foolproof structural system prompts.

Dynamic Chart Component State Synchronization: Initially, the frontend progress chart would appear blank or out-of-sync upon generating a new analysis. We had to re-architect our state management lifecycle in React, ensuring that as soon as the Flask endpoint handles the parallel vision execution and appends the payload to our local database, an asynchronous fetchHistory callback is automatically triggered to force-update the layout.

🎉 Accomplishments that we're proud of

Absolute Data Privacy: Achieved a fully functioning healthcare MVP prototype where medical images and log histories remain 100% confidential and bound strictly to the local host machine.

Flawless Multi-Model Synchronization: Successfully orchestrated a seamless, single-click execution chain that moves from raw image input to vision-text summary, downstream language model triage, and immediate data-visualization rendering.

High-Fidelity UI/UX: Designed a clean, highly modern dark UI that mimics real-world telemedicine workspaces, making intricate AI processing feel accessible and straightforward to everyday users.

📚 What we learned

Local LLM Infrastructure: Gained extensive, practical experience inside the Ollama environment, learning how to handle vision-to-text transitions and optimize inference parameters.

Asynchronous Lifecycles in React: Deepened our understanding of React hooks (useState, useEffect, useRef) when handling heavily delayed, asynchronous dual-model processing from a Python backend.

Data Mapping Strategies: Discovered how to effectively map qualitative, unstructured AI medical text outputs into quantitative, structured UI analytics (like dynamic chart bars based on variable urgency and structural data elements).

🚀 What's next for WoundCare Copilot

Advanced Predictive Healing Analytics: We plan to implement computer vision regression models to calculate precise healing rates across a timeline, projecting predictive lines directly into the progress tracker.

Hardware Optimization & Quantization: Explore further model quantization to make running local vision-language pipelines smoother on lower-end consumer setups or mobile edge devices.

Localized Multi-Lingual Support: Introduce multi-lingual prompt translation so non-English speaking communities can comfortably describe their conditions and receive vital first-aid outputs in their native languages.

Built With

Share this project:

Updates

Submission history