CareOctopus

💡 Inspiration

Caregiving is the ultimate multitasking challenge. A single family caregiver often acts as a nurse, a pharmacist, a dietitian, and a data analyst simultaneously. It feels like you need eight arms just to keep up with medication schedules, changing symptoms, and doctor updates.

Inspired by the Octopus Hackathon's theme of "Intelligent Multitasking," we wanted to build a tool that extends the caregiver's capabilities. We asked: What if an AI could handle the data entry, so the caregiver can focus on the care?

🐙 What it does

CareOctopus is a multimodal AI web application that acts as a central nervous system for caregiving. It uses four specific "Tentacles" to manage health data:

1. Tentacle 1: Vision (Medication Scanner) Instead of typing complex drug names, the caregiver snaps a photo of a pill bottle. The AI (Gemini Vision) instantly extracts the name, dosage, and safety warnings.
2. Tentacle 2: Voice (Symptom Logging) Caregivers can speak naturally (e.g., "Grandpa felt dizzy after lunch but feels better now"). The AI (Gemini Pro) analyzes the audio, extracts medical details, and determines the severity.
3. Tentacle 3: Logic (Dynamic Health Score) We built a "living" algorithm that calculates a real-time Health Score (0-100%). It penalizes for critical events (like Chest Pain) but also recognizes recovery trends over time, giving families instant peace of mind.
4. Tentacle 4: Memory (Reports) One-click generation of professional PDF reports for doctor visits, ensuring no detail is ever forgotten.

⚙️ How we built it

We built CareOctopus with a Mobile-First mindset using the latest web technologies:

• Frontend: Built with Next.js 15 (App Router) for speed and SEO.
• Styling: We used Tailwind CSS and Shadcn UI to create a clean, accessible interface that looks great on both desktop and mobile.
• AI Brain: We leveraged Google Gemini (Multimodal). We used Gemini Flash for instant image recognition and Gemini Pro for complex medical reasoning and sentiment analysis.
• Security: We implemented Next.js Server Actions to create a secure tunnel between the client and Google's API, ensuring API keys are never exposed to the browser.
• Deployment: Hosted on Vercel for instant global availability.

🚧 Challenges we ran into

• Mobile Hardware Constraints: Browsers on mobile phones handle microphones and cameras differently than desktops. We dealt with issues where the microphone wouldn't release control, preventing the camera from opening. We solved this by implementing strict resource cleanup hooks in React.
• The "Scoring" Logic: Initially, our health score was purely punitive—every symptom dropped the score. This felt discouraging. We rewrote the algorithm to include a "Recovery Factor," where positive logs (like "Feeling Well") actually heal the score over time.

🏅 Accomplishments that we're proud of

• True Responsiveness: The app seamlessly transforms from a Sidebar layout (Desktop) to a Bottom Navigation layout (Mobile), making it usable in a crisis.
• Zero-Friction UI: We successfully eliminated the need for keyboards. You can use the entire app with just a camera and a microphone.
• Security: Successfully refactoring the AI calls to Server Actions to make the application production-secure.

📚 What we learned

• Multimodal AI is the future of UX: Text forms are obsolete. Being able to "show" or "tell" the AI what is happening is significantly faster and more accurate.
• State Management: Handling real-time media streams (Audio/Video) in React requires precise state management to avoid memory leaks.

🔮 What's next for CareOctopus

• SMS Alerts: Automatically texting family members if the Health Score drops to "Critical."
• Doctor Portal: A secure link for doctors to view the dashboard in real-time.
• Wearable Integration: Pulling heart rate data directly from smartwatches.

Built With

• google-gemini
• jspdf
• lucide
• next.js
• react
• shadcn-ui
• tailwind-css
• typescript
• vercel