Foru AI: Skin & Health Agent

Foru AI is a multimodal health agent using Gemini 3 reasoning to bridge meds, skin, and diet. It decodes scans into life-saving safety alerts through a persistent, context-aware health coach.

Comment

Inspiration

We noticed a critical gap in personal wellness: information silos. People often use separate apps for skincare, diet tracking, and medication reminders, yet these areas are deeply interconnected. A specific food might trigger a skin flare-up, or a medication might cause photosensitivity. We built Foru AI to be the "Reasoning Layer" that connects these dots. We wanted to move beyond passive tracking and build a practical, unified mobile experience where AI acts as a proactive health agent that "thinks" across different categories to keep users safe.

What it does

Foru AI is an all-in-one health agent that transforms raw visual data into expert-level guidance:

  • Multimodal Scanner: Identifies medications (pills), analyzes skin conditions, and evaluates food nutrition/allergens through high-resolution photo scans.
  • Actionable Safety Alerts: Instead of generic advice, it outputs straightforward warnings (e.g., "Warning: High Dairy - Acne Trigger" or "Photosensitivity Risk: Use SPF 50").
  • AI Assistant & Coach: A persistent "Expert Dermatologist" persona that remembers your scan history for deep, multi-turn consultations.
  • Smart Routine Management: A dynamic dashboard that tracks your Skin Health Score, local UV Index, and manages morning/evening routines with persistent notifications.
  • Comparison Engine: Analyzes "Before & After" photos using Gemini's reasoning to track healing progress over time.

How we built it

  • The Brain: Integrated Gemini 3 Flash/Pro via the google_generative_ai package. We engineered specific System Instructions to enforce a structured, expert persona.
  • The Framework: Developed with Flutter for a premium Material 3 experience using our signature "Foru Green" (#45A17E) palette.
  • The Logic: State management is orchestrated by a unified AppState using the Provider pattern for reactive UI updates across the chat and dashboard.
  • Services: Leveraged geolocator for live UV tracking and flutter_local_notifications for exact-time reminders.

Challenges we ran into

  • The iOS Permission Loop: A major practical hurdle was handling native camera permissions. We faced issues where the system dialog wouldn't trigger due to specific Podfile macro configurations. We solved this by writing robust fallback logic with permission_handler to guide users to settings if they had previously restricted access.
  • Structuring AI Output: Gemini is naturally conversational, but our UI required structured, parseable data. We spent significant time tuning prompts to return valid JSON-mapped data for complex UI elements like "Safety Alert" cards and "Health Scores."
  • Hardware Integration: Fine-tuning camera parameters (imageQuality: 95) was essential to ensure the model could read the fine print on medication labels or analyze skin texture accurately.
  • Async State Handling: Managing the "loading state" during multimodal processing was tricky. We implemented a custom "Reasoning Shimmer" effect to maintain a premium UX while Gemini processed high-resolution image tokens.

Accomplishments that we're proud of

  • Unified AI Persona: Successfully implemented a consistent "Expert" persona that handles everything from dermatology to medication safety without breaking character.
  • Comparison Engine: Built a functional tool that can reason across two different images taken days apart to provide a progress analysis.
  • Premium Aesthetics: Achieved a clean, modern "Glassmorphism" look that makes complex health data feel accessible and calm.
  • Reliable Error Handling: Created a system that gracefully handles network timeouts or API limits, ensuring the user is never left with a frozen screen.

What we learned

  • Prompt Engineering is Engineering: We learned that small changes in the "System Instruction" are just as critical as Dart code for app reliability.
  • The Importance of Context: For AI to be useful in health, it must have memory. Sending a single image isn't enough; sending the image plus the user's past 3 scans creates a much more powerful, personalized insight.
  • Platform Nuance: Gained deep experience in mobile-specific constraints, especially the difference between Android's strict alarm permissions and iOS's notification reliability.

What's next for Foru AI: Skin & Health Agent

  • Agentic Scheduling: Moving from static reminders to "Smart Timing" where the AI suggests the best time to apply skincare based on real-time weather and UV data.
  • Wearable Integration: Connecting with Google Health Connect to correlate sleep and stress levels with skin flare-ups.
  • Context Caching: Implementing Gemini's Context Caching to significantly reduce latency and costs during long-term health consultations.
  • Offline Logic: Exploring Gemini Nano for basic, on-device pill identification to ensure privacy and speed.

Built With

  • camera
  • dart
  • flutter
  • flutter-local-notifications
  • gemini3
  • geolocator
  • google-cloud
  • google-generative-ai
  • openuv
  • sharedpreferences
Share this project:

Updates