glowupclub

Ia using webcam to analyze

## Inspiration

GlowUpClub was born from the insight that professional aesthetic and visagism consultations are expensive and inaccessible for most people. We wanted to empower users with a "pocket mentor" that uses the science of the Golden Ratio ($\phi$) and high-fidelity AI to provide instant, personalized self-care guidance.

## What it does

GlowUpClub is a multimodal AI Live Agent that acts as an aesthetic consultant. By analyzing a user's photo or live stream, it:

Maps 468 facial landmarks via MediaPipe to calculate symmetry and Golden Ratio adherence.
Performs a semantic visual analysis using Gemini 3.1 Flash Lite to detect skin health, jawline definition, and hairline integrity (specifically identifying signs of thinning or balding).
Delivers an instant, structured action plan with recommended procedures and specialist referrals.

## How we built it

We architected a cloud-native solution designed for low latency:

Backend: FastAPI (Python) hosted on Google Cloud Run.
AI Vision: Gemini 3.1 Flash Lite Preview via Vertex AI for multimodal probing.
Geometric Engine: MediaPipe FaceMesh for real-time landmark coordinates.
Infrastructure: Automated deployment using Docker and Terraform for a reproducible environment on GCP.

## Challenges we ran into

The biggest technical hurdle was moving from a traditional chatbot to a Live Agent. We initially experimented with WebSockets and the Gemini Live API, but faced significant rate-limiting and token consumption challenges. We pivoted to a high-fidelity Multimodal Probing strategy, optimizing the prompt engineering to ensure the AI could "see" subtle details like hair density and skin texture accurately without losing performance.

## Accomplishments that we're proud of

Sub-5s Latency: Achieving a full 3D facial analysis and clinical-style report in seconds.
Multimodal Precision: Successfully training the agent to recognize aesthetic nuances (like periorbital hollowing or receding hairlines) that generic models often ignore.
Cloud-Native Stability: Building a robust, containerized backend that scales effortlessly on Google Cloud.

## What we learned

We learned that building "Live" experiences requires a balance between complex real-time streaming and stable multimodal analysis. We also discovered the immense power of Gemini 3.1's long context and vision capabilities when combined with precise geometric data from MediaPipe—the hybrid "Geometric + Semantic" approach is the future of vision agents.

## What's next for GlowUpClub

Real-Time Video Streaming: Transitioning back to the Gemini Live API for "mirror-like" real-time feedback.
AR Try-On: Integrating Augmented Reality to let users visualize the results of recommended procedures (like fillers or new hairstyles) in real-time.
Dermatological Grounding: Implementing RAG to cite specific scientific papers for every recommendation provided by the agent.