gemini-glowchart-agent

Beauty advisor agent: Vertex AI Gemini reads Perfect Corp skin + tone JSON, picks a makeup look + hair color, triggers the virtual try-on. Verbatim citations only.

Comment

Inspiration

Beauty AI is everywhere, but most demos that promise a try-on either paraphrase the cosmetic recommendations into something safe and bland, or hallucinate undertones the API never reported. Skin tone, foundation shade, and a recommended look all need to land verbatim from the structured signal the underlying API returned. If the agent makes up a shade, the user shows up to the makeup counter and the product doesn't match. That's the failure mode this project rules out.

What it does

You upload a selfie. The agent calls Perfect Corp's YouCam Skin Analysis and Skin Tone endpoints in parallel, reads the structured JSON, picks a makeup look and hair color that fits, and triggers the virtual try-on endpoints. Every attribute name, score, undertone, season, foundation shade, and hex color in the final answer is copied byte-for-byte from the Perfect Corp API response.

The final answer is structured into five labeled sections:

  • ATTRIBUTES — top concerns with raw scores
  • UNDERTONE — undertone, season, foundation shade
  • RECOMMENDED_LOOK — lipstick / eyeshadow / blush / hair hex codes
  • EVIDENCE — unedited quotes from the API JSON
  • CONFIDENCE — derived from overall_score; drops to "low" if any required field is missing or overall_score < 50

How we built it

  • Google Cloud Agent Builder (ADK) LlmAgent on Vertex AI Gemini 2.5 Flash
  • Five FunctionTools shaped exactly like Perfect Corp's YouCam API: upload_image, analyze_skin, analyze_skin_tone, apply_makeup, apply_hair_color
  • Streamlit dashboard with selfie upload, deployed to Cloud Run
  • Deterministic stub mode for CI: same tool shapes, hand-written fixtures shaped exactly like the real API
  • System prompt that forbids paraphrasing — quote blocks must be verbatim

Challenges we ran into

  • The agent's instinct is to smooth recommendations into helpful prose; the prompt explicitly forces quote blocks with no rewording
  • Hex code accuracy matters — a single-digit drift turns a warm earth tone into a cool burgundy. The CONFIDENCE rule keeps "low" results honest
  • Two analyze endpoints run in parallel; the agent has to handle the race in stub mode the same way as real mode

What's next

  • Multi-photo composition: same prompt across 3 photos, average the undertone
  • A "before/after" stitched grid the user can download
  • Integration with sponsor's loyalty SKUs once a Perfect Corp tenant is provisioned

Try it

Repo: github.com/MukundaKatta/gemini-glowchart-agent Live demo: https://gemini-glowchart-agent-1029931682737.us-central1.run.app License: Apache 2.0

Built With

  • adk
  • ai-agents
  • beauty-ai
  • cloud-run
  • gemini
  • perfect-corp
  • python
  • streamlit
  • vertex-ai
  • youcam
Share this project:

Updates