Confidence Copilot

Turn a selfie into an explainable beauty and eyewear shopping bundle using Perfect Corp analysis, shopper preferences, and local product logic.

Comment

Inspiration

Confidence Copilot was inspired by a simple retail problem: shoppers often hesitate because they do not know what will actually suit them. A skin score or face analysis result can be useful, but the bigger opportunity is turning that information into a practical shopping decision.

I wanted to build something that felt useful beyond a novelty filter: a mobile-first assistant that helps a shopper understand what to buy, why it was suggested, and how it fits their situation, budget, region, and existing routine.

What it does

Confidence Copilot turns a selfie into an explainable beauty and eyewear shopping bundle.

The user can upload or capture a selfie, select a shopping goal, choose a region, set budget and routine preferences, and mark products they already own. The app then uses Perfect Corp Skin Analysis and, for the glasses path, Face Analyzer to generate a retail confidence profile.

It converts those signals into:

  • Skin-focused product recommendations
  • Face-shape-aware glasses guidance
  • Regional product offer examples for NZ, AU, US, and Other
  • Product ideas from a static merchant-feed example
  • A clear explanation of why each item was selected
  • A mock checkout summary showing how the bundle was built

The public Vercel version runs in safe demo mode so judges can experience the full flow without needing an API key or spending live API credits.

How we built it

We built Confidence Copilot as a mobile-first Next.js app using React, TypeScript, CSS, Vercel, GitHub, Codex, and Perfect Corp APIs.

The app uses server-side API routes for Perfect Corp integration so API keys are never exposed to the browser. Skin Analysis and Face Analyzer requests are handled through controlled backend routes, while demo mode uses deterministic local data for safe public judging.

The recommendation engine is deliberately deterministic rather than LLM-based. It scores products using the selected shopping goal, skin analysis metrics, face-shape guidance, region, budget, routine complexity, and products the user already owns. This makes the recommendations explainable, auditable, and safer for a retail prototype.

We also added a static merchant-feed example to demonstrate how this could later connect to retailer catalogues without scraping or relying on live product APIs during the hackathon.

Challenges we ran into

One challenge was balancing live API capability with public demo safety. We wanted to prove the Perfect Corp integration worked, but also needed to protect API credits and avoid exposing keys. The solution was a safe public demo mode plus controlled live mode for local or private testing.

Another challenge was handling real image-quality issues. Some photos can be too far away, too low quality, or unsuitable for face analysis. We added clearer fallback states so Skin Analysis can still succeed even if Face Analyzer cannot complete.

We also had to make the recommendation logic feel useful without becoming a black box. Instead of adding an external AI layer, we built transparent scoring rules and explanations so users can see why each product was suggested.

Accomplishments that we're proud of

We are proud that Confidence Copilot is more than a raw analysis demo. It turns Perfect Corp analysis into a complete shopper journey from selfie to recommendation to mock checkout.

Key accomplishments include:

  • Live-capable Perfect Corp Skin Analysis integration
  • Live-capable Face Analyzer integration for the glasses path
  • Safe public demo mode
  • Server-side API key handling
  • Mobile-first upload, camera, preview, remove, and reupload flow
  • Region-aware product recommendations
  • Shopper preferences and already-owned product logic
  • Product ideas from static merchant-feed metadata
  • Explainable recommendation reasoning
  • Deployment-ready Vercel build
  • Final branding, documentation, and tests

What we learned

We learned that the most valuable retail experience is not just analysis, but translation. Shoppers do not only need to know that they have a skin metric or a face-shape result. They need to know what that means for the products they should consider buying.

We also learned that deterministic logic can be powerful when the product needs transparency. For this use case, explainable scoring was more appropriate than adding an external LLM, because shoppers and retailers need to understand why a recommendation was made.

Finally, we learned that public demo safety matters. Hackathon projects often need live API proof, but public judging links should be safe, repeatable, and protected from accidental credit usage.

What's next for Confidence Copilot

The next step is connecting Confidence Copilot to real retailer catalogues through Shopify, merchant feeds, or affiliate product APIs. This would allow the app to move from demo product ideas to real purchasable bundles with live stock and pricing.

Future improvements could include:

  • Real retailer catalogue integration
  • Optional user accounts and saved routines
  • More Perfect Corp try-on experiences
  • Full eyewear or makeup try-on support
  • Better product filtering by price, brand, and availability
  • A retailer dashboard for managing recommendation rules
  • A production credit and rate-limit system for live API usage

The long-term goal is to make Confidence Copilot a practical retail assistant that reduces purchase hesitation by giving shoppers clear, explainable, personalised product confidence before they buy.

Built With

Share this project:

Updates