APL COLOR Image Agent

AI-powered image consulting platform that diagnoses personal color, face shape & body type from a selfie — then generates real styling simulations, backed by 12,000+ expert consultation records.

Comment

Inspiration The personal color diagnosis market is growing rapidly, but most professionals still rely on paper palettes and handwritten records. APL COLOR is a professional personal color consulting firm with over 12,000 real consultation PDF records, and we started this project with the conviction that combining this massive dataset with AI could dramatically improve both diagnosis quality and efficiency.

The core question was simple: "Can AI change only the lip color, eye shadow, eyebrows, and glasses while keeping the customer's face 100% intact?" Existing AI image generation had the problem of distorting the face itself, and solving this was our starting point.

What it does APL COLOR Image Agent is an all-in-one AI image consulting platform for personal color professionals.

Data Foundation:

Over 12,000 real consultation PDF records were parsed and refined using a Mac Mini, then stored as structured data in MongoDB This data serves as the foundation for AI analysis and the agent system Customer Diagnosis Flow:

Customers submit their information (face/body photos, personal details) through the landing page Gemini AI automatically analyzes personal color, face shape, and body type to generate an initial diagnosis Experts review, adjust, and finalize the AI diagnosis in the Expert Console AI Image Generation (Gemini Image Generation):

Lip Color: Generates images with recommended lip colors applied directly to the customer's face Eye Shadow: Applies recommended shadow colors to the actual face Eyebrow Styling: AI generates eyebrow shapes suited to the customer's face shape Glasses Recommendation: AI generates images with face-shape-appropriate glasses on the customer All generated images preserve the customer's original face as much as possible Data Analysis Agent:

An AI agent built into the Expert Console autonomously searches the 12,000+ customer DB via Gemini Function Calling Responds in real-time to natural language queries like "Search for female customers in their 30s with Summer Light type" Suggests personalized styling directions based on customer profiles (occupation, age, style preferences) Result Report:

Auto-generates a shareable diagnosis result page for customers Includes Before/After comparison images How we built it Data Pipeline: Over 12,000 real consultation PDF records were batch-parsed using a Mac Mini, converting customer information, diagnosis results, and color palettes into structured data stored in MongoDB. This data serves as the foundation for the AI agent and diagnosis algorithms.

Frontend: Built with Vanilla JavaScript + HTML/CSS (Expert Console, Landing Page, Result Page) for fast loading and deployment flexibility. We built a custom i18n system supporting four languages (EN/KO/JA/ZH).

Backend: RESTful APIs powered by Node.js + Express + MongoDB. Google Cloud Storage manages customer photos and AI-generated images.

AI Models: Gemini serves as the core AI model across the entire system.

Gemini 2.5 Flash: Chat agents (Landing chatbot + Expert data analysis agent) Gemini (Image Generation): Generates lip, shadow, eyebrow, and glasses images based on customer face photos Gemini Function Calling: Enables the Expert agent to autonomously search the customer DB through tool invocation Preset Image Collection & Classification: We collected preset images for image-making professionals to use in the Expert Console (lip colors, eye shadows, nails, hair, glasses, accessories, etc.) and meticulously categorized each image by category, color type, and face shape.

Infrastructure: Cloudflare Pages (frontend) + Google Compute Engine (backend, PM2) + MongoDB Atlas + Google Cloud Storage

Challenges we ran into Achieving accurate demo diagnoses was the biggest challenge. In real professional consultations, results come from a comprehensive assessment of multiple factors — face color, overall atmosphere, contrast level, hue, value, and chroma. However, in the demo diagnosis, we had to work with limited information from a single photo, primarily relying on face color and contrast level. It took significant time to fine-tune prompt engineering and algorithms to produce reasonably accurate results from this limited input.

Refining 12,000 PDF records was no small task either. Parsing consultation PDFs in varying formats and converting them into a structured DB schema was carried out using a Mac Mini. Ensuring data consistency turned out to be far more complex than anticipated.

Collecting and categorizing preset images for experts also took considerable time. Gathering images across categories — lip colors, eye shadows, nails, hair, glasses, accessories — and sub-classifying them by personal color type, face shape, and body type required professional judgment and meticulous attention to detail.

Balancing security with functionality was another important challenge. The AI agent needs to search the customer DB, but sensitive information (phone numbers, emails, photo URLs) must never be exposed to Gemini. We solved this by placing the server as an intermediary and excluding sensitive fields from query results.

Accomplishments that we're proud of Implementing a high-accuracy demo diagnosis from limited input is what we're most proud of. We succeeded in deriving close approximations of the full multi-factor diagnosis using only face color and contrast, producing results that even professionals find reasonable upon review.

Building an AI agent powered by 12,000+ real records is another meaningful achievement. This isn't a simple chatbot — it's a genuine agent that can search the DB in real-time via Gemini Function Calling, and its analysis accuracy improves as more data accumulates.

Digitizing the entire expert workflow is also a significant accomplishment. From customer intake → AI pre-diagnosis → expert confirmation → AI image generation → result report sharing, we built a system that fully replaces the paper-based process.

Systematically categorizing a massive library of preset images into an immediately usable format for professionals is another point of pride.

What we learned Prompt design for extracting the best results from limited data is critical. Even with constrained input like the demo diagnosis, we learned that carefully engineered prompts grounded in patterns from 12,000 real consultations can produce high-probability results.

Function Calling transforms AI into a real agent. System prompts alone have their limits, but providing tools (functions) enables AI to autonomously judge and act. We also learned the security pattern of placing a server as an intermediary.

Data refinement determines 80% of AI quality. Through the process of converting 12,000 PDFs into a database, we experienced firsthand how data consistency and structuring directly govern the quality of AI analysis results.

Collaboration with domain experts drives prompt quality. Incorporating insights from real consultation experience into system prompts, we witnessed the transformation of a general-purpose AI into a specialized professional tool.

What's next for APL COLOR Image Agent Subscription Model & Image Chronicle: The key feature currently under development — customers can subscribe to access cosmetics matched to their profile and newly updated styling content, periodically updating their information to build a personal Image Chronicle over time Expanded Function Calling: Adding getCustomerStats (statistical analysis), getPopularProducts (popular recommended products), findSimilarCustomers (similar customer search), and getColorTrends (trend analysis) to extend agent capabilities Real-time Video-based Diagnosis: Automatically extracting optimal frames from video instead of still photos for more accurate diagnoses AI Learning Feedback Loop: Accumulating data from expert corrections to AI diagnoses, building a system where diagnostic accuracy improves over time Customer Self-Service: A B2C service allowing customers to receive basic diagnoses and view results without an expert

Built With

Share this project:

Updates