Personalisation-meets-VTO

when personalisation meet virtual try on(see your self in it) in one shopping experience

Comment

POV Fit Engine — Don't Just Add It to Your Cart. Fit It with POV.

Inspiration

Online fashion stores offer thousands of garments, but having more choices does not necessarily make shopping easier.

Imagine opening an e-commerce website with 5,000 jackets, dresses, shirts, and other garments. The products might be organised by popularity, price, or newest arrivals, but none of those approaches necessarily answers the question that matters most to an individual shopper:

Which of these products actually matches my point of view?

A shopper who enjoys streetwear, hip-hop culture, oversized silhouettes, and graphic designs may have very different preferences from someone who loves minimalist fashion, contemporary designers, and elegant outfits. Yet both shoppers could encounter the same default product catalogue.

This creates an opportunity for e-commerce businesses: instead of simply displaying a large inventory, they can help each shopper discover the products most relevant to them.

That is the problem I set out to address with POV Fit Engine, a personalised fashion discovery and virtual try-on platform that combines cultural intelligence, machine learning, explainable product ranking, agentic AI, and visualisation.

POV is designed for two connected audiences:

  • Shoppers: Discover garments that reflect their individual tastes, understand why products are recommended, and preview supported garments before making a purchase decision.
  • E-commerce businesses: Present a more personalised product catalogue by ranking garments according to each shopper's predicted style compatibility, rather than relying exclusively on a generic product order.

My vision is to help online fashion stores move from showing shoppers thousands of choices to helping them identify the choices most relevant to them.

The experience follows a simple journey:

Discover → Personalise → Rank → Fit → Decide.

What I Built

POV Fit Engine is a Python and Flask-based fashion discovery and virtual try-on platform designed to connect personalised recommendations with actual product discovery.

The platform brings together:

  • Qloo's cultural taste intelligence.
  • An existing XGBoost recommendation model.
  • A garment-ranking service that calculates a POV Style Compatibility Score.
  • Gemini and Google's Agent Development Kit (ADK) for agentic orchestration.
  • Perfect Corp's YouCam Apparel Virtual Try-On for supported garments.

The key idea is to make product relevance the starting point of the shopping experience.

Rather than presenting every shopper with the same default ranking, POV can organise the available catalogue according to the compatibility signals available for that shopper.

1. An Agentic Fashion Intelligence Layer

I designed POV around a Gemini-powered root-agent architecture using Google's Agent Development Kit (ADK).

The agent coordinates specialised capabilities to support the shopping journey, including cultural intelligence, product recommendations, garment ranking, product information, and virtual try-on.

Instead of treating an LLM as the entire application, POV separates intelligent orchestration from deterministic business logic.

Gemini can help interpret a shopper's intent, coordinate the relevant capabilities, and explain recommendation results. Python services remain responsible for application behaviour, product identifiers, constraints, and compatibility calculations.

This separation makes the system more controllable and creates a foundation for integrating POV into different e-commerce environments.

2. Culture-Aware Personalisation with Qloo

Fashion preferences extend beyond browsing history and previous purchases.

Music, films, designers, cultural interests, aesthetics, and personal expression can all contribute to how people discover and interpret fashion.

Qloo provides POV with a cultural intelligence layer through supported relationships between entities in its taste graph.

For example, a shopper's selected music or film interests can provide additional context for discovering relevant fashion entities and aesthetics, where supported by the available data.

These relationships become additional recommendation signals rather than definitive proof that a shopper will like a particular garment.

This is especially relevant to e-commerce businesses serving new customers or shoppers with limited purchase histories. Where sufficient cultural and product signals are available, POV can explore ways to personalise discovery without relying exclusively on previous transactions.

The goal is to make recommendations more context-aware while maintaining a clear distinction between API evidence and system inference.

3. POV Style Compatibility Score: Intelligent Catalogue Ranking

This is where POV's consumer experience connects directly to its business value.

An online fashion merchant may have thousands of products but limited space to display them prominently. The order in which garments appear can influence which products shoppers discover and explore.

POV introduces a POV Style Compatibility Score, using the existing XGBoost recommendation model and ranking logic to organise garments according to the evidence available for each shopper.

The ranking can incorporate available signals such as:

  • Stated style preferences.
  • Relevant cultural taste signals.
  • Model outputs.
  • Garment attributes.
  • Budget and occasion constraints.
  • Other available product information.

Garments with higher compatibility scores can appear near the top of the personalised catalogue, while other products remain accessible further down the list.

For example, a shopper browsing thousands of streetwear and casual-fashion products could see garments that better align with their stated preferences before less compatible alternatives.

The score is not arbitrarily generated by Gemini. The ranking service calculates compatibility using the available evidence and application logic.

The business proposition: make product discovery more relevant by changing the default question from "What should we display first?" to "Which available products are most compatible with this shopper?"

The ranking can also provide a foundation for evaluating product engagement, recommendation relevance, and conversion performance over time.

The percentage represents predicted style compatibility according to the system's available signals. It is not a guaranteed probability of purchase, a verified physical-fit measurement, or proof that a recommendation will result in a sale.

4. Fit with POV: Explainable Product Recommendations

Ranking products is only part of the experience. Shoppers should also be able to understand why a garment appears near the top of their catalogue.

With Fit with POV, shoppers can explore the signals that contributed to a recommendation and receive a garment-specific explanation of how it relates to their preferences.

For example, the system might explain that a garment aligns with a shopper's stated preference for minimalist designs, its available garment attributes, and other relevant recommendation signals.

This gives shoppers an opportunity to question recommendations, explore alternatives, and refine their preferences.

For merchants, explainability offers a way to make personalised product discovery more transparent rather than presenting an unexplained list of recommended items.

The experience becomes:

Discover a garment → Understand its compatibility → Preview it virtually → Make a more informed decision.

5. Virtual Try-On with Perfect Corp YouCam

A garment may match someone's aesthetic preferences, but they may still want to visualise how it could look on them.

POV integrates Perfect Corp's YouCam Apparel Virtual Try-On to generate visual previews of supported garments using a shopper's photo.

This connects personalised discovery to visual exploration.

Instead of simply seeing a recommendation and deciding whether to click through, shoppers can move from discovering a garment to understanding its recommendation and generating a supported virtual preview.

For e-commerce businesses, this creates an opportunity to make product pages more interactive and help customers make more informed shopping decisions.

Virtual try-on is treated as a visualisation aid, not a guarantee of physical fit, sizing, material behaviour, or garment performance.

6. The Business Opportunity: From Product Catalogue to Personalised Shopping

POV is designed around an e-commerce problem that becomes more significant as product catalogues grow.

A merchant may offer thousands of products, but shoppers have limited time and attention. Generic product sorting cannot necessarily reflect the different tastes of every customer.

POV proposes a personalised ranking layer that can sit between the product catalogue and the shopper's discovery experience.

The intended business value includes:

  • More relevant product discovery: Present products that better align with available shopper preference signals.
  • Reduced decision friction: Help shoppers navigate large catalogues without relying exclusively on manual filtering.
  • Better product engagement: Encourage exploration of products that are more relevant to each shopper.
  • More informed purchase decisions: Connect recommendation explanations with virtual garment previews.
  • A measurable optimisation opportunity: Evaluate whether personalised ranking improves product clicks, add-to-cart rates, conversion rates, or other merchant-defined metrics.

These are intended outcomes to validate through testing, not claims of measured improvements from the current implementation.

A potential integration could use a merchant's existing product catalogue and product metadata, then return a shopper-specific ranking for the storefront to display. The precise integration would depend on the merchant's platform, catalogue structure, and available customer preference data.

In this model, POV is not simply another fashion shopping destination. It is a potential intelligence layer for e-commerce product discovery.

How I Built It

POV combines a Flask application, product data, machine-learning recommendation logic, Qloo taste intelligence, Gemini/ADK orchestration, and Perfect Corp's Apparel Virtual Try-On API.

The architecture separates AI orchestration from application logic:

  • Python and Flask: Application routes, services, and the shopping experience.
  • Gemini + Google ADK: Agentic reasoning and orchestration.
  • Qloo: Cultural entity and taste relationships.
  • XGBoost: Existing machine-learning foundation for product-preference prediction.
  • Garment-ranking service: Compatibility scoring, product ordering, and application constraints.
  • Perfect Corp YouCam: Virtual previews for supported garments.
  • Product catalogue: Garment information and the product metadata required for ranking and try-on.

One of my most important architectural decisions was to avoid turning the entire application into an LLM.

The agent coordinates capabilities, while deterministic Python services remain responsible for predictable operations such as validating product identifiers, applying constraints, and calculating compatibility scores.

This modular approach creates a foundation for evolving POV from an individual shopping experience into a reusable product-ranking capability for e-commerce businesses.

Challenges

Turning Cultural Signals into Fashion Recommendations

A relationship between cultural entities does not automatically mean that a specific garment suits an individual. I needed to distinguish between actual API evidence and inferred relationships while combining signals without inventing explanations.

Making Compatibility Scores Meaningful

A percentage can look authoritative even when its methodology is unclear. I chose not to let Gemini arbitrarily generate the compatibility score. Instead, the ranking logic calculates compatibility from available evidence, creating a foundation for evaluating recommendation quality.

Balancing Personalisation and Business Constraints

A highly compatible garment may not meet a shopper's budget or occasion requirements. A useful ranking system must consider relevant constraints rather than optimising for preference signals alone.

Handling Virtual Try-On Constraints

Virtual try-on APIs have requirements for supported garment categories and image inputs. The application must respect those limitations and handle unsupported garments gracefully.

Building an Agent Without Losing Control

Agentic applications become difficult to control when every decision is delegated to an LLM. I separated responsibilities so that the model coordinates and explains, while application services handle predictable business operations.

What Makes POV Different?

Traditional product catalogues organise choices. Traditional recommendation systems attempt to predict what shoppers might like.

POV connects personalised discovery, cultural context, explainability, and visualisation into one experience.

It aims to answer five questions:

  1. What matches my taste?
  2. Why is this garment recommended?
  3. How does it compare with other available choices?
  4. Can I preview a supported garment?
  5. Does this help me make a more informed shopping decision?

For e-commerce businesses, those questions translate into a practical opportunity: organise large product inventories around shopper-specific relevance rather than relying exclusively on one default catalogue order.

Qloo contributes cultural context. XGBoost contributes machine-learning-based preference prediction. The ranking service calculates compatibility. Gemini/ADK coordinates the experience. YouCam provides visualisation.

Together, these components create a foundation for a more personalised fashion discovery journey.

What I Learned

Building POV reinforced the importance of connecting AI capabilities to real decisions.

Cultural intelligence can provide additional discovery context. Machine learning can help predict product preferences. Deterministic ranking can organise products consistently. Explanations can make recommendations easier to understand. Virtual try-on can make the experience more tangible.

But the real test is not whether the system can produce a percentage or generate an explanation.

It is whether the resulting recommendations are useful to shoppers and valuable to the businesses serving them.

That requires evaluating recommendation relevance and measuring real shopping outcomes rather than assuming that a higher compatibility score automatically produces a purchase.

What's Next?

My next priorities are to:

  • Strengthen Gemini/ADK orchestration and tool integration.
  • Expand the connection between Qloo-derived evidence and garment ranking.
  • Improve compatibility explanations and preference refinement.
  • Evaluate recommendations against a baseline without cultural intelligence.
  • Test how shoppers respond to different product-ranking strategies.
  • Explore integration with existing e-commerce catalogues and storefronts.
  • Measure product engagement, add-to-cart rates, and conversion performance.
  • Investigate whether personalised discovery can help reduce unsuccessful purchases and improve customer confidence.

The central evaluation question is:

Does ranking products around a shopper's point of view create a better shopping experience and measurable value for e-commerce businesses?

Why POV?

Fashion is personal, but online product catalogues are often designed around general categories, popularity, or fixed sorting rules.

Two shoppers can visit the same store, see the same thousands of garments, and want completely different things.

POV Fit Engine explores a different approach: combine cultural intelligence, machine learning, explainable ranking, agentic AI, and virtual try-on to help each shopper discover products that better reflect their individual preferences.

For shoppers, the ambition is to spend less time searching and more time discovering.

For e-commerce businesses, the ambition is to turn large product catalogues into more relevant, personalised shopping experiences.

Don't just add it to your cart. Fit it with POV.

Built With

Share this project:

Updates

Submission history