SoleFit

Inspiration

Online shoe shopping has two unsolved problems at once: will it fit my foot shape, and what will it actually look like on me. Most tools solve one or the other sizing calculators guess a number, virtual try-on tools show a picture but nobody puts them together. That gap is where SoleFit started: a foot-shape-aware fit engine that also lets you see the shoe on yourself before you buy, instead of making you choose between confidence in fit and confidence in look.

What it does

SoleFit walks you through a short, guided flow: who it's for (so suggestions are gender-aware), what you're shopping for (Everyday, Sports & Training, or Medical & Comfort), your foot profile (toe shape, arch, width, self-reported or estimated from an optional photo scan calibrated against a standard ID card for real millimeter measurements), and your region. It scores every matching shoe against your profile and shows ranked results with a visible fit-percentage bar, with the single best match clearly highlighted. Then, for any recommended shoe, you can generate a photorealistic image of yourself actually wearing it, powered by Perfect Corp's YouCam AI Shoes Virtual Try-On API. You don't just get told "this fits" you see yourself in it.

How we built it

The app is Streamlit, restructured as a step-by-step wizard (one decision per screen, with a progress indicator) across focused modules: app.py for the interface, database.py for the shoe catalog, matching.py for the fit-scoring engine, vision.py for photo calibration, and vto.py for the YouCam integration.

Fit matching uses a weighted scoring algorithm that maps self-reported toe shape (Egyptian, Greek, Roman-Square) to the shoe toe-box shapes that fit each pattern best, blends in arch and width scores, and always surfaces ranked options instead of an empty page from strict filtering. Measurement calibration uses OpenCV to detect a standard ID or credit card in-frame as a real-world size reference, converting pixel measurements into actual millimeters instead of guessing. The catalog is a SQLite database of 26 shoes across three categories and three gender classifications, with prices computed from one USD base price per shoe across seven regional exchange rates. Virtual try-on integrates the YouCam AI Shoes API end to end: upload, async task creation, polling, and result display, with plain-language error messages instead of raw API errors when something goes wrong (e.g. "we couldn't detect a face in that photo" instead of a JSON error code).

Challenges we faced

The YouCam integration was the real challenge, and getting it working end-to-end took genuine, iterative debugging against the live API rather than working purely from documentation:

Wrong upload format, three times over. Our first attempt sent file metadata as JSON and got "files is required" back — the API wanted the actual file, not a description of it. Switching to multipart/form-data got a 415 rejecting that content type outright. Sending raw bytes directly got further but still failed. Each error was a real clue, and eventually cross-referencing Perfect Corp's own FAQ revealed the actual documented pattern: request an upload URL, then PUT the file to it with the request body needing a "files" array we'd initially left out entirely. Free image-hosting services blocking cloud servers. As a parallel path, we tried hosting user photos on public file-drop services (catbox.moe, tmpfiles.org, 0x0.st) to get a URL YouCam could fetch. Several of these actively block requests from cloud-hosted servers like the one our app runs on, to prevent abuse so we built a fallback chain trying multiple hosts in order, and ultimately added Cloudinary (built specifically for exactly this programmatic use case) as the most reliable option, with the free hosts as backup. Discovering a real product constraint, not a bug. After the upload pipeline started working, results started failing with error_no_face. Investigating showed every sample photo in YouCam's own Playground was a full-body shot with a visible face — never a feet-only crop. That told us this is intentional: the model needs a face to anchor a realistic full-body render. We adjusted our UI guidance accordingly rather than treating it as something to work around. A quiet rendering bug. Separately, HTML generated deep inside nested Python conditionals picked up enough leading whitespace that the browser's markdown renderer treated it as a code block instead of real HTML, so shoe cards displayed as raw tags instead of styled cards.A good reminder that "the app runs with no errors" isn't the same as "the app renders correctly."

Accomplishments that we're proud of

A fit-scoring engine that's actually useful even with an incomplete or imperfect foot profile, instead of failing silently.

Honest UX around measurement accuracy — the app visibly tells the user whether a measurement is calibrated or an estimate, rather than presenting a guess as fact.

A genuinely novel combination in this space: nobody else pairs foot-shape-aware fit scoring with photorealistic virtual try-on in one flow.

What we learned

A fit-scoring engine that's actually useful even with an incomplete foot profile, instead of failing silently. Honest UX around measurement accuracy and error handling the app tells the user what's actually happening, in plain language, rather than hiding behind a guess or a stack trace. And a genuinely working end-to-end integration with Perfect Corp's YouCam AI Shoes Virtual Try-On API, built through real API debugging rather than a simulated integration.

What's next for SoleFit

Expand the catalog with more built-in shoe photos so try-on works out of the box for more shoes without needing a manual upload. Add saved profiles so returning users don't re-enter their foot profile every visit. Explore a lightweight trained shape-classification model to replace the current heuristic-based estimation from photos.

Built With

• ai-vision
• cloudinary
• computervision
• perfect-corp
• python
• sqlite
• streamlit
• virtual-try-on
• youcam-api