Mint | Devpost

mint logo

Inspiration

Buying second-hand is sustainable and affordable, but online marketplaces lack trust. Photos can hide defects, listings can misrepresent condition, and buyers have little way to verify an item's identity or condition at handover.

We built Mint — Proof, Not Photos to bring 3D computer vision and AI to second-hand commerce.

What It Does

Mint transforms a roughly 25-second iPhone video into a photorealistic 3D Gaussian splat. AI agents identify the product, assess visible condition, retrieve comparable Canadian listings, and generate a listing with an explainable price.

At handover, buyers rescan the item. Mint geometrically aligns the two 3D representations and compares them to determine whether the physical item matches the original scan.

How We Built It

3D Reconstruction: Python, PyTorch, SAM 2, VGGT, and gsplat for video segmentation, multi-view reconstruction, and Gaussian splatting.
GPU Infrastructure: Serverless NVIDIA H100 acceleration via Modal for compute-intensive reconstruction workloads.
Computer Vision: ARKit and Vision for capture tracking; PCA and Iterative Closest Point (ICP) for geometric scan alignment.
AI Agents: Gemini, Google Lens/Shopping via Serper, Jev, and Groq for product identification, condition assessment, pricing evidence, and verification explanations.
Backend & APIs: Python, FastAPI, Supabase, Postgres, Realtime, and Edge Functions.
Interactive 3D Marketplace: Next.js, React, TypeScript, Three.js, and Spark.
Native Scanning App: Swift, SwiftUI, and ARKit.

We combine learned reconstruction models with geometric registration and deterministic decision logic, keeping price calculations and verification thresholds explainable rather than delegating everything to an LLM.

Challenges We Ran Into

Robust 3D Capture: Developing a repeatable scanning workflow and tracking item rotation across video frames.
Geometric Registration: Aligning independently reconstructed, potentially symmetric objects despite differences in orientation.
GPU Latency: Managing H100 cold starts and reconstruction time through caching, parallel stages, and infrastructure fallbacks.
Reliable AI Outputs: Validating product identities and pricing evidence instead of trusting unsupported model predictions.

Accomplishments That We're Proud Of

Built an end-to-end system spanning mobile capture, GPU-accelerated 3D reconstruction, AI-generated listings, and buyer-side scan verification.
Tracked 565/566 frames, achieving a 0.5° median angular error in our recording test.
Produced compact 2.3–3.0 MB Gaussian splats.
Published reconstructed splats in approximately 40–66 seconds after upload.
Built hundreds of automated tests across the marketplace, listing pipeline, and verification system.

What We Learned

Combining learned 3D reconstruction with geometric methods provides a stronger foundation than relying on image similarity alone.
Structured outputs, evidence validation, and deterministic logic make AI-powered systems more predictable.
GPU acceleration is only part of the performance problem; cold starts, data movement, and pipeline orchestration matter too.
Verification thresholds need empirical evaluation against real matching and non-matching objects.