SnapPatch

Logo for our project
Opening page
Front page

Inspiration

We realized that wound care today is slow, expensive, and often not personalized. Custom splints typically require specialist intervention and hours of manual work. We imagined a faster, smarter alternative—what if a simple camera scan could automatically generate a custom splint in real-time? SnapPatch was created to turn that idea into reality.

What it does

SnapPatch lets users scan an injury, detects wound landmarks instantly using AI, and generates a personalized 3D mesh splint ready for recovery. It also allows users to upload rescans and track their healing progress visually.

How we built it

We built the frontend with React, TailwindCSS, Zustand, and react-webcam. The backend uses FastAPI with OpenCV, MediaPipe, and TensorFlow for detection. STL mesh files are generated with OpenSCAD and trimesh, and everything is connected through Firebase Firestore. The platform is deployed using Vercel and Render.

Challenges we ran into

From the very first prototype to our final demo, we battled a host of integration headaches: coaxing MediaPipe and a remote SAM model into a reliable async workflow, wrestling with LibreSSL’s SSL handshake failures when connecting to MongoDB Atlas (forcing us to fall back to in-memory storage and add certifi-based TLS configuration), and stubbing out Auth0 locally so we could build and test protected endpoints without waiting on a fully configured tenant. We also spent hours chasing down subtle fastAPI quirks—properly parsing multi-part uploads (especially filenames with spaces), tuning CORS rules, and reconciling synchronous PyMongo calls inside an async FastAPI app—while squeezing our CV inference and STL-generation steps under tight performance budgets. All of that under the clock of a live hackathon taught us a ton about end-to-end resilience and deployment readiness.

Accomplishments that we're proud of

In just one intense day, we built out a fully functional SnapPatch prototype end-to-end. On the backend, we stood up a FastAPI server with endpoints for injury verification (using a custom MobileNet V2 TFLite classifier), wound segmentation (leveraging MediaPipe and the SAM model), recovery‐timeline recommendations, 3D STL generation with trimesh, OctoPrint integration mock-printing, IoU‐based re-scan comparisons, secure session and share management via MongoDB Atlas (with in-memory fallbacks), JWT-protected routes powered by Auth0, and GPT-4–driven personalized recovery advice. We wrote and passed pytest suites for health, verify, segment, and timeline, and deployed the API locally. Meanwhile, the frontend team integrated React with TailwindCSS, react-webcam for live capture, react-three-fiber for 3D previews, Chart.js for healing timelines, and Auth0 for authentication, tying it all together into a smooth user flow from camera scan to mesh preview.

What we learned

Over the course of the hackathon, we deep-dived into a swath of modern tooling and best practices. We mastered deploying asynchronous Python services with FastAPI and Uvicorn, handling CORS and multipart file uploads, and securing endpoints with JWTs from Auth0. We learned to stitch together computer‐vision pipelines—balancing on-device TFLite inference, MediaPipe landmark detection, and Replicate’s SAM API—and convert masks into printable meshes with trimesh and OpenSCAD concepts. On the frontend, we navigated React state management, integrated real‐time camera feeds, and visualized STL meshes in browser. We also sharpened our MongoDB Atlas usage with Motor and certifi for SSL, practiced writing end-to-end tests, and gained experience with GPT-4 prompt design for domain-specific advice generation.

What's next for SnapPatch

Looking ahead, we plan to harden SnapPatch for real-world use: productionize Auth0 flows, secure our MongoDB deployment, and swap in a persistent object-storage solution (e.g., S3) for STL hosting. We’ll optimize segmentation speed—profiling our SAM pipeline and exploring lightweight on-device alternatives—as well as refine our severity classifier with more training data. On the frontend, we aim to enhance the UI with guided scan feedback, mobile responsiveness, and in-app mesh customization controls. Finally, we’ll initiate user testing with healthcare partners, iterate on UX based on clinician feedback, and explore direct OctoPrint integration for seamless at-home printing of our custom wound-mesh splints.