SkinPin

Inspiration

Skin conditions run in so many families, including some of ours, and watching family members struggle to get timely answers about rashes, lesions, and inflammatory conditions inspired us to build something that puts dermatology-level screening in everyone's pocket. Most people wait weeks for a dermatologist appointment for something that might be urgent. We wanted to change that, which is why we aimed to create a project tailored to the Catalyst for Care track at LA HACKS.

What it does

SkinPin is a fully on-device AI skin analysis app. You point your camera at any skin concern, and within seconds it classifies the condition across 30+ dermatological categories, from eczema and psoriasis to potentially malignant lesions, giving you a triage result: clear, monitor, see a doctor soon, or seek urgent care today. Everything runs locally on your iPhone. No photos ever leave your device.

How we built it

We built SkinPin with React Native (Expo) for the cross-platform UI and trained two custom TFLite models, a 16-class gate model that detects normal skin vs. any condition, and a 23-class disease classifier trained on the DermNet dataset. We integrated the ZETIC MLange SDK via a custom Swift native bridge for on-device neural inference, with TFLite as a fallback. The triage system runs a multi-agent pipeline locally with a vision agent, symptom agent, and consensus triage agent that combine image predictions with user-reported symptoms to produce a final recommendation.

Challenges we ran into

Getting a native Swift bridge to talk to React Native while using the ZETIC MLange SDK was the hardest part, the npm package was 9 months outdated so we built a custom RCT_EXTERN_MODULE Swift bridge from scratch. Training models that generalize across skin tones and lighting conditions was also tough with limited labeled data, and getting sub-second inference on a real device required aggressive image preprocessing optimization.

Accomplishments that we're proud of

• Fully on-device inference, zero network calls for the AI, complete privacy
• Custom Swift with React Native bridge for the ZETIC SDK
• Two-stage model pipeline (gate + disease classifier) that avoids false positives on normal skin
• Multi-agent triage system that combines visual and symptom signals locally
• Clean, production-quality UI that guides users to capture good-quality photos before analyzing

What we learned

Building real-time on-device ML on mobile is hard, as model format compatibility, native module bridging, and inference latency are all real constraints. We learned a lot about TFLite quantization, the ZETIC MLange framework, and how to design a proper fallback pipeline so the app always gives useful output even when one component fails.

What's next for SkinPin

• Make the body map history 3D so users can better track conditions across body zones
• Progression tracking with photo timelines per condition
• Integration with dermatologist referral networks for high-urgency cases
• Expanding the model to cover more rare conditions with better training data

Built With

• c
• eslint
• expo.io
• numpy
• pandas
• react-native
• ruby
• swift
• tensorflow
• tflite
• typescript
• zetic