SkinSight

Front Page
Uploading Clear Skin
Clear Skin Results
Uploading Bad Skin
Bad Skin Results

Inspiration

Every day, millions of people notice changes in their skin, from new lesions, discoloration, or unusual growths. While many are harmless, some can be early indicators of serious underlying conditions, including skin cancer and other systemic diseases. The skin often acts as the body’s first visible warning system, but interpreting these signals requires specialized expertise.

Globally, access to dermatological care remains limited. Long wait times, geographic barriers, and resource constraints mean that many individuals are left uncertain about whether a skin change is benign or something that requires urgent attention. This gap between observation and diagnosis inspired us to build SkinSight, a tool designed to bridge that gap by providing fast, accessible, and intelligent insights into skin lesions.

What it does

SkinSight is an AI-powered system that analyzes images of skin lesions and provides users with preliminary insights into their potential risk level. By simply uploading an image, users receive:

1. A classification of the lesion (e.g., benign vs. suspicious)
2. A confidence score indicating model certainty
3. A visual explanation highlighting the most relevant regions of the image
4. A clear, human-readable explanation with recommended next steps

Rather than acting as a diagnostic tool, SkinSight is designed as a decision-support system, helping users understand when a lesion may require professional evaluation and when it may be safe to monitor.

How we built it

SkinSight was developed as a modular AI pipeline combining computer vision, explainability techniques, and natural language generation.

At its core is a fine-tuned Vision Transformer (ViT) model trained on a dataset of over 10,000 dermoscopic images spanning a variety of lesion types. To improve robustness and generalization, we applied data augmentation techniques such as rotation, scaling, and color normalization, simulating real-world imaging variability. We also addressed class imbalance to ensure the model remained sensitive to high-risk conditions.

To enhance interpretability, we integrated Grad-CAM (Gradient-weighted Class Activation Mapping), allowing the system to generate heatmaps that highlight which regions of the lesion influenced the model’s prediction. This step is critical in building user trust and aligning AI outputs with clinical reasoning.

On top of the vision model, we built a language generation layer that translates technical outputs into accessible explanations. This layer is designed with safety in mind, avoiding definitive diagnoses, clearly communicating uncertainty, and guiding users toward appropriate next actions.

The entire system is structured as a pipeline:

1. Image ingestion and validation 
2. Preprocessing and normalization
3. Model inference
4. Explain ability mapping
5. Language generation
6. Output delivery

This modular design allows for scalability, future improvements, and integration with additional datasets or diagnostic categories.

Challenges we ran into

One of the primary challenges we faced was data quality and diversity. Skin lesion datasets can be biased toward certain skin tones, lighting conditions, or lesion types. Ensuring that our model generalizes well across diverse real-world scenarios required careful preprocessing and augmentation.

Another challenge was balancing accuracy with safety. In a medical-adjacent application, a highly confident but incorrect prediction can be more harmful than a cautious one. We addressed this by implementing confidence thresholds and fallback mechanisms to prevent overconfident outputs.

We also encountered system reliability issues, particularly around latency and resource constraints. Integrating multiple components, vision models, explainability tools, and language models, introduced potential points of failure, such as timeouts or token limits. Designing robust fallback systems was essential to ensure the user always receives meaningful feedback.

Finally, translating complex model outputs into clear and responsible language proved to be non-trivial. We had to carefully design prompts and templates to ensure outputs were informative without being misleading or alarmist.

Accomplishments that we're proud of

In just a short development window, we successfully built a fully functional, end-to-end AI system that combines multiple advanced techniques into a cohesive product.

We are particularly proud of:

1. Achieving strong classification performance with ~85% accuracy
2. Integrating explainability through Grad-CAM to enhance transparency
3. Designing a safe and user-friendly explanation layer
4. Building a modular architecture that supports scalability and future improvements
5. Implementing robust fail-safes, including confidence-based fallbacks and timeout handling

Most importantly, we created a system that is not only technically sound but also thoughtfully designed for real-world use, prioritizing user trust and safety.

What we learned

Through building SkinSight, we gained valuable insights into both the technical and ethical dimensions of AI in healthcare.

Technically, we deepened our understanding of:

1. Training and fine-tuning computer vision models
2. Handling imbalanced and sensitive datasets
3. Implementing explainable AI techniques
4. Designing resilient, multi-stage AI pipelines

Equally important were the lessons in responsible AI design. We learned that in healthcare contexts:

1. Transparency is just as important as accuracy
2. Communicating uncertainty is critical
3. Systems must be designed with failure in mind

We also learned the importance of human-centered design, ensuring that outputs are understandable, actionable, and aligned with user needs rather than purely technical metrics.

What's next for SkinSight

Looking forward, there are several exciting directions for expanding SkinSight.

First, we aim to improve model performance and coverage by incorporating larger and more diverse datasets, including broader skin tones and additional condition types. This will help reduce bias and improve generalization.

Second, we plan to enhance the system’s ability to detect a wider range of skin-related and systemic indicators, moving beyond lesion classification toward more comprehensive skin health analysis.

We also see potential in integrating SkinSight into clinical workflows, enabling collaboration with healthcare providers and supporting telemedicine platforms.

On the technical side, we aim to:

Optimize for real-time, on-device inference Improve explainability with more advanced visualization techniques Strengthen reliability under low-resource conditions

Ultimately, our vision is to make SkinSight a globally accessible tool that empowers individuals with timely, trustworthy insights, helping bridge the gap between early observation and informed medical action.