SURGEONSIGHT

Inspiration

Surgical training depends heavily on expert observation and feedback, but access to consistent, high-quality coaching is limited by time, availability, and cost. Trainees often receive delayed or overly general comments that are difficult to translate into concrete improvements. We were inspired by the gap between how experienced surgeons actually think and teach during procedures and how feedback is typically delivered in training settings. SurgeonSight was built to help bridge that gap by making structured, expert-style feedback more accessible through video analysis.

What the project does

SurgeonSight analyzes surgical training videos using computer vision and AI to extract meaningful technical information from visual data. It converts raw video into concise observation logs, high-level summaries, and key takeaways that reflect what mattered during a procedure. Rather than narrating every action, the system focuses on delivering high-signal feedback at a controlled pace so trainees can review, reflect, and improve without being overwhelmed.

How we built it

We built SurgeonSight as a modular system that separates video handling, analysis, and feedback presentation. Videos are processed through an AI analysis pipeline that interprets visual cues such as instrument usage and procedural flow. The resulting outputs are post-processed to reduce repetition, enforce consistency, and adapt feedback density to the length and context of the video. The frontend was designed to clearly present insights while ensuring analysis only appears once valid results are available. The backend holds our CV model and logic that accurately tracks the user's hands and positioning, comparing the speed and angles to the original video and provides critical feedback.

Challenges we faced

One of the main challenges was managing noisy or repetitive AI output and transforming it into feedback that feels deliberate and expert-driven. We also had to avoid hallucinations, especially in simulated training scenarios, by constraining outputs to what could reasonably be inferred from the video. Another challenge was modeling and programming the CV to accurately track and monitor hand positions and movements. This was an incredible barrier we overcame as a team and enabled our project to implement accurate live feedback and tracking.

What we learned

We learned that effective educational feedback is as much about filtering and structure as it is about analysis. Clear constraints, deterministic behavior, and thoughtful presentation are essential for building trust in AI-assisted tools, especially in medical contexts. Most importantly, we learned that AI can meaningfully support surgical education when it prioritizes clarity, restraint, and alignment with how experts actually teach.