Inspiration
Our inspiration for building Anatotrack was the HoloRay Motion-tracked medical annotations. We have always been interested in the medical field, computer, and software engineering. It is inspiring to be able to create things that can help healthcare professionals, using skills that we have in engineering.
What it does
AnatoTrack is a real-time motion tracking annotation for medical videos such as ultrasound, echocardiography, laparoscopy and POCUS. Unlike static annotations, AnatoTrack allows the annotations to remain anchored to the initial anatomy that it was drawn on even if it or the camera moves.
How we built it
We built AnatoTrack using Python and OpenCV's library. We loaded the medical videos and captured users annotations on a specific frame. Then we applied optical flow tracking so that the annotations moved with the anatomical features across the video in real time.
Challenges we ran into
One of the challenges we faced was occlusion, when a surgical tool temporarily blocked part of the anatomy, the annotation would drift and lose alignment and could sometimes follow the tool instead. Since we use optical flow to track the points, it relied on the visible features for consecutive frames. We tried to fix this by taking a snapshot the feature's details before occlusion and match it later, but the application works best when the anatomy stays visible
Accomplishments that we're proud of
We successfully built a real-time motion-tracked annotation that let users draw directly on the video and aligned it to the moving anatomy
What we learned
We gained experience with computer vision and learned how to implement libraries such as OpenCV to process video and perform motion tracking to create dynamic annotation.
What's next for AnatoTrack
We plan to implement more robust tracking methods which can handle occlusions and cases when the specific anatomy leaves and reenters the frame with more accuracy.
Log in or sign up for Devpost to join the conversation.