I, Davin, was rock climbing one fateful Monday when I made a really dumb move and landed on my wrist from about eight feet high. The first doctor I saw told me that I wouldn't need surgery, but the bones were quite broken and I would still need a cast. However, at a later appointment a few days later, an orthopedic assistant looked at the x-rays and told me that the bones would likely heal in the wrong place, and that surgery would be strongly advised. I was quite upset by this news, but when I met with the lead orthopedic doctor, I was informed that my bones were, in fact, perfectly aligned and that I would be fine with just cast.
I happily go on for another week until I go in for my checkup, where, upon x-raying through the cast again, the bones were, in their opinion, quite misaligned, and that I would need surgery. The orthopedic doctor vetoed this, and suggested that they simply push the bones back into place. So there I stood, in some of the worst pain of my life, as doctors who constantly flip-flopped on whether I actually needed surgery shoved my bones back into place.
I'm not a doctor, and I don't claim to know better than them, but I think that the Broke Detector (tm) would help them make more objective and consistent decisions, hopefully sparing others the pain that I went through.
What it does
Broke Detector is a simple tool to help analyze the site of wrist injury. It helps doctors decide a) whether the bones are broken or not (which is admittedly very simple for humans, but not for computers) and b) provides tools such as a protractor to provide an objective measurement of how misaligned bones are. Broke Detector aims not to replace the orthopedist, but to aid them.
How we built it
The actual detection algorithm is the guts of this project (no machine learning involved). The process is as follows: Thresholding: Reasons for this will be listed in the next section, we did this using color histograms to determine the percentile of certain pixel values. Edge detection: We used a simple Söbel kernel (however, not the default opencv one) to handle edge detection, as it was simple to detect only horizontal edges. Hough transform: This is the guts of the algorithm, which decides where horizontal lines are in the X-Ray. It sounds dumb, but choosing the lowest isolated horizontal line almost always results in finding the break in the bone. Flask: We developed a locally hosted web application which supports uploading from the computer. The HTML formatted site displays the results of the Python calculations.
The desktop interface was written in Python as well, using the wxPython framework for the GUI.
Challenges we ran into
While we were very passionate about doing something involving the X-rays, there were a couple issues with our process. The common state-of-the-art solutions to a problem like this would be stuffing a bunch of data into a faster R-CNN and let it handle the detection. However, we lacked both the volume and consistency of data to make this a reality: many of the research papers that tackled this problem partnered with a medical office, which would provide them a lot of consistent and annotated images to work with. Because of the hodgepodge of different variables that differ with each image we ripped off of Google images, we had to use techniques like color thresholding, as the brightness of what was bone and what was background differed from image to image. Even now, the algorithm isn't quite perfect. It still struggles to detect fractures that are close to the end of the radius/ulna.
Another challenge with this project was that the GUI framework we went with, wxPython is not only slow but antiquated in terms of GUI design. Something as simple as drawing lines on an image proved to be an immense challenge of trudging through documentation to find the obscure function call that enables something that should be the default anyway.
Accomplishments that we're proud of
It works! Surprisingly. Certainly, I didn't expect to have this kind of success with a non-ML solution.
What we learned
ML isn't the end-all and be-all for computer vision. Don't use a slow Python GUI framework, just make a web-app instead. Opencv is fun.
What's next for Broke Detector
More features, of course! Hosting on a public domain, perhaps a database to track patients, and more tools to help doctors analyze the bones objectively. The algorithm is surprisingly robust, and could be applied to other kind of fractures as well.