Skeletons - Rohde&Schwarz

Inspiration

We got inspired by the possibility to take the idea from the immobile QAR50 "on the road" and identify possible use-cases in the real-world while using a mobile, handy microwave image technology device. To us, the social impact of a fast and portable medical imaging technology posed the most fascinating challenge at the HackaaTUM!

What it does

Our project is an early PoC showing the possible use-case for ambulances and emergency doctors to detect possible injuries and fractures through clothing (shirt, trousers, gloves, helmet). We process the image of a palm taken with the QAR50 and can detect "misplaced" / possible injured fingers. This way, medical personell is able to get quick access to information about the condition of the patients limbs.

How we built it

The project is solely built with Python. Preprocessing is mostly done with OpenCV. Labeling with the open source tool LabelImg. For training of the bounding box regression to locate possible injuries we used PyTorch and a Binary Image Classifier (derivative of a Unet encoder art) to classify non-injured vs. misplaced.

Challenges we ran into

Extracting good pre-processed data from the raw microwave data. We encountered problems using Canny edge detection, ConvexHull, etc. to get a reasonable binary (skeleton) representation of the microwave imagery. Besides that, our first idea - mapping palms to skeleton representations did not work out as planned. Another important challenge is the small dataset to train our classification and bounding box model.

Accomplishments that we're proud of

Being able to finish the hackathon with an interesting use-case and providing a PoC from start (taking a microwave palm image) to end (classification and bounding box).