Rebuild

• 

  Rebuild Project Website

Inspiration

For Fintectual's Hack the Heat 2022 hackathon, we knew we wanted to implement Machine Learning into our project and our common interest was healthcare so after further discussion, we decided to make this series of apps that appeal to the both of us catered towards rehabilitation patients.

What it does

Rebuild is a brand for rehabilitation patients to access rehabilitation therapy while receiving live feedback 24/7.

How we built it

We first brainstormed an idea as to what problem we wanted to solve. We found a common ground of healthcare and decided upon rehabilitation, in which we then pooled all our ideas and began wireframing. We knew we wanted to utilize machine learning in our project and so we decided on using the action classifier and object classifier to identify different physical therapies (hand and body) for rehabilitation patients. Gathering all the videos and images took a while considering there are only 2 people in our team, but once we finished we trained the machine through CreateML. Throughout this process, we utilized Apple’s “Recognizing objects in live capture” and “Detecting human actions in a live video feed” sample code and adjusted it to integrate our own machine learning models into iOS apps and StackOverflow to help us solve our problems. Our final steps were to make the UI design of the app on Xcode.

Challenges we ran into

We initially used feature points to annotate our images, however, the JSON file could not convert to CreateML (which was the machine learning program we were using) which is why we had to switch to annotating with shapes.

Accomplishments that we're proud of

We started this project full of inspiration and ideas, and we had a lot we wanted to implement into our project. We are proud of the fact that we were able to complete all the functions originally suggested for our app, especially during the small timeline given. We are also very proud of how high our accuracy is for both of our apps. Overall, we are very proud of how hard we worked over the course of this weekend!

What we learned

This project taught us how to use 2 types of machine learning utilized in this app: object classification and action classification. Prior to this hackathon, we had minimal knowledge on the topic of machine learning, and this experience was a learning curve for us. We also learned how to efficiently work as a team and communicate difficulties.

What's next for Rebuild

Rebuild has a lot of potential ahead! We hope to involve physical therapists through adding the function of goal setting and enabling the therapists to set goals for their patients and track their progress. We also hope to branch out into more types of physical and mental therapy apps utilizing more types of machine learning!

Built With

• createml
• figma
• plainsight
• stack-overflow
• swift
• xcode