We were inspired to build this project by the increasing dis-compforatablility of therapist and doctors of not able to treat their patients especially children and teens because of the current lock down and COVID-19 safety measures. Thousands of children undergoing issue after accidents, brain surgery, Parkinson's disease , etc.. are stranded with no help to continue their recovery exercises. We found an opportunity for making something for the future. After some research we found that gaming are the most effective way of improving children and teens recovery as it makes the process fun and enjoyable.This is built in their subconscious mind and gaming makes them more inclined towards recovering faster than usual doctor visited training. Thus, We wanted to build a gaming system that has some hardware at users(patients) end which can be used by the patient for gaming. The intelligent iot device gets the data realtime and helps the patient play games. This data can be further collected and provided to the doctor/therapist for personal analysis and thus therapist can analyze the timely recovery of patients get all health related data in this side using an app or web. This can help him access the child more closely by sitting remote areas and during restrictions like we have now. This also enables doctors across the world to treat patients and help improving the medical network.
What it does
We have an iot device at the patients end. The patient wears this during the teraphy time in his hands. The patient can login to the system using his email and password. Then either go to instructors by learn button where the child can learn diffrent exercises or can click play option. We catogorized diffrent excercies in diffrent levels for fun way of interaction for the patients. For demo purpose we have only used one level, which we plan to expand to diffrent levels and add time based features. Now the user can do the exercies and the iot device will capture the data and send it in a csv format, which we change into json and is parsed to a dictionary in our system and hence our software can get the track of movements made by the patient and help him with moving object, balancing ,etc.. Severe jumps, or level failure detected can be noted. Now when a patient perform this the data generated later which is added to firebase database. Our teraphist can basically take this data from any remote area nad analyze it which gives him perfect way of treating the patient by understanding deeply how the patient is improving and giving ideas about how to procced in future.
How I built it
We are 3 developers building this project - Rafi Rasheed started off building the hardware together to make them communicate the way we want. Rafi integrated NodeMcu - ESP8266 with MPU6050 - Sensor and got the track of actions performed by patients. Which he send to Siddharth for the game actions. Rafi used MicroPython a new language to him for many of initial works and also integrated some files in python due to lack of complete documentation in micropython. Siddharth worked completely on the game development. Siddharth had never done game development before and never used language GDScript or framework Godot. He studied it for a day and later buld the game for the next day. Siddharth also integrated the firebase auth with godot and send receiving data from patient to Anas. Anas worked completely on the UI/UX of the game and he made the android app for doctor/therapist, and integrated firebase backend to it. We also used EchoAR and integrated AR video innto gaming system that is like an instruction to the kids.
Challenges I ran into
Overall we faced a lot of challenges. The best challenge was we were using a language or trying to build a platform that we had never done in our life. Siddharth had never worked on game development and Rafi had never worked with micropython. Anas had new experience working with game design. Overall the new adaptation was a big challenge. Further we faced some challenges like authenticating firebase with a game development engine like Godot due to lack of any libraries. Integrating Augmented reality into gaming system.
Accomplishments that I'm proud of
We never thought we could make an entire new software without experience in a span on 2 days. We had planned this project just when the hack started , but at end we are proud we could learn tons of knowledge and add the stuff to our resume.
What's next for Tremor Therapy
For this demo we had taken the json and automated it for gameplay due to lack of time. We next plan to integrate realtime gameplay with video calling feature for therapist. Further we plan to integrate firebase MLKit so that we can have in depth data analysis and better decision making for the doctors. We also plan to make it open source so that tons of awesome developers can fork and contribute to make it better and everyone at any part of world can heal any other person.