When in emergency situations which require first aid and swift care, the proper treatment may not be known or panic may make treatment unorganized. Also, certain communities may not have access to proper first aid training, making them more vulnerable when accidents happen. For example, an American Heart Association study showed that while 65% of white Americans are trained in CPR only 13% of both Black and Hispanic Americans are trained in CPR. This demonstrates the disparity in first-aid preparedness for communities of color in the United States that can risk the safety and health of individuals in dire need. Our app's purpose is to correct this by providing free interactive AR tutorials to treating injuries. These tutorials are meant to be used both in emergency situations, and as first aid practice. Most importantly, our app streamlines the first-aid process by consolidating easy step-by-step first aid information into one source that can easily be accessed, saving precious time in an emergency.
What it does
Using AR modeling, Medic.AR can demonstrate and teach people the proper way to act in first aid situations along with calling 911 in case of serious injuries that require expertise outside of the current bystanders. Medic.AR allows the user to input the current injury in question, identify the injury using image segmentation, and ultimately use AR modeling to demonstrate the steps needed to care for the injury.
How we built it
We used the language Swift in Xcode to code the app and we used the echoAR service to provide the AR modeling and 3D models.
Challenges we ran into
Our team was not very familiar with Swift or Xcode so we spent more time absorbing and learning information at the beginning rather than actual development. Along with this, we had trouble getting the echoAR technology to properly display models at times because we also wanted to segment the injured area simultaneously. However, we eventually solved this problem by dividing this process into multiple parts.
Accomplishments that we're proud of
We're most proud of the fact that we finished developing an app in a language we're not familiar with. We are also proud of learning how to use echoAR to create an interactive first-aid tutorial, and using its image segmentation feature. Although we were unable to implement it, we also made progress to create an ML model in TensorFlow to identify different body parts in an image using transfer learning. We even created our own dataset for training.
What we learned
We learned a significant amount about the format and organization of Xcode along with some intricacies of Swift. We also learned how to implement new technologies such as EchoAR.
What's next for Medic.AR
We hope to expand the amount of identifiable injuries along with the detail and accuracy of the AR modeling instructions.