EMSnap

EMSnap is an app that speeds EMS response by letting users take pictures. It uses deep learning to generate a prompt and send location data for instant EMS requests.

Comment

Inspiration

One of our members, Josh, was introduced to the Hackathon last Monday. On the walk home, he passed by a small accident on Washington where the police and EMS services just arrived in time to "save the day"! This made Josh think about if there were ways to speed up the response time for different types of emergencies and that's how we got our idea.

What it does

Open the app, take a photo of a serious incident, press send to send the photo for deep learning processing and identification. After a couple of seconds our DeepSeek agent will generate an EMS prompt for you to view and allow you to regenerate the prompt if you want, if not, press the send to EMS button on the bottom right to perform a text-to-911 call. After doing so the 911 operators will be contacted successfully, and nothing else is needed on your end!

How we built it

React-Native: Open source UI software framework developed by Meta. It is commonly used to develop apps for Android, IOS, macOS, and Windows. Expo: Expo/EAS is an open-source framework built on top of React Native that simplifies and streamlines the development of cross-platform mobile applications using JavaScript/TypeScript. TensorFlow/Keras: Tensorflow Keras is the high-level API for building and training deep neural networks for TensorFlow efficently and seemlessly. NodeJS/TypeScript: TypeScript is a superset of Javascript that introduces static typing and is the main programming language of this project. Using Node.js as a runtime environment and node package manager for managing third-party dependencies. DeepSeek: We utilized Deepseek's extremely fast and efficient LLM's to generate EMS messages on the fly. Python: Main programming language used to develop our own deep learning model in the Jupyter environment. (Code for this model is also provided in the folder dl_model for you to view!)

Challenges we ran into

Most of our challenges came from the mobile app development - from deciding on the architecture, to creating the UI/UX for the application, to finally integrating the deep learning model into our mobile app EMSnap.

Accomplishments that we're proud of

The state transition system of our mobile app development (to make execution based on states way easier). In addition, how the mobile app actually came out was quite good in our opinion. Finally, the biggest accomplishment we are proud of is the deep learning model itself and how well it performs on the data trained/tested with a validation/test accuracy close to 98%.

What we learned

Definitely not to do Mobile App Development!!!

Jokes aside, we learned a lot about architecture for mobile app and deep learning. Definitely a lot about the more specific aspects of mobile app (and really how hard it is).

What's next for EMSnap

Rate limit SMS's (shouldn't be allowed to spam EMS) Send SMS messages instantly, instead of opening iMessage and then sending. Implement greater error checking (not allowing users to press buttons while prompt generation or sending messages is currently happening). Adding your own little bits to the EMS prompt generated/selecting a topic associated when you submit a picture to the app.

Built With

Share this project:

Updates