Responder

logo

Inspiration

Despite the importance of emergency cell services (i.e., calling the police, ambulance, etc.), the current process for contacting these services has massive drawbacks in accessibility and versatility. Callers who are speech-impaired have always been inherently disadvantaged in making emergency calls due to the nature of voice-calling, and the requirement to audibly "speak" to a first responder has empirically hindered effective calls seeking help (https://www.cnn.com/2019/11/23/us/911-pizza-domestic-abuse-call/index.html). While small-scale and region-specific solutions exist for text-based emergency services, there is a universal need for a highly adaptable and simple application that solves these issues across a wide range of use cases, opening the door to a wide range of enhanced rescue functionality.

What it does

Responder is an iOS application that takes advantage of T-Mobile's YNA API to make remote calls with live transcripting and text-to-speech capability. It essentially wraps calls that are made by YNA endpoints and possesses the additional capability to process the audio being sent and the audio being received in the WebSocket connection. Callers have messaging capability, and anything they type will be converted to audio and fed through the connection. They can also now communicate with first responders visually, as audio sent back can be processed via local device speech-to-text, thus enabling live transcript functionality.

How we built it

The iOS application was built with Swift, and since this is a T-Mobile track project, we also used the "Your Number Anywhere" API.

Challenges we ran into

Ramping up to "Your Number Anywhere" was difficult, there were many moving parts with authentication and establishing the WebSocket connection within the frontend application. It was also difficult to conceptualize and implement a "live" backend to help with audio processing, because we wanted to avoid re-inventing the wheel with establishing our own web RTC in the backend, and Firebase cloud functions are susceptible to cold starts, which hinders its ability to continuously listen for connections. In the end, properly scoping our project and narrowing down the system design was a huge challenge.

Accomplishments that we're proud of

We're proud of being able to understand and work within the constraints of "Your Number Anywhere" in such a short period of time and conceptualizing the integration of the API into our own service. In general, we're really proud of being able to conceptualize an application that, if genuinely implemented, we believe could actually impact the way that emergency services are contacted today. There was a lot of consideration and comprehensive brainstorming that went into producing the idea.

What we learned

In working extensively with "Your Number Anywhere", we learned a lot about establishing WebSocket connections, authenticating using various access tokens, and deriving socket URLs (the basics). In addition, we also gained valuable refreshers on general API development principles.

What's next for Responder

Considering this project was just an interactive mockup of our eventual application, our most immediate goal is to invoke YNA endpoints into a functioning application and make real calls to emergency responders. The next step would be to integrate local device text-to-speech and speech-to-text functionality once calls can be established. Finally, to accommodate our locale-specific communications, we aim to eventually support real-time translation and regional hotline mappings (to automatically call the correct responder numbers in various locations).