HearMeSafe

A silent guardian when traditional safety falls short - discreetly records audio and sends alerts with your exact location to trusted contacts if you ever feel unsafe.

Comment

HearMeSafe: Unlocking a Safer Tomorrow for Everyone

Use case

HearMeSafe is designed as a silent guardian for those moments when traditional safety measures, like panic buttons, fall short - especially when your hands aren't free, but your voice is. The app uses an audio recording feature that, when activated, captures any threats and automatically sends an alert to your designated contact. This alert includes your precise location and a description of the threat, ensuring your trusted friend can quickly respond and get you the help you need. It's like having a personal safety net, ready and listening whenever you feel vulnerable.

Whether you're walking home late or in an uncomfortable situation, HearMeSafe ensures you're never truly alone. The app transforms personal safety by empowering individuals to use their voice as a tool for protection, removing the need to physically engage with a device. Users can simply press the record audio button when going out, and the app will continue to record and send alerts. In this way, help is accessible even when your hands are occupied or you can't reach your phone. This is a crucial step toward a world where everyone can move through life with confidence, knowing that help is just a word away.

What our application does

With HearMeSafe, we're transforming personal safety by empowering individuals to use their voice as a tool for protection and remove the need to physically engage with your device. The user can just press the record audio button when going out and the app will continue to record and send alerts. In this way the help is accessible even when your hands are occupied or you can’t reach your phone. It’s a crucial step toward a world where everyone can move through life with confidence, knowing that help is just a word away.

Architecture Overview

HearMeSafe currently operates with a streamlined authentication system that allows users to log in through a secure FastAPI backend hosted on Google App Engine. The system employs robust security measures, including password hashing and JWT token authentication, while storing user data efficiently in Google Memorystore.

Before using the app, users must complete an initial setup process that requires granting microphone and location permissions. During this setup, users input essential information including their safeword, personal email, and a trusted contact's email address who will receive alerts in case of emergency.

The app's core functionality revolves around continuous monitoring through 5-second audio fragments, using Google's YAMNet model via MediaPipe to intelligently filter and process only speech-related audio. Simultaneously, the app tracks the user's location every 5 seconds, integrating with maps to help trusted contacts navigate to the user's location if needed.

For threat detection, the system employs a sophisticated process where speech is converted to text using Google Cloud Speech-to-Text API. This text, along with audio classification labels and preset safeword, is analyzed by the Gemini API to determine threat levels on a binary scale (0 or 1).

When a threat is detected, the app initiates a careful alert protocol, starting with a 10-second window during which users can cancel the alert if needed. If the alert isn't cancelled, the system proceeds to notify the trusted contact with detailed information including the threat description and location data. Importantly, the alert system continues to function even if the phone is damaged or disabled, ensuring user safety remains uncompromised.

All these features work together to create a comprehensive personal safety system that prioritizes both user privacy and effective emergency response. The app maintains a careful balance between continuous monitoring and efficient resource use, while ensuring that help is readily available when needed.

Challenges & Accomplishments

Developing this user-activated safety app that is discreet and reliable presented unique challenges. One of the main technical hurdles was ensuring the app's recording and tracking features remained active when the phone is locked, as many devices restrict background activity. Balancing efficient audio processing with accuracy was also a challenge, as the team needed to filter out irrelevant audio without missing important details. Building a seamless experience that ensures user safety without sacrificing privacy required extensive testing and adjustments.

Built With

Share this project:

Updates