Hearth

User Flow
Landing Screen
Real-Time Translation Screen
Support Screen
Guided Prompts Screen
Conversation Transcript
Harmful Language Detection
Safety Alert: Potentially Harmful Language

Inspiration

Emily and Jarny work at the front desk of The Bloom Group, a low-barrier women’s shelter in Vancouver. When we called them, we expected to hear about challenges such as access to shelter housing, mental health counselling, or digital resources. Instead, the most immediate need they identified, and the first thing Emily mentioned when we asked, “What would make your job easier?”, was interpreters.

For women in shelters, communication is not just a matter of convenience. It is directly tied to safety and access to care. Many residents are navigating crisis situations such as domestic violence, housing instability, or displacement, often while managing emotional stress, trauma, or caring for children. In these moments, the ability to clearly express their needs and understand the support available to them is critical.

However, many shelters serve individuals from diverse linguistic and cultural backgrounds. Language barriers can make already difficult situations even more isolating. When communication breaks down, it affects everything from intake and safety planning to accessing resources and receiving emotional support.

Through our conversations with staff at The Bloom Group, we identified a clear and urgent need for a more reliable and accessible communication solution.

Shelter workers shared that:

Language barriers are a daily challenge, especially among residents from diverse backgrounds, including refugees and immigrants
Existing tools such as Google Translate are often unreliable in real-world scenarios
Many languages, particularly African and Middle Eastern languages, are poorly supported
Shelters often lack the resources to hire professional interpreters
In some cases, staff are unable to identify which language a resident is speaking

Incorrect Translation by Google Translate

Google Translate output showing how inaccurate translations can distort meaning in sensitive or context-dependent conversations, leading to confusion and miscommunication.

Google Translate interface highlighting limited language options, which can exclude low-resource and underserved language communities.

Beyond our interviews, data reinforces the scale of the issue:

Around 45% of shelter residents are non-native English speakers
About 87% have low to medium digital literacy

These constraints mean that most existing solutions are not designed for real shelter environments. When communication fails, people may struggle to access shelter spaces and essential services, mental health support becomes harder to deliver, and residents can feel less safe, less understood, and less empowered.

What It Does

Most translation tools were built for the world's dominant languages. Hearth was built for the ones left out.

Hearth is a full-stack mobile communication and translation tool designed for women's shelter residents and staff. It is built as a Progressive Web App (PWA) with Next.js, React, TypeScript, and CSS on the frontend and FastAPI and Python on the backend. When a resident speaks, OpenAI's Whisper, a multilingual speech recognition model that runs locally on-device, first detects the spoken language and then transcribes the speech into text. The transcribed text is then translated using Cohere's Tiny Aya, a family of multilingual translation models that also supports local deployment and is optimized for low-resource languages such as Amharic, Hausa, and Swahili. The translated text can be read aloud through the browser's Web Speech API, which can also support voices built directly into the device. Harmful language detection is run continously in the background, sending an SMS alert to staff through the Twilio API if harmful langauge is detected to support timely follow-up and de-escalation. Conversations can also be saved as transcripts in local storage, letting residents and staff revisit or delete past sessions directly on-device to help protect their privacy. Rather than acting as a one-way translator, Hearth functions as a low-friction communication layer that enables real-time, two-way conversations between shelter residents and staff.

Core Features

Real-time, two-way communication through a speech-to-text, translation, and text-to-speech pipeline
Automatic language detection, so conversations can begin even when the spoken language is unknown
Broad coverage of 70+ languages, including many underserved, low-resource languages
Four selectable translation models, each optimized for a different language group (Auto, Earth, Fire, and Water)
Flexible voice or text input, so residents and staff can choose what works best
Guided conversation prompts tailored to women's shelter environments and conversations
A low-friction, intuitive interface that requires little to no training

Designed for Safety and Privacy

Built for shared-device use in low-resource shelter environments
Speech-to-text, translation, and text-to-speech models and services can all support local deployment
No account creation required and transcripts are stored on-device to reduce barriers and protect privacy
Harmful language detection that alerts shelter staff by SMS, supporting timely response and de-escalation

Beyond Translation: Thinking About Empowerment

Enabling residents to communicate their needs, concerns, and experiences more clearly
Supporting staff to ask more effective, context-aware questions
Creating space for meaningful communication beyond simple information exchange

How We Built It

We made deliberate technical decisions to align with the realities of shelter environments, where privacy, accessibility, and reliability are essential.

Tech Stack

Frontend: Next.js, React, TypeScript, and CSS, built as a Progressive Web App (PWA) to provide accessible, low-friction use across devices
Backend: FastAPI and Python, supporting efficient processing of real-time interactions
Speech-to-Text: Whisper by OpenAI, an AI transcription model used to automatically detect the spoken language and transcribe speech to text
Translation: Tiny Aya by Cohere, an AI multilingual translation model optimized for low-resource languages
Text-to-Speech: Web Speech API, used to convert translated text to speech using built-in voices
SMS Alerts: Twilio API, used to send SMS alerts when harmful language is detected
Database: Local storage, used to retain conversation transcripts locally on-device

Challenges We Ran Into

Balancing privacy and functionality by selecting models and services that support local deployment wherever possible while maintaining reliable performance
Integrating multiple AI models across the speech-to-text, translation, and text-to-speech pipeline into a single, cohesive real-time conversational system
Ensuring speech-to-text, translation, and text-to-speech systems perform reliably across diverse languages
Designing for low digital literacy, ensuring the interface remains intuitive and requires minimal training

Accomplishments That We’re Proud Of

Built a privacy-first system that uses models and services that support local deployment, avoids account creation, and stores transcripts locally
Integrated multiple AI models and services into a unified, real-time user experience
Designed for real-world edge cases often overlooked by existing tools, including low digital literacy, limited context, and diverse language needs
Made intentional product and technical decisions to ensure the solution is practical, accessible, and deployable in real women's shelter environments

What We Learned

Recognized that the challenges faced by women in shelters extend beyond access to information and is shaped by communication, trust, and day-to-day constraints that affect how they seek help and interact with staff
Explored new technologies and AI models, including Tiny Aya by Cohere and Whisper by OpenAI, applying them in a real-world system to support multilingual communication
Developed the ability to evaluate and combine AI models based on their strengths for translation, speech recognition, and multilingual support, ensuring they work effectively together
Strengthened collaboration skills by integrating, debugging, and refining multiple AI components as a team

What’s Next for Hearth

Several potential enhancements could extend the functionality of the application:

Conducting an ethics-focused study on deploying AI in sensitive, real-world environments
Fine-tuning models with trauma-informed language, women's shelter-specific terminology, and improved handling of sensitive topics
Expanding safety guardrails with a more context-aware approach to detecting harmful language
Supporting code-switching to better reflect real-world multilingual communication
Introducing new interaction modes, including one-to-many "townhall" and broadcast-style communication
Continuing to test directly with women's shelters like The Bloom Group to gather real user feedback
Improving deployment in low-connectivity environments to ensure consistent and reliable performance