UniHear

A community-run platform that translates news into multiple languages and ASL, making global stories accessible to everyone, regardless of language or hearing ability.

Comment

Building a Multilingual + ASL News Translator: A Story of Accessibility and Innovation

What Inspired Us

Our inspiration came from a simple, personal observation. Whenever my grandparents visited us, they would always prefer reading and listening to news in their native language, even though they understood English perfectly well. The familiarity of tone, rhythm, and emotion in their own language made the experience more human.

Similarly, when we interacted with members of the Deaf community, we noticed a parallel challenge. Many individuals who rely on ASL (American Sign Language) often find traditional video captions or text summaries limiting.

This realization sparked our inspiration:
What if there were a platform where everyone (no matter their language or hearing ability) could access news in the format that felt most natural to them?

What We Built

We developed a multilingual and ASL translator for YouTube videos and online articles.
Our platform converts any content into:

  • ASL video translations,
  • Multilingual text summaries, and
  • Audio summaries in different languages.

We envisioned it as a community-driven platform, where people can upload or share local news stories. With an integrated Gemini-powered profanity checker, the content remains safe and inclusive for all audiences.

The goal: a global, multicultural news hub (like a “Wiki for the world’s voices”) that prioritizes accessibility and community engagement.

How We Built It

We started development in VS Code, managing collaboration through GitHub for commits, pulls, and version control. Initially, we tried Roo Code to accelerate our workflow, but soon realized the limitations of auto-generated code.

After spending five hours debugging errors from Roo Code’s output, we decided to pivot. We rewrote the entire backend ourselves , then refined and optimized it with help from Claude and ChatGPT. Also, we pivoted from Postgres to a JSON storage system.

This mix of human intuition and AI assistance ultimately gave us a robust, working prototype (better than solely AI)

What We Learned

The most valuable lesson we learned:

AI is a tool, not a developer.

While AI can accelerate ideation and debugging, relying solely on it can slow you down.

We also deepened our understanding of:

  • Cross-cultural accessibility,
  • Multimodal translation workflows, and
  • The real meaning of inclusive technology.

Challenges We Faced

  1. Debugging AI-generated code – our initial AI-generated backend had dependency issues and type errors that took hours to resolve.
  2. ASL video generation – balancing accuracy with natural expression required extensive testing.
  3. Multilingual summarization – ensuring summaries maintained tone and emotion across languages was complex.

Reflection

This project taught us one thing, patience. It was not easy going through the same file 20 times, and still not being able to spot the error but we persisted, and everything fell into place.

In mathematical terms, if

$$ A = \text{Accessibility}, \quad L = \text{Language}, \quad C = \text{Community}, $$

then our vision can be expressed as:

$$ \text{Inclusive Technology} = A + L + C $$

Share this project:

Updates