Empangeni Chat: Connecting Communities

Empangeni-LAN-Chat: Fostering inclusivity and understanding through accessible communication technology

Comment

Empangeni Chat: Connecting Communities

Inspiration

Growing up in Empangeni, I witnessed how language barriers could hinder communication and create misunderstandings. This, coupled with the desire to empower women and promote inclusivity, inspired me to develop Empangeni Chat: Connecting Communities. This AI-powered tool aims to bridge communication gaps and foster a more connected and equitable community.

What it does

Empangeni Chat is a LAN chat application designed for inclusivity and accessibility. It provides:

Real-time translation: Seamlessly translates between Afrikaans, English, and Zulu, the dominant languages in our community. Voice support: Offers text-to-speech and speech-to-text capabilities, making the app accessible to those with visual impairments or literacy challenges. OCR for handwritten notes: Digitizes handwritten notes, enabling their inclusion in the chat and translation process.   Rephrasing and tone adjustment: Helps users craft messages that are clear, respectful, and culturally sensitive. Conversation analysis: Provides insights into communication patterns, helping users understand and improve their interactions.   Gemini Knowledge Graph: Provides access to a vast knowledge base that can explain complex concepts in an understandable way, perfect for educational assistance and broadening understanding

How we built it

We leveraged the power of cutting-edge AI technologies:

Gemini: For advanced language understanding, culturally nuanced translation, and rephrasing capabilities. Azure Cognitive Services: For accurate speech-to-text, text-to-speech, and OCR functionalities. Flask: To build a robust and scalable backend. HTML, CSS, and JavaScript: To create a user-friendly and accessible frontend. Vertex AI: Utilized for cost-effective and time-efficient TPU acceleration for fine-tuning language models.

Challenges we ran into

Limited Afrikaans/Zulu proficiency in Azure LLMs: While Azure offers powerful language models, we found their proficiency in Afrikaans and Zulu to be less robust than needed for our Empangeni community. Langchain's lack of Gemini support: Integrating Gemini with other tools proved challenging due to Langchain's lack of direct support. Building custom pipelines required significant effort. Lack of cost-effective TPU acceleration on Azure: We require TPU acceleration to fine-tune our models. While Azure offers GPUs, we found Vertex AI's TPUs more cost-effective and time-efficient. GitHub Copilot's open-source friction: While Copilot can be helpful, we encountered instances where it introduced errors into our code, creating debugging challenges. Model Zoo limitations: We found the Azure model zoo to be less comprehensive compared to Vertex AI, making it challenging to find relevant starting points for our specific use case.

Accomplishments that we're proud of

Effective communication bridge: We built a tool that facilitates clear communication across language barriers. Culturally sensitive design: By incorporating Gemini's understanding of cultural nuances, we ensured respectful and accurate translations. User-friendly interface: The application is intuitive and accessible to individuals with diverse technical backgrounds.

What we learned

The power of AI for social good: We witnessed how AI can address real-world challenges and create positive social impact. The importance of user-centric design: Building a truly useful application requires a deep understanding of the target users and their needs. The value of collaboration: Working together allowed us to leverage each other's strengths and overcome challenges effectively.

What's next for Empangeni Chat: Connecting Communities

Expand language support: Add support for more languages spoken in South Africa. Enhance accessibility: Integrate features like text-to-Braille and sign language support. Develop a mobile app: Make the application accessible on smartphones and tablets. Explore offline functionality: Enable basic functionality even without an internet connection. We believe Empangeni Chat has the potential to transform communication within our community and beyond. We're excited to continue developing this project and making a positive impact on the lives of others.

Built With

Share this project:

Updates