AfyaYangu

Project Icon

Inspiration

The inspiration for this project came from the pressing need to improve access to healthcare information for marginalized communities, particularly those that don't have easy access to digital resources or face language barriers. Many communities in Africa, including rural areas, lack access to quality health information in their native languages. Bridging this gap using technology seemed like an impactful way to enhance their well-being. Additionally, I had a personal goal to see how technology can be used to improve the health lifestyle of individuals

What it does

Translation of health information to native languages. This is to improve access to health information to members of marginalized communities. (This was done to provide translated sources of information for the chatbot)
Provision of a smart and integrated chat assistant on the website.
Prescription Information Extraction: Automatic extraction of information on prescriptions such as dosage and medication.
A SMS gateway to enable individuals without smartphones/laptops access to the information. (This feature cannot be deployed to production due to high costs involved.)
Use of Computer Vision to accurately track exercises performed by an individual.

How we built it

The project was built using a combination of modern web technologies and AI tools. We used a multi-layered approach, where each feature addressed a specific need of the target community:

Health articles were manually translated by individuals who speak the native languages. This was due to the difficulty in finding translated articles, along with the need to provide accurately translated information.
The chat assistant was powered by a speech-to-text module and a text-to-speech module, ensuring that users with limited literacy could still interact with the system.
The SMS gateway utilized a cloud-based messaging API to allow for text-based information dissemination to users without smartphones. We utilized Africa's Talking API to connect the workflow to users.
The Computer Vision feature was developed using MediaPipe and TensorFlow, tracking physical exercises with precision and feedback in real-time.

Challenges we ran into

Lack of information in native languages: There are almost no health articles written in Kikuyu. To cope with this problem, I had to scour through hours of YouTube videos from local television stations (e.g., Inooro TV) to gain native translations.
Lack of resources for native languages: Speech models are predominantly trained for English speakers. However, for African languages like Swahili, there is a lack of open-source models. The only significant one I found was Facebook's mms-ttw-sw for Swahili, which is still limited in scope compared to resources for English or other widely spoken languages.

Accomplishments that we're proud of

Successfully creating a functional system that translates health information into native languages, making it more accessible to marginalized communities.
Integrating a chat assistant that provides smart health advice, reducing barriers to information.
Building the Computer Vision component to monitor exercises with a high degree of accuracy, giving users real-time feedback on their movements.
Implementing the SMS gateway to ensure users without access to advanced devices can still obtain critical health information, despite the challenges of full deployment due to cost.

What we learned

We learned that while technology can solve many problems, addressing issues in marginalized communities often requires an understanding of local contexts, languages, and cultures. Additionally, the gap between resource availability in major languages versus native African languages is significant, especially in the AI and healthcare fields. We also learned how to integrate multiple technological solutions (NLP, computer vision, SMS gateways) into a cohesive platform that addresses the needs of people in resource-limited settings.

What's next for AfyaYangu

Expanding the language base to include more native African languages, ensuring that more communities can access health information in their own tongues.
Seeking partnerships with telecom companies to reduce the costs associated with the SMS gateway, allowing us to launch the feature at scale.
Improving the AI assistant’s ability to provide culturally relevant health information by incorporating feedback from local healthcare professionals.
Enhancing the Computer Vision exercise tracker to cover more types of exercises and provide detailed health and fitness insights based on user behavior.