BhashaAI

Inspiration

The inspiration for BhashaAI is deeply personal. While learning English myself, I faced many challenges—expensive platforms and the lack of personalized guidance. I often struggled to find a tool that could adapt to my unique learning needs, making the journey frustrating at times. During those moments, I used to imagine a platform that could help me learn English in a way that felt personalized, interactive, and accessible anytime, anywhere. I strongly believe that language learning should be available to everyone, regardless of their financial situation. Thanks to advancements like Chrome's built-in AI capabilities, creating a platform that offers free, offline language learning is now possible. With this in mind, I decided to build BhashaAI—a solution that brings AI-powered, personalized language learning to everyone, making it engaging, inclusive, and completely accessible.

What it does

• Personalized Learning: Users can learn English, Spanish, and French through AI-generated articles that are personalized based on their selected topics and interests. Articles are available in three difficulty levels—easy, medium, and hard—allowing learners to start at their current skill level. Additionally, the Custom Levels feature lets users fine-tune the difficulty even further, ensuring the content aligns perfectly with their unique learning goals and progress.
• Real-Time Conversations: Users can interact with AI for grammar lessons, personal discussions, or scenario-based sessions, tailored to their interests. Additionally, users can choose personalized open-ended conversations, where the AI generates discussion topics based on the interests selected by the user. This makes the conversations engaging and relevant, helping users practice real-life scenarios or explore topics they are passionate about.
• Vocabulary Building: A vocabulary-enhancing feature where users can click on any word in an article to get definitions, synonyms, antonyms, translations, example sentences, and even listen to the word for correct pronunciation. Users can also save these words using the Saved Words feature, allowing them to revisit and revise their vocabulary later. This helps users retain new words effectively and strengthen their language skills over time.
• Offline Support: Learn anytime, anywhere, even without internet, leveraging Chrome's built-in AI capabilities.
• Language Accessibility: The platform supports three interface languages—English, Spanish, and French—allowing users to navigate the website in their preferred language while also learning any of the supported languages.
• Interactive Features: Includes a range of tools designed to reinforce learning:
  
  • Article Buddy Chatbot: A conversational assistant that helps users understand complex sentences, key phrases, or difficult concepts within an article. Users can ask the chatbot any question related to the article content, and it provides clear explanations, breaking down complex ideas into simpler terms. This feature enhances comprehension and provides instant, interactive support for learners.
  • Audio Playback: Users can listen to articles to improve their pronunciation and listening skills.
  • Article Summary: Users can view a brief, AI-generated summary of the article, allowing them to quickly grasp the main points without reading the entire content.
  • Quizzes and Summary Challenges: Users can test their understanding through quizzes and summary challenges that provide AI-driven feedback on their summaries.

How we built it

• Tech Stack:
  • Entirely built using React.js and Chrome's built-in AI APIs.
  • Used IndexedDB to persist user-generated data, such as article content, vocabulary, saved words, and generated quiz questions, ensuring seamless access and continuity of learning.
  • Used Local Storage to save user preferences, including website interface language, learning language, and user-selected interests, to provide a personalized and consistent experience.
• Prompts for AI Integration:
  Custom prompts were crafted for features like:
  • Article Generation
  • Real-Time Conversations
  • Article Buddy Chatbot
  • Quiz and Summarization Challenges
• Wikimedia Integration:
  
  • Used Wikimedia APIs to display relevant images in articles.
    
  • During article generation, we asked AI to provide relevant image keywords, which are then used to fetch images from Wikimedia.
• Multilingual Support:
  
  • The Prompts API does not natively support multiple languages. To address this, the following workaround was used:
    
    • Articles are first generated in English.
      
    • Used Translation APIs to translate articles into Spanish and French based on the user's learning language.
• APIs Used:
  • Prompts API: For generating articles, retrieving vocabulary, building the chatbot, handling conversations, and creating quizzes and challenges.
  • Translation API: For translating articles into Spanish and French for language learning and localizing the website interface.
  • Summarize API: For generating article summaries in the Article View Page.
  • Wikimedia API: For fetching relevant images based on keywords.

Challenges we ran into

• Lack of JSON Response Support:
  
  • Many of our scenarios required structured JSON responses, which the Chrome inbuilt AI doesn’t natively support.
    
  • Using prompt engineering, we achieved a 90% success rate.
    
  • For the remaining 10%, we implemented an exponential retry mechanism, where the AI call is retried up to five times if parsing the output into JSON fails.
    
  • This approach ensures consistent results even when initial attempts encounter issues.
• Word Count Accuracy:
  
  • Ensuring exact word count was challenging due to the limitations of the Chrome inbuilt AI.
    
  • While we optimized prompts to improve accuracy, further enhancement isn’t feasible as it depends on the AI model itself.
• Unexpected Errors in Prompts API:
  
  • Initially, we encountered issues like “not supported error” or “untested language error” while using the Prompts API.
    
  • Resolving these errors often required restarting Chrome or, in some cases, the entire system.
• Duplicate Article Generation:
  
  • When generating random articles, we initially encountered duplicates where the same title and content were regenerated.
    
  • To address this, we updated the prompt to include a list of previously generated article titles, instructing the AI not to repeat them.
    
  • This significantly improved accuracy, though duplicates still occur rarely.
• Token Limit in Chrome’s Inbuilt AI:
  
  • The AI enforces a token limit per session, which can be exhausted during prolonged interactions with the Article Buddy Chatbot or Real-Time Conversations.
    
  • Currently, when tokens are nearly depleted, we prompt users to create a new session, which unfortunately loses the context of previous sessions.
    
  • Future Plans:
    
    • Store user interactions in IndexedDB or local storage.
      
    • If tokens are finished, retrieve stored data, summarize it, and pass it as additional context to the new session.
      
    • While this approach will eventually fill tokens with context after multiple sessions, It is expected to significantly extend session length and improve user experience compared to the current implementation.

Accomplishments that we're proud of

When we found out about this hackathon just four weeks before the deadline, we weren’t sure if we could finish the project on time. We had so many features in mind, and the timeline seemed incredibly tight.

However, in just three weeks, we managed to design and develop a fully functional platform packed with innovative features. The final week was dedicated to documentation and video creation, all while managing a tight schedule to ensure everything was completed on time.

We’re proud of creating a platform that combines AI-powered personalization, offline accessibility, and multilingual support to make language learning easier and more engaging for everyone. From the seamless integration of Chrome’s built-in AI to interactive features like the Article Buddy Chatbot, quizzes, and real-time conversations, BhashaAI represents a significant step toward breaking barriers in language learning.

We’re incredibly proud of what we’ve built in such a short time. This journey has been both challenging and rewarding, and we’re thrilled to have brought our vision to life in just four weeks. We can’t wait to see the positive impact BhashaAI will have on learners worldwide.

What we learned

• Practical Implementation of Web Speech and Recognition APIs:
  
  • While I had prior knowledge of the Web Speech API and Recognition API, this project provided the first opportunity to implement them practically.
    
  • It was rewarding to see these APIs in action and understand their real-world applications.
• Prompt Engineering:
  
  • This project gave me extensive hands-on experience with prompt engineering.
    
  • I explored various patterns, techniques, and approaches to craft effective prompts that aligned with our requirements.
    
  • Testing and iterating on these prompts taught me how to optimize them for better results.
• Understanding Chrome’s Inbuilt APIs:
  
  • I gained a deeper understanding of Chrome’s inbuilt AI APIs, including their features and limitations.
    
  • Learning how to leverage these APIs effectively while working around their constraints was a significant takeaway.
• Using Wikimedia API:
  
  • This project introduced me to the Wikimedia API for fetching relevant image details.
    
  • Integrating it into the platform and seeing how it enhances the user experience was a valuable learning experience.

What's next for BhashaAI

• Better Error Handling:
  
  • We plan to improve error handling across the website to ensure a smoother user experience and fix any bugs that might arise.
• Prompt Fine-Tuning:
  
  • While our current prompts work well, we believe there’s still room for improvement.
    
  • Fine-tuning them further will help us achieve even better results.
• User Feedback and Feature Enhancements:
  
  • Gathering feedback from users will be a key focus as we continue to improve BhashaAI.
    
  • Based on their suggestions, we aim to:
    
    • Integrate the platform with the latest news or article data to provide fresh and engaging content.
      
    • Develop a Chrome extension that allows users to save articles directly to BhashaAI for further learning and exploration.
      
    • Introduce a feature for English teachers to download articles along with quizzes and challenges as a PDF sheet, enabling offline or classroom use.
      
    • Continuously add more features that align with user needs and enhance their overall learning experience.
      
  • This feedback-driven approach will ensure that BhashaAI evolves to meet the expectations of its users while offering innovative and practical solutions for language learning.
• Support for More Languages:
  
  • Currently, the platform supports English, Spanish, and French.
    
  • In the future, we plan to expand support to include more languages, making BhashaAI accessible to a broader audience.
• Responsive Design:
  
  • For the hackathon, we focused exclusively on the desktop version due to Chrome’s built-in AI only being available on desktop Chrome.
    
  • Going forward, we aim to create a responsive design to support multiple devices, including mobile and tablet.
• Enhanced Offline Functionality:
  
  • Currently, BhashaAI operates in pseudo-offline mode—users need an internet connection to load the website initially. Once loaded, it remains usable offline. Moving forward, we plan to implement service workers to cache the website's pages during the initial load when the user is online. For first-time users, an internet connection will still be required to load the website. However, once cached, returning users can access the website entirely offline, making the experience seamless and consistent.
  • Currently, the Web Speech Recognition API is used for speech recognition in the Real-Time Conversations feature, but it doesn’t support offline mode. We plan to explore alternative solutions to enable offline speech recognition, making this feature more accessible and seamless for users without an internet connection.
• Improved Summarization Feedback:
  
  • While our summarization feedback works well in most cases, we’ve identified areas where accuracy can be improved.
    
  • By refining the prompts and testing different techniques, we aim to make this feature even more reliable for users.

Built With

• browser-local-storage
• chrome-build-in-ai-apis
• indexeddb
• javascript
• react
• wikimedia-apis