Inspiration
The inspiration for AudioNewsByte came from the real-world challenges faced by international students and immigrants like Rajesh, who despite being proficient in English, struggle with understanding American accents and following local news. Many newcomers to English-speaking countries face a significant barrier in staying informed about their local communities, not because they lack interest, but because language and accent barriers create an information gap. This challenge is particularly acute when trying to follow lengthy news broadcasts or understand cultural context, leading to a sense of disconnection from their new community.
What it does
AudioNewsByte is a comprehensive multilingual news platform that:
Converts long-form news videos (approximately 60 minutes) into concise 2-3 minute summaries Supports over 50 languages, ranging from major languages (English, French, German) to less commonly supported ones (Thai, Tamil, Swahili) Allows users to ask follow-up questions about news content in their preferred language through voice input Provides audio responses to questions in the user's chosen language Sources latest news content from major platforms like YouTube, including reputable channels such as BBC and CNN Offers a refresh feature to download and access the most recent news updates Maintains the context and quality of news while making it linguistically accessible
How we built it
The technical architecture consists of several sophisticated components:
News Summarization:
Developed a custom LLM called "LLama 3.2-News" Fine-tuned on LLAMA base model using LORA Trained specifically on the XSUM-BBC news summarization task Model is publicly available on HuggingFace (SP2001/LLama3.2-NewsFinetune)
Speech Processing:
Implemented Whisper tiny model for audio transcription and translation Utilized OpenAI TTS model for converting text responses into natural-sounding audio
Question-Answering System:
Leveraged the fine-tuned LLama 3.2-News model for processing and responding to user queries Integrated with the news transcript database for contextual answers
Challenges we ran into
While not explicitly mentioned in the content, the implied challenges likely included:
Training a specialized LLM for news summarization while maintaining accuracy and context Handling multiple languages and accents effectively Ensuring accurate transcription and translation across 50+ languages Maintaining audio quality and natural speech patterns in different languages Managing the processing of long-form content into concise summaries Developing an efficient system for real-time question-answering
Accomplishments that we're proud of
Key achievements include:
Successfully developing and deploying a custom-trained LLM (LLama 3.2-News) Supporting an impressive range of 50+ languages, including less-resourced ones Creating an interactive, conversation-based news experience Achieving significant compression of news content (60 minutes to 2-3 minutes) while maintaining relevance Making the model publicly available for the community Building a complete end-to-end system from news sourcing to interactive audio responses
What we learned
The development process likely provided insights into:
Large Language Model fine-tuning techniques Challenges in multilingual audio processing The importance of user experience in language learning and news consumption Technical aspects of speech-to-text and text-to-speech conversion The complexities of news summarization
What's next for AudioNewsByte
Potential future developments could include:
Expanding language support beyond the current 50+ languages Enhancing the summarization model's performance Adding more news sources beyond YouTube Implementing additional interactive features Developing offline functionality Creating personalized news feeds based on user interests and language preferences Adding features for language learning and vocabulary building Expanding to other forms of media content beyond news Developing partnerships with news organizations for direct content access Implementing community features for shared learning and discussion
Log in or sign up for Devpost to join the conversation.