AI.nchors

Inspiration

Text-based news/articles are cool… but a bit “old school”. We want to push the boundaries and bring something new to the table. We saw a massive opportunity in leveraging deep learning in media creation and consumption that will likely change the equation for the entire industry. We are excited to take part in shaping this new future with the ultimate news experience.

What it does

Do you easily get bored reading long articles? We got you. AI.nchors is an AI-powered news anchor that will read and summarize everything for you. This isn’t your average text-to-speech app, we use deep learning technologies that will produce a synchronized audiovisual representation from a library of popular individuals. We create an AI that powers this individual to be your anchor, speaking with voice synthesized from text and lips synced to their voice, making the experience of reading any information on the internet as engaging as ever.

Feature Details

• Select a famous or historical individual to be your news anchor
• Voice match, gesture, and lip-sync of the anchor to the content
• Sentiment analysis on the news to know whether it's positive or negative
• Summarize long content into smaller reads
• Fact-checking using Google Fact check across multiple news sources
• Scrape and synthesize media across the web as content for your anchor
• Text to Speech synthesis to narrate your news
• Translate using Google API
• Personalization and storage for your curated content

Software Architecture

How we built it

• NextJS and TailwindCSS for frontend
• Serverless functions and cloud functions for backend
• Firebase for auth
• Firebase storage for media content and processing
• Firestore for accounts, persistence
• Cloud functions to funnel data through the backend
• Cloud vision to read visual media
• Google Fact Checker API for news verification
• Cloud Natural Language for sentiment and summarization
• Google Translate API for language change
• Vocoders for narrating the content

Challenges we ran into

Creating believable digital humans, both visually and in voice, is extremely hard to do as we’re hardwired to spot even the slightest errors in visual representation and speech. Fact check API is a bit quirky.

Accomplishments that we're proud of

A lot of Google Cloud usage than we've had before.

What we learned

Learned to use Google fact check and expanded use on various ML API from Google Cloud.

What's next for AI.nchors

A chrome extension with all this functionality will be nice.

Built With

• cloud-functions
• cloud-natural-language
• cloud-vision
• fact-check
• figma
• firebase
• firebase-auth
• firebase-storage
• firestore
• framer-motion
• google-translate
• nextjs
• tailwindcss
• text-to-speech