Pulse.AI

Sensing the pulse of PopCulture in real-time

Comment

Pulse AI

Sensing the pulse of pop culture in real-time.

Inspiration

Pop culture lives and breathes on Twitter. With trends rising and falling by the hour, we wanted a way to visualize how people feel and what themes are emerging around any given topic. Our idea was to create an AI-powered dashboard that reveals the emotional and semantic landscape of social media in a glance—and even anticipate where it might go next.

What it does

Sentiment Analysis

Pulse AI scrapes Twitter for real-time tweets based on a topic you choose. It then:

  • Analyzes sentiment of each tweet using VADER.
  • Embeds the semantic meaning using Sentence-BERT.
  • Applies UMAP for dimensionality reduction.
  • Uses KMeans to cluster tweets into themes.
  • Extracts top keywords to name each cluster.
  • Displays everything on an interactive Plotly chart showing sentiment vs. semantic similarity.

Smart Predictive Questions

Pulse AI also offers users a selection of smart, context-aware predictive questions generated based on their original query. The user can pick one of these questions to explore further. Pulse AI then uses the just-collected tweet data to make a real-time prediction that reflects the general public opinion or trend surrounding that question.

Virality Prediction — Built from Scratch

One of our proudest features is a custom-trained virality prediction model that analyzes each cluster on the sentiment chart. This model, built and trained by us from scratch, predicts how likely each thematic cluster is to go viral based on:

  • Tweet frequency
  • Sentiment strength
  • Semantic coherence within the cluster

The results are visualized on a dynamic graph where:

  • X-axis = sentiment (negative ↔ positive)
  • Y-axis = semantic proximity
  • Node size = volume of tweets

This gives users a clear sense of which themes are gaining traction and could explode soon.

How we built it

We structured the project with modular components for flexibility and scale:

  • Tweet Scraper: Uses tweepy to access Twitter’s recent search endpoint.
  • NLP Pipeline:
    • SentenceTransformer for embeddings
    • VADER for sentiment scores
    • UMAP for visualization
    • KMeans for clustering
  • Smart Prediction Module:
    • Generates predictive questions using NLP
    • Uses real-time tweet data to generate forecasts
  • Custom Virality Model:
    • Trained on labeled tweet cluster data
    • Outputs cluster virality scores and projections
  • Web Interface: Flask app with routes for input and visualization
  • Visualization: plotly for an interactive 2D scatter plot
  • UI: HTML templates + custom CSS with a dark mode aesthetic
  • Storage: Saves all processed data to JSON with timestamps

Challenges we ran into

  • Rate limits on the Twitter API made us rethink how to batch requests and introduce fallback handling.
  • Tuning UMAP + KMeans to give meaningful, distinct clusters for different types of topics.
  • Designing and training our own virality prediction model under time constraints.
  • Balancing usability and depth in the smart prediction question interface.

Accomplishments we're proud of

  • Built a fully functional NLP-based visualization tool in 36 hours.
  • Designed an elegant UI that non-technical users can interact with.
  • Developed a custom-trained virality prediction model for Twitter data.
  • Introduced smart user-driven prediction features to guide insight discovery.

What we learned

  • How to integrate sentence embeddings and clustering models into a web app.
  • Handling external API limitations and fallback mechanisms.
  • Designing predictive ML models for real-time social media insights.
  • Structuring teamwork efficiently around modular components.

What's next for Pulse AI

  • Live tweet streaming for real-time updates
  • Expanding the smart prediction engine with GPT-based question generators
  • Side-by-side topic comparison
  • Toxicity and bias detection
  • Mobile-optimized frontend

Built With

  • Python
  • Flask
  • Tweepy
  • SentenceTransformers
  • VADER Sentiment
  • UMAP-learn
  • Scikit-learn
  • Plotly
  • HTML + CSS

Team

  • Vaibhav Maloo
  • Ishan Vaish
  • Sidhant Singhvi
  • Atharva Kedia

Built With

Share this project:

Updates