WhatsApp Chat Analyzer (ChatLens AI)

Turn your conversations into intelligence with AI-powered analytics, semantic search, RAG, insights, and interactive exploration.

Comment

Inspiration

Conversations contain a huge amount of information, but most chat applications only provide basic search and message history. Important patterns, discussions, decisions, topics, and trends can easily get buried inside thousands of messages.

We wanted to build something that could go beyond simply counting messages or generating charts. This idea led to ChatLens AI, a conversation intelligence platform that transforms exported chat history into an interactive, searchable, and AI-powered knowledge source.

The goal was simple: make conversations understandable, searchable, and queryable with AI.

What it does

ChatLens AI turns exported WhatsApp conversations into an interactive intelligence platform.

It provides traditional analytics as well as modern NLP and RAG capabilities.

Conversation Analytics

  • Total messages, words, media, links, and active users
  • Daily, weekly, monthly, and hourly activity
  • Activity heatmaps
  • Most active users
  • Conversation-starting patterns
  • Response-time analysis
  • Longest inactivity periods
  • User participation statistics

NLP & Content Intelligence

  • Word-frequency analysis
  • Word clouds
  • Emoji analysis
  • Sentiment analysis
  • Sentiment trends over time
  • Topic detection and clustering
  • Language detection
  • Multilingual conversation analysis
  • Automatic translation for supported languages

Semantic Search

Instead of searching only for exact keywords, ChatLens AI supports meaning-based search.

For example, searching for:

"discussion about exams"

can retrieve messages discussing exams even when the exact word "exam" is not present.

This is powered by embeddings and vector-based retrieval.

AI Conversation Assistant

Users can interact with their chat history using natural language.

They can ask questions such as:

  • "What were we discussing about the project?"
  • "Summarize this conversation."
  • "What were the main topics this month?"
  • "When did we discuss the trip?"
  • "Who was involved in this discussion?"
  • "What were the important decisions?"

The system retrieves relevant conversation context and uses Retrieval-Augmented Generation (RAG) to produce grounded responses.

Ask This Moment

Users can select a specific point or time period in the conversation and ask questions about what was happening around that moment.

This provides contextual, timestamp-aware conversation analysis instead of treating the entire chat as one large document.

Media & Link Intelligence

ChatLens AI also analyzes shared content, including:

  • Images
  • Videos
  • Documents
  • Audio
  • Stickers
  • GIFs
  • URLs

Links can be categorized by domain and analyzed to identify frequently shared sources and users.

AI Summaries & Reports

The platform can generate:

  • Daily summaries
  • Weekly summaries
  • Monthly summaries
  • Topic summaries
  • Conversation summaries
  • AI-generated insights
  • Exportable PDF reports

How we built it

The project started as a Streamlit-based WhatsApp chat analytics application and was progressively transformed into a complete conversation intelligence platform.

The overall architecture follows this pipeline:

WhatsApp Export
      ↓
Smart Chat Parser
      ↓
Message Cleaning & Normalization
      ↓
Conversation Analytics
      ↓
NLP Processing
      ↓
Embeddings
      ↓
Vector Database
      ↓
Semantic Retrieval
      ↓
RAG Pipeline
      ↓
AI Conversation Assistant
      ↓
Interactive Dashboard & Reports

Core technologies

Frontend / Application Layer

  • Streamlit

Data Processing

  • Python
  • Pandas
  • NumPy

Visualization

  • Matplotlib
  • Interactive Streamlit components

NLP

  • Sentence Transformers
  • Language detection
  • Sentiment analysis
  • Text preprocessing
  • Topic analysis

Semantic Search & RAG

  • Text embeddings
  • Vector database
  • Retrieval pipeline
  • LLM-based response generation

Additional Processing

  • URL extraction
  • Emoji analysis
  • Multilingual translation
  • PDF report generation

The application maintains the original message timestamp, sender, and message content while deriving additional fields such as date, month, weekday, hour, language, sentiment, topic, and message type.

For semantic retrieval, conversation content is transformed into vector representations. Given a user query (q), relevant messages are retrieved according to embedding similarity:

[ sim(q,d)=\frac{q\cdot d}{|q||d|} ]

The retrieved messages are then supplied as contextual evidence to the RAG pipeline before generating the final response.

Challenges we ran into

1. Parsing real-world WhatsApp exports

WhatsApp exports are not always perfectly uniform. Messages can contain:

  • Multi-line text
  • Media placeholders
  • Deleted messages
  • System notifications
  • Different date formats
  • Different time formats
  • Different year representations

Building a reliable parser that could handle these variations required careful regular-expression design and message reconstruction.

2. Handling multilingual conversations

Real conversations frequently mix languages and may contain English, Hindi, Marathi, Hinglish, emojis, links, and other content in the same conversation.

We needed to separate language detection, translation, sentiment analysis, and original-message preservation so that translation would not destroy the original conversation data.

3. Scaling analysis for large conversations

Processing thousands or hundreds of thousands of messages can become expensive and slow.

We addressed this by introducing:

  • Data preprocessing
  • Cached computations
  • Efficient DataFrame operations
  • Chunking
  • Vectorized analysis
  • Reusable embeddings
  • Retrieval instead of sending the entire conversation to an LLM

4. Making RAG useful instead of simply adding a chatbot

One of the biggest challenges was ensuring that the AI assistant actually answered questions using the uploaded conversation rather than generating generic responses.

The solution was to build a retrieval pipeline where relevant messages are selected before generation.

User Query
    ↓
Query Embedding
    ↓
Similarity Search
    ↓
Relevant Messages
    ↓
Context Construction
    ↓
LLM
    ↓
Grounded Answer

5. Connecting analytics with AI

Traditional analytics and generative AI provide different types of information.

Charts are excellent for numerical patterns, while RAG is better for understanding conversational content.

Combining both into one application required designing a system where:

structured analytics + unstructured conversation retrieval + AI generation

work together rather than functioning as separate features.

6. Privacy considerations

Chat data can contain highly personal information. This made privacy an important engineering consideration.

The project was designed around processing exported conversations as user-controlled data and minimizing unnecessary persistence of raw conversation content.

Accomplishments that we're proud of

We are proud that ChatLens AI evolved from a simple chat-analysis dashboard into a complete conversation intelligence platform.

Some of the major accomplishments include:

  • Built a robust WhatsApp conversation parser
  • Created an interactive analytics dashboard
  • Added user-level conversation analytics
  • Implemented activity heatmaps and temporal analysis
  • Added sentiment and multilingual analysis
  • Added semantic search
  • Built a vector-based retrieval pipeline
  • Implemented RAG-based conversation question answering
  • Added timestamp-aware Ask This Moment functionality
  • Added topic discovery and conversation summarization
  • Added media and link intelligence
  • Added AI-generated reports
  • Designed the system with privacy considerations
  • Combined traditional data analytics with modern AI techniques in one application

Most importantly, the project demonstrates that an exported conversation can be transformed from a static text file into an interactive knowledge source.

What we learned

Building ChatLens AI helped us understand that creating a useful AI application involves much more than connecting an LLM to a user interface.

We learned how to:

  • Process messy real-world text data
  • Design reliable data pipelines
  • Work with embeddings and vector databases
  • Build semantic retrieval systems
  • Design RAG pipelines
  • Evaluate retrieved context
  • Handle multilingual text
  • Combine structured and unstructured data
  • Optimize processing for large datasets
  • Design privacy-aware AI workflows
  • Convert analytical requirements into interactive visualizations
  • Build a complete AI product rather than an isolated model demo

We also learned an important engineering lesson: good retrieval and data preparation are just as important as the LLM itself.

What's next for ChatLens AI

The current version focuses primarily on exported WhatsApp conversations, but the architecture can be extended into a broader conversation intelligence platform.

Future improvements include:

Multi-platform support

Support additional conversation sources such as Telegram, Discord, and other exported chat formats through a common message schema.

Advanced conversation graphs

Build interaction graphs showing relationships between participants, conversation frequency, replies, and communication patterns.

Better topic intelligence

Use more advanced clustering and hierarchical topic modeling to automatically organize long-running conversations into meaningful subjects.

Local AI mode

Integrate local LLMs through Ollama so users can perform AI-powered analysis without sending conversation content to external AI providers.

Advanced multimodal analysis

Extend the system to understand images, documents, voice messages, and other media shared inside conversations.

Conversation memory

Allow users to maintain multiple analyzed conversations and build a searchable personal conversation knowledge base while maintaining strict privacy controls.

Advanced AI reports

Generate professional reports containing statistical insights, conversation summaries, detected topics, trends, and AI-generated explanations.

Ultimately, the goal is to evolve ChatLens AI from a WhatsApp analyzer into a privacy-conscious, multi-platform conversation intelligence system capable of helping users search, understand, analyze, and interact with their digital conversations.

Built With

Share this project:

Updates