MemoryLens

🧠 Overview

Most AI prompts are too long—and we don't know which parts actually matter.

As AI systems become more widely used, issues like cost, latency, and reliability are becoming critical. However, these problems are often treated as model limitations, rather than input design problems.

MemoryLens was built to reveal how much of a prompt is actually used—and how much is wasted.

💡 Inspiration

Through analyzing AI-based systems, I observed a recurring pattern:

increasing token costs slow response times unpredictable outputs A key insight emerged:

The problem is not only the model, but the way we communicate with it.

I wanted to make this invisible layer measurable.

⚙️ What it does

MemoryLens analyzes LLM prompts and:

visualizes token usage identifies important vs unused context compresses prompts while preserving meaning compares original vs compressed outputs In many cases, prompts can be reduced by 30–50% without affecting the result.

Compression is presented as a suggestion with full visual context, never as automatic deletion — users see every flagged sentence and make the final call.

🏗 How I built it

The system is designed to be transparent and reliable:

deterministic token analysis (tokenization, structure) heuristic-based importance scoring web-based visualization dashboard optional LLM usage only for compression suggestions This minimizes reliance on black-box AI behavior.

🚧 Challenges

1. Lack of transparency LLMs do not expose which parts of the input influence the output.

→ Addressed by approximating importance using:

structural signals repetition positional context

1. Trust and reliability Reducing input size risks losing meaning.

→ Solved with:

side-by-side comparison similarity estimation user-controlled validation

1. Human-AI interaction Users often cannot understand why an AI produces a result.

→ Improved through:

visual explanations direct input-output mapping intuitive UI

1. Honest limits of rule-based analysis Can a rule-based system really decide what an LLM "needs"?

Ceremonial personas ("You are an expert...") are statistically verbose, but a short functional persona ("Respond as a pediatrician") can genuinely shift output. MemoryLens cannot tell these apart semantically — so I designed it not to.

→ Resolved through design, not over-claiming:

every deletion candidate stays visible (grey + strikethrough) nothing is auto-deleted; the user remains in control safety through visibility, not automation 📊 Key Insight A significant portion of AI input is not used effectively.

This inefficiency directly impacts cost, latency, and environmental footprint.

🚀 What I learned

AI limitations are often interaction design problems transparency improves trust in AI systems reducing complexity can improve both performance and reliability admitting what a tool cannot do is part of making it trustworthy

🔮 What's next

• Embedding-based learning loop: Upgrade from TF-IDF to semantic embeddings with a continuous feedback loop that improves compression accuracy over time.

• LLM-powered verification: Add optional LLM comparison to validate that compressed prompts produce equivalent outputs, moving from estimated to verified compression quality.

🧩 Why it matters

As AI becomes infrastructure, efficiency and accountability matter more.

MemoryLens provides a way to:

reduce unnecessary computation improve system transparency build more trustworthy AI workflows without changing the underlying model.

Built With

• chart.js
• fastapi
• javascript
• python
• react
• tiktoken