🧠Inspiration
Large Language Models are powerful, but inefficient.
While building LLM-based applications, we noticed a recurring pattern: prompts were getting longer, more complex, and increasingly expensive—without clear evidence that all parts were necessary.
This raised a simple but important question:
Which parts of a prompt actually matter?
We realized that most developers optimize model choice, architecture, and latency—but rarely optimize the prompt itself, even though it directly impacts both cost and performance.
MemoryLens was born from this gap.
🚀 What we built
MemoryLens is a prompt efficiency analysis and optimization tool that helps developers:
Identify unnecessary tokens Visualize importance within prompts Compress prompts without losing meaning Quantify cost savings In our demo, MemoryLens achieved:
42.3% token reduction $438 saved per 100K requests Compression is presented as a suggestion with full visual context, not as automatic deletion — every flagged sentence stays visible, and the developer decides what to cut.
🏗 How we built it We designed MemoryLens as a DevTool-first architecture, making it easy to integrate into real workflows.
Core components: Python SDK
analyze_prompt() — measures token usage and reduction potential compress_prompt() — generates a shorter equivalent prompt check() — enforces token limits for CI/CD CLI Interface
memorylens analyze prompt.txt memorylens compress prompt.txt memorylens check prompt.txt --max-tokens 500 FastAPI Backend
Exposes /analyze, /compress, /trace endpoints Prompt Analysis Engine
Token-level analysis using tiktoken Importance estimation for prompt segments Heuristic + structural compression
đź“Š Key Insight
Our experiments revealed a consistent pattern:
A large portion of prompts (30–50%) does not contribute meaningfully to the output.
This suggests that prompt engineering today is:
Over-specified Redundant Cost-inefficient MemoryLens makes this inefficiency visible—and actionable.
🧪 What we learned
Cost optimization is an underexplored dimension of LLM systems Developers lack tooling to reason about prompt efficiency Small prompt changes can lead to significant cost savings at scale DevTools with clear ROI are easier to evaluate and adopt Admitting what a tool cannot do is part of making it trustworthy We also learned that:
Clarity beats complexity—both in prompts and in tools
⚠️ Challenges we faced
- Measuring "importance" Determining which parts of a prompt matter is non-trivial.
We approached this by:
Analyzing token contribution heuristically Observing structural patterns Comparing before/after outputs
- Preserving meaning during compression Naive truncation breaks prompts.
We needed to ensure:
Semantic consistency Task intent preservation Output stability
- Making it usable in real workflows A web interface alone was not enough.
We solved this by:
Building a CLI for instant use Adding CI/CD integration (check) Designing SDK functions for easy embedding
- Honest limits of rule-based analysis Can a rule-based system really decide what an LLM "needs"?
Ceremonial personas ("You are an expert...") are statistically verbose, but a short functional persona ("Respond as a pediatrician") can genuinely shift output. MemoryLens cannot tell these apart semantically — so we designed it not to.
We addressed this through design rather than over-claiming:
Every deletion candidate stays visible (grey + strikethrough) Nothing is auto-deleted; the developer remains in control Safety through visibility, not automation
🎯 What's next
Embedding-based learning loop: Upgrade from TF-IDF to semantic embeddings with a continuous feedback loop that improves compression accuracy over time.
LLM-powered verification: Add optional LLM comparison to validate that compressed prompts produce equivalent outputs, moving from estimated to verified compression quality.
đź’ˇ Closing Thought
Most of the prompt is not used.
MemoryLens helps developers see—and remove—that waste.
Log in or sign up for Devpost to join the conversation.