RepoLens — Understand Any GitHub Repository with Fewer Tokens

Inspiration

Large Language Models have transformed how developers understand unfamiliar codebases, but they come with an expensive problem: context.

Most AI coding assistants answer repository questions by sending huge amounts of code to the model. As repositories grow, token usage grows with them—making responses slower, more expensive, and often filled with irrelevant information.

We asked ourselves:

What if the AI never needed to read the entire repository in the first place?

That question became RepoLens.

Instead of treating token optimization as an afterthought, we built RepoLens around a retrieval-first architecture where only the most relevant parts of a repository are selected, then compressed with Paritok, before any LLM ever sees them.

What it does

RepoLens helps developers understand any public GitHub repository using natural language.

Simply paste a GitHub repository URL and RepoLens:

Analyzes the repository structure
Builds an intelligent repository index
Retrieves only files relevant to the user's question
Optimizes that context using Paritok
Sends a much smaller, cleaner context to the language model
Returns accurate, repository-aware answers

Instead of wasting tokens on thousands of unrelated files, RepoLens focuses on only what matters.

Why Paritok is the Core

Paritok isn't an optional optimization layer—it is the heart of RepoLens.

Our retrieval pipeline is intentionally designed so that relevant repository context is collected before compression, allowing Paritok to optimize only the highest-value information.

This results in:

🚀 Faster responses
💰 Lower AI costs
✂️ Fewer tokens
🎯 Better signal-to-noise ratio
🧠 More accurate answers

Without Paritok, RepoLens loses one of its biggest advantages.

How we built it

RepoLens is built with Next.js 15, TypeScript, and a multi-stage repository retrieval engine.

                 GitHub Repository
                        │
                        ▼
             Repository Analysis Engine
        (metadata, tree, languages, commits)
                        │
                        ▼
              Intelligent File Index
                        │
                        ▼
         Universal Retrieval Engine
   ┌───────────────────────────────────────┐
   │ Metadata Ranking                      │
   │ Content Ranking                       │
   │ Symbol Extraction                     │
   │ Import Graph Analysis                 │
   │ Popularity & Related File Expansion   │
   └───────────────────────────────────────┘
                        │
                        ▼
          Most Relevant Repository Context
                        │
                        ▼
              Paritok Compression Layer
                        │
                        ▼
                 OpenAI Language Model
                        │
                        ▼
               Repository-aware Answer

The retrieval engine combines several complementary ranking strategies instead of relying on simple keyword search or embeddings.

These include:

Metadata ranking
Lightweight content matching
Symbol extraction
Import graph analysis
Related-file expansion
Popularity scoring
Conceptual document boosting

Together, these techniques help RepoLens locate the files most likely to answer a developer's question while keeping context extremely small.

Challenges we ran into

One of our biggest challenges was building an intelligent retrieval engine without relying on embeddings or vector databases.

We needed a solution that was:

Lightweight
Fast
Deterministic
Token-efficient

Other challenges included:

Understanding repository architecture
Ranking important files above utility files
Handling monorepos and workspace aliases
Finding related files through imports
Keeping retrieval accurate while reducing token usage

Balancing answer quality with token efficiency required multiple iterations of the ranking pipeline.

Accomplishments that we're proud of

We're proud that RepoLens became much more than a GitHub search tool.

During the hackathon we built:

A universal repository retrieval engine
Symbol-aware search
Import graph analysis
Popularity-based ranking
Related-file discovery
Intelligent context selection
Seamless Paritok integration
A polished developer-friendly interface

Most importantly, we demonstrated that better retrieval leads to dramatically better token efficiency.

What we learned

This project reinforced one important lesson:

Better AI isn't always about using a larger model—it starts with sending the model better context.

We also learned that understanding a repository requires much more than filename matching. Repository structure, imports, symbols, documentation, and relationships all contribute to finding the right information.

Building RepoLens gave us valuable experience designing retrieval systems and integrating Paritok into a real-world AI workflow.