ContexTrain

• 

  ContexTrain front page

• 

  ContexTrain Thinking

• 

  ContexTrain Answer

Inspiration

What inspired me to build this project was my experience using AI. After having long conversations with different AI agents, I noticed that they all had one problem: eventually, they would start losing context and giving me incorrect answers.

I decided to focus this idea on software development because developers often need to explain their project structure, files, and code to AI coding assistants before getting useful answers.

What it does

ContexTrain scans the selected project directory, reads the relevant files, and splits their contents into 50-line chunks. Each chunk is converted into an embedding using Sentence Transformers, allowing cosine similarity to identify the 10 chunks most relevant to a user's question.

The overall process is:

Project → File Indexing → Embeddings → Semantic Search → Relevant Context → Llama 3.2 → Answer

How we built it

To build the project, I used Sentence Transformers with the all-MiniLM-L6-v2 model to turn the project text into embeddings. When a user asks a question, the question is also converted into an embedding.

I then use cosine similarity to compare the question with the project content and retrieve the most relevant results. The retrieved context is then passed to Llama 3.2, which I run locally through Ollama.

I used FastAPI for the backend and Jinja2 for the web interface.

Challenges we ran into

In this project, I faced three main challenges.

My first challenge was that a basic keyword search wouldn't really solve the problem I wanted to address. A user could ask a question using completely different words from those used in the code or documentation. To combat this, I used all-MiniLM-L6-v2 to create embeddings and cosine similarity to find chunks that were semantically related to a user's question.

The biggest challenge I encountered was during testing. I asked: "Where does the application handle API routes?" The model initially answered that there was a routes directory, even though there wasn't one. This meant the AI was making up information to satisfy the question.

Therefore, I changed the prompt to explicitly tell the model to only use the provided project context and to say that it doesn't know when the answer isn't available.

This highlighted an important issue for me: having an LLM isn't enough; the context supplied to it matters.

My third challenge was that I initially considered using the Anthropic API, but API usage would require paid credits. After researching free local LLM options, I switched to Ollama, allowing me to run Llama 3.2 locally during development without needing to spend credits on hosting the AI.

Accomplishments that we're proud of

One of the main accomplishments I'm proud of is building a complete AI system that goes from project indexing and retrieval through to generating a final response.

I'm also proud that testing exposed a real limitation of LLMs: even when the model is given project information, it can still produce an incorrect answer if it isn't properly instructed.

What we learned

Through this project, I learned how to use scikit-learn, Sentence Transformers, and the Ollama API. I also gained a better understanding of how embeddings and semantic search can be used to retrieve information that is related in meaning rather than just matching exact keywords.

One of the biggest things I learned was that getting an AI model to produce an answer is not the same as getting it to produce a reliable answer. The quality and relevance of the context provided to the model can have a major effect on its response.

What's next for ContexTrain

The next step for ContexTrain would be to improve the quality and reliability of the retrieved context. This could include improving how project files are chunked and indexed so that the system can retrieve more precise information for different types of questions.

I would also like to improve the system's ability to understand larger projects and provide more useful context as the project grows.

Built With

• anthropic
• fastapi
• html
• jinja
• llm
• python
• python-multipart
• scikit-learn
• sentence-transformers
• uvicorn