Inspiration
While I was building Cognivia, Retrieval-Augmented Generation (RAG) was becoming an increasingly prominent approach in the AI field, particularly for building knowledge-grounded conversational applications. The growing adoption of RAG-based chatbots made me interested in understanding what happens beyond simply connecting a chatbot to an LLM.
What particularly inspired me was the complete RAG pipeline: document processing, chunking strategies, vector embeddings, information retrieval, and LLM-based generation. I became interested in understanding how the quality of retrieved documents affects the quality and accuracy of the final response.
As I explored the technology further, I became particularly interested in the retrieval side of RAG and how different retrieval techniques could complement one another. This led me to implement a hybrid retrieval approach combining BM25 and semantic search, followed by Reciprocal Rank Fusion and reranking, before passing the most relevant context to the LLM.
That exploration ultimately led to the development of Cognivia, where I could apply these concepts in a practical academic assistant designed to answer questions using college-related documents. The project objective was to provide accurate and context-aware responses to students, applicants, and faculty through a knowledge-grounded architecture.
What it does
Cognivia is a knowledge-grounded academic assistant built using a hybrid Retrieval-Augmented Generation (RAG) pipeline. It combines BM25-based lexical search with semantic vector search to retrieve relevant information from academic and institutional documents. The results from both retrieval methods are combined using Reciprocal Rank Fusion (RRF) and further refined through a reranking stage to improve the relevance of the retrieved context. The selected context is then combined with the user's query and passed to a Large Language Model (LLM), which generates a context-aware response grounded in the retrieved information.
How we built it
Cognivia was built as a hybrid Retrieval-Augmented Generation (RAG) system using Python and Django for the application/backend layer, Milvus for vector-based retrieval, and HTML, CSS, and ReactJS for the frontend. The project also uses LangChain and local LLMs through Ollama as part of the overall RAG architecture.
The document-processing pipeline begins with large PDF documents, which are chunked into smaller pieces. Embedding models are then used to convert the chunks into vector representations, which are stored and indexed for semantic retrieval. The architecture shown on page 5 specifically illustrates the document chunking, embedding, vector creation, Milvus storage, query processing, augmented-query stage, and LLM generation flow.
For retrieval, Cognivia uses a hybrid search strategy consisting of two complementary retrieval methods:
BM25 lexical search retrieves documents based on keyword and term-level relevance.
Semantic search uses vector embeddings to retrieve content based on semantic similarity, with the vectors stored in Milvus.
The ranked results from these two retrieval approaches are then combined using Reciprocal Rank Fusion (RRF). RRF produces a unified ranking by considering the positions of documents across the individual retrieval result sets.
After fusion, the candidate results are passed through a reranker, which further evaluates their relevance and prioritizes the most useful context for the user's query.
The selected context is then incorporated into the augmented-query stage, where the original question and retrieved information are prepared as input for the Large Language Model (LLM). The LLM uses this contextual information to generate the final response. This overall query-to-retrieval-to-generation flow is consistent with the RAG architecture illustrated in the project's component diagram.
The backend is implemented using Django, while the frontend uses standard web technologies. The documented execution process involves starting the Django server using python manage.py runserver, after which the frontend sends user queries to the backend. The backend processes the queries through the RAG pipeline and returns the LLM-generated response to the frontend.
The application was developed and tested locally using Visual Studio Code, Python virtual environments, and Ollama for local model execution. The development process also required resolving Python-version and environment-configuration issues.
Challenges we ran into
One of the major challenges during the development of Cognivia was setting up a consistent development environment for the RAG pipeline and local language models.
The first challenge was model selection and installation. We used Ollama to run language models locally and evaluated suitable models based on the project requirements and their compatibility with the RAG workflow. The project models including Llama 3.2, Mistral, and embedding models such as mxbai-embed-large.
A second challenge was Python environment and version compatibility. The virtual environment was associated with Python 3.9, while the project was initially being executed with Python 3.13. This mismatch resulted in installation and execution issues. The problem was resolved by selecting the appropriate Python interpreter in Visual Studio Code and ensuring that it matched the virtual environment.
We also encountered platform-specific differences in virtual-environment setup, particularly between Windows and macOS. Since development was performed on macOS, the environment creation and activation commands had to be adjusted accordingly.
Another technical challenge was designing the retrieval stage so that the system could handle both exact keyword-based matches and semantically similar content. To address this, Cognivia uses a hybrid retrieval approach combining BM25 lexical search and semantic vector search. The results from both retrieval methods are combined using Reciprocal Rank Fusion (RRF), after which a reranking stage further refines the retrieved candidates before they are passed to the generation stage.
During execution, we also encountered issues related to the Pylance extension in VS Code, which affected code suggestions and type checking. Verifying the interpreter and environment paths helped resolve these issues. The project also required awareness of platform-specific Python execution commands, such as python versus python3.
Overall, the development challenges involved both engineering the retrieval pipeline and maintaining a stable development environment for the application and its supporting models.
Accomplishments that we're proud of
One of the accomplishments I am most proud of is seeing the final RAG pipeline come together and work end-to-end in the frontend. It was rewarding to see the system I engineered retrieve relevant information and transform it into a meaningful response through the complete retrieval and generation pipeline.
I am particularly proud of implementing a robust RAG architecture incorporating hybrid search, Reciprocal Rank Fusion (RRF), reranking, and LLM-based generation. Bringing these individual components together into a functioning application gave me a much deeper understanding of how retrieval quality directly influences the final response generated by the language model.
Seeing the architecture translate from an implementation into a working frontend experience was a significant achievement for me, because it demonstrated that the different stages of the pipeline could operate together as a complete system rather than merely as isolated components.
Important: the PDF you provided explicitly supports the RAG, embedding/vector retrieval, Milvus, augmented-query, and LLM-generation components, but it does not document hybrid search, Reciprocal Rank Fusion, or a Reranker. So the wording above preserves those claims because you have stated that you implemented them, but they should be treated as your additional project information, not as claims verified by the PDF.
What we learned
Developing Cognivia gave us practical experience in designing and implementing a Retrieval-Augmented Generation (RAG) system and understanding how retrieval quality influences the quality of the final generated response. The project combines document processing, retrieval, contextual augmentation, and language-model generation to answer academic queries using a defined knowledge base.
One of the most important things we learned was that retrieval quality is fundamental to RAG performance. A single retrieval strategy may not always be sufficient because users can express the same information need through exact terminology or through semantically different wording. This led us to implement hybrid search, combining BM25 for lexical retrieval with semantic search for meaning-based retrieval.
We also learned how Reciprocal Rank Fusion (RRF) can be used to combine independently ranked results into a unified ranking. This allowed the system to benefit from both keyword-level and semantic retrieval rather than depending entirely on either approach.
Another important learning was the role of reranking. Initial retrieval can produce a broader candidate set, while the reranker can further evaluate those candidates and prioritize the most relevant context before generation. This helped us understand that an effective RAG architecture is not simply “retrieve and generate”; it can involve multiple stages of relevance refinement.
We also gained practical experience with local LLM deployment using Ollama, model selection, embedding models, vector databases, and the integration of these components into a Python/Django application.
Finally, the project reinforced the importance of development-environment management. Python-version compatibility, virtual environments, package installation, interpreter selection, and platform-specific configuration all had a direct impact on the reliability of the development process.
What's next for Cognivia
The next stage for Cognivia is to further develop it into a more capable and extensible AI assistant. The project's original objective identifies extensibility and integration with advanced Generative AI models as an important future direction.
Future development can focus on expanding the knowledge base and supporting a broader range of complex queries. The current system is designed around knowledge-grounded responses derived from Django document, Physics text book and Deep Learning documentation.
Further work could also focus on improving the retrieval and ranking pipeline, including continued evaluation and tuning of BM25, semantic retrieval, RRF, and reranking so that the system consistently selects the most relevant context for different types of queries.
Another area for future development is improving the overall maintainability and robustness of the application and its development environment, particularly around dependency management, configuration, and deployment. The development process demonstrated that environment inconsistencies can introduce significant implementation challenges.
The longer-term goal is to evolve Cognivia from a working RAG application into a more reliable and extensible production grade assistant which can be scaled while retaining its core principle of grounding responses in institutional knowledge.
