Inspiration
We asked ourselves the question: "Why shouldn't we just use GPT4 to analyse the entire log file?" From our point of view there are two main reason:
- The large the context size the worse an LLM performs, a log file is very large.
- One query with such a large context costs a lot of money and is very computationally expensive.
A RAG based approach is the obvious way out for both of these issues.
What it does
- Split the entire log file into chunks (usually chunks are identified by the program running).
- Embed chunks with an embedding model (see MTEB leaderboard)
- Calculate similarity between embedded user question and chunks
- Use top k chunks as context for the LLM (Mistral-7B) together with the user question.
How we built it
The entire project is build with python. The backend makes heavy use of Pytorch and the Hugging Face Transformer Library, while the frontend is build with streamlit.
Challenges we ran into
- Efficiently embedding the large log files within seconds
- Prompting our LLM correctly to get the desired behaviour
- Nicely visualise the output and important information within the log files ## Accomplishments that we're proud of Making a fully functional RAG App for chatting with log files!
What we learned
*Working together as a team, delegating tasks to parrallelize the progress. *Improved understanding of almost all the frameworks and libraries used in our project. *The importance of "Programming in English" via LLM Prompting, i.e the amount of difference a good prompt can make when it comes to desired behaviour and LLM performance
What's next for Rohde&Schwarz LogGPT
*The log file should be more easily available in the frontend. Whenever the LLM references line number or excepts it should be possible to directly jump to those, both to fact check the output and to continue reading within the log file from that point. *There are likely a lot of speed gains to be made, both in the embedding and generation process.
Log in or sign up for Devpost to join the conversation.