QRT challenge: Log Analyser

Inspiration

Log analysis is a common task in software development, debugging, and monitoring, and the challenge by QRT inspired me to build a tool that simplifies this process by providing real-time analysis.

What it does

The Log Analysis Tool is designed to simplify the extraction and analysis of log output generated by various applications. It provides a range of functionalities to make log analysis efficient and flexible. Here's an overview of what the tool does:

Real Time Log Analysis -Filters log entries: -Time Difference Calculation ## How we built it It is a Python program with Command Line interface Log Parsing: In the log file handler, we parsed log entries, extracted relevant data using regular expressions, and stored them in a data structure for further analysis. Filtering and Highlighting: We implemented filtering options that allowed users to specify important information, both through exact matches and regular expressions. You can define what information to filter and it does so by finding the keyword or counting the number of errors, debugs, information, etc Time Difference Calculation: We developed a component for calculating the time difference between two specified points in log entries.

Challenges we ran into

Real-Time Processing: Implementing real-time log file monitoring and processing was complex. Ensuring that new log entries were detected and analyzed as they were appended to log files required careful synchronization. Data Extraction: Parsing log entries and extracting relevant information was challenging, particularly when dealing with varying log formats and data structures. Additionally, We did not have time to build a GUI. We have a basic code but nothing that can be implemented in our submission. Additionally, we felt a lack of time to design code for analysing multiple log files at once.

Accomplishments that we're proud of

To be able to learn so many novel concepts in a span of 24 hours including regex, real-time monitoring, and data parsing.

What we learned

Problem Solving: We honed our problem-solving skills by addressing a real-world challenge. Developing a tool to analyze logs is a practical problem often encountered in software development. Real-Time Data Processing: We learned how to continuously monitor and process data as it's generated, which is a valuable skill for various applications beyond log analysis. Log Parsing and Data Extraction: Parsing log entries and extracting relevant information was a central challenge. We became proficient in regular expressions and other parsing techniques, which are applicable in many data extraction tasks. Overall, this hackathon project provided a rich learning experience, enabling us to tackle a practical problem with real-time log analysis. This will be a valuable asset for future projects and challenges.