LogLens

Inspiration

I wanted to branch out into Machine Learning, and this project is my first step toward that goal. I wanted to create an ML-based project, and since I am taking CS3377, I started learning about servers, and I was fascinated by how backend systems worked beyond the frontend. Thus, I decided to combine them.

What it does

LogLens is a CLI log analyzer and anomaly detector. The user uploads their log file, and the tool handles the rest. First, a regex-based parser reads each line and extracts structured fields: timestamp, log level, endpoint, HTTP status code, and response time. Rejecting any line that does not follow the format. Once the Logs are successfully parsed using SQLite the logs are then stored in the database via SQLAlchemy. From there, two independent anomaly detection methods run against the stored data. The first is statistical: it calculates the mean and standard deviation of all response times and flags any entry that falls more than 2 standard deviations above the mean using a Z-score formula. The second is an Isolation Forest model from scikit-learn — an unsupervised ML algorithm that learns what "normal" looks like and flags entries that deviate from that pattern across all features simultaneously. Both detectors return human-readable explanations, for example: "Response time 1200.0ms is 2.9 standard deviations above mean (194.5ms)". Finally, FastAPI exposes everything through a REST API with 6 endpoints and auto-generated Swagger documentation at /docs. There is also a dark-themed web dashboard at /dashboard that displays all logs, level breakdowns, response time statistics, and highlights anomalous entries visually.

How I built it

I started with what I knew, Python, and built the log parser first since it had no external dependencies. From there, I added SQLite storage through SQLAlchemy, which required learning how ORMs work and how to map Python classes to database tables. I then implemented the FastAPI layer to expose the data over HTTP, which was entirely new territory for me. The anomaly detection phase followed, where I learned how to encode categorical data for scikit-learn and use Isolation Forest for unsupervised detection. Finally, I wired everything into a CLI using Click. The project was built phase by phase, with a full test suite written at each stage — 31 tests total using pytest.

Challenges I ran into

The biggest challenge was FastAPI. The framework is powerful but has a lot of moving parts Pydantic schemas for validation, dependency injection, async file uploads, and response models all had to work together. I also ran into an issue early on where an accidental terminal command deleted my entire loglens/ source folder, which taught me the hard way to commit to GitHub early and often.

Accomplishments that I'm proud of

The project as a whole is a significant milestone for me. I successfully learned and applied FastAPI, SQLAlchemy, scikit-learn, and Click, none of which I had used before starting this project. I also implemented two independent anomaly detection methods and cross-validated their results, which felt like a genuine engineering decision rather than just following a tutorial. Writing 31 automated tests across 5 components was also new for me and gave me a much deeper appreciation for test-driven development.

What I learned

Throughout the project I have learned how to use:

1. FASTapi
2. Scikit-Learn
3. SQLAlchemy
4. Pytest

This was also my first project at this scale, which meant I had to think carefully about how pieces connect rather than just writing code in a single file.

What's next for LogLens?

The natural next evolution is moving from detection to prediction and alerting. Specifically:

1. Real-time log streaming — instead of uploading a file, pipe live server logs directly into LogLens as they are written
2. Alerting system- send email or Slack notification automatically when anomalies are detected over a threshold.

Built With

• click
• fastapi
• python
• scikit-learn
• sqlalchemy
• sqlite