About the Project
Modern Large Language Model (LLM) applications are powerful, but operating them safely in production is difficult due to limited observability. Teams often struggle to monitor LLM behavior, runtime performance, and security risks in real time. This project demonstrates an AI-powered security log analyzer with end-to-end observability, built using Google Cloud Vertex AI (Gemini) and Datadog. It shows how LLM applications can be monitored, secured, and operated reliably using modern observability practices.
Inspiration
The project was inspired by working with log analysis and security systems where traditional monitoring tools fail to provide visibility into AI-driven behavior. As LLMs become core components of applications, it is critical to monitor not only application health but also LLM-specific signals such as latency, token usage, and abnormal behavior.
How We Built It
Logs are ingested through an API hosted on Google Cloud Run Logs are analyzed by Gemini on Vertex AI for threat detection and severity classification Runtime metrics, logs, traces, and LLM telemetry are sent to Datadog Datadog dashboards visualize system health and security signals Detection rules automatically trigger alerts and incidents for anomalies or threats The focus was on clear integration and actionable observability rather than heavy infrastructure.
What We Learned
LLM applications require different observability signals than traditional systems Datadog can effectively monitor AI and security-related telemetry Well-designed detection rules are critical for actionable alerts Google Cloud services enable scalable and reliable AI architectures
Challenges Faced
The main challenge was designing observability for LLM behavior, not just application performance. Another challenge was balancing real-world relevance with simplicity within a limited timeframe.
Future Scope
Real-time streaming log ingestion Automated remediation using Datadog incidents Advanced prompt risk and hallucination detection Cost and token usage optimization
Datadog (APM, Logs, Metrics, Alerts, Dashboards)
Log in or sign up for Devpost to join the conversation.