Kairos AI: Real-Time Global & Local Impact Intelligence

• 

  kairos_architecture

Inspiration

In times of local or global crises, coordination is everything. Whether it's tracking wildfire movements, distributing relief supplies, supporting local small businesses, or identifying community volunteer needs, the data exists—but it is buried under a mountain of raw, unstructured news and articles.

For students, non-profits, and local communities, building a system to monitor this data is incredibly challenging. Running dedicated GPU instances to run large language models 24/7 is financially impossible. Traditional serverless infrastructure helps with cost, but introduces severe latency (cold starts) and loses observability metrics due to container suspension.

Kairos (meaning the right or opportune moment) was built to bridge this gap. It is an end-to-end, zero-cost-idle intelligence pipeline that consumes raw news, extracts structured community-level insights using a custom-fine-tuned Llama 3 model, and delivers it to a real-time dashboard—all while utilizing high-performance serverless telemetry that prevents data loss without increasing latency.

What it does

Kairos operates in two distinct phases to provide an intelligent, real-time pipeline:

1. Model Fine-Tuning Pipeline (Phase 1): It reads raw documents from Google Cloud Firestore, uses a synthetic data factory script to query Gemini 1.5 Pro to generate ~100 high-quality training pairs, and fine-tunes a base model using QLoRA. The resulting lightweight adapter is hosted on the Hugging Face Model Hub.
2. Real-Time Inference & Ingestion Pipeline (Phase 2):
   • Automated Scraper: A GCP Cloud Scheduler runs hourly to invoke a scraper (news-scraper-v2) which queries NewsAPI for top news topics.
   • Serverless LLM Inference: The raw article text is sent to our serverless KAIROS API running on Modal (leveraging NVIDIA L4 GPUs). The fine-tuned Llama 3 model (kairos-llama3-finetune) processes the text and outputs structured JSON containing categories, severity levels, local impacts, and recommended community actions.
   • Real-Time Storage: The analysis is saved back to Firestore.
   • Live Dashboard: A responsive web dashboard uses the Firestore SDK to listen for real-time updates, displaying immediate notifications of community impact opportunities or alerts.

To ensure the serverless inference engine is production-ready, we implemented a custom OpenTelemetry observability pipeline with a Global Telemetry Cache (for rapid warm starts) and Instant-Flush Telemetry (forcing trace/log delivery via force_flush before container suspension).

How we built it

We utilized a modern, decoupled cloud and AI architecture:

• LLM Fine-Tuning & Hosting: Python, QLoRA, Gemini 1.5 Pro (for training data synthesis), and Hugging Face.
• Serverless Inference: Modal Container with NVIDIA L4 GPUs, FastAPI, and OpenTelemetry Python SDK (Tracer & Logger Providers).
• Data & Ingestion: Google Cloud Platform (GCP) Cloud Scheduler, GCP Cloud Run, Google Cloud Firestore (as a real-time database and document store), and Python news-scraper-v2 calling NewsAPI.
• Observability: OpenTelemetry standard semantic conventions for Generative AI (tracking prompts, completions, and token costs).
• User Interface: HTML5, CSS3, JavaScript, and the Firebase/Firestore Web SDK for live streaming data.

Challenges we faced

• Telemetry Loss in Serverless Functions: Traditional OpenTelemetry exporters push data in the background. In serverless containers (like Modal), execution is paused as soon as the response is returned, which causes trace spans and logs to be cut off and lost. We solved this by using a finally block to invoke provider.force_flush() and logger_provider.force_flush(), forcing all data to export synchronously before container suspension.
• Warm-Start Latency: Initializing OpenTelemetry providers on every container invocation added massive overhead. We resolved this by creating global cache variables that lazily initialize the telemetry pipeline only if it doesn't already exist, keeping subsequent warm requests extremely fast.
• Fine-Tuning Data Scarcity: To get a model that accurately categorizes complex global events into local impact opportunities, we needed precise training data. We solved this by building a synthetic dataset generator using Gemini 1.5 Pro, transforming raw documents into 100+ highly tailored Q&A instruction pairs.

Accomplishments that we are proud of

• Successfully bypassed the serverless "black hole" of telemetry, achieving 100% trace and log delivery on ephemeral GPU instances.
• Created a fully automated, hourly news ingestion loop that transforms raw web articles into clean, structured database records with zero manual intervention.
• Designed a system that costs $0.00 to run when idle, making it accessible to non-profits and students.

What we learned

• Mastered OpenTelemetry GenAI semantic conventions, including tracking input/output tokens, temperature, and specific model metadata (gen_ai.usage.prompt_tokens, gen_ai.usage.completion_tokens).
• Gained a deep understanding of Modal's serverless container lifecycle and the difference between cold and warm starts.
• Learned how QLoRA adapters allow running specialized LLM tasks with low memory footprints.

What's next for Kairos

• Local Language Support: Expanding the fine-tuning dataset to support multi-lingual article translation and analysis.
• Cost-to-Impact Analytics: Adding automatic cost-tracking to show exactly how much GPU execution cost was spent per extracted community opportunity.
• Push Notifications: Integrating web push alerts so community organizers are pinged immediately when high-severity local crises or critical volunteer needs are detected.

Built With

• css3
• fastapi
• gemini-1.5-pro
• google-cloud-firestore
• google-cloud-run
• google-cloud-scheduler
• grpc
• html5
• hugging-face
• javascript
• llama-3
• modal
• opentelemetry
• otlp
• python
• pytorch
• qlora