AgenticBI

AgenticBI: Event-driven AI analytics on AWS. Upload a CSV to S3 and Lambda containers invoke Bedrock + Strands agents to detect anomalies, generate charts, and deliver insights in real time.

Comment

Inspiration

Traditional Business Intelligence (BI) dashboarding is slow, rigid, and demands constant manual oversight to spot critical data changes or anomalies. As data engineers, we wanted to build a "zero-overhead" analyst—an autonomous workflow that requires no infrastructure management, no hardcoded database schemas, and no manual chart-building. We were inspired to bridge enterprise event-driven cloud data pipelines with advanced autonomous AI agent workflows.

What it does

  • AgenticBI is an autonomous, serverless AI data analyst that monitors data streams completely unattended.
  • Automated Ingestion: The moment any arbitrary CSV dataset is dropped into an Amazon S3 input directory, a serverless event trigger wakes up our engine.
  • Autonomous Analysis: An AI Agent handles data cleaning, dynamically understands the data context regardless of the column names, and isolates anomalous data points or spikes.
  • Telemetry & Actions: The pipeline automatically exports a structured JSON report containing a metric breakdown and clear plain-language operational mitigation steps, alongside a custom Matplotlib visualization chart.

How we built it

We engineered a modern, decoupled cloud architecture designed to process heavy analytical tasks smoothly:

  • The Core Engine: Built as an OCI-compliant Docker Container Image hosted on Amazon ECR and executed via Python 3.12 AWS Lambda.
  • The AI Brain: The Strands Agents SDK orchestrated our agent tools, communicating via streaming responses (ConverseStream) with Amazon Bedrock's Nova Lite model.
  • Data & Visualization Engines: Pandas and NumPy were utilized for rapid column mapping and anomaly scanning, while Matplotlib generated real-time telemetry graphics.

Challenges we ran into

Building an autonomous system inside serverless boundaries presented major roadblocks:

  • The 250 MB Deployment Wall: Packaging pandas, numpy, matplotlib, and an AI agent SDK into standard AWS Lambda layers completely crushed the strict 250 MB unzipped storage limit, resulting in constant deployment failures. -- We overcame this by completely migrating our workflow into Docker container images, unlocking a 10 GB storage headroom.
  • AWS Serverless Boot Timeouts: Heavy Python analytics packages caused our container to take slightly over 10 seconds to initialize, triggering hard Lambda Phase: init Status: timeout crashes. -- We solved this by optimizing resource configuration limits up to 1024MB of memory and extending the timeout windows.
  • Strict Cloud Security and Marshaling Errors: Fine-tuning complex IAM policies for cross-service communication (S3 object access, Bedrock streaming permissions) required precise configuration, alongside resolving strict JSON string serialization constraints when exporting custom Python agent objects.

Accomplishments that we're proud of

  • Universal Domain Agnosticism: The system is truly generic. It can digest financial sales ledgers, IoT hardware logs, or healthcare tracking records without a single line of code changing, automatically figuring out the underlying data context.
  • Overcoming Infrastructure Limits: Building a production-ready, highly advanced AI pipeline entirely within browser-based environments like AWS CloudShell.

What we learned

We discovered that traditional serverless strategies are no longer sufficient when dealing with modern generative AI stacks. Heavy mathematical extensions demand containerized architectures. We also mastered the complexities of event-driven infrastructure, understanding how to securely map IAM permissions to autonomous code loops using AWS CLI automation.

What's next for AgenticBI

The immediate next logical phase is automating the deployment steps we engineered. We plan to wrap our entire architecture—including the input/output S3 buckets, folder structures, specific IAM execution policies, and container configurations—into a modular, repeatable Terraform configuration file. This will allow any team to spin up an isolated version of AgenticBI in their own AWS account in less than 60 seconds.

Built With

Share this project:

Updates

Submission history