illuminAIte - AI Powered Data Conversations

A sample session showing a graph of random data

Inspiration

Data is everywhere in security. You have data about vulnerabilities, alerts, threats, forensic artifacts. It’s stored in a variety of formats, behind a variety of consoles, applications, command line tools, etc.

Making sense of that data is difficult. You need to format it, analyze it, decide what is useful, etc.

What it does

What if you could use AI to have a conversational interaction with your data in a way that lets you:

Rapidly get an overview
Have a conversation about the data
Gain insights, graphs, browser the data adhoc

IlluminAIte (a play on illuminate suggested by AI) aims to accomplish this by bringing data to an AI agent in conjunction with simple tools in a way that allows you to meet your data where it lies (.csv, .json files), get insights as quickly as possible and do it without having to write complicated queries or learn yet another language.

The best security tools are simple ones that do a job well. IlluminAIte aims to be a straightforward and easy to way to harness the power of AI to make sense of your data.

How we built it

Built using python, agno, shiny to run locally. It allows you to grab whatever csv or json files aide in your current situation (incident response, vuln management, forensics, etc) and tie them together in a conversational fashion saving you from all those sed/grep/awk/cut/sort/wc journeys.

Challenges we ran into

Models with tools are new. Models with repeatable outputs from tools in a way that is repeatable in UI and agentic frameworks are challenging. Multimodal tools like graphs and data grid display are not quite yet possible. Every model reacts differently to tools, prompts, environments, tool history, etc. Some are amazingly resilient even with errors in tools, some give up or hallucinate tools.

The open source repos behind this change frequently, often with breaking changes, often with enabling features. uv pip install git+ is a lifesaver!

Accomplishments that we're proud of

It works! Using particular models like gemini 1.5 flash delivered via vertex you can have a reliable conversation with data with context, memory and it will correct itself when it makes errors.

What we learned

A ton about individual models, tool calling, shiny for declarative UI using only python, duckDB's amazing flexibility and the power of LLMs + tools.

What's next for illuminAIte - AI Powered Data Conversations

Duckdb is a powerful proxy for access to local and other sources of data via community plugins. It would be interesting to experiment with duckDB->bigquery or other data lake sources.

Finding a reliable way to create a plotly tool that AI could use. Lots of async/await and reactive locks in the way and AI doesn't quite know or care about the UI layer.