EcoTwin AI

An AI digital twin that spots pollution spikes in real time, explains them using sensor data and camera images, warns residents by voice, and closes the loop when it's fixed.

Comment

Inspiration

Cities already collect a lot of environmental sensor data, but a spike on a chart doesn't protect anyone by itself. Someone has to notice it, figure out whether it's real, decide what to do, warn the people nearby, and then follow up until it's fixed. Each of those steps is usually manual, and a location that keeps having problems gets treated as a brand-new event every time. I wanted to see how much of that chain an AI agent could handle, from the first abnormal reading all the way to "this is fixed."

What it does

EcoTwin AI is an environmental digital twin for urban sensor networks. When a sensor (here, water turbidity at a runoff outfall) spikes:

  • Detects the anomaly in real time from a Tiger Data hypertable
  • Reasons about it with Gemini, using both the sensor trend and a camera frame of the site
  • Remembers the location's history and flags repeat offenders as a systemic pattern instead of an isolated event
  • Plans a response using only the resources that actually exist for that zone
  • Warns residents with a short spoken broadcast from ElevenLabs
  • Closes the loop: once readings stay safe, the incident is marked RESOLVED automatically
  • Rewards the response crew with a Solana bounty (mock, or a real devnet transaction)

You can also talk to it. Ask "What's the status of the downtown basin, and do we have field crews nearby?" and it answers out loud from live data.

How we built it

  • Tiger Data (TimescaleDB): sensor readings live in a hypertable. Every 2 seconds, a sliding-window query compares each node's last 10 minutes with the hour before. A node is flagged if it crosses its safety limit or its recent average jumps far above baseline:

$$ z = \frac{\bar{x}{\text{10 min}} - \mu{\text{baseline}}}{\sigma_{\text{baseline}}} \geq 3 $$

  • Gemini: gets the live numbers, a 5-minute trend, the camera frame, the node's incident history, and its list of response resources. It has to answer in a fixed JSON schema, so the pipeline never breaks on a malformed reply.
  • ElevenLabs: turns Gemini's two-sentence script into a voice alert, and also speaks the voice console's answers.
  • Pattern memory: an agent_memory table in Tiger Data. 2+ incidents at a node in 30 days = systemic, decided by a rule rather than a model guess.
  • Solana: resolved incidents trigger a transfer plus a memo naming the incident, sent on devnet with free test SOL (with a mock fallback).
  • FastAPI dashboard: a live chart, the incident panel with the camera frame, a voice console, and a simulated crew wallet page.

The sensors are simulated, but every other part of the pipeline makes real calls: live database queries, Gemini, ElevenLabs and Solana devnet.

Challenges we ran into

  • The model I started with was retired. The first real run returned a 404 from Gemini, so I made the model name a setting and pulled the list of models my key could actually use.
  • The AI invented infrastructure. An early action plan told crews to "engage automated containment gate valves" that don't exist. I fixed it by storing each zone's real resources in the database and only allowing Gemini to plan with those; anything else has to be "escalate to ".
  • Alerts repeated, then arrived late. A cooldown timer re-alerted on long spikes and blocked fresh ones. I replaced it with once-per-event alerting that re-arms only after readings recover.
  • Overlapping voices. Multiple players and tabs were playing the same warning. I added guards so each alert plays exactly once.
  • Couldn't sign up for Backboard, so I moved long-term memory into Tiger Data itself, which ended up simpler and faster.

Accomplishments that we're proud of

  • Sensor spike to spoken public warning in about 10 seconds, with timing for each step shown on the dashboard
  • Action plans that stay grounded in what the city actually has
  • A full closed loop: detect, explain, warn, resolve, reward
  • A voice console that answers from live data and says "I don't know" when the data doesn't cover it, instead of making something up

What we learned

  • How hypertables and sliding windows make real-time anomaly detection a single query
  • That grounding an AI agent in real constraints matters more than making its output sound impressive
  • Edge-triggered vs. level-triggered alerting, and why the difference matters to the people receiving the alerts
  • How a blockchain transaction is built and signed from scratch, and how to keep a test system from ever touching real funds

What's next for EcoTwin AI

  • Real hardware sensors and live camera feeds instead of simulated ones
  • Many nodes across a city, with a map view
  • Crew dispatch with real locations, so "nearby" becomes a real answer
  • SMS alerts to residents, using the notification resources already modeled
  • Smarter baselines that learn daily and seasonal patterns

Built With

Share this project:

Updates

Submission history