Smarty Plants

www.smartyplants.nyc

Inspiration

This was our team's first time building with hardware, and the environmental track felt like the perfect track to allow us to dive into that world. We decided to work with what we had: cardboard we found lying around, a plant we rescued, and whatever hardware was available at the event. Being resourceful became part of the project. If we couldn't buy it, we built it.

What it does

Smarty Plants turns an ordinary houseplant into a talking, self-watering, science-teaching classmate.

Senses: Soil moisture, light, temperature and humidity sensors, plus a webcam, watch the plant around the clock and report back every few seconds.
Thinks: An AI agent reads the sensors, looks at the plant's photos, and checks the local weather forecast to decide what the plant needs.
Cares: When the soil is dry, a servo-powered arm tips water into the pot. If rain is on the way, it skips watering and tracks how much water it saved.
Talks: Kids can chat with their plant, and it answers in its own personality, out loud, about how it's feeling and why. Ask it for a drink and it pours itself one. If it's already had plenty, it explains why too much water hurts and asks if you're sure.
Teaches: A live camera feed and charts of moisture, sunlight and leaf growth turn plant care into a hands-on lesson in photosynthesis, data and responsibility.

How we built it

Smarty Plants has four layers:

Sensors + servo → Arduino → Raspberry Pi → Cloud backend + AI agent → React dashboard

Arduino UNO R4 + Grove sensors: Reads soil moisture, light and a DHT11 (temperature and humidity), streams one line of data every 2 seconds over USB serial, and slowly tilts a servo to pour water when it receives a WATER command. It never waters on its own; the backend makes that call.
Raspberry Pi 4: Python scripts read and average the Arduino's serial data, buffer readings in a local SQLite database (so nothing is lost if the connection drops), stream the webcam, and score each photo for brightness and "% green" with Pillow and OpenCV. The Pi polls the backend for queued commands like water or photo.
Backend (FastAPI on Railway, in Docker): Stores sensor readings in Tiger Data (Postgres + TimescaleDB hypertables, using time_bucket() for chart averages), relays the live camera as an MJPEG stream, and runs the AI agent.
AI agent (Google Gemini): A hand-written tool loop gives the plant tools to read its sensors, pull reading history, check the Open-Meteo forecast, look at its latest photo, send phone alerts through ntfy, and water itself. It also runs routine check-ins on a timer when nobody is chatting.
Voice (ElevenLabs): The plant's replies are streamed as speech, with repeated lines cached to save credits.
Dashboard (React + Vite + Tailwind): Clerk sign-in, live meters, an animated plant face that shows its mood, the live camera, Recharts history charts, and a chat panel built with the Vercel AI SDK.

Challenges we ran into

The public Wi-Fi was our biggest enemy. Connecting to the Raspberry Pi on the venue network was unreliable, so we had to find workarounds just to talk to our own hardware. Moving the backend to the cloud turned out to be the fix: instead of reaching into the Pi, the Pi reaches out to the cloud, uploading data and picking up commands. That made the whole system far more stable. We also built the Pi to keep readings locally and catch up whenever the connection came back.

Accomplishments that we're proud of

We built a full-stack application that connects to real hardware and runs on real-time data, from a sensor in the soil all the way to a plant you can talk to in the browser. For a team building with hardware for the first time, getting every piece talking to every other piece was a huge win. More than anything, it was a learning experience, and we definitely want to keep building with hardware.

What we learned

We learned how a Raspberry Pi actually works: serial communication with an Arduino, managing a webcam, and running background services on a tiny computer. We also got to take everything we know from AI engineering and embed it into a makeshift, cardboard-and-tape project that interacts with the physical world. Seeing an AI agent decide to water a real plant, and then watching the servo actually pour, was something else.