OraclePlant
Inspiration
I’ve always been fascinated by the intersection of IoT, AI, and human-plant interaction. I wanted to create a project that brings plants “to life,” giving them a voice to express their needs and moods. Watching smart gardens and AI assistants inspired me to combine sensor data with AI-generated messages and text-to-speech technology.
What it does
OraclePlant is an interactive IoT system that monitors your plant’s environment and generates fun, personalized audio messages about its mood. Using sensors for temperature, humidity, light, and soil moisture, the system calculates the plant’s "mood" and uses AI (Gemini) to create a short message. This message is then converted to speech using ElevenLabs TTS, so your plant can “talk” to you in real-time.
How we built it
- Hardware & IoT: Sensors simulate plant conditions (temperature, humidity, light, moisture).
- Server & API: Node.js + Express handles incoming sensor data, mood calculations, AI generation, and TTS audio creation.
- AI Integration: Gemini API generates fun, plant-specific messages based on sensor readings.
- Text-to-Speech: ElevenLabs API converts the AI-generated messages into audio files that can be played in a web dashboard.
- Dashboard: A simple front-end polls the server and plays new audio whenever the plant's mood changes.
Challenges we ran into
- Managing asynchronous calls to Gemini and ElevenLabs APIs without blocking the server.
- Avoiding duplicate audio generation and handling cooldowns between messages.
- Handling 401 and other authentication errors with ElevenLabs API.
- Ensuring the front-end updates reliably with new audio and sensor data.
- Sometimes problems are not your fault — it’s a learning process, and debugging those issues was key.
Accomplishments that we're proud of
- Successfully gave a “voice” to plant sensor data.
- Seamless integration between IoT simulation, AI, and text-to-speech.
- Implemented a robust polling system on the dashboard that avoids overlap and plays audio reliably.
- Learned to handle API rate limits, authentication errors, and asynchronous programming in Node.js.
- Discovered the limits of ElevenLabs API first-hand — yes, I “broke” it!
- Learned that Gemini TTS is a paid feature, which taught me to work within API constraints.
What we learned
- How to combine IoT data with AI-generated content and TTS.
- Best practices for API error handling and cooldown management.
- How to build a responsive web dashboard that reacts to real-time data.
- The importance of debugging asynchronous workflows in Node.js.
- University networks can be tricky, sometimes rejecting seemingly reasonable requests.
- Unexpected API behaviors can happen — like “unusual activity detected” messages — and you learn to adapt.
What's next for OraclePlant
- Add real sensors for a physical plant setup.
- Enhance the AI messages with more personality and context awareness.
- Include multi-plant support, so each plant can have a unique voice.
- Explore mobile notifications or integration with smart home devices for plant care alerts.
- Add Spotify integration for “in-the-mood” plants, exploring studies that suggest sound affects plant metabolism.
- Build a mesh network of multiple plants so they can “communicate” with each other.
- Could this be used in aquariums to monitor water parameters, meaning water can be saved by only doing water changes when needed.
Built With
- audio-playback-with-html-`<audio>`-element
- axios
- built-with-**languages:**-javascript-(node.js)
- but-gemini-and-elevenlabs-are-cloud-apis-**databases:**-none-(state-is-kept-in-memory)-**other-tools:**-web-browser-for-dashboard
- css-**frameworks-&-libraries:**-express.js
- dotenv-**platforms:**-raspberry-pi-(simulated-sensors)-**apis:**-gemini-api-(ai-message-generation)
- elevenlabs-tts-api-(text-to-speech)-**cloud-services:**-none-(all-processing-is-local)
- html
- in
- python
- script
- simulation
