ImagineAg

Farmers new and old rely on outdated planning methods. ImagineAg, a multi-modal agent, converts voice instructions into interactive 3D farm layouts—allowing real-time design with a simple phone call.

Comment

Inspiration

For the past few years, I’ve worked with schools and farmers to set up green tech and sustainability education platforms. As we built virtual farms for them, we noticed a common theme—most farm planning was still done with pencil and paper, whether it was a backyard garden or hundreds of acres. This highlighted a major gap: farmers needed a simpler, more intuitive way to visualize and adapt their layouts without relying on complex 2D/3D tools.

What it does

ImagineAg is a multi-modal agent that allows farmers to design 3D farm layouts through a simple phone call. It processes voice input, matches it to available Unity assets, and generates a structured 3D farm layout in real time. This removes the need for complex software, making farm planning more accessible, efficient, and collaborative to a group of members who are traditionally slow to adopt advanced technology.

How we built it

ImagineAg integrates:

  1. Voice processing & structured prompting via a multi-modal agent and Whisper
  2. Gemini API to interpret spoken input and match it with Unity assets
  3. Together API for additional AI-assisted reasoning 3a. Leverage Weave to monitor LLM progress/tracking
  4. Unity Cloud Functions to generate and load 3D environments dynamically
  5. Unity3D as the visualization and interaction layer

Challenges we ran into

  1. Getting consistent results from Gemini when matching spoken descriptions to available models
  2. Handling cases where the requested farm features don’t have a matching asset, requiring better fallback solutions
  3. Ensuring the generated layouts feel natural and usable while keeping processing lightweight

Accomplishments that we're proud of

  1. Successfully leveraging off-the-shelf AI tools to streamline an overlooked but critical process
  2. Reducing the technical barrier for farmers by replacing complex UI with simple voice interaction
  3. Bridging the gap between traditional planning methods and AI-powered automation

What we learned

  1. AI-generated 3D environments can significantly improve workflows for non-technical users
  2. Voice-driven interactions are powerful but require robust fallback mechanisms to handle missing assets
  3. There’s a real need for more accessible 3D planning tools in agriculture and other industries

What's next for ImagineAg

  1. I’m looking for my next opportunity(Ya boy need a job). I wanted to also integrate MM agents in my IoT controller that connects to these virtual worlds(digital twin). However, the need for accessible AI-powered farm planning remains, and this could easily be expanded into a standalone design tool or integrated into existing design/planning platforms.

Built With

Share this project:

Updates