Project Green Thumb

Restoring ecosystems one yard at a time.

Comment

Inspiration

The idea for Project Green Thumb came from a problem we kept running into: a great amount of people want yards that look good while also supporting the environment, but the tools available today usually focus on one or the other.

Most landscaping tools either give you generic plant recommendations or create beautiful visualizations without considering whether those plants actually belong in the local ecosystem. After talking with landscapers and an ecologist, we realized that things like native ranges, county-level data, hardiness zones, sunlight, slope, and water conditions need to be part of the design process from the beginning.

We wanted to build something that could be useful to homeowners, landscapers, and environmentalists without requiring someone to be an expert in horticulture.

That became Project Green Thumb: a platform that combines AI, ecological research, and landscape design tools to help people create outdoor spaces that look great while supporting native plants and healthier local ecosystems.

What it does

Project Green Thumb is an AI-powered landscape design app that helps take someone from understanding their property conditions to creating a complete planting plan they can actually use.

The home screen pulls live, plant-relevant weather data from Open-Meteo. Instead of only showing temperature, we include information such as soil conditions, vapor pressure deficit, UV levels, evapotranspiration, and hardiness-related data that can affect how plants grow.

Users can search through our curated plant database using intelligent full-text search. Each plant has a detailed profile that includes ecological information, growing conditions, and visual maps showing its natural range.

One feature we spent a lot of time on is our mapping system. We display county-level wild-range data and USDA hardiness zones directly on Apple Maps. This allows users to see where a species naturally occurs and whether it is appropriate for their location instead of relying on overly broad range maps.

The Design section is where everything comes together with the GPT 5.6 Sol model.

Users can start with a blank property, import a 3D model, create a plant palette, or use LiDAR to scan an existing yard.

While the scan is processing, the app asks questions inspired by conversations with professional landscapers. We then analyze the property for things such as plantable area, slope, elevation, and water retention.

From there, the system can recommend native and climate-appropriate plants and automatically place them within the landscape while considering constraints such as foundation setbacks, mature canopy size, and plant spacing.

Users can then manually adjust the layout and view additional site information such as:

  • Sun paths
  • Drainage and water retention
  • Elevation
  • Plant spacing
  • Other environmental overlays

Once the design is finished, it can be exported as a PDF that a homeowner or landscaper can take directly to the property.

One of the newest features we are especially excited about is our photorealistic 3D concept rendering pipeline using OpenAI's gpt-image-2 model.

Instead of asking an AI model to invent an entirely new backyard, we first generate meter-scaled planting guides directly from the measured yard geometry. Those guides preserve the real camera position, buildings, terrain, plant locations, scale, and occlusion.

We then use the image edits API to turn those planting guides into realistic botanical visualizations.

The result is a concept render that looks much more like a finished landscape while still staying grounded in the actual scanned property.

How we built it

We built the MVP through a lot of experimentation and iteration, with a significant amount of development done using Codex and GPT-5.6 alongside technologies such as computer vision, MapKit, ARKit LiDAR, and our ecological plant database.

Data

A major part of Project Green Thumb has always been the data.

Over the past several years, we have researched and collected information covering more than 200,000 plants.

For the current MVP, we focused on approximately 200 deeply curated species with detailed ecological information, including county-level distribution data.

Seeing several years of plant research finally become part of an interactive product has been one of the most rewarding parts of the project.

Plant visuals

To create realistic 3D plant assets, we use SAM 3.1 to segment licensed plant photography and isolate high-quality plant cutouts.

We then use SAM 3D Objects to turn strong 2D references into 3D assets.

One challenge we are still working through is distribution. High-quality plant models can quickly make an app extremely large, so we are exploring cloud-based hosting rather than shipping more than a gigabyte of models directly with the application.

We are also careful about image licensing throughout this process.

Maps

For ecological range data, we preprocess county geometry and hardiness information into mobile-friendly map overlays.

These overlays are displayed using Apple Maps so users can quickly compare where a plant naturally occurs with the climate conditions of their own location.

A lot of work went into making these maps accurate without overwhelming the device with thousands of polygons at once.

Scanning and landscape design

For property scanning, we built an Ultra-mode LiDAR capture workflow using ARKit scene reconstruction.

The scan is converted into geometry that we can analyze on-device using DEM-style terrain processing.

From that data, we calculate information such as:

  • Slope
  • Elevation
  • Plantable areas
  • Water-retention patterns
  • Site constraints

Those results feed into the recommendation and layout systems.

Plants can then be automatically positioned while accounting for mature plant size, spacing requirements, foundations, and other constraints.

The full workflow, from scanning a yard to generating an editable landscape plan and exporting a PDF, lives inside the Design section of the app.

Codex also played a major role in helping us rapidly build and iterate on this workflow.

Concept renders

For our photorealistic concept rendering system, we intentionally made rendering a separate step rather than something that happens automatically.

The workflow is:

  1. Capture or load the measured yard geometry.
  2. Generate plant cutout views at the correct real-world scale.
  3. Preserve the camera, terrain, structures, placement, and occlusion.
  4. Optionally add design notes or stylistic guidance.
  5. Send the prepared image through the gpt-image-2 image edits API.
  6. Save the resulting photorealistic concept render.

This gives us a presentation layer on top of the ecological data and measured landscape design.

The underlying layout stays grounded in real geometry, while the final image helps a homeowner or client understand what the finished landscape could actually look like.

Challenges we ran into

One of the hardest problems was building an intelligent outdoor scanner.

Outdoor environments are much more difficult than scanning a clean indoor room. There can be vegetation, uneven terrain, large open areas, noisy LiDAR data, and a huge amount of geometry.

Our Ultra scanning mode needed to remain detailed without exhausting the phone's memory.

We eventually built a system using disk-backed frame storage, chunked exports, and strict vertex budgets so larger outdoor scans could be processed more reliably.

Search was another major challenge.

Earlier versions of our search system were slow and did not work well with a database containing horticultural names, scientific taxonomy, ecological characteristics, and common-language queries.

We ended up rebuilding the search experience so results could be intelligently ranked across the plant database.

Another difficult problem was generating realistic landscape visualizations without letting AI completely redesign the property.

It is easy for an image model to create a beautiful backyard. It is much harder to make it respect the actual building, terrain, camera angle, plant placement, scale, and occlusion from a real scan.

After a lot of experimentation, the gpt-image-2 editing pipeline gave us a way to enhance the planting visualization while keeping the original property geometry intact.

The mapping system also presented its own challenges.

County-level plant ranges and hardiness overlays contain a large amount of geographic data, and sending thousands of raw polygons directly into MapKit can quickly hurt performance.

We had to preprocess and optimize that data so the maps remained both accurate and responsive on a mobile device.

Accomplishments that we're proud of

What we are most proud of is that Project Green Thumb became much more than a chatbot that recommends plant names.

We built the foundation of an AI-driven ecological landscape design platform.

The current system includes:

  • A modern liquid-glass interface
  • Live plant-relevant weather data from Open-Meteo
  • Intelligent plant search
  • County-level native range maps
  • USDA hardiness overlays
  • LiDAR yard scanning
  • Landscaper-inspired site questionnaires
  • Ecology-aware plant recommendations
  • Automatic plant placement
  • Editable landscape layouts
  • PDF plan exports
  • Sun, elevation, and water-retention overlays
  • A pipeline for creating curated 3D plant assets
  • Photorealistic concept renders powered by gpt-image-2

We are also proud of the depth of the ecological data behind the platform.

Our MVP currently exposes around 200 highly curated species with county-level information, backed by a much larger plant dataset containing information on more than 200,000 plants.

That data is what allows the AI recommendations to be based on real ecological information rather than simply generating plausible-sounding plant suggestions.

What we learned

One of the biggest things we learned is that high-quality data matters just as much as having a powerful AI model.

For ecological landscape design, information such as native range, county-level occurrence, hardiness, invasiveness, plant characteristics, and site conditions is what makes a recommendation trustworthy.

That information has to be researched and validated. It cannot simply be generated from a prompt.

We also learned that outdoor scanning is not just a computer vision problem. It is an entire systems problem involving memory management, mesh quality, confidence, storage, export, terrain analysis, and mobile performance.

Another important lesson came from working with generative AI.

AI visualization works much better when the model is given a measured, world-scale design as its foundation.

Rather than asking AI to invent a landscape, we provide it with the actual property geometry and plant placements and use the model to enhance that information visually.

Finally, combining feedback from landscapers and an ecologist with rapid iteration using Codex helped us move from an idea into something that can now be demonstrated from beginning to end.

What's next for Project Green Thumb

Our next goal is to continue expanding the curated plant database and improving the quality of AI-generated landscape recommendations and plant placement.

We also want to push the AR experience further so users can visualize proposed plants directly in their real yards before anything is planted.

Another major focus is making property scanning more automatic. Ideally, a user should be able to scan their yard and have the system identify usable planting areas, understand site conditions, recommend plants, and generate an initial layout with as little manual setup as possible.

On the 3D side, we plan to move plant models and generated renders to cloud-based hosting. This will allow us to maintain a much larger library of high-quality assets without making the mobile application unnecessarily large.

We also plan to continue improving the gpt-image-2 concept rendering pipeline, making it easier to turn a scientifically grounded landscape plan into a realistic client-facing visualization.

Ultimately, the goal is to take what is currently a working local prototype and turn Project Green Thumb into a production platform that makes ecological landscape design accessible to homeowners, landscapers, and communities at a much larger scale.

Built With

Share this project:

Updates