FloraCast 3D

Split-Compare Earth for Vegetation Forecasting + Plant Identification

Comment

FloraCast 3D: A Global Phenology and Plant Identification System

FloraCast 3D is a comprehensive system designed for global phenology monitoring and advanced plant identification, presented through an interactive 3D globe view (CesiumJS). It integrates diverse environmental data, machine learning capabilities, and artificial intelligence to offer profound insights into Earth's vegetation health and dynamics.

Key functionalities of FloraCast 3D include:

  • NASA GIBS imagery layers with adjustable opacity and a time slider for historical analysis.
  • Dual-map compare mode for side-by-side data visualization.
  • NDVI/EVI charts and specialized phenology tools for vegetation health assessment.
  • Weather (NASA POWER) charts providing meteorological context.
  • iNaturalist photo/observation overlays (points + heatmap) for biodiversity insights.
  • An in-app ML forecast panel for predicting vegetation trends.
  • An AI assistant panel (optional; requires your own API key) for intelligent data interpretation.
  • Plant ID (Conservation): A core feature enabling users to upload plant/flower photos for remote identification suggestions using online APIs.

Plant ID is probabilistic. It serves as a preliminary identification tool, and verification with field guides or experts is recommended for conservation projects.

Project Story

Inspiration

FloraCast 3D was conceived from a fundamental recognition of the critical need to monitor and understand Earth's ecological systems. In an era marked by rapid environmental shifts, the precise tracking of plant Phenology and overall vegetation health is paramount. Observing the limitations of traditional, often localized monitoring approaches, we envisioned a unified platform. This platform would harness cutting-edge geospatial technologies, advanced machine learning, and artificial intelligence to deliver an intuitive and holistic perspective on global vegetation health. The ambition was to empower Conservation efforts and ecological research by providing an immersive 3D globe view, allowing users to explore and analyze environmental data with unprecedented depth and clarity, moving beyond conventional 2D representations.

What it does

FloraCast 3D functions as a sophisticated phenology monitoring and plant identification system, offering multi-dimensional data analysis through its interactive 3D globe view (powered by CesiumJS). The system is designed to facilitate the monitoring of vegetation health, accurate identification of plant species, forecasting of ecological trends, and intelligent decision support. Its integrated capabilities include:

  • Layers: Users can seamlessly integrate and visualize various satellite imagery Layers sourced from NASA GIBS. These layers, including true-color imagery and vegetation indices like NDVI/EVI, can be adjusted for opacity and explored across historical timelines using an intuitive time slider, enabling dynamic observation of Earth's surface transformations. A Dual-map compare mode further enhances analysis by allowing simultaneous comparison of different datasets or temporal snapshots.
  • Flower Photos: The system incorporates Flower Photos and observational data from iNaturalist, presenting them as interactive overlays on the globe. This feature provides rich visual context for species distribution and flowering patterns in specific geographic areas, serving as a valuable resource for both casual exploration and scientific inquiry.
  • Plant ID (Conservation): A cornerstone of FloraCast 3D, this feature offers remote Plant ID capabilities. Users can upload images of plants or flowers, which are then processed through advanced online APIs such as Pl@ntNet or Plant.id (Kindwise) to suggest species identifications. This tool is specifically designed to aid Conservation initiatives, providing a rapid initial assessment that can be further validated by experts, thereby supporting biodiversity protection and ecological studies.
  • Vegetation Index: The system provides comprehensive charts for key Vegetation Indexes, including the Normalized Difference Vegetation Index (NDVI) and the Enhanced Vegetation Index (EVI). These indices are fundamental metrics for assessing vegetation health and vigor. Users can analyze historical trends at specific locations, gaining insights into seasonal plant growth cycles and responses to environmental stressors.
  • Phenology: At its core, FloraCast 3D is a Phenology monitoring system. By synthesizing satellite data and vegetation indices, it enables users to track the spatial and temporal dynamics of plant life cycle events—such as budding, flowering, fruiting, and senescence. This capability is crucial for applications ranging from agricultural management and climate change research to broader ecosystem health assessments.
  • Weather: Integrated Weather data from the NASA POWER project allows users to access and visualize historical temperature and precipitation records for any chosen location. Presented in clear chart formats, this data helps in understanding the profound influence of meteorological conditions on vegetation growth and ecological processes.
  • ML Forecast: The system features an ML Forecast panel that leverages client-side machine learning (TensorFlow.js) to predict future NDVI/EVI trends over a 30-day horizon. Users can select various satellite products (e.g., MODIS or VIIRS) and target indices to train models, providing proactive insights for early warning of vegetation anomalies or strategic planning in agriculture and environmental management.
  • AI Assistant: An optional AI Assistant panel offers an intelligent interface for users. Powered by the OpenAI API (gpt-4o-mini), this assistant can respond to queries, analyze current map data, and provide context-aware suggestions, such as identifying suitable plant species for a given area. This enhances user engagement and facilitates deeper data interpretation and informed decision-making.

How we built it

FloraCast 3D was engineered using a robust and modern web technology stack, meticulously chosen to deliver high performance, scalability, and a rich user experience:

  • Frontend: The immersive user interface and the interactive 3D globe view are primarily driven by CesiumJS, a powerful open-source JavaScript library renowned for its capabilities in rendering world-class 3D geospatial visualizations on the web. The overall user interface and interaction logic are crafted using pure JavaScript, HTML, and CSS, ensuring a lightweight, responsive, and accessible design. For enhanced compatibility and specific mapping functionalities, leaflet-compat-cesium.js was integrated. Data visualization and charting components are expertly handled by Plotly.js.
  • Backend: The backend services are constructed with Node.js and the minimalist Express.js framework. Its primary role is to serve as an efficient API proxy, particularly for the plant identification features. It securely manages sensitive API keys and acts as a crucial intermediary between the frontend and external plant identification services, such as Pl@ntNet and Plant.id (Kindwise). Additionally, the backend orchestrates data requests from various sources, including iNaturalist and NASA POWER.
  • Data Sources:
    • NASA GIBS: Provides a comprehensive array of global satellite imagery layers, offering both real-time and historical data for environmental monitoring.
    • NASA POWER: Delivers extensive global meteorological datasets, essential for detailed weather analysis and its ecological implications.
    • iNaturalist: A vital source of global biodiversity observation data, contributing a wealth of plant photos and geolocated observations.
    • Pl@ntNet / Plant.id (Kindwise): These platforms provide the core plant species identification APIs that power FloraCast 3D's recognition capabilities.
  • Machine Learning: The ML Forecast functionality is implemented using TensorFlow.js, a JavaScript library that enables the training and deployment of machine learning models directly within the client's browser. This allows for localized and efficient forecasting of NDVI/EVI trends.
  • Artificial Intelligence: The AI Assistant feature is powered by the OpenAI API (specifically gpt-4o-mini), providing sophisticated natural language understanding and generation capabilities to enhance user interaction and data interpretation.

This architecture ensures a clear separation of concerns, with frontend code residing in the public/ directory and backend service logic in the server/ directory, promoting ease of development, maintenance, and future expansion.

Challenges we ran into

Developing FloraCast 3D presented a series of intricate challenges that required innovative solutions:

  • Optimizing 3D Globe Performance: While CesiumJS offers unparalleled 3D rendering, maintaining fluid performance when integrating numerous satellite imagery layers, iNaturalist observation points, and dynamic time slider animations was a continuous technical hurdle. We addressed this through meticulous data loading strategies, efficient rendering pipelines, and progressive detail management.
  • Seamless Multi-source Data Integration: The integration of disparate data from NASA GIBS, NASA POWER, and iNaturalist, each with unique formats and update frequencies, demanded sophisticated data processing and precise coordinate transformations to ensure accurate overlay and synchronized display on the 3D globe.
  • Robust Plant Identification API Integration: Integrating external APIs like Pl@ntNet and Plant.id (Kindwise) for plant identification involved complex tasks such as secure API key management, handling image uploads efficiently, robust error management, and standardizing the presentation of diverse identification results. Ensuring high accuracy and rapid response times, especially with varying qualities of user-submitted images, was a significant challenge.
  • Client-side Machine Learning Implementation: Deploying ML Forecast models using TensorFlow.js in the browser required a delicate balance between model complexity and browser performance constraints. Developing appropriate model architectures, fine-tuning training parameters, and effectively handling the time-series nature of historical data to generate meaningful predictions were key technical difficulties.
  • Intelligent AI Assistant Context Management: Designing the AI Assistant to accurately interpret user queries and provide contextually relevant information based on the current map view and loaded data was crucial. Maintaining conversational state and ensuring the AI's responses were both precise and helpful presented a significant challenge in achieving truly intelligent interaction.
  • Intuitive User Experience Design: Crafting an intuitive and uncluttered user interface for a system that presents multiple data layers, complex control options, and advanced analysis tools on a 3D globe was a paramount design consideration. The goal was to empower users without overwhelming them with information.

Accomplishments that we're proud of

We take immense pride in the following achievements of FloraCast 3D:

  • Groundbreaking 3D Global Visualization: The successful creation of an immersive 3D globe view that transforms traditional mapping into an engaging and intuitive exploration of Earth's environmental data.
  • Unified Multi-functional Data Platform: The seamless integration of geospatial, meteorological, and biodiversity data from leading authoritative sources (NASA, iNaturalist) into a single, comprehensive Earth observation platform.
  • Effective Plant Identification System: The development of an efficient and user-friendly Plant ID feature, which holds significant potential for Conservation efforts, serving as a powerful tool for citizen science and biodiversity protection.
  • Pioneering Client-side ML Forecasting: The implementation of client-side ML Forecast for NDVI/EVI directly within the browser, showcasing the robust capabilities of web technology in delivering proactive data analysis and predictive insights.
  • Intelligent AI Assistant Integration: The successful integration of an AI Assistant that provides intelligent data querying and analytical support, significantly lowering the barrier to data interpretation and enhancing the overall user experience.
  • Modular and Scalable Architecture: The adoption of a modular design, allowing for independent development and maintenance of various functional panels (such as Layers, Flower Photos, Plant ID, Vegetation Index, Phenology, Weather, ML Forecast, AI Assistant), thereby establishing a solid foundation for future growth and expansion.

What we learned

Throughout the development journey of FloraCast 3D, we acquired invaluable knowledge and insights:

  • Mastery of CesiumJS: Gained profound expertise in leveraging CesiumJS's advanced features, including sophisticated layer management, precise timeline controls, performance optimization techniques, and ensuring compatibility with other mapping libraries like Leaflet.
  • Advanced Geospatial Data Handling: Significantly enhanced our capabilities in processing and visualizing large-scale geospatial datasets, encompassing data transformation across diverse projection systems, efficient tile service integration, and optimized data rendering.
  • Best Practices for Secure API Integration: Learned the intricacies of securely and efficiently integrating third-party APIs, including robust API key management, effective rate limit handling, comprehensive error recovery mechanisms, and the standardization of data formats.
  • Client-side Machine Learning Realities: Gained a deeper understanding of the immense potential of TensorFlow.js for in-browser machine learning, alongside a clear appreciation for its practical limitations concerning model size, training duration, and computational resource demands.
  • AI-driven UX Design Principles: Explored innovative approaches to embed AI technology within the user interface, transforming it into a truly helpful assistant that provides contextually relevant and intelligent feedback, rather than a generic chatbot.
  • Interdisciplinary Collaboration Imperative: Reinforced the critical importance of integrating knowledge from diverse fields such as geographic information science, ecology, computer vision, and machine learning, highlighting its essential role in addressing complex environmental challenges effectively.

What's next for FloraCast 3D

The future trajectory for FloraCast 3D is rich with potential, and we envision expanding its capabilities in several key areas:

  • Advanced Phenology Models: Future efforts will focus on developing more sophisticated Phenology models. This includes incorporating a wider array of environmental factors (e.g., soil moisture, light intensity) and integrating cutting-edge machine learning techniques to achieve even greater accuracy and granularity in phenological event predictions.
  • Expanded Data Source Integration: We plan to continuously integrate additional high-resolution satellite data, ground-based sensor data, and citizen science observation data. This will further enrich the available data Layers and expand the dimensions for analysis.
  • User-Defined Analysis Tools: Empowering users with the ability to upload their own geospatial data and providing customizable analysis tools is a key objective. This could include features for regional vegetation change trend analysis or the development of specific species distribution models.
  • Mobile Application Development: Exploring the development of native mobile applications will enable users to perform real-time plant identification and data collection directly in the field, significantly bolstering Conservation efforts.
  • Community Collaboration Features: Introducing robust user collaboration functionalities, such as sharing observation data, facilitating the creation of Conservation projects, and enabling expert validation of plant identification results, will foster a vibrant biodiversity community.
  • Enhanced ML/AI Capabilities: We aim to apply more complex machine learning models, including deep learning architectures, to refine image recognition and prediction tasks. The AI Assistant's capabilities will also be expanded to support more in-depth ecosystem analysis and sophisticated decision-making.
  • Educational and Outreach Modules: Developing dedicated educational modules will serve to popularize knowledge about phenology, biodiversity Conservation, and remote sensing technologies among the general public, thereby raising environmental awareness and engagement.

Through these continuous advancements, FloraCast 3D is poised to evolve into an even more powerful, intelligent, and user-centric platform for global phenology monitoring and plant conservation.

Quick Start

To get FloraCast 3D up and running on your local machine, follow these steps:

Option A — Full app (recommended: includes Plant ID)

  1. Install Node.js 18+. You can download it from nodejs.org.
  2. In the project root, create .env from .env.example and add your key(s).
  3. Start the server: bash cd server npm install npm start
  4. Open your web browser and navigate to: http://localhost:5173

Option B — Static only (no Plant ID)

Open public/index.html directly in a browser.

Some features that depend on requests may be blocked by the browser when opened as a local file. Use the server for best results.

API Keys

To enable full functionality, you will need API keys for the following services. Set them in your .env file (created in the Quick Start section):

  • Pl@ntNet: Set PLANTNET_API_KEY in .env.
  • Plant.id (Kindwise): Set PLANTID_API_KEY in .env (optional).
  • OpenAI: Set OPENAI_API_KEY in .env (optional, for AI Assistant).

Notes on the 3D Globe

This build uses CesiumJS via CDN. Internet access is required to load Cesium and some imagery tiles.

Project Structure

The project is organized into the following main directories:

  • public/ – Frontend (HTML/CSS/JS)
    • leaflet-compat-cesium.js – Leaflet API compatibility layer mapped to Cesium
    • plantid.js – Plant ID panel logic
    • app.js – Main application logic
  • server/ – Express server + proxy endpoints for plant ID

Troubleshooting

  • Globe is blank: Check that your browser allows WebGL and that the Cesium CDN is reachable.
  • Plant ID says "API key not set": Add the required key in .env and restart the server.
  • CORS errors: Run via npm start instead of opening the file directly.

Technologies Used

  • Frontend: HTML, CSS, JavaScript, CesiumJS, Leaflet, Plotly.js, TensorFlow.js, Marked.js
  • Backend: Node.js, Express.js, Axios, Multer, Dotenv
  • APIs: Pl@ntNet API, Plant.id (Kindwise) API, OpenAI API

External Resources & Data Credits

FloraCast 3D leverages a variety of external data sources, APIs, and libraries to provide its comprehensive phenology monitoring and plant identification capabilities. We extend our gratitude to the providers of these invaluable resources:

  • NASA Global Imagery Browse Services (GIBS): Provides a vast collection of global satellite imagery layers, enabling dynamic visualization of Earth's surface changes over time. [https://gibs.earthdata.nasa.gov]
  • NASA Prediction Of Worldwide Energy Resources (POWER): Offers global meteorological datasets, crucial for analyzing weather patterns and their impact on vegetation. [https://power.larc.nasa.gov]
  • iNaturalist: A global citizen science project providing biodiversity observation data, including a rich collection of plant photos and geolocated observations that enhance our understanding of species distribution. [https://www.inaturalist.org]
  • Pl@ntNet: An online platform and API for plant identification through image recognition, supporting our Plant ID feature. [https://my-api.plantnet.org]
  • Plant.id (Kindwise): Another robust API for remote plant species identification, offering an alternative provider for our Plant ID functionality. [https://api.plant.id]
  • OpenPortGuide: Provides weather-related overlay data, such as wind streams and precipitation, for enhanced environmental monitoring. [https://weather.openportguide.de]
  • OpenStreetMap Tiles: Offers foundational map data for general geographic context. [https://openstreetmap.org]
  • OSM Nominatim: A geocoding service used for converting addresses and place names into geographic coordinates. [https://nominatim.openstreetmap.org]
  • CesiumJS: An open-source JavaScript library for creating world-class 3D globes and maps, forming the core of FloraCast 3D's immersive visualization. [https://cesium.com/cesiumjs/]
  • Leaflet: A lightweight JavaScript library for mobile-friendly interactive maps, used for compatibility layers within the application. [https://leafletjs.com/]
  • Plotly.js: A high-level, declarative charting library used for rendering interactive data visualizations, particularly for vegetation indices and weather charts. [https://plotly.com/javascript/]
  • TensorFlow.js: A JavaScript library for training and deploying machine learning models in the browser, enabling client-side NDVI/EVI forecasting. [https://www.tensorflow.org/js]
  • OpenAI API: Powers the AI Assistant feature, providing advanced natural language understanding and generation capabilities. [https://openai.com/docs/api/]
  • Marked.js: A Markdown parser and compiler, used for rendering Markdown content within the application, particularly for AI Assistant responses. [https://marked.js.org/]
  • Node.js: The JavaScript runtime environment used for the backend server. [https://nodejs.org/]
  • Express.js: A fast, unopinionated, minimalist web framework for Node.js, forming the foundation of our backend API. [https://expressjs.com/]
  • Axios: A promise-based HTTP client for the browser and Node.js, used for making API requests from the backend. [https://axios-http.com/]
  • Multer: A Node.js middleware for handling multipart/form-data, primarily used for file uploads in the plant identification process. [https://github.com/expressjs/multer]
  • Dotenv: A zero-dependency module that loads environment variables from a .env file into process.env, used for secure API key management. [https://github.com/motdotla/dotenv]
  • FormData: A web API interface that represents an HTML form and its fields, used for constructing multipart/form-data requests. [https://developer.mozilla.org/en-US/docs/Web/API/FormData]

Built With

Share this project:

Updates