Inspiration

"Augmented Reality Gardener" takes care of the crops for you!

It is an automated serverless plant care management system, that monitors plant-related sensor data, uses weather forecasts, Data Mining and Machine Learning to predict & perform irrigation tasks using PID (Proportional Integral Derivative) watering control.

Features wireless, Low Power Wide Area Networks (LPWAN) kilometer-range IoT communication with LoRa (Long Range) message-passing. Presents real-time data using Augmented Reality.

alternativetext

What it does

Monitors

Locally, using HomeAssistant:

  • Soil: Humidity, Temperature, Fertility and Sun levels.

On the cloud:

  • Air: Atmospheric pressure, Relative humidity.
  • Weather: Local (lon/lat) weather future reports for rain/temp/humidity.

Data Mining:

  • Aggregates & interpolates (forward/backward fills missing) weather data, using Dark Sky API (https://darksky.net/dev).

Predicts:

  • Future metrics using multi-variate time-series prediction with Machine Learning (non-linear Artificial Neural Network regression).

Acts:

  • Gradually waters the plants with soil moisture% sensor feedback (using a software PID controller to open a water valve).

Repositories

Augmented Reality IoT Gardener (https://bitbucket.org/account/user/temp3rr0rprojects/projects/AUG ):

  1. Gardener Azure ML C2D Function (https://bitbucket.org/temp3rr0rprojects/gardener-azure-ml-c2d-function ): Invokes the Time-Series Azure ML Studio service and sends a Cloud2Device (C2D) message with the predicted statistics of the soil moisture % of the next 48 hours.

  2. Gardener Computer Vision (https://bitbucket.org/temp3rr0rprojects/gardener-computer-vision ): Detects a model image in a video feed. Draws real-time computer vision information & the model's translated geometry rectangle of the 3D plane. Also attaches the latest sensor/plant data from CosmosDB.

  3. Gardener IoT Hub Device (https://bitbucket.org/temp3rr0rprojects/gardener-iot-hub-device ): Azure cloud I/O communication (D2C and C2D) with .Net Core 2.0 and CosmosDB storage. Proportional Integral Derivative (PID) controller: Performs gradual watering using Soil Moisture % readings (https://en.wikipedia.org/wiki/PID_controller ).

  4. Gardener Azure ML Studio Data Mining (https://bitbucket.org/temp3rr0rprojects/gardener-azure-ml-studio-data-mining ): A python script for data mining of weather data (from Dark Sky API), used in Azure Machine Learning studio. Forward/backward fill interpolation is applied to missing data.

  5. Gardener PID Controller (https://bitbucket.org/temp3rr0rprojects/gardener-pid-controller ): Gardener Proportional Integral Derivative (PID) controller: Performs gradual watering using Soil Moisture % readings. Proof of concept project for simulation & calibration runs.

  6. Gardener IoT Edge ESP32 (https://bitbucket.org/temp3rr0rprojects/gardener-iot-edge-lora-esp32 ): RestFul API Control end-point for water valve and water flow sensing on a TTGO ESP32 LoRa SX1276 868/915MHz Bluetooth WiFi.

  7. Gardener IoT Edge LoRa MKR1300 (https://bitbucket.org/temp3rr0rprojects/gardener-iot-edge-lora-mkr1300 ): LoRa end-point for water valve control on an Arduino MKR 1300.

Technologies

  • Augmented Reality
  • Internet Of Things (IoT)
  • Data Mining:
    • Time-series data interpolation and forward/backward fill.
    • Weather Forecasts (48 hour window)
  • Machine Learning (ML):
    • Artificial Neural Networks (ANN)
    • Hyper-parameter Tuning
    • Cross-Validation ML Training
    • Multivariate Time-series Regression
    • Real-time Soil Humidity predicton (48 hour future window)
  • Control Theory:
    • Proportional Integral Derivative (PID)
  • Computer Vision:
    • Oriented FAST and rotated BRIEF (ORB) feature & descriptor detection
    • Hamming distance descriptor matching
    • Homography using Random sample consensus (RANSAC)
    • 3D-to-2D plane perspective transform & warping
    • Canny Edge detection
    • Classification & match thresholding
  • Communication:
    • LoRa (Long Range) for wireless, Low Power Wide Area Networks (LPWAN) IoT kilometer-range message passing interface
    • Representational State Transfer Application Programming Interface (RESTful API)
    • Advanced Message Queueing Protocol (AMQP)
    • Device-To-Cloud (D2C)
    • Cloud-To-Device (C2D)
    • Machine-To-Machine (M2M)
    • Bluetooth Low Energy (BLE)
    • WiFi
  • Azure:
    • IoT Hub
    • Functions
    • Machine Learning Studio Service
    • Cosmos DB

Languages, SDKs & Libraries

  • C#:
    • Net.Core 2.0
    • Azure.Documents
    • Linq
    • Json
    • Net.Http
    • Timers
    • Asynchronous Tasks
    • Collections.Generic
  • Python:
    • numpy
    • cv2 (Open Computer Vision)
    • pandas (for Data Mining)
    • requests
    • datetime
  • Arduino Ino
    • LoRa.h
    • WiFi.h
    • WiFiClient.h
    • WebServer.h
    • ESPmDNS.h
    • aREST.h
    • SPI.h

Hardware

Built With

Share this project:
×

Updates