Augmented Reality Gardener logo
View of the tomato garden.
Azure IoT Serverless Architecture: Augmented Reality Gardener
Everything started from a seed!
Automated Watering. PID controller gradually achieves target soil moisture.
Tomatoes seed kit.
Soil Moisture Machine Learning prediction. Sample results from an Artificial Neural Network regression model.
Artificial Neural Networks had min Time-Series error. Cross-Validation and Hyper-parameter tuning model training vs other regression models
MiFlora BLE Sensor. Tracks plant data (Soil: Humidity, Conductivity, Solar exposure, Temperature)
Water Flow Sensor (Hall Effect). Measures the amount of water provided in mL.
Soil moisture % ML Service. The best trained Neural Network model, uses weather forecast & data mining to predict the next 48 hours.
Solenoid Valve. Provides water from the tank to the plant.
Relay Switch. Controls the 12 volt water valve.
ESP32 LoRa Wifi. Communicates between Windows IoT core (WiFi) and the remote valve (LoRa) at a kilometer-range (LPWAN)
Raspberry Pi 2B. Runs the Windows IoT Core operating System.
Raspberry Pi 3B. Runs the Hass.io operating system.
Arduino MKR1300 Lora. Communicates with ESP32 (LoRa) and controls the solenoid water valve at a kilometer-range (LPWAN).
Radana tomatoes. Sweet flavored and pear-shaped.
Water tank. Holds 5 liters of water.
Green Envy tomatoes. Grape sized cherry tomatoes.
Zlatava tomatoes. Round/heart shaped.
Common tomatoes. Seed extracted from supermarket's tomatoes.
Taiko tomatoes. When ripe, has bright yellow banana color.
"Augmented Reality Gardener" takes care of the crops for you!
It is an automated serverless plant care management system, that monitors plant-related sensor data, uses weather forecasts, Data Mining and Machine Learning to predict & perform irrigation tasks using PID (Proportional Integral Derivative) watering control.
Features wireless, Low Power Wide Area Networks (LPWAN) kilometer-range IoT communication with LoRa (Long Range) message-passing. Presents real-time data using Augmented Reality.
What it does
Locally, using HomeAssistant:
- Soil: Humidity, Temperature, Fertility and Sun levels.
On the cloud:
- Air: Atmospheric pressure, Relative humidity.
- Weather: Local (lon/lat) weather future reports for rain/temp/humidity.
- Aggregates & interpolates (forward/backward fills missing) weather data, using Dark Sky API (https://darksky.net/dev).
- Future metrics using multi-variate time-series prediction with Machine Learning (non-linear Artificial Neural Network regression).
- Gradually waters the plants with soil moisture% sensor feedback (using a software PID controller to open a water valve).
Augmented Reality IoT Gardener (https://bitbucket.org/account/user/temp3rr0rprojects/projects/AUG ):
Gardener Azure ML C2D Function (https://bitbucket.org/temp3rr0rprojects/gardener-azure-ml-c2d-function ): Invokes the Time-Series Azure ML Studio service and sends a Cloud2Device (C2D) message with the predicted statistics of the soil moisture % of the next 48 hours.
Gardener Computer Vision (https://bitbucket.org/temp3rr0rprojects/gardener-computer-vision ): Detects a model image in a video feed. Draws real-time computer vision information & the model's translated geometry rectangle of the 3D plane. Also attaches the latest sensor/plant data from CosmosDB.
Gardener IoT Hub Device (https://bitbucket.org/temp3rr0rprojects/gardener-iot-hub-device ): Azure cloud I/O communication (D2C and C2D) with .Net Core 2.0 and CosmosDB storage. Proportional Integral Derivative (PID) controller: Performs gradual watering using Soil Moisture % readings (https://en.wikipedia.org/wiki/PID_controller ).
Gardener Azure ML Studio Data Mining (https://bitbucket.org/temp3rr0rprojects/gardener-azure-ml-studio-data-mining ): A python script for data mining of weather data (from Dark Sky API), used in Azure Machine Learning studio. Forward/backward fill interpolation is applied to missing data.
Gardener PID Controller (https://bitbucket.org/temp3rr0rprojects/gardener-pid-controller ): Gardener Proportional Integral Derivative (PID) controller: Performs gradual watering using Soil Moisture % readings. Proof of concept project for simulation & calibration runs.
Gardener IoT Edge ESP32 (https://bitbucket.org/temp3rr0rprojects/gardener-iot-edge-lora-esp32 ): RestFul API Control end-point for water valve and water flow sensing on a TTGO ESP32 LoRa SX1276 868/915MHz Bluetooth WiFi.
Gardener IoT Edge LoRa MKR1300 (https://bitbucket.org/temp3rr0rprojects/gardener-iot-edge-lora-mkr1300 ): LoRa end-point for water valve control on an Arduino MKR 1300.
- Augmented Reality
- Internet Of Things (IoT)
- Data Mining:
- Time-series data interpolation and forward/backward fill.
- Weather Forecasts (48 hour window)
- Machine Learning (ML):
- Artificial Neural Networks (ANN)
- Hyper-parameter Tuning
- Cross-Validation ML Training
- Multivariate Time-series Regression
- Real-time Soil Humidity predicton (48 hour future window)
- Control Theory:
- Proportional Integral Derivative (PID)
- Computer Vision:
- Oriented FAST and rotated BRIEF (ORB) feature & descriptor detection
- Hamming distance descriptor matching
- Homography using Random sample consensus (RANSAC)
- 3D-to-2D plane perspective transform & warping
- Canny Edge detection
- Classification & match thresholding
- LoRa (Long Range) for wireless, Low Power Wide Area Networks (LPWAN) IoT kilometer-range message passing interface
- Representational State Transfer Application Programming Interface (RESTful API)
- Advanced Message Queueing Protocol (AMQP)
- Device-To-Cloud (D2C)
- Cloud-To-Device (C2D)
- Machine-To-Machine (M2M)
- Bluetooth Low Energy (BLE)
- IoT Hub
- Machine Learning Studio Service
- Cosmos DB
Languages, SDKs & Libraries
- Net.Core 2.0
- Asynchronous Tasks
- cv2 (Open Computer Vision)
- pandas (for Data Mining)
- Arduino Ino
- Tomatoes! Plant Theatre Wacky Tomato Kit (https://www.amazon.co.uk/gp/product/B01897NVEC/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1)
- MiFlora Plant Sensor (https://xiaomi-mi.com/sockets-and-sensors/xiaomi-huahuacaocao-flower-care-smart-monitor/)
- Solenoid Water Valve (https://www.amazon.de/gp/product/B07171LNYR/ref=oh_aui_detailpage_o08_s00?ie=UTF8&psc=1)
- Hall Effect Water Flow Sensor (https://www.adafruit.com/product/828)
- TTGO ESP32 LoRa SX1276 868/915MHz Bluetooth WiFi (https://www.amazon.com/ILS-Bluetooth-Internet-Antenna-Development/dp/B07BK225LT)
- Arduino MKR WAN 1300 (LoRa connectivity) (https://store.arduino.cc/mkr-wan-1300)
- Raspberry Pi 3B (https://www.amazon.co.uk/gp/product/B01CD5VC92/ref=oh_aui_detailpage_o06_s00?ie=UTF8&psc=1)
- Raspberry Pi 2B (https://www.amazon.co.uk/gp/product/B00T2U7R7I/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1)
- 5 Liter water container
- Relay Switch (https://www.adafruit.com/product/3191)