Wata-Conscious

Consciously using water

Comment

Inspiration

I believe that once you become conscious of something, you can either do something or do nothing. If you care about your community or the people around you, it becomes difficult to for you not to anything. I also believe that a small effort by each member of a large community will result in a big change collectively. Thus if we can get 10% of the world population to use water consciously, conserving even a little, the collective effect will be huge. That is the aim of this project. To make people aware of their and members of their community water consumption.

What it does

Wata Conscious allows users to monitor their water consumption in real time. It also allows them to see how their water consumption compares with other members in their community.

How I built it

Wata Conscious consists of a number of different technologies and sensors.

Artik 5

I wrote a python program that runs on the artik. The role of this python program is to continuously log the water consume by the user via a water sensor. It reads the digital signal from a flow sensor attached to one of its GPIO pins and sends the reading to the kakfa producer. The code is shown below: 

import requests
import os
import time
from time import gmtime, strftime

ADC_PORT = "/sys/devices/126c0000.adc/iio:device0/in_voltage0_raw"
READ_PORT_COMMAND = "cat " + ADC_PORT
# get the sensor id from the envXS
SENSOR_ID = 1
API_URL = 'http://45.56.109.38:8090/'
REQUEST_STRING = API_URL + str(SENSOR_ID) + "/"

def send_data(value, timestamp, sensor_id):
    r = requests.post(REQUEST_STRING,data = {'timestamp': timestamp, 'value': value, })


def main():
    # read port
    # send data to rest api
    while True:
      water_usage =  os.system(READ_PORT_COMMAND)
      print("value " + str(water_usage) )
      time_stamp = strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime())
      send_data(water_usage, time_stamp, SENSOR_ID)
      time.sleep(1000)

if __name__ == "__main__":
    main()

Apache Kafka producer

I wrote an Apache Kafka application that receive messages from the artik and sends them to the log streamer. Using Apache Kafka will allow the application to scale to millions of users easily. The code for the producer is shown below. It implements a rest api, that accepts post from the artik board. Upon receiving these post it then put it in the kafka message queue. 

from kafka import KafkaProducer
from flask.ext.api import FlaskAPI

app = FlaskAPI(__name__)

from flask import request, url_for
from flask.ext.api import FlaskAPI, status, exceptions

app = FlaskAPI(__name__)
kafka_producer = KafkaProducer(bootstrap_servers='localhost:9092')


def sensor_repr(sensor_log):
    return {
        'text': sensor_log
    }

@app.route("/<int:sensor_id>/", methods=['POST'])
def log_readings(sensor_id):
    """
     log sensor reading
    """
    if request.method == 'POST':
        print("sending message to kafka")
        log = str(request.data.get('text', ''))
        kafka_producer.send('wata', log)
        return sensor_repr(log), status.HTTP_201_CREATED

if __name__ == "__main__":
    app.run(debug=True, host="0.0.0.0",port=8090)

Log Streamer

This was written using Apache spark, the purpose of the log streamer is to preprocess the user water consumption logs and store them into the application mysql database. It consume the messages place in the message queue by the Apache Kafka producer. It then writes the logs to the dashboard database. The code below is the listing for Log Streamer.

#######################
# This spark app receives user water consumption messages and writes them
# to the dashboard mysql database
#
# how to run
# spark-submit  --jars spark-streaming-kafka-assembly_2.10-1.6.1.jar  spark_sensor_log_streamer.py
# format of the input messages should be => timestamp, sensor-id, value
# this message will get stored in the mysql database, which can be later
# queried by the dashboard flask app
########################

## Imports
from pyspark import SparkContext
from pyspark.sql import SQLContext
from pyspark import SparkContext, SparkConf
from pyspark.streaming import StreamingContext
from pyspark.streaming.kafka import KafkaUtils
import json

## Module Constants
APP_NAME = "MADD CHALLENGE LOG STREAMER"

ZOOKEEPER = "localhost:2181"
TOPIC = "wata"

## Closure Functions
def save_logs_to_mysql(kafka_stream):
    logs = (kafka_stream.map(lambda message : log[1].split(","))
            .map(lambda log : Row(date = log[0], sensor_id= log[1], value= int(log[2]))))

    logs_df = sqlContext.createDataFrame(logs)
    mysql_url="jdbc:mysql://wata?user=wata&password=!wata"
    logs_df.write.jdbc(url=mysql_url, table="", mode="append")



## Main functionality
def main(sc):
    sql = SQLContext(sc)
    stream = StreamingContext(sc, 1) # 1 second window
    kafka_stream = KafkaUtils.createStream(stream, ZOOKEEPER, "wata-log-streamer", {TOPIC: 1})
    save_logs_to_mysql(kafka_stream)
    logs_df.show()
    stream.start()
    stream.awaitTermination()


if __name__ == "__main__":
    # Configure Spark
    conf = SparkConf().setAppName(APP_NAME)
    conf = conf.setMaster("local[*]")
    sc   = SparkContext(conf=conf)
    # Execute Main functionality
    main(sc)

The application dashboard

This is a web application written in flask. This app allows users

  1. to create an account and register thier water consumption sensor
  2. to view their water consumption via a dashboard
  3. to compare their usage against other users.

Challenges I ran into

  1. The artik board that was sent to me was bad. I spent quite a lot of time before I determined that it was bad. This was done with the help of artik support. The replace board took sometime to arrive.

  2. I was really stretch for time , I got a new Job, I had a new born. Thus all the time I had allocated to this project had to be reallocated to the new baby and the new job.

  3. Most of the time nothing worked :-), the artik documentation was not very clear. unlike the raspberry pi, there was little or no support material outside of the artik samsung site.

Accomplishments that I'm proud of

  1. Building a highly scalable IOT application
  2. Designing and building a dashboard
  3. Learning how to use the artik board
  4. Actually submitting a project, while taking care of a 8 months old.

What I learned

  1. How to interface a water sensor to artik
  2. How to use the artik board features i.e (GPIO, networking facilities)
  3. Updating a the artik board image
  4. Writing rest api's using flask
  5. theming web application
  6. creating a realtime dashboard

What's next for WataConscious

what conscious is still a work in progress. I want to complete the following features:

  1. visualizing this consumption data in a user accessible dashboard, having the following functionality:

    • provide a running cost of the water consume
    • provide a future projected cost (1 month, 3 mths , 1 year etc)
    • compare the user water consumption with other users
    • show total consumption of all users
    • visualize the environmental impact
      • greenhouse gas emission
      • cost to city
    • provide the use with actions he/she can take to reduce his/water consumption
    • show the impact of the action both for the user and the city

  2. Allow the user to share invite other persons to use the application

  3. allow the user to actual transfer the money he/her saved by reducing water consumption to a bank account

  4. future :partner with government institutions, so that users can put the money they save into there retirement savings.

Built With

Share this project:

Updates