Computer Vision detection of 2 dirty "cups"
Elderly Assistant 101
Early Elderly Assistant Architecture schematic scribble.
If IMU detect a fall, an Automatic Alert email is sent
Communication with DHT22 (Temperature & Humidity), connected to Home Assistant (Raspberry pi 2)
View from the camera that detects dirty dishes/cups
Voice enabled Events
GPGPU Computer vision runs on Jetson Tk1
Philips Hue Bulb
Philips Hue Hub (controls bulbs)
Smoke & Barometric pressure sensors, running on an ESP8266
Several years ago, a close family member deceased from a heart attack, without having the ability to call for help. This could have been prevented with today's applied technology.
Elders usually face difficulties in keeping up with technology. Internet of Things (IoT) platforms, even though they might be helpful, they are also difficult to control and interact with. Older people may enjoy an easier and safer every-day life with voice-interacting monitoring IoT platforms.
What it does
Elderly Assistant 101 is a:
Internet of Things device, that can monitor senior citizens and help them with household activities and chores.
- Fall Detection: Using the internal Inertial Measurement Unit (IMU) of arduino 101.
- Automatic alert/call for help: Via email i.e to a physician or family.
- Step Counter: Using the internal IMU of arduino 101.
- Sink Monitor: Computer vision that counts dirty cups or dishes.
- Smoke Detection: Via a gas & smoke detection sensor.
- Light Management: Philips hue bulb control.
- Tap-to-Call for Help: Triggered by the double-tap IMU event of arduino 101.
- Speak-to-Call for Help: Triggered by asking help via the Snips voice AI interface.
- Household Metrics:
- Humidity (%).
- Temperature (Celsius).
- Barometric Pressure (in kPa).
How we built it
- Genuino 101 (Microcontroller).
- ESP8266 (Microcontroller).
- Raspberry pi 3 (Embedded ARM computer).
- Nvidia Jetson Tk1 (Embedded ARM-GPGPU accelerated computer).
- Logitech C920 (USB camera).
- Philips Hue bridge and light bulbs.
- DHT22 (Humidity & Temperature sensor).
- BMP180 (Barometric Pressure sensor).
- MQ-2 (Smoke/Gas leakage detector).
- USB microphone & headphones.
- USB power bank.
Techniques & Technologies:
- Voice Recognition.
- Computer Vision.
- Restful Application Programing Interface (API).
- Publisher/Subscriber architecture.
- Inertial Measurement Units.
Software & libraries:
- Snips (AI Voice platform, https://snips.ai/).
- OpenCV (Open Computer Vision, http://opencv.org/).
- Python (https://www.python.org/).
- Node.js (https://nodejs.org/en/).
- Access to Home Assistant platform sensors (open-source home automation, https://home-assistant.io/).
- SQLite (https://www.sqlite.org/).
- PostgreSQL (https://www.postgresql.org/).
Challenges we ran into
The compatibility & versioning between the different software libraries.
Accomplishments that we're proud of
Coupling a wearable device with Voice User Interface and passive/active event handling.
What we learned
Given enough patience and drive, multiple technologies can be bind together.
What's next for Elderly Assistant 101
Integration with more automated abilities. And possibly a future integration with Telemedicine & a City Of Things ecosystem.