VR portion of project
Brenda and Autonomous
Morrison and Autonomous
Social Local Mobile Robot (SoLoMoRo) Silicon Valley (HBO TV show); robots singing, robots helping sick people, 2001 a Space Odyssey, Virtual Reality, autonomous AI, IoT, and cloud messaging,
What it does
SoLoMo Robot is a personal assistant for the elderly that recognizes emotional states and automatically alerts the patient's care giver. This device is the first to combine Virtual Reality and Internet of Things to track the patient's position and provide an accurate location of the subject. After the robot triggers an alert, the care giver will be able to see the patient via telepresence and perform diagnostic triage.
How I built it
SoLoMoRo uses the APIs bellow:
- PubNub Connecting HTC Vive positioning detector with our robot assistant for automatic patient's tracking via the PubNub IoT API.
- Autonomous: Using the robot's API to automatically control the patient's movement and provide telepresence capability.
- Nexmo: Sending notifications via SMS API to alert the patient's doctor.
- Affectiva: Detecting the patient's emotions and analyze its emotional state.
- HTC Vive: Tracks accurate patient's position relative to the robot device.
Challenges I ran into
Getting the robot to start and stop by itself with automatic distance tracking and then sing a happy song after seeing a sad face was challenging Getting the robot to sync with the Vive motion sensors and match the VR world view on the headset
Accomplishments that I'm proud of
Combining APIs: Unity VR, Android SDK (phone and tablet), Autonomous AI Android robot, Nexmo, Pubnub remote device control for IOT, and cloud latency with position tracking in VR
What I learned
Many new APIs and integrations for IoT control, mood detection, and sms messaging integrated with VIVE, Android, and Autonomous AI
What's next for solomoro
Working to integrate more autonomous messaging, voice and position tracking into the application to help the caregiver to make an accurate diagnosis