Daydream Lighthouse

• 

  VR Application

• 

  Tracking server

• 

  Training set generation

Inspiration

There is head tracking for advanced headsets like the HTC Vive, but not for WebVR, or Google Daydream.

What it does

Converts a RGB camera into a tracker for the Google Daydream. Users can even choose to build their own classifiers for different headsets.

How we built it

On the front end, there is ReactVR, which calls through express.js to our backend. The server is a python OpenCV application that uses our classifier to determine the location of the headset in space.

To generate the classifier, we used OpenCV and C++ to automate building of a training set. Then the collection, selection, and training of the samples was automating using a bash script.

Challenges we ran into

• Debugging Chrome for Android
• WebVR blocks XMLHttpRequests
• Mac times out ports
• Mac does not communicate well with Android
• Little documentation for WebVR APIs
• Cross origin request denial
• Automating the sample generation
• Poor suppression of false positives
• Bad OpenCV documentation
• Failure of markerless tracking
• Data plumbing
• Request limit on ngrok
• Sensor drift on Android accelerometers
• Bracket wrangling in Python3 and Javascript
• Debugging Chrome for Android
• Damaged USB-C port on the only vr ready phone for Google Daydream
• Staying awake
• Downloading and installing the same version of opencv
• Tabs versus spaces
• Debugging Chrome for Android
• libcurl problems in C++
• scrapping C++ and starting over in python
• proxy time outs

Accomplishments that we're proud of

It works on every platform, except in VR!

What we learned

• Commit and push often

What's next for Daydream Lighthouse

Waiting for more stable WebVR!

Built With

• bash
• c++
• express.js
• github
• javascript
• machine-learning
• mongodb
• mongoose
• node.js
• npm
• numpy
• opencv
• python
• react
• react-native
• reactvr
• requests
• ssl
• vr