VR exercise motion controller

A bluetooth motion controller that can be used in VR for exercise and gaming

Comment

Inspiration

Chris had worked on a Google Cardboard game before, as well as worked heavily with the Google Daydream pre-release, using an extra phone as a controller. When trying to figure out something that showed off the Intel Curie's specific features (accelerometer, gyroscope, Bluetooth Low energy), we had thought some kind of wearable device to track movement would be useful to show off those features. Additionally, we could use the power of arduino to add more inputs to make it into optionally a full VR game controller as well. So you can use it to exercise in VR using the google cardboard, or use it as a motion sensitive game controller in VR, just on the phone, or directly through your desktop using the serial output (and later BLE on desktop as well)!

What it does

The VR exercise motion controller is able to track its orientation (rotation in all 3 axises), as well as has a joystick to allow movement in VR or just regular games/ applications. It also has buttons that can be used in either portrait orientation (using the motion controller like a wand, using the joystick with your thumb, using the Z button on the back for gripping), or in landscape (using the motion controller like a wheel of a car, using the joystick with your thumb, and the 2 other buttons as conventional A and B buttons - such as for jumping and hitting).

How we built it

We first started with just trying to get the orientation displayed in 3d with reasonable accuracy, using the accelerometer and gyroscope. Then we added the joystick, and mapped the X,Y and switch on it. We added several buttons to test them out, then planned how many we would need based on the Vive controller, Wii controller, and the Daydream controller, which meant one for gripping, one for resetting orientation due to drift, and 2 for conventional A and B buttons.

We soldered all the wires into the joystick module, attached it to the arduino. We soldered 10K ohm resistors to each of the buttons, as well as ground, input signal, and 5V line, and connected them to the arduino as well.

Next we built the frame out of cardboard, constraining it to the proportions of the Arduino 101 board, with a breadboard shield attached to the top, and affixed it to the bottom using velcro. We looked at the other motion controllers to get a good distance for the thumb joystick from the bottom of the controller (about 4.5" to center). We then added the Z button on the back of the device using hot glue, and glued the joystick in as well. We then created the top of the controller to cover the dimensions left over, and glues in the reset button near the joystick, and the A and B buttons towards the bottom, again using spacing relative to what the wii controller uses.

Next we assembled everything together, using hot glue to attach all the buttons and joystick to their positions on the cardboard frame. Lastly, we held the top of the controller on using rubber bands as an easy way to get access to the arduino. We had tried velcro, but it added too much to the width of the controller.

Challenges we ran into

The first challenge was trying to come up with ideas that used the Intel Curies strengths, as well as had a large potential impact on daily lives. The original idea was to experiment with all the grove sensors, as well as the BLE and accelerometer/ gyroscope. We ended up deciding to use the 3d position as a controller with extra sensors, but were planning on submitting multiple ideas, but ran out of time.

The next challenge was trying to get the orientation in 3d to stay reasonable accurate. At first the yaw was drifting several degrees per second. We eventually found the code to calibrate the offset of the gyroscope and accelerometer, which massively slowed down the drift. It was reasonably accurate, but still ended up getting out of sync after a while, or drifting slowly over time. We had to play with the values related to the Madgwick filter to get more acceptable results. It was difficult finding other articles about breaking down the different filters you can use to get the 3d position from the sensors. Most forums stated that it wouldn't be super accurate without a magnetometer, but there weren't any on Amazon that had good reviews, so we ended up getting an adafruit-9 dof, and were planning on using it for a secondary controller connected to the primary, so we could visualize both hands in VR, but ran out of time. So we would prefer to find a way to limit the amount the controller drifts compared to where it actually is, but ran out of time, but the current results are acceptable, and we added a reset button to reset the orientation whenever it drifts too far.

The BLE was also a problem on the Android end. We only found one article from Android on interfacing with BLE, but most sites references pre-built apps for receiving/ sending over BLE. We ended up getting a backend that worked reasonably in Unity, but it would only allow us to receive one characteristic at a time, so we would have prefer to fix that limitation.

The BLE also wasn't perfectly reliable on the arduino end, most of the time working with no problems, but occasionally the arduino would stop responding, and obviously ran into a crash, but we weren't able to track down and figure what was happening. It wouldn't happen when connected via serial, so is a problem when sending over BLE, but very inconsistent. But it would have been nice to have more time to track it down and fix it.

Accomplishments that we're proud of

We're proud we were able to get the 3d positioning working reasonably well, and interfaced with bluetooth, working wirelessly with the phone in Unity.

It was awesome to see the project coming together, from just starting with the arduino connected to the computer to having the case built, and the controller looking similar to other motion controllers.

We were happy that we found a project that met most of the requirements of using the BLE and motion sensors, as well as could be used in daily lives by allowing people to exercise in VR as part of their daily routine.

What we learned

We learned a lot about arduino. We spent a lot of time learning about each of the grove sensors, the built in Curie modules, and just about how to use the arduino in general. We hadn't used an arduino previously, and only have brief experience with a Parallax micro controller many years ago, so everything was new to us.

We learned how to test each sensor out using the included examples, how to communicate over the serial port, over BLE, how to connect sensors, and how to read out values over both interfaces. We also learned how to construct a prototype box using spare cardboard we had from amazon, and that the buttons/ joystick/ board can be held in place well with hot glue. We also learned that it would have been better to finish the first project with what we had, instead of spending so much time learning how all the sensors worked first. If we had done that, we might have had more time to finish and submit more projects (as well as polish up each one more/ get rid of all the bugs).

What's next for VR exercise motion controller

The next thing will be to fix all the little issues that ocassionally come up, so the controller is perfectly reliable. Then from there we would add the second motion sensor, to use similar to the nunchuck on the wii, so we can track both hands rotation in VR, and build a body for that as well. Then we would add a battery to make it completely wireless. We would then look for a smaller board to use in the controller, to get the size down a little bit, and use the 101 for prototyping future projects. Finally, we could get a final body 3d printed, and assembled, and look into seeing where we could get it mass produced, and see about starting a kickstarter, or some kind of other crowd funding, to see if more people are interested in having inexpensive, customizable and programmable VR controllers they could use with both hands.

Built With

Share this project:

Updates