RotoVR, 3DS Edition

Using a 3DS as a VR controller to control the RotoVR chair.

Comment

Inspiration

what if da three dee ess was on the vee arr

But, for real - VR controllers aren;t suitable for spinning. Steering wheels are a solution, but feel intimidating, and can't be easily connected to mobile first headsets such as the Meta Quest. Gamepads are a good option, but, the 3DS fully expanded is strangely similar in form factor to a Formula 1 racing wheel - and, we can rely on players having experience with this kind of movement control through games such as "Mario Kart 7"

What it does

Bridge from the 3DS to Unity, allowing for rotational gyro data to be read. This is then connected via the Roto SDK to allow for a experience where rotating the 3DS will spin the player.

How we built it

On the 3DS: there is a simple web server built in C++, making use of libcrtu. This listens for the HTTP header for /x /y and /z paths, and will then return the appropriate data.

The 3DS is then connected to a iPhone mobile hotspot, being used as an access point. A mac is then connected to both the mobile hotspot via USB, and, to a fast wifi connection. Then, a SSH tunnel is created on a remote server, making use of TailScale - this SSH tunnel allows for the 3DS to be accessed from the open internet - this is required for the headset, which makes use of a different wifi network, and on which the mobile hotspot wasn't functional.

In Unity, we then poll this data, and do some signal processing on the raw values, to create an interpolated stream of values, keeping in mind latency, as well as input smoothing + deadzones. This data is then applied to rotate a transform, which is then synchronised using the Roto SDK.

Challenges we ran into

  • Networking! Originally used ngrok, but got hit by their free tier having a usage limit - moved to SSH tunnels instead, so we have near complete control over the network solution, with only Tailscale being the main point of failure in this scenario
  • VR SDK - Unity's XR plugin has a wonderful bug on the latest 2023.3.x stream, which assumes your machine has VisionOS support, causing compilation issues. Manually modifying the preprocessor directive fixes this, indicating Unity is incorrectly flagging machines as supporting VisionOS - however, I found using the legacy Oculus SDK to be a better fix short term
  • Limitations in time! This is my first 24 hour hackathon - while I got the main project achieved, I feel I could've gone further with the actual game project. However, I'm not particularly worried about this, given this was a soly project , and, the main objective was achieved, with a small sample game.

Accomplishments that we're proud of

  • Latency! The total latency when using Unity is around 50 - 100ms, which is totally in the range of acceptable for a Hackathon project, especially one with so many moving parts

What we learned

  1. 3DS is totally viable to run tiny little servers on!
  2. The 3DS form factor is a surprisingly good form factor for VR steering controllers. Face & shoulder buttons combined with tilt is very similar to a F1 steering wheel :)
  3. (Relearned) that so many debugging tools assume your devices can access each other on local networks - which is a pain when working on university networks
  4. 3DS homebrew toolchain is really rad, libctru my beloved

What's next for RotoVR, 3DS Edition

  1. Look into better streaming methods with less overhead, including streaming button inputs
  2. Look into making use of the 3DS in passthrough mode in headset, such that the touchscreen can be used to interact with the environment - mixing a easy to use pattern with VR.

Built With

Share this project:

Updates