Even the expert typists out there can fumble up on the keyboard if they need to hit a special key or two while in VR. It is silly to think that you have to rip yourself out of the VR world just to see where your hands are on a keyboard.
What it does
keypacitance adds a whole new layer of touch input into your computer by way of capacitance touch sensing. By adding capacitive touch sensors to the individual keys on your keyboard, a new way of interacting with your computer can be realized. The potential applications and neat projects you can do with this are as numerous as developers are creative. That being said, our hack focuses on one specific application that we think is pretty cool, VR.
The keypacitence VR application allows you to type on your keyboard unimpeded while experiencing VR. Wether you are browsing the web, writing some software, or playing games (not necessarily VR!) with your VR headset on you will never have to rip off that headset to see where the hell your hands are again. By simply resting your hands on or dragging your fingers across the keys you are able to see (in VR) exactly which keys are at your fingertips on a VR keyboard in the application. No need to blindly type a key to see where you’re at or take off your headset. Working on your desktop in VR just became so much easier!
How we built it
There were 3 main parts to then project. 1) Building an Arduino circuit to detect the capacitance of the keys and deliver that information to the computer. 2) Writing the Unity code to interpret the information on serial connection and translate it into color changes for the keys. 3) Designing a virtual keyboard in Unity and interfacing that with the code written in step 2.
We used a simple RC circuit design to turn the keys into capacitance sensors. By using one pin on the Arduino as a voltage source, and a second as a sensor, we're able to determine a "charge time" associated with the natural capacitance of the circuit. A finger touching the copper foil changes the capacitance of the circuit and therefore its charging time. The arduino code detects this change, and above a certain difference indicates the "key" as being on. Every 10 milliseconds, all the state of all 3 keys is calculated, and the output is sent from the Arduino to computer via a serial connection.
We then pull this data using a C# script in Unity. We treat each key on the virtual keyboard as its own object, and set conditions for the color they manifest based on the information we've pulled from the serial port. If the Arduino data indicates a key is being touched, we change its color. Since we treat each key as its own object this method is easily scalable (provided enough power; there is a voltage drop in the source pin associated with the capacitive charging due to multiple key presses) and we are able to register multiple independent key-presses simultaneously.
Challenges we ran into
Learning how to use Unity was most definitely our biggest challenge. Just plain unfamiliarity with how the software works led to man slow downs. Whether it was just creating (many) 3D objects, getting Unity to interface with the Arduino properly, or mirroring the desktop in our application, Unity was our biggest bottle neck.
Accomplishments that we're proud of
- A pretty simple implementation of the capacitive key circuit.
- Interfacing the Arduino and the Unity Engine.
- Figuring out how to simultaneously script multiple objects in Unity.
- Snazzy bronze key aesthetic.
- A horrific but fitting pun for a project title
What we learned
A bit of how Unity works.
What's next for keypacitance
Probably just complete capacitance coverage of the keyboard so we can move beyond the proof of concept stage for starters. Then maybe other neat applications for the tech.