Inspiration
As hackers, as creators, we face the limits of technology often, although there are certain boundaries that we are ready to overcome. In our case, we found a problem we want to solve: we want to connect Reality one device at a time.
What it does
We’ve created a framework in C++ and C# for Bluetooth communication between Unity and external devices and tested it with Arduino sensors. This is a tool that didn’t exist and allows developers/creators to shorten the implementation time for Bluetooth dramatically.
To test our framework we created a unique demo where we’ve connected a living plant with different sensors (moisture, humidity, light…) These sensors connect in real-time with a data visualization experience in VR.
In the demo, the user is able to see and feel how the plant is feeling thanks to a series of particle systems that reflect in real-time the state of our plant.
You may think that a plant can’t make much difference, but our framework will allow developers, creators and makers to connect all kinds of systems from entire aquaponics vertical farms, new custom made controllers for diversely abled users or new tools for surgeons, among many other uses.
How we built it
We worked in parallel development. Taylor and Alireza developed, tested, & debugged several .NET native Bluetooth libraries to integrate microcontroller communication with Unity for XR projects. While Pilar and Mónica worked on the demo scene, search and tweak assets, and wrote the scripts that will interact with the demo plant. Finally, we merged our two separate scenes together with Unity BT scene and spent time debugging functionality.
Challenges we ran into
The main issue we ran into was the System.IO.Ports namespace for C#/Unity. For this class it was able to maintain an open BlueTooth connection, it would briefly connect and then drop. We believe this comes from a complex threading issue of how the Serial class maintains its connection. To make sure our project still succeeded we used a USB communication as windows binds these to COM ports the same way it does with BlueTooth. This allowed us to make a framework that works the exact same way. The other issue is since we had a lot of personal hardware there were many different ways in which we had to interface ranging from the sensor data to communication protocols.
What we learned
We learned a lot about the lower levels of C# programming, and how to use lower-level functions.
What's next for uTooth
What is next is to either wait till System.IO.Ports is updated our create our own serial communication libraries which may be easier. The data transfer was also much slower than it could be, sending smarter packers and moving around data in a more efficient way would allow for much high fidelity sensors such as videos and audio.
Log in or sign up for Devpost to join the conversation.