Our implementation of VR w/ Camera
Coding up the IMU and Motors...
Building a robotic arm was definitely interesting!
We took a great interest in the increasing popularity of the XR movement. Currently, there are controllers that need either external sensors or are bulky, and one must use these to interact with the XR environment. We wanted to change that. We wanted a sensor that could interact with the XR environment without having the need for external sensors or bulky equipment. Using our passion for robotics and our inquisitory questions for XR, at this hackathon, we created a product that utilizes the best parts of XR (the immersiveness, the reality, the controllers) and incoporated some of our own ideas (robotic arm, re-vamped controller, camera for AR) to create a unique products that will open new paths for XR technology.
What it does
We incoporated multiple IMUs as our sensors for data collection. Then we read the data from Unity and parsed the data in Unity using a Kalman Filter to clean out data and then output it in Euler angles that can then be read by the motors. The motors will take information provided to them and turn in a specified direction in a specified rotation by the code. We put a camera on top of the robotic arm to show the possibilities of adding modules to our arm. We use Arduinos for both data capturing and the motor portion, and Unity was used for the both the data parsing and the image acquisition, which was then sent to our VR headset to be viewed remotely.
How we built it
We combined our many expertises to furnish the product. We had to use the Arduino C code IDE to help do the motor code and the data acquisition. Both required superior EE skills and coding experience to complete. We had those already wired and on this hackathon, we focused on cleaning out more of the data and adding a new component to our system: the VR headset. We had to use Unity to interact with the headset and we put a video display on there to show our future plans.
Challenges we ran into
Some challenges we ran into were downloading the Unity SDK for the VR headset and making it work. It was difficult as nobody on our team had a lot of prior experience with Unity and so we had to do all the VR things from scratch. It was definitely helpful to have mentors in the vicinity who could help us develop and add the video to our VR. Another challenge was making our motors run. With our limited electronics experience, we were dumbfounded at what was happening as we are unsure of whether it was an electronics problem or a software problem. It was definitely difficult debugging and we are still currently debugging our motor problems.
Accomplishments that we're proud of
We are definitely proud of being able to incoporate the Unity portion as we had to experience with the VR headset. It was also a difficult leap forward in doing the motor code as we are trying to refine it with our limited C++ experience. Having to recreate a robotic arm at the event was definitely also a big accomplishment.
What's next for KalmanKidz's Glove Project
Next, we are planning on adding a lot of AR elements in to it, including maybe incorporating ML for object detection etc. Also, we are thinking about providing a lot of different demos on Unity to help out with the VR and adding a lot of different demos will allow us to have more possibilities with using our glove. To further develop the glove, it is imperative that we gain some sort of VR interactive device as none of us have any. With the second place prize, we would be able to further test our thing in VR even after the hackathon. For this reason, we hope you choose to award the VR headset to allow us to continue our testing and hopefully implement this in the future. Thank you!