When thinking of how to better equip the human in a workplace that is being taken over by robotics, we've decided to grab the best of both worlds. Why not integrate robots with humans? Morever, studies have consistently shown, co-bots, so called robots that work with humans and enhance their abilities, acheive higher productivity while keeping humans relevant in the workplace. Below, we will highlight the dreams we have, the struggles we went through to acheive it, and our future plans. That being said, I invite you to join us on our journey!
The glove motion capture idea has already been around. So called VR haptic gloves track hand motion as well as induce "feeling" in one's hand when interacting inside virtual environments. By all means, the motion tracking technology has been around for some time. However, the gloves are very expensive, almost as expensive as a new gaming console, and they were bulky, which tired the hand and did not immerse the gamer inside the game. Now how is this relevant to our project and why did we still decide to do it? We responded that: 1) even though we are beginners in the arduino technology (electrical in general), high schoolers, as well as slighly incompetent, we believed that the more technological intesive tasks could be easily done, leaving only the math behind it, and 2) this would provide an opportunity for us to grow as fellow hobbiest and create something that would add value to society. Thus, we choose the glove as it allowed us to focus more on the algorithm side, something we thought we could do, while slowly incorporating applicable technology. The result was to be a glove that could help save lives, enhance user experience, and also bolster humans in the workplace.
So we got started on the project as soon as possible, eagerly jumping in for a chance to learn. We started by ordering some of the parts we thought we needed and researched on the unfamiliar electronics parts to gain an understanding of what we would need to do. Splitting our time between school, clubs, extracurriculars, and this project we read up on Kalman Filters, Mahony Filters, Mahogany Filters, Quarternions, and other associated mathematical formulas that were needed to accurately predict the state of the user's hand. Learning electrical was equally challenging, as we started from scratching, moving form what volts/current/resistance are, to more higher level topics like the difference between arduino micro, mini, and nano. Much of our time was spent trying to gain a grasp on a subject we have never seen. When it came time to start connecting wires and trying out our code, we realized that things didn't go according to plan.
The code didn't work for us so we had to constantly change it, look for other source code, look at our electrical diagram, and overall, troubleshoot to find where we messed up. When we started to get data out of it, we used Unity to conver the raw data into usuable vectors that could manipulate 3D objects. Hence, quarternions were introduced, a concept so abstract that it would take us a full month to slowly grasp onto 4D numbers, and armed only with calculus, we weren't in the best position to learn college level abstract math. Despite our hinders, we managed to pull through and get some sort of response from Unity and our IMU. We were ecstatic at having the in-game 3D object respond to IMU data from external stimuli.
Even with our energetic attitude and perserverance, we did not have enough time to produce a workable stage 2 product and coupled with other problems, our prototype is only a simple prototype and still needs work well. However, we aren't discouraged. We know that there is potential in our work and we plan to continue it in order to learn while giving back to the community. We hope that the inclusive community of fellow arduinoists will helps us along our journey and we thank you in advance.
"Have the courage to follow your heart and intuition" - Steve Jobs
Wish us the best of luck!
- Benjamin Nguyen, Jeremy Li, Banyan Nguyen
P.S. Big thanks to Kris Winer for his amazing IMU code. We couldn't have done it without him!