Team lead: Igal Nassima
Team lead's telephone number: 646 696 9286
We're located: (Floor 6, table 7, on the corner where the balcony entrance door is)
Category: Commerce/Industry (Architecture/Engineering/Construction/Productivity/Industry/Commerce)
"The physical plane, in emanationist metaphysics taught in Neoplatonism, Hermeticism, Hinduism, and Theosophy, refers to the visible reality of space and time, energy and matter"
As we subscribe ourselves to worlds created or altered in Virtual or Augmented Reality, we are leaving and redefining our current plane of reality. In this mode of transition, it will become important to explore how our existing bodies simultaneously interact with physical and virtual environments.
The Reality Plane begins to explore our interaction in Mixed Reality through added sensory cues in a familiar input device, a tablet.
What it does
Reality Plane presents a completely blank surface onto which information can be projected in mixed or virtual reality. It has the potential to support natural interactions (touching, sliding, pinching, et cetera) and provides haptic feedback for interactions. Developers can create their own experiences while users experience a more natural mode of interactions (as compared with air gestures or game controllers)
The orientation of the Reality Plane allows for different types of information to be displayed. In the demo game we created, when orientated horizontally a game map appears and displays the location of crystals to collect. Vertical orientation allows you to engage with the crystals that are floating in your environment. When you catch the crystals you simultaneously feel a buzz and hear the sound that each crystal emits.
Our demo game and tablet shows how it is possible to seamlessly integrate Arduino with Hololens, Vuforia and Unity to add tactile experience in addition to visual, audio, and motion.
How we built it
- Hololens as the primary mixed reality device.
- We created a Unity plugin for communicating with BTLE devices
- We created an Arduino Circuit, and integrated it into a custom fabricated tablet with a vibration motor to create haptic feedback
- We integrated Vuforia AR tracking into Hololens to be able to track the location of the tablet
- We also created an interactive "game" displaying some of the potential of the device.
Challenges we ran into
We built a BLE library for unity from scratch, which took a long time to get working with consistency--it's already a great start but there's more to do
We had to design haptic tablet in multiple iterations and test it to create a balanced physical UX. The viewport scale of
Hololens creates a challenge in terms of designing the interaction between tablet, navigation and gestures that change the environment at any time.
Vuforia performance create a delay in the content movement, we designed in the interface to minimize its use.
One of our team members, Dale, had to drop out due to sickness. Feel better Dale!
Accomplishments that we're proud of
- We created a new BLE library for Unity Windows 10 UWP.
- Designing a tablet from scratch, fabricating with haptic responsive circuit.
- Integrating Vuforia and Hololens to create a unique game experience.
- Having something to present in 48 hours
What we learned
- Lots about GATT and the UWP API
- Creating AR experiences in Unity
- It's possible to create and connect devices to Unity
- Learned about creating markers for Vuforia
- 3D Software Cinema 4d
- Got much deeper into HoloLens Mixed Reality capabilities
- Design considerations building a physical haptic interface, such as weight, balance, etc.
What's next for Reality Plane
- Adding IR Tracking, IMU, and other sensors and inputs to the tablet to make the experience even more compelling
- Using the rotation of the tablet to trigger menu items