Last Christmas, one of our team members saw her grandmother who now lives in a nursing home. She is now bedridden with a very limited mobility, but she is still so alive inside. Our teammate could see the liveliness in her eyes. She talked about her hopes to move, travel and share memories with us. Her vision is the inspiration of this project. Moreover, everyone has grandparents, and this application will let you build shareable memories with elders in augmented spaces, and eventually create a new virtual heritage between you and them.
Elders suffer from social isolation by staying in a limited space for a long time, which eventually causes many complications such as loss of muscle strength and mental illness. We decided to build AR Intimate messenger that connects younger generations with elders through technology. How can we create an affective environment to reconnect generations, and fulfill their hopes to share memories with us?
What it does
SeniAR is an AR application for family members to create an AR space using photogrammetry technique and 3D templates, and send the new space to elders using AR Cloud. Elders receive the augmented reality messages along with text messages from their family. They can immerse into the environment, be relaxed, exercise, listen to music and sound and share memories with their family, creating intimate feelings. We believed that interaction in AR allows elders to be more engaged to situations, and exercise their bodies in a more entertaining way. For restricted and disabled people, eye-tracking on Hololens2 seems the resolution of this isolation.
You will craft a shareable memorial environment with landscape and registered audio message to send to your grandmother laying on her bed and using an HOLOLENS2 in her nursing home.
Here are some steps:
Capture - 3D scan objects, plants, portraits and more using photogrammetry software.
Share - a user can create an AR environment from choosing templates on the application, then add 3d models such as plants and trees or your 3d scanned models.
Why AR Cloud?
We believe that using AR cloud, we could create different layers of AR templates environments that can be easily shared and be shared by friends and family, and users to be immersed into the environments. Private shareable and open-source environments are provided.
How we built it
We built the project in Unity. The 3D assets were obtained for free on Sketchfab, free Unity assets, and video resources from Youtube. We experimented with some other ways to share the file with Aframe and AR Cloud. We used _ dispaly.land _ and _ capture _ to demonstrate our photogrammetry idea. Lastly, we used Hololens2 to augment our environments!
Challenges I ran into
Accomplishments that I'm proud of
We are proud of our idea and concept. Each person got his /her role in the team and brought amazing creative inputs.
What we learned:
How to cooperate in a diverse environment, and how to develop it with Hololens2. A new understanding of family relationship. Eye-tracking tool.
What's next for SeniAR
We want to further develop eye tracking AR for people with mobile disabilities. We also initially wanted to create a 3D layers of a Nursing Home (having layers of user's data, meditation room, fitness center like an AR Cloud 3D map), but later we decided to focus more on the social interaction. We want to create AR environment where elders can exercise, meditate, and enhance social interaction.