SeniAR

Logo Design
Mobile 3d environment builder
Yinggui's AR message to his grandmother
Social impact for elders

Inspiration

Last Christmas, one of our team members saw her grandmother who now lives in a nursing home. She is now bedridden with a very limited mobility, but she is still so alive inside. Our teammate could see the liveliness in her eyes. She talked about her hopes to move, travel and share memories with us. Her vision is the inspiration of this project. Moreover, everyone has grandparents, and this application will let you build shareable memories with elders in augmented spaces, and eventually create a new virtual heritage between you and them.

Problems

Elders suffer from social isolation by staying in a limited space for a long time, which eventually causes many complications such as loss of muscle strength and mental illness. We decided to build AR Intimate messenger that connects younger generations with elders through technology. How can we create an affective environment to reconnect generations, and fulfill their hopes to share memories with us?

What it does

SeniAR is an AR application for family members to create an AR space using photogrammetry technique and 3D templates, and send the new space to elders using AR Cloud. Elders receive the augmented reality messages along with text messages from their family. They can immerse into the environment, be relaxed, exercise, listen to music and sound and share memories with their family, creating intimate feelings. We believed that interaction in AR allows elders to be more engaged to situations, and exercise their bodies in a more entertaining way. For restricted and disabled people, eye-tracking on Hololens2 seems the resolution of this isolation.

You will craft a shareable memorial environment with landscape and registered audio message to send to your grandmother laying on her bed and using an HOLOLENS2 in her nursing home.

Here are some steps:

Mobile users

A user choose from two options - Capture and Share.

Capture - 3D scan objects, plants, portraits and more using photogrammetry software.
Share - a user can create an AR environment from choosing templates on the application, then add 3d models such as plants and trees or your 3d scanned models.

Send the message to your family or friends using AR Cloud!

Hololens users

An alarm pops up saying that you have a new message!

Open the message, and AR environment will appear along with the voice message. She can read it, if she can hear well.

Immerse into the AR environment by interacting with the objects.

Why AR Cloud?

We believe that using AR cloud, we could create different layers of AR templates environments that can be easily shared and be shared by friends and family, and users to be immersed into the environments. Private shareable and open-source environments are provided.

How we built it

We built the project in Unity. The 3D assets were obtained for free on Sketchfab, free Unity assets, and video resources from Youtube. We experimented with some other ways to share the file with Aframe and AR Cloud. We used _ dispaly.land _ and _ capture _ to demonstrate our photogrammetry idea. Lastly, we used Hololens2 to augment our environments!

Challenges I ran into

We only had one developer, and no professional 3D animator. So, we went to the workshops on Wednesday, and watched Youtube tutorials.

We wanted to have some interviews with elders, but the interviews were very limited to our family members.

We did not have enough time to use photogrammetry SDK on our mobile application, but we were able to visualize our ideas.

The entire team members didn’t have experience in Hololens2. We struggled to get used to the equipment, and had to call mentors several times. We are glad that the AR environment is running and looking amazing!!

Accomplishments that I'm proud of

We are proud of our idea and concept. Each person got his /her role in the team and brought amazing creative inputs.

What we learned:

How to cooperate in a diverse environment, and how to develop it with Hololens2. A new understanding of family relationship. Eye-tracking tool.

What's next for SeniAR

We want to further develop eye tracking AR for people with mobile disabilities. We also initially wanted to create a 3D layers of a Nursing Home (having layers of user's data, meditation room, fitness center like an AR Cloud 3D map), but later we decided to focus more on the social interaction. We want to create AR environment where elders can exercise, meditate, and enhance social interaction.