"I'm so stressed out at school. I can hear voices. I don't know, but I feel very sad. _ _I feel suicidal. I need help on Mathematics." Most of these quotes are pretty common among kids, it’s just that they don't talk to anyone because they don't know if it’s a concern/issue. Some of these signs need immediate care and attention, if unattended might lead to serious consequences. Augmented reality in this application creates a safe environment for the users to engage and let out their feelings, issues, or concerns to a virtual host.

What it does

Selp - is an Augmented Reality application which is triggered, based on Selp stickers at schools and colleges. On image recognition, a host will engage with the user to find more details about the concern. Selp host can handle a wide variety of education subjects to emotional, mental, and physical topics. After collecting relevant information, the original host will transfer the user to another host or pass the information to the right channel.

How we built it

This application was built using Sumerian AR & VR AWS platform. AR scene was built using the platform with lights, an AR camera and an AR anchor. This app uses image recognition to trigger a Sumerian host with polly and state machine to interact with the user.

Challenges we ran into

Getting the host to appear on image recognition of the "Selp" logo was a definite challenge. Initially, the image that was used had lot of objects, but then changing it to a simple logo made it easier. However, Xcode kept complaining about it. Due to the recent policy changes, creating an AR app with unmuted sound was a challenge with no references online anywhere.

Accomplishments that we're proud of

This application whether installed as an app on phones, tablets, or accessed via web is very crucial to the younger generation and will make them feel like they have a virtual friend to talk to or let it out at any given time.

What we learned

Augmented Reality, ARkit, 3d Modeling, Sumerian, AWS services, AWS Polly, Host Gestures, creating an AR scene, Image targets. We learnt that there are lot of kids who could use this service. No matter how much parents take care of their children, kids still need someone that they can talk to about anything. Open minded and freely.

What's next for Selp

We will take this to schools and showcase the use/case scenarios. We will work with topic specialists to route the issues for some topics which need immediate attention. We will need to work with authorities to route the channel. Selp would also get "voice" commands instead of typing text.

Built With

  • sumerian
  • arkit
  • hosts
  • polly
  • state-machine
  • cognito
  • image-recognition
  • lex
Share this project: