• The COVID-19 pandemic brought a complex array of challenges which had mental health repercussions for everyone, including children and adolescents.
  • Grief, fear, uncertainty, social isolation, and parental fatigue have negatively affected the mental health of children.
  • Fun and entertainment was in sheer need to overcome the stress and monotonicity during the Covid period.

What it does


  • Augmented reality (AR-Buddy) is a socially interactive model which resides in a real-world environment
  • It superimposes a pre-created playful environment on top of a user’s actual environment.


  • The main objective of utilizing AI in gaming is to deliver a realistic gaming experience for players to battle against each other on a virtual platform.
  • In addition, AI in gaming also helps to increase the player’s interest and satisfaction over a long period of time. Here the player competes with our pre-trained ML model and is scored accordingly.

How we built it


  • Imported Vuforia in Unity, then an image target was provided such that Vuforia Engine can detect and track the surface on which it can superimpose the 3d character.
  • The AR character can perform various actions like walk, kick and run. And the user can control its actions through various buttons.
  • We added animator transitions and controlled its movement and actions using c#.
  • After this, we modified each time stamp for handling the actions in order.


  • We had four labels: rocks, paper, scissors, and none.
  • We had used transfer learning where the squeeze net model is used. and later preprocessed it according to models input from the feed of webcam.
  • The ml bubby classifies our moves according to the label and makes its own moves.
  • It is a self-learning model. After each round, it learns the game in order to win.
  • For detecting the user moves we extracted the region of interest within the rectangle and gave predictions to the user interface.

Challenges we ran into

  • Game object translation with respect to the camera frame.
  • Animated transition between phases.
  • In web, with OpenCV packages, we had difficulty translating with videos so we used a user interface for ml prediction and web camera feed.
  • Area of the region played a very critical role in detecting the user's input images.

Accomplishments that we're proud of

Apart from web tech stacks, we tried something new in the field of augmented reality, and we are really proud of the fact that we were able to finish this project.

What we learned

  • Use of augmented reality and artificial intelligence to enhance user experience.

What's next for Pseudo Bud

  • Integrating AR Buddy with Artificial intelligence to make our buddy more interactive and smarter so that it can interact according to the events.
  • Adding more features to AR Buddy to increase engagement.

Built With

+ 4 more
Share this project: