We all know how boring it is to stay at home during this pandemic lockdown. It's Winter, and most of us are accustomed to going out and chilling with our friends which unfortunately isn't possible this winter. Although we are connected virtually, with that, the enthusiasm level of enjoying Christmas with our near ones has touched the surface, which simultaneously led to the depression of minds, especially the young ones. Besides, not only the young ones are badly affected, but also those who are physically or mentally inaccessible because of certain disabilities. Our project WintAR bridges the gap and makes video conferencing more intuitive and fun for not only kids and toddlers but also for everyone!

What it does

WintAR is a TF.js based gesture recognition interface to spice up Video Conferencing calls during your Winter Holidays ☃️❄️. WintAR is easy to run & requires no custom application/software installation. Just click on View Demo & directly use it on the browser! Supported across multiple video conferencing applications like Zoom, Google Meet, Microsoft Teams etc.

WintAR runs on both on PC & mobile directly on the browser (on Canvas), although PC is always recommended for a much better experience! ✨

How I built it

  • Javascript
  • TF.js
  • ml5.js
  • p5.js
  • HTML
  • CSS

Challenges I ran into

We had a lot of issues in configuring the model. Apart from that bringing the confidence score of sensing those fingers was a bit challenging too. Also, it was a bit difficult for us to collaborate in a virtual setting but we somehow managed to finish the project on time.

Accomplishments that I'm proud of

To be able to finish the project in time in spite of having our internal exams :)

What I learned

A lot of things, both summed up in technical & non-technical sides. For the technical side, the documentation of ml5 was a bit lengthy, but since we were three, we divided the task among ourselves to figure out the exact part which we needed to run the project. We also gained some UI/UX skills while one of us was busy building the frontend side of the project. It was a great experience working overnight & watching MLH's mini-events! Not to mention, Stackoverflow was the gem for us while we're troubleshooting some complicated issues late-night.

What's next for WintAR

We're going to go through many changes & planning to add the following updates to the project in the future,

  • Improve the UI.
  • Add more features regarding conferencing software integration.
  • Background blur / remove feature.
  • Add extra filter and particle effects

Built With

Share this project: