Inspiration

COVID-19 has transformed distance learning. As high school students, we felt isolated from our peers and teachers, and decided to create SmartRoom: an immersive and feature rich application for students and teachers.

What it does

SmartRoom connects students with teachers through an interactive and immersive 3D classroom. Students can move around in the classroom, sit in chairs, and see each other. SmartRoom features an unique smart dashboard, which teachers may use to receive AI-powered feedback from students, administer real-time quizzes, and give lectures on a live whiteboard.

How we built it

For the front-end we used: Three.js for the 3D environment, HTML/JavaScript/CSS (SASS) for our webapp, and bundled it with Parcel. We also used Google Cloud Storage to manage personal photos uploaded by users, and Blender for animating the 3D character models. For the back-end, we used: Node.js/Express for our server, Socket.io (websockets) for communication, and IBM Watson’s Tone Analysis API for our smart feedback. We used Heroku to deploy our application (smartroomvr.herokuapp.com)[ http://smartroomvr.herokuapp.com/]

Challenges we ran into

  • Lighting our 3D scene was difficult since we had to balance performance with quality of lighting.
  • There were some issues when we were loading our HTML pages in separate chunks since we wanted to make a single page application.
  • Structuring our code was a challenge because we had to combine a 2D web interface with a 3D environment, as well as write significant back-end code to support it all.
  • Combining multiple FBX animations into a single GLTF one in Blender also took a lot of time to learn.
  • Google Cloud Storage was giving us a CORS error when uploading photos, so we had to manually modify the server through the Google Cloud Terminal to allow CORS with all domains.

Accomplishments that we're proud of

Kirtan:

  • I made a sleek and aesthetic dashboard from scratch -- no templates, no frameworks.
  • I built my own, custom classification between Constructive/Destructive criticism derived from IBM Watson’s 5 tone classifications (joy, anger, analytical, etc.)

Deepak:

  • I made an intuitive landing page that allows students and teachers to select their role and enter a room number to join a classroom.
  • I build the real-time 3D environment from scratch using Three.js and Socket.io.
  • I learned about Vector projections from 3D to 2D and how to combine multiple FBX animations into a single GLTF file.

What we learned

  • We learned a lot about server side logic and the Socket.io library for websockets
  • We learned about 3D graphics concepts like lighting, model animation, and rendering
  • We learned how to use IBM Watson’s tone analysis API
  • We learned how to use Google’s Cloud Storage API along with modifying the bucket itself through the Google Cloud terminal

What's next for SmartRoom

  • We will replace the images (profile pictures) with a Web RTC video so that students and teachers feel even more connected with one another.
  • We will also integrate the whiteboard into the 3D environment so that the experience is much more immersive.
  • We will implement our 3D environment on a VR platform for an even more immersive experience.

Built With

Share this project:

Updates