Fuse Screen

Augmented Reality guide to destination
Augmented Reality towards the Atrium
3D model and QR code to augmented reality guide
Home Screen w/ facial recognition
Passive screen when no face detected

Inspiration

Our project aims to reinvent the front desk experience by creating a digital receptionist that welcomes and directs visitors within a building using an interactive 3D model. This assistant goes beyond a static information kiosk; it offers dynamic directions similar to those found in malls and recognizes George Mason University (GMU) students, faculty, and event attendees, providing a personalized and engaging experience.

What it does

When a visitor arrives, the digital receptionist offers assistance. Guests can state the name of the event they are attending, for example, "the Cloud Force event." The program then checks if the event is in progress, welcomes them, and provides 3D directions using augmented reality and 3D modeling to the meeting room. This visual navigation experience mimics the guidance seen in modern shopping malls, making it easier for guests to follow.

When Mason GMU students and faculty reserve a room or a lab the receptionist can clarify the reservation details and provides 3D directions to the meeting room.

How we built it

AAAAAAAAAAAAAAAAAAAAAAHHHHHHHHHAHHHHHHHHHHHHHHHHH We used React, Typescript, Azure AI, Text-To-Speech, React Three(a 3d engine), an augmented reality software and a facial recognition library.

Challenges we ran into

We really wanted to use Microsoft Face for facial recognition but we could get access in time. We also ran into so errors implementing the Azure AI and Text-To-Speech but we were able to sort most of them out.

Accomplishments that we're proud of

We designed a wide variety of features and while we haven't scaled the data behind it, implementing a totally encompassing app that covers the whole building wouldn't take nearly as much work as setting the groundwork did.

What we learned

We expanded our skills in React, Typescript, and TailwindCSS. We learned how to design 3D objects in React. We got better at implementing Azure models and how to make Natural Language processing models.

What's next for Fuse Screen

For GMU students and faculty, the experience will become even more seamless. If they reserve a room, the assistant will allow them to check in upon arrival. The system will send a notification to the relevant room, alerting the occupants with a custom message about the next party’s arrival. While this notification feature faced some technical infrastructure, it remains a future enhancement we aim to implement. Improved graphics and animations with the 3D objects.