HoloWorld

FUnnnn picture of a BEE
Generated 3D Model Based on Destination City
AR of the Generated Town
Computer Vision Recognition of the Boarding Pass Contour
User Interface for Signed In Users
Itinerary List Generated Based on User Ticket, with Google Map

Inspiration

We are inspired by the mission statement of American Airlines of improving passenger experience, as well as past experiences of waiting long hours in the airport bored out of our own minds.

What it does

The program scans the boarding pass, and generates a hologram of the destination city, consisting of several notable landmarks, which also serve as a mini-itinerary for passengers to consider visiting after they land there.

How we built it

We primarily used Computer Vision to track the position of the ticket in real time, and Augmented Reality to "bring it to real life": a model that can interact with the user by moving the ticket or the camera.

Challenges we ran into

Some of the most significant challenges are managing API calls and merging the GitHub branches. We are limited to only a few usages, and it wasn't long before we need to set up a new API key. Eventually though, we came up with alternatives to use those calls sparingly - such as asking the OCR procedure to take a picture of the ticket, rather than a scan, which involves continuously capturing and making calls to Gemini.

Additionally, because each member worked on separate features with their unique style and choice of language, putting them altogether became a major challenge. Some of our members are skilled in working with Python, others are more comfortable with Node.js, so when we tried to combine our efforts into a workflow, we had a difficult time trying to transition from one member's output to another member's input. We solved the problem, however, by articulating and visualizing the workflow, gaining a better understanding, and making the cooperation easier.

Accomplishments that we're proud of

We are particularly proud that our hologram rendered in real time.

What we learned

The project is a very meaningful to us in both technical and strategical aspects. On one hand, we gained new knowledge on a variety of fields, such as computer, but also experimented much with Artificial Intelligence prompts, and how to be smart about using the tokens (like asking the AI to answer in a certain format, making it both shorter and easier to parse)

As for the strategy, we learned to be flexible about our product, especially one that involves high technical complexity. Regardless of how well and thoroughly we plan the project, there are always issues - some of which we cannot resolve in a short period of time. That is a time when we reflect upon our workflow and discuss a work-around that guarantees functionality while minimizing compromise in user experience.

What's next for HoloWorld

The next step could be to use some of the technologies in an actual airport. One such example would be determining the destination - instead of relying on OCR, which can make mistakes (it apparently read "Laguardia" as "Calcutta" at some point), we can scan the bar code and retrieve flight information from there instead.