Cygnus

• GIF

  Cygnus Animated Logo

• 

• 

Inspiration

The idea for Cygnus came from something that happened to my little sister a few years ago. She and her friends had been planning a two-week trip to Italy, Greece, and Spain for months — flights, hotels, the Amalfi Coast, Barcelona, everything. The night before the flight, while checking in on the airline's app, her passport wouldn't scan. She called the airline and learned something none of us had ever heard of: many countries enforce a "three or six-month rule," requiring your passport to be valid several months beyond your travel dates. Because her passport fell inside that window, the airline told her entry could depend on the customs officer when she arrived. She decided not to risk it. Her friends left without her. She lost a year of excitement and a lot of money.

That moment stayed with me. When this hackathon came up, it popped back into my head — and I realized that with the power of Gemini, I could finally build something that would have helped her.... and many other people. The UI Navigator category felt like a perfect fit: it's genuinely hard to find the State Department's official international travel guidelines, and an AI agent that can show you exactly where to look could save a lot of people from the same situation.

What it does

Cygnus solves a very real problem: nearly 40% of travelers are unaware that passport requirements vary by destination, leading to boarding denials and lost travel costs. Cygnus acts as a "UI Navigator" and real-time Live Agent. Cygnus monitors your browser via screen sharing and detects when you're searching for or booking international flights. The moment it identifies a destination — from a country name, airport code, or booking site like Google Flights, Kayak, or Expedia — it speaks to you directly, alerts you to the entry requirements for that country (passport validity, blank pages, visa status), and offers to open the official U.S. Department of State page for your destination.

How it works

Cygnus uses Gemini Live's Multimodal capabilities to process a real-time screen capture and microphone audio simultaneously — giving it both visual and conversational awareness. Cygnus monitors the user's browser for any activity that relates to searching for international flights, including research, or booking by looking for country names, airport codes, and flight booking websites, including Google Flights, Kayak, Expedia, etc.

The moment Cygnus identifies a destination — from a country name, airport code, or booking site like Google Flights, Kayak, or Expedia — it uses Gemini Live to speak to the user directly, alerting them that they should always check the entry requirements for their destination country (passport validity, blank pages, visa status). Cygnus then offers to either fetch the information about your country from the State Department website, or open the official U.S. Department of State page for your destination for you.

How I Built It

The backend is a FastAPI server deployed to Google Cloud Run, connected via WebSocket. It uses Playwright for browser automation when Cygnus needs to open URLs or navigate on the user's behalf. The API key is stored and retrieved securely using Google Secret Manager. This worked alone by itself, but I wanted a console/something to look at for testing reasons. To get a web console up and running, I cloned the Live Web API Console from GitHub into a local frontend/ directory. The entire deployment pipeline is also automated with Google Cloud Build cloudbuild.yaml.

To give users a more visually appealing experience, I used Google AI Studio to create a React app. I then connected it to the backend and deployed it to Google Cloud Run as well as Google AI Studio.

Challenges I ran into

API Connection Errors

The biggest technical challenge was the Gemini Live API connection — getting bidirectional audio and screen streaming to work reliably in the browser required careful handling of PCM audio encoding/decoding, frame rate tuning, and WebSocket lifecycle management. I also had to debug a deployment issue where environment variables weren't being correctly injected into the Vite build at Cloud Build time (the wrong build arg names were being passed), which caused the deployed app to fail silently. Working with a GCP business account added some IAM and permissions complexity to the Cloud Build and Secret Manager setup.

Submodule Errors

I realized that the Frontend/ directory appeared to be locked on GitHub and I could not push and pull it like a regular directory. I did some research, and learned that this happened because I was missing a very important .gitmodules file. This meant, the judges might pull an empty repository from my project.

What I learned

I learned how to use Google Gemini to monitor the user's browser for any activity that relates to searching for international flights, including looking up flights, booking flights, and researching flights.

Submodule Lesson

As mentioned above, I realized I committed my frontend/ directory incorrectly. I did not want the judges to pull in an empty frontend/ folder, so I removed it before submission, but it is available on a separate branch and can be installed by running, git submodule add https://github.com/google-gemini/live-api-web-console [folder-name].

Gemini Live

I learned a tremendous amount about the Gemini Live API — specifically how to stream PCM audio at 16kHz in, receive 24kHz audio out, and schedule audio chunks without gaps or overlaps using the WebAudio API. I also deepened my understanding of building secure, automated GCP deployments using Cloud Build, Cloud Run, and Secret Manager together.

Accomplishments

The core Cygnus experience works end-to-end: it watches your screen, recognizes an international flight, and speaks to you — all in real time. I'm especially proud of the fact that I had this idea just last week and shipped a working, deployed MVP in time for the deadline. Getting the Gemini Live multimodal pipeline (vision + voice, simultaneously) working in a browser without any transcription lag was the hardest part, and it works.

I was able to build a working Cygnus MVP that uses Google Gemini to monitor the user's browser for any activity that relates to searching for international flights, including looking up . Most significantly, I got this idea last week, and I'm proud of myself for sticking to it with the tight deadline.

🚀 What's next for Cygnus

Chrome Browser Extension

• Now that I have a working MVP, I plan to turn it into a Chrome browser extension that will allow users to use Cygnus without having to install it on their computer or use it in a web browser.

Google Flights Integration

I think in the future, Cygnus could be an excellent addition to Google Flights. I believe it would be a great way to help users avoid passport issues when booking international flights, and it would greatly enhance the Google Flights experience.

Built With

• gemini
• python
• react
• tailwind
• typescript