Pack Pro

Pack Pro Architecture Diagram

Overview

Pack Pro is an immersive AI packing assistant that transforms the stressful process of preparing for travel into a guided, real-time experience.

Using a live multimodal agent powered by the Gemini Live API, Pack Pro can see what you’re packing, hear what you say, and guide you through the process step-by-step.

Instead of manually searching packing lists or repeatedly checking travel blogs, users simply start a conversation and begin packing. The AI assistant watches the suitcase through the device camera, understands spoken instructions, and provides contextual guidance in real time. Pack Pro continuously adapts its recommendations based on the trip details, luggage constraints, and the items it observes being packed.

The result is a natural, agentic interaction where the AI acts like a knowledgeable travel companion—helping users pack smarter, avoid mistakes, and ensure nothing important is forgotten.

The Problem

Packing for travel involves juggling multiple constraints:

destination and weather
trip duration and itinerary
airline luggage rules
suitcase capacity
remembering essential items

Most existing tools rely on static packing lists or generic travel checklists. These tools are not personalized and cannot see what the user is actually packing.

Packing is a physical, visual task, but current solutions only operate in text.

Our Solution

Pack Pro introduces a multimodal AI packing assistant that works during the actual packing process. Users describe their trip conversationally:

“I’m going to Tokyo for five days for a business trip.”

The assistant gathers trip details and generates a personalized packing list. As the user begins packing, the AI uses the camera feed to understand what items are present and keeps track of what has been packed.

During packing, the assistant can:

automatically mark items as packed when it sees them
suggest better packing arrangements
provide folding and space-saving tips
warn about airline restrictions
recommend carry-on vs checked placement

The result is a natural workflow where the assistant acts like a knowledgeable travel companion helping you pack.

Why XR

Packing is a physical activity that benefits from spatial guidance.

Using XR allows the assistant to provide contextual help while the user is actively packing rather than forcing them to switch between apps or lists.

Pack Pro currently runs on Android XR, but the experience is designed to translate naturally to AI glasses and future XR devices, where hands-free guidance during everyday tasks becomes even more useful.

Challenges we ran into

First time working with the Gemini Live API: Getting familiar with real-time multimodal streaming, session handling, and response patterns required a learning curve.

Maintaining responsiveness with growing context: As conversation history, packing state, and function calls increased, keeping latency low and responses fast became challenging.

Function calling reliability: Function calls occasionally triggered repeated or looped responses, which disrupted the natural flow of conversation and required additional handling on the backend.

What's next for Pack Pro

We plan to expand Pack Pro with more advanced spatial guidance and contextual assistance:

Spatial packing overlays: Visual highlights inside the suitcase showing where items should go for better organization and weight distribution.

Guided packing cues: Text and graphical overlays that provide step-by-step suggestions while packing.

Contextual folding guides: When clothing items are detected, the assistant can surface short videos or visual guides demonstrating efficient folding or rolling techniques.