Inspiration
Every friend group has a trip that never happened. It dies in the group chat or people bail out last second, mostly because no one wants to say that it is too expensive, or one person ends up booking on their own card then spend weeks chasing everyone else for their part. So we wanted a planning experience that would feel like sitting around a globe with everyone having a say. The greatest part? No one's budget is on display, either everyone agrees or no one goes.
What it does
Each group member briefs their AI agent on: budget, availability, and Interests. Then the agents on behalf of the members have a meet at the chart table. On meta Quest the table occupies the space as your real table in mixed reality. If need be others can also join on a phone or laptop.
The agents propose trips, push back / agree with other opinions to find something that would work for the whole group. The the agents ensure that no information regarding budget or other sensitive details is disclosed, while still acting with the member's best interest. Then the two best options are played out as small cities, a day at a time so that everyone gets a visual of what to expect. Finally, everyone puts down their seals, and their share gets approved and charged to their card.
How we built it
Headset: three.js with WebXR running in the meta quest browser, avoids the hassle to install any apps. It uses passthrough mixed reality, hit-testing and anchors to put the chart on a real table. Phones: React and Vite. Every screen works on its own, so a member without a headset can still get the full experience. Server: Node, Express and Socket.io. All devices at the same table are synced in real time. The mates/Agents: Gemini writes each mate's lines inside a strict protocol. A privacy filter checks the lines before they are spoken, so no sensitive information is discolsed. Voices: ElevenLabs gives every seat its own voice which is paced similar to a real life convo. Memory: Backboard lets the mate remember each user across trips. Storage: MongoDB Atlas keeps every tip, so the table can pick right up from where it was left off. Payments: A real call to the Visa developer sandbox validates each card when the owner seals their deal. We also built a model of Visa intelligent commerce's agent cards, a capped travel only card per person. Prices: RouteStack for live hotel and flight prices where available.
Challenges we ran into
Mixed Reality and the browser: Keeping a chart locked to a real table is a lot harder than it sounds, to the point that our first versions kept jumping around everytime you pinched the controller. We ended up rebuilding the placement so the chart would only move when you explicitly place it. AI are not People: The mates originally spoke "I am in a rush! Get me out of here" They talked too fast and kept repeating the exact same things. In the later implementation we slowed the pace and added a natural filler and wrote more ways to say all the lines. Privacy: A booking has to fail for everyone without anyone learning whose card declined/who bailed out. We designed the public view so a backed out seal looks the same as the other seals.
Accomplishments that we're proud of
- A full trip from who is in to the whole itinerary booked, and running it across a phone, laptop and a meta Quest.
- Any mix of headphones and phones can share one table, and each person only sees their own numbers.
- All or Nothing works well with the privacy, allowing people to backout safely without upsetting others.
What we learned
- Spatial Computing shines when it puts people around a shared object. The table did more for the feeling since it gave a feel of planning alongside others.
- Agents are a lot more trustwothy when the code is checking what is said, than just relying on prompts.
- Consent and Privacy have to be designed and don't work as a settings page.
What's next for project
- Real booking through Visa intelligent commerce once we have production access, and live inventory for more cities.
- Smart glasses, so the table can come along wherever the group is. Even later down the line maybe a tour guide comes alongside with personalized planning.
- Implementing the Everyone agrees, to shared rent, event plannings and for group gifts.
META CHALLENGE: Who it is for: Friend groups, families and roommates planning some trip together, especially for groups scattered across cities or sometimes even countries. How it strengthens human connection: TripDeck turns the stressful chat thread into an actual moment together. With headsets, friends from different places sit around the same table in their homes and discuss a group trip. Everyone's voice is heard including the one who couldn't make it. Money stops being the thing that splits the group, nobody has to announce their budgets and nobody gets stuck paying for others. Why AI is essential: The toughest part of a group planning is the negotiation between private constraints that no one wants to say out loud. A shared spreadsheet is not capable of doing so, because everyone's numbers are out in the open. Personal AI agents can hold each person's private terms, argue for them and find a trip that fits everyone, without making anyone feel uncomfortable. Gemini does the reasoning and ElevenLabs does the talking by giving each agent a voice of their own, and finally Blackboard lets the agents remember the trips to maximize user comfort.
Log in or sign up for Devpost to join the conversation.