Compass

agent systems
main page
flights
resturants
hotels
tours
payment
maps
start

Inspiration

If you've ever planned a vacation before, you know how difficult it is. Vacation planning takes an average of 10 to 30 hours of research and booking for a typical trip, often spread over 3 to 6 months. We were motivatived by the question of: what if we could shorten this process to 5 minutes? Introducing Compass, an app consisting of a workflow of AI agents to automate the entire vacation planning process, from researching the options, calling for reservations, and automatically paying, so that your next vacation can be done within the time to make a coffee.

What it does

End-to-end planning: Compass researches and synthesizes a complete trip plan from flights, hotels, restaurants, to activities from simple user preferences
Automatic bookings: Compass has an agentic caller who automatically books hotels withou you having to speak a single word
Optimized itinerary: Compass builds an optimized day-by-day route and visually displays them and travel-mode recommendations on a map
Human-in-the-loop: At each stage, Compass allows live approvals and selections by a human (YOU!).
Payments: Compass processes payments (Stripe flow) and executes transaction workflows.

How we built it

Compass is built with a multitude of technologies:

Full stack - React frontend with FastAPI and uvicorn as ASGI server
Vapi - Voice call orchestration platform
ElevenLabs - for voice synthesis
Webhooks and Ngrok - Ngrok for providing a public HTTPS tunnel, and webhooks for real-time, live call transcript streaming
LangGraph - Multi-agent orchestration framework (and LangChain for tool use binding) for researching, API calling, validating, and payment
Fetch.ai uAgents and AgentVerse - for hosting and orchestrating 8+ different agents on the cloud
ChromaDB - Vector database for RAG
Gemini API - LLM used for agent reasoning
OpenStreetMap - for visually displaying planned routes and locations
Pydantic - Data validation and serialization
Stripe - Payment processing

Challenges we ran into

Live call transcript streaming: There were very little documentation for our specific case of webhooks in Vapi; setting up and properly sending real-time data required complex handling of polling in the frontend and webhook management in the backend
Rate limits: Vapi has very strict limits on the number of outbound calls per day (on free tier); we probably created 6 accounts just to use on testing making outbound calls
Figuring out how to run multiple APIs, making agents, and coordinating them on agentverse

Accomplishments that we're proud of

We built a project with a large number of features including 8+ different agents with individually coded tool calls. We also built a fast, real-time call transcript streaming service to make agentic processes more transparent to the user, instead of simply summarizing the results of an agent's actions.

What we learned

How to use webhooks to transfer real-time data, and that one should use ngrok when providing server access to the public
Combining AI agents that each achieve a specific purpose works very effectively
Simulating real-world call flows is costly and rate-limited; build lightweight mocks for CI
Build more lightweight prototypes instead of testing in too many files with too many libraries and APIs

What's next for Compass

Compass plans to add a gcal integration to directly add planned events to your calendar. We are also planning to more easily adapt to each user's personal background by learning from search histories, habits, etc.