presto

presto logo concept

Inspiration

"Welcome to Rosa's Cafe Temecula! How may I help you today?"

"Yea how many calories are in the burrito supreme? Is the corn in the corn tortilla Non-GMO? "

Luis can feel himself start to sweat as he contemplates the next course of action--does he make up some answer to appease the cranky lady at the drive through window? Or should he admit he doesn't know and possibly lose a customer?

Even worse, if the customer does order the non-popular corn tortilla, he'd likely have to run to the kitchen to tell the chef to specially prepare some because they do not keep them in stock or on hand most days.

Luis and the many other service industry workers manning an understaffed team act as our inspiration behind presto.

What it does

Presto is a dynamic, voice-enabled AI assistant designed to revolutionize the drive-thru, phone, and online restaurant ordering affair. Presto enables customers to receive instant, accurate responses to questions about menu items from nutritional information to ingredients and allergens. For restaurant owners, Presto offers a scalable solution to manage customer inquiries efficiently, ensuring consistent and reliable information delivery whilst freeing up staff to focus on higher-value tasks. Simply upload a PDF of your menu, and Presto is ready to go.

How we built it

We built this using Next.js for the frontend with a modern, responsive design. The backend uses Google Gemini AI (gemini-2.0-flash-exp) to process PDF nutrition guides and extract menu information. We integrated with Supabase for database management, storing restaurant data and menu items with proper foreign key relationships. The system uploads PDFs to local storage, processes them through Gemini AI to extract structured menu data, and stores everything in Supabase for instant querying. We implemented comprehensive error handling, retry logic for API quota limits, and detailed status messaging throughout the user experience.

Explanation of the Flow:

The user speaks to the Next.js application in their browser.
The Next.js application, specifically the restaurant/[id] page, requests restaurant and menu data from its own Next.js API route (/api/restaurant/[id]).
This API route then queries our Supabase DB for the specific restaurant and menu item information.
Supabase returns the requested data to the Next.js API route, which then passes it back to the Next.js application in the browser.
Once the restaurant and menu info is loaded, the VAPI.ai SDK is initialized. The fetched menu context (menu items and their nutritional info) is dynamically embedded into the VAPI assistant's system prompt.
The user's subsequent speech is captured by the VAPI.ai SDK and sent to the VAPI.ai Platform.

Challenges we ran into

One of the biggest challenges was getting Google Gemini AI to consistently return structured data from PDF nutrition guides. Initially, the AI was returning SQL queries with placeholders, which caused parsing errors. We solved this by switching to JSON output format and implementing markdown code block cleaning to ensure reliable parsing. We also faced API quota exceeded errors and implemented exponential backoff retry logic to handle these gracefully. Another challenge was ensuring the frontend properly handled the asynchronous nature of PDF processing while providing real-time status updates to users. We solved this by implementing proper timeout handling and status checking mechanisms.

Accomplishments that we're proud of

We successfully created a system that can transform any PDF nutrition guide into an intelligent AI customer service agent in minutes. The AI can answer complex questions about dietary restrictions, allergens, nutrition facts, and make personalized recommendations. We built a robust system that handles edge cases, API limitations, and provides a seamless user experience. The integration between PDF processing, AI analysis, and database storage works flawlessly, and we've created a scalable solution that any restaurant can use to improve their customer service.

What we learned

We learned the importance of choosing the right AI model and output format for reliable data extraction. We discovered that JSON output is much more reliable than SQL for AI-generated structured data. We also learned about handling API rate limits and implementing proper retry mechanisms for production-ready applications. The project taught us about the complexities of PDF processing and the importance of comprehensive error handling in AI-powered applications. We gained valuable experience in integrating multiple technologies (Next.js, Google Gemini AI, Supabase) into a cohesive system.

What's next for presto

We plan to expand the AI capabilities to handle voice interactions for drive-thru systems, integrate with existing POS systems for seamless order processing, and add multi-language support for diverse customer bases to interact in their native languages. We're also working on implementing real-time menu updates and seasonal item handling. Future features include customer preference learning, loyalty program integration, and advanced analytics for restaurant owners to understand customer behavior and optimize their menus accordingly.

Additionally, our original vision for presto included more than just saving orders, but tracking these orders to smartly stock inventory (i.e warning employees and calculating when they might be running out).