IBTROS

Seamless shuttle booking and calling using telegram, azure ai, copilot and azure cloud resources.

Comment

IBTROS (Institute Based Shuttle Management System)

Inspiration

My name is Jeremiah and I’m the founder and CEO of IpTrust. We built IBTROS to revolutionize how students, workers, and parents commute to various institutes such as schools, churches, banks, and hospitals. Inspired by the power of Azure AI Foundry and the innovative GPT-40 API, we set out to create an intelligent, multimodal shuttle ordering system that leverages text, voice, and image processing. Our vision is to empower users with an interactive, conversational experience that makes ordering a ride as simple as sending a message on Telegram.

What it does

  • Shuttle Ordering: Users can order a ride directly from the Telegram Bot UI by entering natural language queries (e.g., “I want to go from MFM to DBI”). Our system intelligently processes both abbreviated and full institute names.
  • Multimodal Interaction: Users can choose to interact via text, or opt for speech-based conversations enhanced by Azure’s text-to-speech (TTS) and speech-to-text (SST) capabilities. They can also submit images to get additional contextual responses.
  • Dynamic Query Processing: Using Azure OpenAI’s GPT-40 API, our system distinguishes between ride orders and general inquiries, ensuring that users receive the most accurate results.
  • Accessibility & Engagement: The platform provides extra features such as a live avatar (powered by Azure Cognitive Services) that responds in multiple modalities, making the interaction both engaging and accessible.

How we built it

  • Telegram Bot UI: The primary user interface is built on Telegram. Users trigger the process by searching for “hypertrust” and start interacting with the bot.
  • Azure OpenAI GPT-40 API: All natural language queries are routed through our query processing layer, where GPT-40 intelligently filters and interprets the request.
  • Cognitive Services Live Avatar: For an enriched experience, users have the option to engage with a live avatar that uses SST, TTS, image recognition, and text processing to deliver interactive responses.
  • Backend Integration: Our system uses SQLite or Azure Postgres to store institute data (e.g., price and location), and returns this information based on filtered queries.
  • Order Workflow: Once a ride is ordered, the system generates a unique order number for confirmation. The driver or “captain” then uses this order number to confirm the ride.
  • IBTROS Demo: Users can easily see the entire flow—from entering the query to receiving shuttle details—in a demo that highlights our natural language processing and multimodal capabilities.

Challenges we ran into

  • Natural Language Variability: Users express ride details in various forms. Filtering and accurately matching these inputs to our database was a complex challenge.
  • Multimodal Integration: Combining text, voice, and image processing in real time required seamless integration of multiple Azure AI services.
  • User Experience: Ensuring an intuitive, accessible, and responsive user interface—whether users choose text-only or interactive speech—was critical.
  • Scalability: Balancing rapid processing times while integrating Azure OpenAI, Cognitive Services, and database solutions demanded careful architectural planning.

Accomplishments that we're proud of

  • Seamless Azure AI Integration: Successfully leveraging Azure AI Foundry and the GPT-40 API to deliver both transactional ride orders and rich, multimodal conversations.
  • Enhanced User Experience: Delivering a natural, engaging, and accessible platform that adapts to various user interaction modes.
  • Robust Filtering Engine: Implementing advanced algorithms that accurately process and match diverse user inputs to the correct institute destinations.
  • Scalable Architecture: Building a backend that efficiently integrates lightweight databases with powerful cloud-based AI services.

What we learned

  • User-Centric Design is Key: Prioritizing ease of use and accessibility ensures that advanced AI capabilities are truly beneficial.
  • Interoperability Drives Innovation: Seamless communication between Azure services (OpenAI GPT-40, Cognitive Services, and databases) is essential for delivering a cohesive experience.
  • Feedback Fuels Improvement: Continuous user feedback is invaluable for refining natural language processing and multimodal interactions.
  • Responsible AI Matters: Balancing cutting-edge innovation with ethical considerations and user privacy is an ongoing commitment.

What's next for IBTROS

  • Enhanced Customization: Introduce features that allow users to personalize their ride experience further, such as adjustable waiting times.
  • Broader Service Integration: Expand to support additional institutes and integrate with more transportation services.
  • Refined Multimodal Capabilities: Continue enhancing our live avatar and conversational AI for even more natural and context-aware responses.
  • Increased Scalability: Optimize backend processes to handle higher traffic and ensure robust performance as the user base grows.
  • Hackathon & Beyond: We’re entering the Azure AI Developer Hackathon with a focus on the “Best Use of Azure AI” category. By showcasing our innovative use of Azure AI Foundry, GPT-40, and integrated multimodal interactions, we aim to win the hackathon and accelerate our mission to transform institute-based shuttle services.

Ready to take your AI skills to the next level? We are too. Join us in revolutionizing transportation with the power of Azure AI!

Built With

Share this project:

Updates