Mr-T-AI

Instead of manually assigning drivers and analyzing routes, dispatchers can simply talk to Mr. T like a human assistant. Basically, Jarvis for trucks

Comment

💡 Inspiration

Modern trucking operations still rely heavily on manual processes — spreadsheets, phone calls, and gut instinct. Dispatchers have to constantly balance driver availability, cost efficiency, and route optimization under time pressure.

We wanted to reimagine this experience:

What if dispatching felt like talking to an intelligent assistant instead of managing a dashboard?

Inspired by systems like Jarvis, we built Mr. T, an AI copilot that turns complex logistics decisions into simple conversations.

🚀 What it does

Mr. T is a voice-enabled AI dispatch assistant that allows users to:

  • Ask for available drivers in real time
  • Assign drivers using natural language
  • Compare driver options based on efficiency and cost
  • Receive explanations backed by real operational data

Example:

“Assign Maria Jen to Dallas”
“Who is the best driver for Phoenix?”

Mr. T:

  • Understands the request
  • Queries real data from NavPro
  • Makes intelligent decisions
  • Explains why using metrics like deadhead miles and cost

All of this happens through a natural voice interface, making logistics feel conversational and intuitive.

🛠️ How I built it

We built Mr. T using a hybrid AI + deterministic architecture:

  • Frontend: React (Vite) for real-time interaction and UI
  • Backend: FastAPI for orchestrating logic and API calls
  • LLM: Google Gemini for intent parsing and response generation
  • Voice: ElevenLabs for realistic, Jarvis-style speech
  • Data Layer: NavPro API for real driver, trip, and routing data

System flow:

  1. User speaks or types a command
  2. Gemini parses the intent into structured data
  3. Backend applies deterministic logic (driver ranking, cost analysis)
  4. NavPro API executes actions (trip creation, data retrieval)
  5. Gemini generates a natural explanation
  6. ElevenLabs converts it into speech

🚧 Challenges I ran into

  • LLM reliability: Early models struggled with consistent intent parsing
    → Fixed by switching to Gemini with structured schema outputs

  • Making AI responses believable:
    Generic responses felt fake and unconvincing
    → Solved by grounding responses in real metrics (miles, cost, comparisons)

  • API integration complexity:
    NavPro required proper driver setup and understanding of data structure
    → Built a service layer to normalize and handle API responses

  • Voice experience:
    Basic TTS felt robotic
    → Integrated ElevenLabs for natural, conversational voice

🏆 Accomplishments that I'm proud of

  • Built a fully working AI agent that interacts with real logistics data
  • Successfully integrated multiple complex systems (LLM + API + Voice)
  • Created a natural, conversational interface for a traditionally complex workflow
  • Delivered data-backed AI decisions, not just generic responses

📚 What I learned

  • LLMs work best when combined with deterministic systems
  • Structured outputs are critical for reliable AI applications
  • User experience matters as much as intelligence — voice made a huge difference
  • Real data integration transforms a demo into a product

🔮 What's next for Mr-T-AI

  • Real-time alerting for delays, breakdowns, and compliance risks
  • Predictive driver assignment using historical data
  • Multi-load and fleet-wide optimization
  • Fully autonomous dispatch workflows
  • Deeper integration with real-time tracking and analytics systems

Built With

Share this project:

Updates