Jarvis

Inspiration

We were inspired by the limitations of Google Workspace tools (e.g., Google Calendar, Gmail). Despite their utility, they require manual clicks and lack natural language understanding (NLU). Users must navigate interfaces instead of issuing simple commands like "Schedule dinner with Alex tonight." Jarvis was conceived to bridge this gap—transforming human language into automated actions, mimicking a conversational personal assistant like a next-gen "J.A.R.V.I.S." from sci-fi.

Features:

Architectural Design: Building a multi-agent system requires nuanced orchestration. The root agent must intelligently decompose complex prompts (e.g., "dinner plans + weather check + email") into standalone tasks, while sub-agents (Calendar, Email, Mobility) must specialize without overlap.
API Integration: Working with third-party APIs (Google Calendar, Gmail, weather/transportation services) demands robust error handling, authentication protocols, and data standardization.

Project Structure:

Frontend: Developed an audio interface to capture user speech and transcribe it to text.
Backend:

Root Agent: Decouple the user prompts into subtasks and identify the workflow(e.g., "check availability," "check weather," "send email").
Sub-Agents:
- Calendar Agent: Interacts with Google Calendar API to verify availability (e.g., "Is 7 PM free?").
- Email Agent: Uses Gmail API to draft/send confirmation emails.
- Mobility Agent: Fetches weather/transportation data via external APIs.
Workflow: For the user’s dinner request, Jarvis would:
- Check calendar availability via the Calendar Agent.
- Pull Richardson’s weather via Mobility Agent.
- Compose/send an email via Email Agent—all through a single spoken command.

Challenges:

Agent Coordination: Ensuring sub-agents share context and call expected API (e.g., Calendar Agent called add event and fetch email to invite people).