Nova Explorer

Your AI copilot for the final frontier

Comment

NOVA Explorer

Your AI copilot for the final frontier

Inspiration

I've always been fascinated by the idea of conversational interfaces that go beyond chat bubbles. When I saw Tambo's generative UI capabilities, I wondered: what if an AI could control an entire dashboard instead of just sending text?

Space exploration games like FTL and Star Trek's LCARS interfaces inspired the vision—a command center where you speak to your AI copilot and watch the ship systems respond in real-time. Not a chatbot with a game attached, but a living dashboard where AI is the interface.

What it does

NOVA Explorer is a voice-controlled space exploration game where you command a starship through an AI copilot named NOVA.

Key Features:

  • Voice Input — Speak commands naturally: "Scan for nearby systems", "Travel to Proxima"
  • Voice Output — NOVA speaks back using ElevenLabs TTS
  • Generative UI — The AI updates the dashboard directly: star maps populate, planet cards appear, events trigger
  • Persistent Dashboard — No chat history; just a command center that evolves
  • Sub-Agents — Cerebras-powered agents analyze and strategize
  • Dynamic Events — Discoveries, dangers, alien contact—all with meaningful choices

How I built it

┌─────────────────────────────────────────────────────────┐ │ NOVA Explorer │ ├─────────────────────────────────────────────────────────┤ │ Frontend │ AI/Backend │ │ ─────────────────│────────────────────────────────── │ │ Next.js 16 │ Tambo (Generative UI) │ │ React │ Cerebras (Sub-agents) │ │ Tailwind CSS │ ElevenLabs (TTS) │ │ Brutalist UI │ Daytona (Code Execution) │ └─────────────────────────────────────────────────────────┘

Architecture:

  1. Tambo Integration — Components wrapped with withInteractable() allow the AI to update them directly. The star map, ship status, planet cards, and events are all AI-controllable.
  2. Game State Layer — A React context with localStorage persistence keeps game state (ship stats, discovered systems, planets) synchronized between AI tool calls and the UI.
  3. Custom Tools — Built 9 game tools for the AI:
    • travel_to_star, scan_systems, scan_planet
    • land_on_planet, takeoff
    • trigger_event, resolve_event
    • update_ship, select_planet
  4. Voice Pipeline:
    • Input: useTamboVoice() hook for speech-to-text
    • Output: Custom ElevenLabs API route + React hook for auto-speaking AI responses
  5. Sub-Agents — Cerebras-powered agents handle complex analysis tasks like route optimization.

Challenges I faced

  1. Generative UI State Management

The hardest part was syncing AI-generated updates with React state. When the AI calls travel_to_star, multiple things must update atomically: star map, ship fuel, location status, and potentially trigger an event.

Solution: A global state accessor that tools can call, which dispatches CustomEvents that React components listen to.

  1. Null Safety in Dynamic Data

The AI sometimes generates incomplete data (null rewards, missing choice IDs). This caused runtime crashes.

Solution: Defensive filtering everywhere: rewards.filter((r): r is { type: string; amount: number } => Boolean(r && r.type && r.amount != null))

  1. Voice Output Timing

Auto-speaking every AI response created issues—speaking over itself, repeating messages, speaking during loading states.

Solution: Track message IDs, only speak completed messages, and provide a toggle for user control.

  1. Dashboard Layout That Doesn't Scroll

Fitting star map + location status + command input + suggestions in the left column while keeping planet/event + ship status + AI response in the right column—without scrolling—required careful flex layout tuning.

What I learned

  1. Generative UI > Chat UI for immersive experiences — Having the AI control the interface directly creates a fundamentally different (better) experience than chat.
  2. Tools are the new API — Designing good AI tools is like designing good APIs. Clear names, typed schemas, and atomic operations.
  3. Voice changes everything — Adding voice input/output transformed it from "a game with AI" to "an AI you interact with naturally."
  4. Defensive coding for AI outputs — Never trust AI-generated data structures. Always validate and filter.

What's next

  • Multiplayer co-op exploration
  • Procedural storylines with memory
  • Ship upgrades and crew management
  • More event types and alien civilizations

Built With

Share this project:

Updates