goobs

Create chibi AI agents (called goobs), give them skills and tools, and watch them work and roam in a living 3D workshop

Comment

Goobs — The Agent Workshop

Learn how AI agents work by building, training, and deploying them as living 3D characters.

Alternate video (gives full project demo/walkthrough): https://youtu.be/HnQYPQtNf5s

The Problem

AI agents are the most powerful tool most developers have never learned to use. The barrier isn't access to LLMs — it's understanding how agents actually work. What is a system prompt? How do skills differ from tools? Why does model choice matter for different tasks? Current tools show agents as text in a terminal or a chat bubble — abstract, intimidating, and hard to grok.

What Goobs Does

Goobs makes AI agent concepts tangible. You create chibi 3D AI characters called "goobs," give them custom skills and tools, chat with them, and watch them work in a real-time isometric scene. A progression system with challenges guides you from basic concepts (creating an agent, adding a skill) to advanced capabilities (writing files, running bash commands, building and executing scripts). Every action earns XP, unlocks new tools, and promotes you through tiers — turning agent mastery into a game.

Judging Criteria

Technical Execution

Three distinct systems running in real time, synchronized through a shared runtime state:

3D Scene (React Three Fiber + drei + FBX)

  • Isometric 3D workshop with OrbitControls, auto-rotate, and workstation markers
  • Agents spawn at drop position when dragged from a sidebar — becoming tangible 3D characters
  • Finite state machine drives 9 animation states: idle, walking, thinking, working, sitting, celebrate, error, idle_long, and an easter egg
  • Selection ring + camera focus on click, floating name badges, per-material color customization (skin, shirt, pants via FBX material mapping)
  • Walking animation with constant-speed interpolation when agents route between workstations

LLM Orchestration (Next.js API routes + SSE streaming)

  • OpenAI-compatible provider with configurable base URL and API key (encrypted at rest via AES-256-GCM)
  • Dual-mode chat: streaming plain responses AND streaming responses with tool calls
  • Custom SSE event types (text, tool_call, tool_result, done) for real-time UI updates
  • Tool registry with 4 built-in tools: read_file, write_file, list_files, exec_bash
  • Sandboxed workspace at ~/.goobs/workspaces/{agentId}/ with path traversal protection
  • Multi-turn agent runtime that chains LLM calls with tool execution

Progression & Evaluation Engine

  • Session-based progression engine with 8 challenges across 3 tiers
  • Counter-based and event-triggered challenge completion with immediate XP awards
  • Workshop challenge runner with double evaluation: deterministic keyword/rubric checks + rubric scoring
  • Challenge runs persisted to SQLite via Prisma for audit trail
  • Tool unlock gating — filesystem tools locked until Tier 1 complete, bash until Tier 2

The hardest part: keeping all three systems in sync. A tool call fires progression events, which update the challenge dock, which may unlock tools, while the 3D character plays a thinking animation, all within the same user action.

Innovation & Creativity

Agents as 3D characters. Every other agent builder shows agents as text in a terminal or a chat bubble. Goobs turns them into customizable chibi characters with pastel colors, floating name badges, and FSM-driven animations. You don't configure an agent — you create a companion.

Skills as user-written markdown. Skills aren't checkboxes or dropdowns. You paste raw markdown knowledge blocks that teach your agent what it needs to know. This mirrors how you'd actually document knowledge for an AI — and it means skills are infinitely flexible.

Tool gating as a game mechanic. Filesystem and bash tools aren't available from the start. You have to earn them by completing Tier 1 challenges. This creates a natural learning curve: first understand agents and prompts, then unlock read/write, then unlock execution. The "Tools Unlocked!" modal feels like a power-up.

Progression that teaches. The 8 challenges map directly to real agent concepts you'd use professionally: system prompts, skill assignment, model selection, tool use, multi-tool orchestration. Players don't memorize — they learn by doing.

Impact & Usefulness

Real agents doing real work. These aren't simulated characters — they're real LLM agents connected to an OpenAI-compatible provider. The code they write runs. The files they create persist. The bash commands they execute produce real output. Every chat message is a real inference call.

Educational by design. A complete beginner walks through: creating an agent → understanding system prompts → adding skills → deploying to a scene → chatting → using tools → running workshop challenges. Each step builds on the last. By the end of a 5-minute demo, you've experienced prompt engineering, skill assignment, tool configuration, and model selection.

Any LLM, any API key. Goobs works with any OpenAI-compatible provider. Bring your own key, pick your model. No vendor lock-in, no hidden costs.

Session-based + persistent state. Progression lives in sessionStorage — perfect for demo kiosks and repeatable walkthroughs. Challenge runs persist to SQLite for audit. One-click reset wipes everything clean.

Presentation & Demo

Visual from second one. The rainbow animated "goobs" launch screen immediately signals this isn't another terminal tool. The 3D isometric scene with auto-rotate draws the eye. Chibi characters bouncing in a workshop make people smile.

Clear story arc. Create → deploy → chat → level up. Every demo has a beginning (problem statement), middle (agent creation + tool use), and end (progression payoff + closing pitch). The challenge dock provides a visual progress bar that the audience can read at a glance.

The "level up" moment is filmable. When a challenge completes, a toast fires with XP, the agent does a celebration animation, and the challenge dock updates. This is the moment judges reach for their phones.

Fun factor. Chibi AI characters with pastel colors, skin/shirt/pants customization, bouncy idle animations, and a celebration dance. It's the only project in the room that makes people smile while teaching them LLM architecture.

Gamification Track

Goobs qualifies for the gamification track through its complete progression system:

8 challenges across 3 tiers:

  • Tier 1 — Onboarding: Create Agent, Add Skill, Deploy to Workshop, First Chat (140 XP total)
  • Tier 2 — Tools: Field 2 Agents, Write a File, Execute Bash Command (210 XP total, unlocks filesystem + bash)
  • Tier 3 — Automation: Build a Script (200 XP)

3 workshop challenges that test agents against real tasks: Change Prompt (poem with keyword checks), Code Writer (Python function with file + bash tools), Multi-Tool (REST API with tool chaining).

Double evaluation engine: deterministic pass/fail (keyword presence, minimum length, forbidden terms) + rubric scoring (code lines, docstrings, tool quality). Both must pass for final pass.

Rewards: XP badges, toast notifications, tool unlocks, progression tier promotion, 3D animation callbacks (celebrate on pass, error on fail).

Player goal: Complete all 8 challenges, unlock all tools, build a team of specialized agents that can handle any task.

Stack

Layer Technology
Frontend Next.js 14 (App Router) + React 18 + TypeScript
3D Rendering React Three Fiber + drei
Styling Tailwind CSS + Catppuccin-based custom theme
Database Prisma + SQLite
LLM Provider OpenAI-compatible endpoint (configurable)
Animation FBX model loader with NLA strip animation blending
Security AES-GCM encrypted API key storage

Built With

Share this project:

Updates