ShibaVerse

Meet Kuro, an AI Shiba Inu you talk to, drop into your real room in AR, and visit in VR. All in the browser, no app store. Companionship, calm, and focus in one link.

Comment

Inspiration

Two problems kept colliding for us: people are lonelier and more distracted than ever, and every "AR pet" or companion app hides behind an app-store install that nobody finishes. The Shibainu Hacks AR/VR theme was the perfect excuse to fix both at once, with the internet's favorite dog.

What if your companion was one link away, could actually talk to you, and could step out of the screen into your real room? Meet Kuro, a Shiba Inu who lives in your browser.

What it does

ShibaVerse is a browser-native WebXR companion. No install, no app store, no API keys.

  • Home Studio — a cinematic real-time 3D scene. Kuro looks at your cursor, breathes, blinks, wags his tail, and reacts to you.
  • AR — real WebXR immersive-ar on supported phones, with a webcam passthrough fallback everywhere else. Kuro trots onto your real floor/desk.
  • VR — real WebXR immersive-vr on a headset, with an orbitable immersive VR world (purple sky + infinite grid) on desktop.
  • Talk to Kuro — type or speak to him (Web Speech recognition). He replies in-character with a voice (speech synthesis) and a synthesized bark.
  • Care loop — Feed, Play, and Pet drive Happiness, Energy, Bond, XP, levels, and treats.
  • Focus with Kuro — a Pomodoro session where Kuro sits beside you and celebrates (and rewards treats) when the timer completes.

Kuro understands intent: say "let's focus", "show me in my room", "I'm stressed", "tell me a joke", "play", or "good boy" and he routes it to a real action (start AR, start a focus session, play, comfort you, and more).

How we built it

We started with a structured ideation pass (7 concepts scored on feasibility, impact, novelty, demo-ability, and scope) and picked a "universal companion" that absorbs the best of a fitness coach, a focus buddy, and a mood mirror into one app.

Stack: React + TypeScript + Vite, three.js with @react-three/fiber and drei for real-time 3D, @react-three/xr for WebXR sessions, Tailwind CSS for the UI, the native Web Speech API for voice, WebAudio for the bark, and Zustand as the single source of truth that wires the UI, the 3D scene, and the XR sessions together.

Kuro is 100% procedural. He is modeled entirely in code from rounded boxes, cones, spheres, and a torus tail, then rigged by hand: idle bob and breathing, a pointer-tracking head, timed blinks, a mood-scaled tail wag, ear twitches, and per-reaction routines (jump, spin, eat, love, sit, tilt). There are no external 3D model files, so he always loads instantly and is entirely original.

WebXR with graceful fallbacks. We detect isSessionSupported and enter real immersive AR/VR when possible; otherwise we fall back to webcam-composite AR and an orbitable VR preview, so the experience never breaks in a live demo on any device.

On-device brain. The personality and intent engine runs locally with zero keys, so the demo is bulletproof. We left a clean SHIBA_SYSTEM_PROMPT hook so it can be upgraded to a full LLM by returning { reply, reaction, mood, command }.

Design. The interface is a cinematic-dark spatial UI (a VisionOS-style glass system) with a single purple accent, custom SVG icons, and tuned micro-interactions, so it feels like a premium product rather than a hackathon sketch.

Challenges we ran into

  • Making a recognizable Shiba out of primitives. Getting the mameshiba brow spots, curled tail, and pointy ears to read as "Shiba" (not "generic dog") from boxes and cones took real geometry tuning.
  • WebXR device variance. AR/VR support differs across phones, laptops, and headsets, so we built layered fallbacks (real WebXR → webcam AR / VR preview) and handled the secure-context and session-lifecycle rules.
  • Keeping the demo bulletproof and keyless. We deliberately made the brain, voice, and bark run fully in the browser so nothing can fail on stage.
  • Trimming the XR library. The XR package ships a multi-megabyte device emulator; we disabled it and used our own fallbacks, cutting the initial load dramatically.

Accomplishments that we're proud of

  • A single install-free link that delivers real AR, real VR, voice, an AI personality, and a game loop together.
  • A completely original, procedurally-generated Shiba with a hand-built animation system.
  • A never-breaks live demo thanks to graceful degradation on every device.
  • A genuinely premium-looking spatial UI.

What we learned

  • WebXR fundamentals: secure contexts, session lifecycle, AR passthrough, and hit-testing.
  • Procedural geometry and rigging animation entirely in code with react-three-fiber.
  • The native Web Speech and WebAudio APIs for keyless voice and sound.
  • How to fuse an evidence-based design system with strict brand discipline to ship something that looks shipped.

What's next for ShibaVerse

  • Plug in a real LLM brain (the hook is already in place).
  • Tap-to-place hit-testing in AR and hand-tracking in VR.
  • Webcam emotion detection for true mood mirroring.
  • A persistent bond (local save), more mini-games, and a shared multiplayer ShibaWorld.

Built With

Share this project:

Updates