KO.rbit

Title Screen
Main hub
Object view
Fighting
Arena
Nav menu
Solar view
Grok stats

Inspiration

We wanted the lightsaber duel everyone acts out in the living room, with your real body in the fight and a real planet under your feet. We also wanted space to be real: a universe built from real catalogs, and fights on terrain NASA actually measured. And we wanted to see how far we could get in just 24 hours using AI tools working beside us.

What it does

KO.rbit is a real universe to explore and a full-body saber duel to fight in. Your laptop shows the world and tracks your body. Your iPhone is your lightsaber using its internal gyroscope.

Explore a real universe of 31,266 objects: stars, exoplanets, galaxies, planets, satellites, asteroids, comets and SpaceX spacecraft.
- Fly freely, search every object, jump straight to it and read its fact card.
- Find the Tesla Roadster, Europa Clipper, Psyche and Hera in their orbits around the Sun. Crew Dragon and Cargo Dragon are docked at the ISS.
- Speed up the orbits with a 1x / 60x / 600x time-lapse.
- Ask LYRA to guide you, and see where other pilots are in space.
Commission a pilot, jump into a 3D cockpit and challenge a rival.
Planetary roulette rolls Mars or Neptune, 50/50, then a travel cinematic flies you in.
Fight with your body. MediaPipe tracks your pose. Scan a QR code, hold to ignite your phone saber, aim by moving the phone, and swipe to step around your rival.
Relive the KO. If you opt in, a Perceptron-powered cinematic replay recuts the fight and ends on the finishing blow.

Real space data

Exploration catalogs (counted from the JSON files on main):

Stars, 24,835 (460 named): HYG Database v4.1 (Astronexus, CC BY-SA 4.0). Everything within 100 pc, plus named stars out to 3,000 pc.
Exoplanets, 5,301 around 3,894 host stars: NASA Exoplanet Archive, PSCompPars table via TAP.
Galaxies, 1,034: Gil de Paz et al., GALEX Ultraviolet Atlas, via NASA/IPAC NED.
Planets and the Moon, 9: NASA Planetary Fact Sheets.
Satellites, 20: the ISS and Hubble (CelesTrak) plus 18 Starlinks (TLEs from a community mirror).
Small bodies, 60: 42 asteroids and 18 comets from the NASA/JPL Small-Body Database API.
SpaceX spacecraft, 7: Crew Dragon, Cargo Dragon and TESS from CelesTrak, plus four heliocentric craft (Tesla Roadster, Europa Clipper, Psyche, Hera) from JPL Horizons orbital elements.
SpaceX fleet facts: vehicles, boosters, capsules and Starship flights from Launch Library 2.

Arena terrain.

Mars: real NASA MOLA elevation data for Jezero Crater. It's a 2.5° (about 148 km) crop of the PDS product MGS-M-MOLA-5-MEGDR-L3-V1.0, resampled to 256 × 256. You fight on the actual heightmap (vertically exaggerated 2.6×).
Neptune: stats come from NASA's fact sheet: 11.2 m/s² gravity, 72 K, winds from 0 to 580 m/s, a 16.11 h day. The fight floor is a fictional station platform, and the clouds are procedural.
Gravity is shown for each world. It doesn't change combat yet.

How we imported it.

Python importers (such as crop_mola.py and import-spacex.py) pull each source at build time. A provenance manifest records SHA-256 hashes and counts, and ATTRIBUTION.md plus an in-game credits dialog list every source.
About orbits: satellites and spacecraft use a two-body Kepler solver on real orbital elements from a dated 2026-10-18 snapshot. Not live tracking or SGP4, and satellite altitudes are scaled for visibility.

How we built it

Authoritative Node + Socket.IO server. It owns the lobby, health and every hit. Camera detections are commands, not proof. The laptop streams pose and the phone streams aim, and a 33 ms tick with 10 substeps checks blade against body, so fast swings can't slip through a guard.
Combat rules:
- 100 health, 15 damage per hit, 0.65 m knockback.
- Stamina 100: in/out steps cost 14, sidestep dodges cost 24, regen is 28/s after a 700 ms delay, and an empty bar leaves you winded until 40.
- Guards: a swing within 30° of a held guard breaks it and stuns the defender for 700 ms. After a clash, the defender can't swing back for 250 ms.
- 3 s countdown, rematch (round resets, devices stay paired), forfeits after a 15 s reconnect grace, and a draw if both players drop.
Three.js renders the cockpit, both arenas and the explore universe.
Perceptron replay: an opt-in local webcam clip (up to 45 s) with separate upload consent. Director cuts (orbit, low, overhead) add speed ramps, sparks and impact holds, ending on the KO.
Audio: a scene-aware CC0 soundtrack, plus a phone hum that follows swing speed.
Polish branch: Ally's hit VFX (sparks, shockwave, victory camera) lives on our saturn branch and isn't on main yet.

Tech stack

Client: React 19, TypeScript, Vite, React Three Fiber, Three.js
Body and saber: MediaPipe Pose on the laptop, iPhone gyroscopic sensors as the blade, QR pairing
Server: Node.js, Socket.IO. The server owns health, stamina, guard breaks, and every hit
AI: Grok Voice for live LYRA, Grok text-to-speech for duel cues, Grok Imagine for her hologram, Grok Bot for research, design, and sound, Perceptron Mk1.5 for the KO replay
Data: NASA, JPL, CelesTrak, and HYG catalogs, imported at build time with SHA-256 provenance

Tracks targeted

SpaceXAI

Grok Voice (Grok API): powers LYRA, our live voice copilot. She can read the match but can't change it. Your own AI companion that walks and guides you through the game, talks to you, answers questions, and narrates.
Grok Imagine: built the LYRA you see. We generated portrait options, picked the hologram, and used Imagine for her idle animation.
Grok Bot: a whole crew: not one chatbot. Agents ran all weekend beside the programmers. They researched datasets and UI, rated plans, wrote build prompts, designed the brand, storyboarded victory and defeat, redesigned saber setup, curated the sound pack, synthesized jump and landing sounds, audited branches, and drafted this writeup. We wrote the game while they kept research, sound, and previews moving so coding never stopped
Cursor: Agent sessions built the camera pipeline, phone saber, 3D arena, catalog explorer, LYRA, and the replay director. Cloud agents audited the repo so planning and code stayed on the same tree
Real space datasets: NASA, JPL, CelesTrak and HYG catalogs power all of exploration (31,266 stars, exoplanets, galaxies, planets, satellites, Starlinks and the Tesla Roadster) plus our Mars arena, built on real NASA elevation data.

Perceptron

A full replay path, not a caption on a video. Opt in and the laptop records a silent clip only while you are fighting, capped at 45 seconds. The arena motion is kept locally too, so the knockout still exists after that cap. Upload is a second consent, and only for the match you just finished.
Perceptron Mk1.5 returns the earlier scenes: time ranges, captions, and boxes that follow the player. The game turns those into the cut, with new cameras, slow motion, and sparks. The last shot is always the server-confirmed finishing blow, so the model cannot invent the winner.
Proved on the live API with a synthetic silent clip: one movement interval and eight timestamped boxes. MediaPipe tracks the body during the fight and is not this model.
This video can be uploaded and saved to the computer

Challenges we ran into

Two sabers diverged. One line paired body tracking with phone aim, and another made a camera-slash projectile on an older movement grid. We converged on the phone saber, but the rebrand branch still sits on the old line.
The contracts drifted apart. Our shared contracts package stayed empty, so every branch edited the server's types directly, and the lines stopped being wire-compatible.
The phone gyro needs HTTPS. iOS gates motion data behind secure pages and a permission tap, so we ran self-signed certs and Cloudflare/ngrok tunnels whose hostnames kept changing.
Mid-hack pivot from a gesture-based rock and blaster duel to the phone saber. Originally was supposed to be real world position tracking computer vision battle game
The replay failed quietly. Stopping the recorder without a flush saved an empty file, and black bars on the result screen made a good take look frozen. Fixing that was recorder engineering plus a new result layout, not a one-line API call.

Accomplishments that we're proud of

A two-device, full-body duel where the server decides every hit, with real stamina, guard breaks and rematches.
31,266 real objects from public NASA, JPL, CelesTrak and community catalogs, with SHA-256 provenance and in-game credits.
A Mars arena on actual NASA MOLA Jezero terrain.
A cinematic KO replay with opt-in recording and separate upload consent.
An AI crew next to the coders: research, previews and audio happened in parallel while we shipped code.
Favorite moment was the first landed hit between two phones after half the hackathon struggling

What we learned

Treating camera input as a request made fights fairer and easier to debug.
Agree on shared contracts in hour one.
Real data needs provenance. Hashes and attribution make it trustworthy.
Short-lived tokens and read-only tools keep an AI copilot safe, and firm shared rules keep an AI crew aligned.

What's next for KO.rbit

Gravity that matters: make Mars and Neptune gravity change how combat feels.
Replay QA on a full recorded duel and phone browsers.
Shared contracts and a full two-laptop, two-phone test pass. -** Expand with even more data**
**Improve fighting mechanics and add what we wanted in the beginning, blasters, force pushing, terrain manipulation

Overall this is the project we are the most proud of creating ever so we hope everyone likes it