Amulith

Title screen: pick Bloom or Blight, or place a Lenny figure on the shrine
Level 2, Mushroom Hollow, played as Blight
Shrine Grove as Blight: night falls and the veins spread
Sleepy and angry Lenny, the Jetson Orin Nano, the shrine camera and the Arduino relic
Level 3, Rootdeep; this Amulith line came live from the Jetson
Amulith on the TV at Knight Hacks IX, run entirely by the Jetson
Judge mode: real numbers from the device (power, temperature, tokens per second)
The shrine camera on the Jetson tells sleepy from angry Lenny
A Bloom pulse turns Blight soil back into grass; light marks the next shard
Shrine Grove with Lenny, T.K. and the gem shard; Amulith speaks live from the Jetson

Inspiration

Knight Hacks IX gave us a story: the gem shattered, the forest grew too fast, T.K. lies moss-covered beside Lenny, and "something left behind by the gem is still awake." We wanted that something to be real, so we put it inside the board on the table. Amulith is a language model running on the Jetson's GPU, housed in a papercraft stone hill, and it needs no internet to talk to you.

What it does

Amulith is a top-down 2.5D forest game in an HD-2D pixel style. You pick a side, Bloom (free T.K. and Lenny by destroying the shards) or Blight (gather the shards and keep the forest corrupted), and play as the Wanderer through three levels: Shrine Grove, Mushroom Hollow and Rootdeep, each with its own intro card and lore from Amulith. A pulse converts nearby ground to your faction, a lantern widens or narrows how far you see (and how far enemies notice you), and enemies of the opposite faction chase you with A* pathfinding and reconvert the land. Find and resolve three of six shard sites to end the run.

Amulith itself speaks every so often and when events happen. It lies to and tempts Bloom players and flatters Blight players. The language model returns a small JSON object (line, mood, how corruption spreads, what spawns, optional hint) that the game reads. Asking Amulith for a hint costs you land. A physical Arduino relic (knob for the lantern, button for "Ask Amulith", green LED as its heartbeat) duplicates the controller actions, and a webcam is aimed at the printed shrine so that placing a sleepy or angry Lenny figure can choose your faction (controller and keyboard picks are the fallback). A judge-mode overlay shows real numbers from the device (GPU use, tokens per second, power, temperature) and shows n/a when a reading is missing.

What is live and what is scripted: the director lines come from the on-device model when it answers in time. If it times out or errors, a scripted director with the same schema takes over, and the dialogue box labels which one spoke. Shrine interaction lines in this build are scripted and labelled as such.

How we built it

Game (game/): Vite, TypeScript and Three.js. A curved heightfield diorama with billboarded pixel sprites, a land shader (Bloom grass and flowers, Blight soil and creeping veins), day-for-Bloom and night-for-Blight lighting, a real lantern point light and one post-processing pass (tilt-shift, glow, vignette, grade). Gamepad API plus keyboard.
Jetson service (remnant/): Python with FastAPI and one WebSocket. It runs the LLM director, the scripted fallback, a tegrastats telemetry stream and the Arduino serial bridge. The message contract between game and service lives alone in protocol/, and a mock service let the game be built before the device was ready.
The model: llama.cpp built with CUDA, serving Qwen2.5 Instruct (Q4_K_M GGUF) downloaded from the Hugging Face Hub. Output is constrained by a JSON schema.
Relic (firmware/relic/): one Arduino Uno sketch (knob on A0, button, LED with software PWM) talking over USB serial. We treat the Arduino as a USB serial device and do not wire its 5V outputs to the Jetson's 3.3V GPIO.
Art: sprites and props are pixel art finished in Aseprite from AI-generated sources (GPT Image 2.5), directed by Juan. Lenny and T.K. are pixel art made in Aseprite from the official Knight Hacks IX character art. Every generated file is listed in game/public/art/CREDITS.md. The two music loops and seven sound cues are original CC0 procedural audio made for the game.
Team workflow: two humans, each with AI coding agents working in separate lanes with file ownership, a shared DECISIONS.md and TEAM-LOG.md, and reviewed pull requests.

Challenges we ran into

The game and the model share one small GPU. Rendering halved the model's speed (19.0 to 9.6 tokens/s, measured on the Jetson). Lowering the render resolution did not help because the cost is per frame, so the game now draws every third frame while Amulith is thinking, which brought generation back to 15.3 tokens/s.
Switching the director to Qwen2.5-1.5B and dropping the hint fields from the schema when no hint is requested took real lines from about 6.5 to 8.5 s down to about 2.5 s (hints about 3.5 s). Speculative decoding with a small draft model did not help on the Orin Nano.
The desktop first rendered on the CPU because the login screen held the NVIDIA device. A 4K TV also reported 3840 x 2160 at 1x, so the UI scales to screen height and the renderer caps at 1080p.
The UCF guest network dropped the Jetson more than once.
Art direction changed from a paper storybook look to HD-2D partway through, after a teammate shared references.

Accomplishments that we're proud of

A language model that runs entirely on the device and drives real game events, with a fallback that keeps the game playable every time.
Honest instrumentation: judge mode reports measured device numbers only.
A physical relic and shrine that are part of play, not decoration.
Accessibility built in: subtitles for every line, faction colors told apart by light and shape rather than red versus green, a reduced-motion setting and on-screen controller prompts.
Two people and several coding agents merging reviewed pull requests in separate lanes without overwriting each other.

What we learned

On a shared edge GPU, frame cost matters more than resolution, and the right fix can be to give the GPU to the model while it thinks.
Constraining a small model to a JSON schema makes it dependable enough to drive a game.
A scripted fallback with the same schema is what lets you demo with confidence.
Clear file ownership and a written decision log let agents and humans work in parallel.

What's next for Amulith

Tuning Mushroom Hollow and Rootdeep so runs last longer before the enemies close in.
Hardening the shrine camera for changing room lighting at new venues.
A demo video, and the game art placed into the Figma brand guide.

Landing page: https://cstljuan.github.io/amulith/

Credits

Lenny, T.K., the gem and the Knight Hacks IX lore belong to the Knight Hacks design team (Lena Tran, Gabriela Zambrano, Christina Nguyen, David Navarrete, Amira Bhuiyan). Amulith is a non-commercial fan project made at their event.