Quarrylings

Inspiration

What if a mixed-reality game did more than place objects in your room? Quarrylings turns a real space into the home of a tiny society that grows, works, builds, and remembers.

What it does

Quarrylings is a Quest-first passthrough AR creature-colony game. The player plants a world seed, raises families of Quarrylings, manages food, water, rest, homes, resources, tools, baskets, meeting spaces, and persistence, then shrinks down to experience the settlement at creature scale.

When a same-variant colony reaches its awakening threshold, its members gather and sing before requesting a bounded group goal. Unreal remains authoritative: it validates the response, presents the goal for player approval or one retry, saves the result locally, and uses deterministic fallback behavior when cloud access is unavailable.

How we built it

The game is built in Unreal Engine 5.8 with C++, Meta XR, OpenXR, hand-first interaction, and a mobile-forward rendering baseline. The tracked Unreal client talks to a deployed Azure Functions broker. The broker uses Azure-hosted OpenAI models for structured group-brain decisions and Azure AI Content Safety for safety checks; secrets stay out of the game repository.

The current single-player route uses GPT-4.1 mini by default, GPT-5.4 mini for escalation, GPT-4.1 for vision-capable work, o4-mini for bounded reasoning, and text-embedding-3-large where embeddings are needed. GPT-5.6 Sol was used through Codex for the Build Week development work; it is not presented as the game's default runtime model.

What was new for OpenAI Build Week

Quarrylings existed before the submission period. After July 13, 2026, we used Codex with GPT-5.6 Sol to meaningfully extend it: refining the player-scale home loop, expanding colony housing and player-facing goals, building a higher-fidelity shrunk-world 3D asset pipeline, investigating a separate mobile AR target without breaking Quest, diagnosing Unreal build issues, and auditing the project for release and submission readiness.

Primary Codex build thread: 019f49fe-cdf5-7cb1-93a1-c9e71eab184c.

Challenges

Mixed reality makes every gameplay assumption physical. Scale, reach, room boundaries, passthrough composition, device performance, saves, and cloud failure all have to remain understandable and safe. We also had to keep AI suggestions bounded so a model never becomes the authority over the simulation.

Accomplishments

We have a signed Quest Alpha build, an Unreal-to-Azure group-brain client, a deployed and smoke-tested broker, persistent colony-brain records, player approval/retry UX, deterministic offline fallback, and a code-first public repository with a durable validation workflow.

What we learned

The strongest role for AI in a living simulation is to propose intent, not mutate state. The game should own rules, validation, privacy, persistence, and recovery; the model should add variety inside those boundaries.

What's next

Next steps are a polished public gameplay capture, broader Quest headset testing, judge Alpha access, higher-fidelity shrunk-world art integration, and continued work on the separate mobile AR path.

Built With

• android
• arkit
• azure-ai-content-safety
• azure-functions
• azure-openai
• c++
• codex
• engine
• gpt-4.1-mini
• gpt-5.4-mini
• gpt-5.6-sol
• meta-quest
• meta-xr
• mixed-reality
• o4-mini
• openxr
• powershell
• python
• text-embedding-3-large
• unreal-engine