SIGIL — Drawings You Can Run

Draw a System. Awaken a World. Upload a hand drawn system, watch its actual ink become a living simulation, then trigger crises, intervene, rewind, and compare alternate futures.

Comment

Inspiration

I started SIGIL with a simple question:

What if a drawing was not just a picture of a system, but the first layer of programming it?

Most people can draw a system long before they can program one. A student can sketch a colony, a cell, a watershed, or an imaginary kingdom and explain how its pieces connect. Turning that drawing into something interactive normally requires modeling software, domain knowledge, and a lot of code.

The idea that locked the project in for me was the idea that the ink lifts off the page and becomes the world.

I wanted that transformation to be literal, not a crossfade into an unrelated scene. But I also did not want to build a visual trick. If the drawing becomes a world, that world should actually run. It should have resources, dependencies, failures, decisions, and consequences.

That became SIGIL.

What it does

SIGIL turns a hand drawn or digitally illustrated system into a living, interactive simulation.

In the Mars Colony experience, I can upload an amateur or crayon style drawing. GPT5.6 Sol reads the visible shapes, labels, arrows, spatial regions, and likely system roles. SIGIL validates that interpretation and compiles it into a structured WorldSpec.

The important boundary is that GPT5.6 interprets the drawing, but it does not invent or execute simulation code. Deterministic TypeScript owns the system's rates, capacities, resources, rules, events, metrics, and state.

When I press "AWAKEN", SIGIL samples the visible marks from the actual uploaded image. Those marks retain their source positions and colors, lift off the paper as particles, travel toward their interpreted targets, and assemble into the colony's structures and conduits.

Then the world starts running.

The Mars colony includes power generation, battery storage, oxygen, water, agriculture, medical support, rover charging, communications, and a network of dependencies between them.

I can trigger a dust storm that reduces solar production, drains the battery, and forces critical and discretionary systems to compete for limited power.

I can then enter an Invocation such as:

"Preserve medical power. Disable rover charging and cut agricultural lighting by fifty percent."

GPT5.6 converts that request into a bounded structured plan. I can inspect the targets and actions before applying it. Once applied, the deterministic engine (not the model) produces the verified consequences.

SIGIL also lets me rewind to the same branch point, try a different intervention, and compare two alternate futures called Echoes. Both branches run to the same simulated horizon, so their resource levels, system uptime, survival outlook, benefits, and costs can be compared directly.

Mars Colony is the complete executable demonstration. SIGIL also contains original Living Cell, Watershed Ecosystem, and Arcane Realm foundations, all built around the same world, Trace, event, intervention, and renderer architecture.

How I built it

SIGIL is a browser based TypeScript application built with Next.js, React, Three.js, React Three Fiber, WebGL2, Zod, Zustand, and the OpenAI Responses API.

The source ink Awakening uses custom GPU driven particle rendering. Instead of creating one component per pixel, SIGIL extracts and samples the drawing into bounded particle buffers. Each sampled mark retains its original source coordinate, source derived color, semantic Trace, and deterministic target.

The simulation is a fixed step deterministic runtime. It handles production, consumption, storage, relationship capacity and loss, consumer priority, timed events, metrics, snapshots, replay fingerprints, rewind, and branch comparison. The renderer receives derived presentation state but never becomes the authority for the simulation.

GPT5.6 Sol is essential in two places:

  1. It interprets uploaded drawings into a strict structured representation.
  2. It translates natural language Invocations into bounded actions.

Every model result is validated before it reaches the executable world. Invalid or ambiguous results become visible Fractures instead of silently entering the simulation.

I used Codex as my primary implementation partner throughout Build Week. I did not treat it as a one shot code generator. I broke the project into implementation batches, established contracts and safety boundaries, reviewed results, tested visible behavior, and changed the workflow when coordination became too slow. Batches are interpreted instructions for codex created by GPT5.6 Sol based on iterative conversations between GPT5.6 and I. I give it my vision, ensure it understands me, the it creates the instructions to pass off to codex.

Codex helped build and integrate the repository, WorldSpec contracts, server routes, source image pipeline, renderer, simulation engine, Chronicle, Echo system, tests, public deployment, and submission video tooling. I made the core product, architecture, design, safety, scope, and acceptance decisions.

The current submission build has hundreds of passing unit tests, end to end browser coverage, a public Vercel deployment, and a bundled demonstration that remains usable even when the live model path is unavailable.

Challenges I ran into

Making the transformation real

The hardest visual requirement was preserving continuity between the source drawing and the awakened world.

The source image could not simply fade out while a 3D scene faded in. The actual marks had to remain visible, separate from the source plane, travel toward corresponding world elements, and remain understandable during a continuous camera move.

Letting AI interpret without letting it control the truth

A language model is useful for understanding an irregular drawing, handwritten labels, and ambiguous arrows. It should not be responsible for generating arbitrary runtime code or changing protected simulation rules.

The solution was to make GPT5.6 the semantic compiler front end while deterministic code remains the execution engine.

Replaying alternate futures fairly

Echoes only mean something when they begin from the exact same state and run for the same amount of simulated time. Snapshots, seeded behavior, fixed timesteps, immutable branches, and replay fingerprints were necessary to make the comparison trustworthy.

Building under the deadline

I originally tried splitting work across multiple Codex threads. The coordination and verification overhead became slower than using one primary implementation thread. I simplified the workflow, preserved the useful contributor work, and moved the remaining implementation into one continuous Core & Integration session. This project was less of a "Build Week" for me, and more of a "Build for 4 Days" because I found out about this so late. Luckily for me, Codex can compress the work of dozens over a year into a few hours.

Accomplishments I am proud of

  • A real amateur crayon drawing can be uploaded and interpreted by GPT5.6. As well as digital artwork of course.
  • The uploaded drawing's actual pixels participate in the Awakening.
  • The Mars world is a functioning deterministic simulation, not a prerecorded sequence.
  • Natural language interventions become inspectable bounded actions.
  • Rewind restores the exact branch state.
  • Echoes produce different, repeatable outcomes from different decisions.
  • The application distinguishes live, cached-live, bundled, and fallback provenance honestly.
  • The public demo works without an account.
  • The same architecture supports engineered, biological, ecological, and fictional world foundations.

What I learned

The most important technical lesson was that AI and deterministic software do not need to compete for authority.

GPT5.6 is strongest where meaning is ambiguous: reading drawings, interpreting language, identifying relationships, and explaining uncertainty.

Deterministic code is strongest where consistency matters: state, resources, timing, replay, validation, and consequences.

The most interesting version of SIGIL comes from combining both.

I also learned that visual spectacle and technical trust can reinforce each other. The ink becoming the world gets attention, but the ability to stress that world, change it, rewind it, and compare futures is what makes the idea useful.

What's next

The immediate next step is making Living Cell, Watershed Ecosystem, and Arcane Realm fully executable through the same shared runtime.

After that, I want to build:

  • richer drawing correction tools;
  • direct spatial editing of awakened worlds;
  • Realistic Visuals
  • teacher created lessons and assignments;
  • prediction before intervention activities;
  • shareable World Capsules;
  • a World Pack SDK;
  • calibrated data backed simulations;
  • collaborative worlds;
  • a public library of systems;
  • tablet and mobile drawing support.
  • More deterministic system support in the immediate future but eventually procedurally generated systems so that any system can be made and interpreted accurately.

The long term idea is simple:

Drawing should be a real interface for building, testing, and understanding systems.

Built With

Share this project:

Updates