Inspiration

What if a child could step into a fairy tale, question its characters, and change what happens next?

We wanted to make reading and creative writing feel like play, without turning AI into a machine that does the imagining for the child. A blank page asks, "What story will you write?" A forest with a wolf waiting nearby asks, "What happens next?"

That became Once upon a Play: the child supplies the idea; the software turns it into a scene they can explore, change, and keep.

What it does

Once upon a Play is a browser-based story adventure. Our first tale is an original retelling of Little Red Riding Hood, playable as Red, Gray the wolf, or a Visitor. Different roles reveal different perspectives, and authored choices affect later scenes and three possible endings.

Players can talk to characters through scene-aware AI dialogue, then write their own next scene. "A hackathon appears in the forest" becomes a staged environment with characters, props, and dialogue—not just another paragraph in a chat window.

Children can also photograph a drawing and bring it into the story as an animated character, create a custom companion, or choose a ready-made one. ElevenLabs reads scenes aloud with a storyteller voice and distinct voices for the characters, while the text remains available on screen.

Exploration leads to hidden objects and six minigames. Creating a scene earns three coins; players can keep their coins or spend them in the Story Store on 30 decorations and 20 character accessories. Purchases, placements, and progress persist in the browser. This gives the story a simple, tangible budgeting choice: spend on something now, or keep the coins for something else. It is an in-game economy, not a real-money financial product.

At the ending, the adventure becomes an illustrated storybook that can be printed or saved as a PDF. No account or installation is required to try the desktop-browser demo.

How we built it

The frontend uses React, TypeScript, Vite, Zustand, and PixiJS. We built 36 layered 2.5D backdrops and 54 props in code using SVG, CSS, and compositing. Procedural sprite animation lets an uploaded drawing move like a built-in character. Zustand and localStorage preserve the story, conversations, characters, coins, purchases, and scene decorations.

An Express/Node.js backend deployed through Vercel Functions handles dialogue, scene planning, and narration. Anthropic Claude generates contextual dialogue and structured scene plans. Instead of inventing arbitrary assets, it selects from known backdrop, prop, and character IDs and a fixed set of allowed story flags. The server validates those outputs and their coordinates before rendering. Suggested conversation replies cannot directly set story flags.

ElevenLabs supplies the spoken narration and character dialogue. Voice is part of the story experience: the child can read a scene, hear its characters, and continue writing what happens next.

We also implemented input and output screening, checks on suggested replies, request limits, and automated tests covering scene planning, map validation, branching, drawing import, safety filtering, and storybook creation. These are prototype safeguards, not a claim that every possible interaction is safe.

Challenges we ran into

The hardest problem was making a scene playable rather than merely making its description sound convincing. We chose reusable assets over generating a fresh background image for every prompt, so the model could arrange a consistent world instead of repeatedly changing its visual style. The resulting pipeline turns a free-form idea into structured scene data that the engine can check and render.

Continuity was another challenge. Visiting the store should not erase a character's position, accessories, or conversation. Moving those details into one persistent store made the environment feel like the same place when players returned.

We also had to prevent repeated reward claims and tune content screening so ordinary fairy-tale language did not unnecessarily interrupt play.

Accomplishments that we're proud of

We built a playable loop: explore, talk, choose, invent, earn, customize, and finish with a storybook. The child's own drawing can join the cast, and the voice layer gives the characters distinct spoken identities.

We also took the project's central engineering question beyond the demo: how do we turn a generated description into an environment that actually works?

Within one day during SteelHacks, we wrote a companion research paper for the LEAP Workshop at CoRL 2026, From Stories to Executable Worlds: Control-Certified Layout Repair for Prompt-Generated Simulations.

The paper extends our generate-and-validate design principle to control-aware simulation layout repair. It studies changing declared geometric variables while keeping the task, agent, controller, event order, and timing fixed. Under its stated feasibility and model assumptions, the repair procedure reaches certification in finitely many updates with bounded net layout displacement.

In a separate MuJoCo evaluation of three structured stress cases, control-aware repair completed 9/9 trials, compared with 3/9 for geometry-only repair and 0/9 for unchanged layouts. These are scoped simulation results—not a benchmark of the browser game, an educational user study, or a guarantee for arbitrary generated worlds.

Read our companion paper.

What we learned

Creative freedom does not require giving a language model control over everything. A constrained set of assets can support many different scenes while keeping their structure inspectable. The model can improvise dialogue and propose a scene; the game engine should still own what becomes valid state.

We also learned to separate three questions: does a scene look plausible, does it render, and can the intended task actually be completed? Our game addresses structured rendering and story-state validation; the companion research investigates the physical-execution question in a controlled simulation setting.

What's next

We want to add more tales, voice input, cross-device saves, and parent-facing controls. We also want to test the experience with families and educators rather than assume that engagement or learning benefits have already been demonstrated.

Our initial product hypothesis is a creative-reading platform for families, followed by classroom and library use. Potential business models include additional story packs and educator tools. The reusable scene engine and a child's personal cast could support continued creation across many stories; demand and willingness to pay are still questions to validate.

Built at SteelHacks XIII

We are I'm Steel Hungry, the team behind Once upon a Play. We thank SteelHacks XIII, its organizers, and the event sponsors for making this build weekend possible. ElevenLabs is directly integrated into our prototype's voice experience.

We would be glad to acknowledge SteelHacks XIII and the relevant sponsors and technology providers in the companion manuscript, with wording that accurately reflects their contributions.

Play the demo · View the code · Read the paper