Vifu

An open-source Agent Runtime and visual creator for AI-native games.

Comment

Inspiration

Game creators can already build compelling AI characters, but turning those characters into a reliable game runtime still means coordinating providers, state, tools, endpoint security, publishing, and observability. I built Vifu to make that workflow one coherent creator experience.

What it does

Vifu is an Apache-2.0 Agent Runtime and visual creator for AI-native games.

A creator can:

  • connect agent providers through the Vifu Agent Gateway;
  • manage project agents, personas, capabilities, and versions;
  • author a headless game backend in Canvas;
  • edit interactive anime stories in a CapCut-style Short Drama timeline;
  • keep Canvas and Short Drama synchronized through one runtime graph;
  • localize a story into English, Japanese, and Simplified Chinese;
  • preview choices, free-text agent interactions, host actions, and endings;
  • publish an immutable release behind a stable project endpoint;
  • call the runtime from a web game or game engine and inspect execution logs.

The included sample, Last Train to the Moon, is a complete interactive anime drama. It has five central characters, two free-text Agent interactions, branching state, a physical hold action, generated portrait media, character voices, music, and three possible endings. The repository includes its editable source, assets, a full gameplay capture, and a dependency-free web host that consumes only the public Vifu endpoint.

How I built it

The core runtime is written in Rust. It compiles a versioned graph into a deterministic release, validates routes and schemas, runs choices and effects, persists durable sessions in PostgreSQL, and exposes HTTP, SSE, and WebSocket contracts through the Vifu server. The Agent Gateway multiplexes provider traffic over one authenticated connection.

The creator Dashboard is a Next.js and React application. Canvas provides the general headless runtime graph, while Short Drama gives narrative creators a timeline, media tracks, cast management, localization, Preview, and publishing workflow. Both edit the same game source.

Docker Compose runs PostgreSQL, the runtime, Agent Gateway, and Dashboard as one self-hosted stack.

How I used Codex and GPT-5.6

I organized the work as a repeatable research-to-product loop rather than a one-shot coding prompt.

First, I opened focused Codex sessions to define the target creator, research their workflow, and compare relevant open-source runtimes, agent systems, game tools, and papers. GPT-5.6 helped gather and structure the evidence; I selected the references and made the product decisions.

Second, GPT-5.6 helped turn the accepted research into a written specification. I reviewed it, then used Codex Plan Mode to produce a staged implementation plan with an explicit user-visible result.

Third, Codex implemented the plan across Rust, Next.js, PostgreSQL, Docker, and tests. I used Agent Browser to operate the real Dashboard while it was being built. I watched the session, reported UX and runtime problems immediately, and repeated the loop until the same product could create and run the sample game.

Finally, I used the same workflow to prepare this submission: improve the reusable editor, build the game in Vifu, verify the endpoint in a separate host, capture real gameplay, and assemble the final demo from the saved Short Drama timeline.

The full first-person development record and verification evidence are in BUILD_WEEK.md.

Challenges

The hardest part was maintaining one semantic source across a visual graph, a narrative timeline, preview sessions, published releases, and an external host. Agent output also had to be constrained enough for deterministic state updates while preserving expressive free-text play.

The creator workflow exposed issues that unit tests could not: media that existed in the timeline but did not render in Preview, unclear node relationships, duplicated concepts, and flows that were technically complete but too difficult for a first-time creator. Building a real game inside Vifu turned those gaps into concrete fixes.

Accomplishments

  • A complete open-source runtime and self-hosted creator stack.
  • One synchronized Canvas and Short Drama source model.
  • Durable interactive sessions with choices, Agent effects, host actions, localization, and branching endings.
  • A full sample game produced through the creator workflow.
  • A separate web host proving that the published endpoint is the product contract.
  • Final verification covering 137 Rust tests, focused Dashboard tests, TypeScript, clippy, production build, Docker health, browser operation, and video inspection.

What I learned

Creator tooling should be verified by creating, not only by testing components. The most productive use of Codex and GPT-5.6 was the continuous loop between research, a reviewed spec, implementation, real browser use, and targeted correction.

What's next

Next steps include native rendered-video export, additional game-engine hosts, more reusable runtime node types, deeper trace visualization, and polished project import/export.

Built With

Share this project:

Updates