UnrealAssetLab

UnrealAssetLab turns Unreal Engine work into a guided flow: open a project, describe the task, approve a bounded plan, run Codex when needed, and execute safe, traceable Unreal operations.

Comment

Inspiration

Unreal Engine development is powerful, but the workflow is fragmented across chat, editor windows, terminals, scripts, logs, and manual validation. Even a small change can require several tools and repeated context switching.

UnrealAssetLab explores a simpler model: open an Unreal project, describe the task in a project-scoped conversation, review a bounded plan, approve execution, and keep results, logs, reports, and Unreal launch connected to the same workflow.

What it does

UnrealAssetLab is a local desktop application for guided Unreal Engine automation.

The current workflow supports:

  • opening and managing Unreal projects;
  • project-scoped conversations with multiple chats;
  • attachments and durable conversation history;
  • bounded task planning and explicit user approval;
  • Codex execution when code changes are required;
  • deterministic validation without unnecessary Codex usage;
  • a local WorkerBridge restricted to registered operations;
  • durable execution history, logs, reports, and output links;
  • opening the selected project directly in Unreal Engine.

The demo also includes a UE 5.8 interaction project with working camera control, WASD movement, and Door, Button, Lever, Pickup, Terminal, and Chest interactions.

How we built it

The desktop interface is built with Tauri and a web frontend. It communicates with a local WorkerBridge that exposes only registered, allow-listed operations.

Codex was used as the main development agent with the verified model gpt-5.6-terra. It helped inspect the repository, implement the desktop workflow, fix Windows path handling, connect project chats to durable runs, repair the Unreal gameplay demo, and generate validation and submission evidence.

The execution model is intentionally split:

  1. The user describes a task.
  2. UnrealAssetLab creates a bounded plan.
  3. The user explicitly approves it.
  4. Codex is invoked when code work is required.
  5. Deterministic registered validations can bypass Codex.
  6. WorkerBridge executes the approved operation.
  7. Results and evidence remain attached to the originating chat.

Challenges

The main challenge was making the system reliable across several boundaries:

  • desktop UI to local worker communication;
  • Windows path normalization;
  • durable project and chat state;
  • Unreal Editor process management;
  • explicit approval and safety boundaries;
  • distinguishing static validation from real gameplay behavior.

A previous validation pass confirmed configuration and compilation but missed gameplay regressions. We corrected the interaction dispatch and vertical-look configuration, added stronger instrumentation, and performed a final real Play-mode verification.

Accomplishments

  • A usable local desktop workflow rather than a command-line prototype.
  • Project-scoped chats and persistent execution history.
  • Automatic WorkerBridge startup.
  • Explicit confirmation gates.
  • Deterministic WorkerSmoke routing without unnecessary model usage.
  • Successful internal WorkerSmoke regression run in UE 5.8.
  • Direct opening of the selected Unreal project.
  • A manually verified playable interaction demo.
  • Installer and standalone Windows executable.
  • Traceable evidence, manifests, hashes, and validation reports.

What we learned

AI-assisted development becomes more useful when the model is not treated as an unrestricted shell. The strongest workflow combines a capable coding agent with bounded plans, explicit approval, allow-listed execution, persistent evidence, and honest distinctions between automated and manual verification.

What's next

Future work includes:

  • validation adapters for arbitrary selected Unreal projects;
  • broader registered Unreal operations;
  • richer asset-generation workflows;
  • provider abstraction for additional models;
  • stronger automated PIE functional testing;
  • optional MCP and ChatGPT App integrations.

WorkerSmoke currently validates an internal regression fixture, not every external Unreal project.

Built With

Share this project:

Updates