Unreal Engine Visual Effects Designer for AI Game Developers

A force multiplier for aspiring AI game developers with no VFX background, helping them vibe-code visually compelling VFX in games, ship faster, and move closer to professional studio standards.

Comment

Inspiration

AI has made it possible for people with no traditional game-development background, but with a compelling imagination, to build playable games. Hence, many games can be built with lower skill barriers and faster time to market.

However, I noticed a major gap between making a game functional and making it feel visually compelling.

Many AI-generated games still rely on basic geometry, flat materials, generic lighting, and simple visual effects. This becomes especially noticeable in games built around magic, combat, superpowers, or spectacle. A fireball may technically work, but it may still look like an orange sphere. A laser may deal damage, but it may not communicate charge, force, impact, or residue, and is only represented as a straight blue rectangle.

This has led to accusations from critiques about an influx of AI slops in gaming. Especially in the MMORPG community, there is usually heavy emphasis on the aesthetics and flashiness of dopamine inducing spell effects to make a player feel powerful. A flat dull rectangular laser beam just doesn't cut it.

Personally, I am a very visual person and power fantasy through spectacular and impactful spells or superpower effects is the number one element I look out for when paying for a game. Anything less than that doesn't make it to my cart.

Professional studios differentiate themselves from this by hiring experienced VFX artists who understand composition, timing, materials, motion, lighting, performance, and countless failure modes accumulated over years of production. New and aspiring developers usually do not have access to that experience.

Hence, the onus is on beginner game developers who want to use AI to vibe code their games to at least get the AI to make their games look aesthetically comparable to professionally developed games, so as to minimize the chances of being labelled AI slop, and to overcome the stigma of AI generated games in the general gaming industry.

However, this is easier said than done, because it leads to a new rabbit hole of pain points: getting AI to know how to come close to AAA standards involves grounding the AI with a great deal of technical contextual knowledge of using professional game engines like Unreal Engine, imparting human artistic tastes, and countless days of iteration and token expenditure. These pre-requisites itself is time-consuming and complex, and it deviates from our paradigm of using AI to spend less effort on technical implementation while spending more time on the creative effort of the game building: the lore, the wild imaginations of how a magic spell effect should look like etc.

That's where I'm stepping in to help with my creation: "Unreal Engine Visual Effects Designer for AI Game Developers", which serves as a force multiplier for new developers looking to gain a foothold in the industry, with minimized effort spent on the AI-led technical implementation of their high quality VFX aesthetics, while at the same time being able to focus more on the creative aspects of their work.

The goal is not to claim that this one AI tool can make a beginner fully equivalent to an experienced professional studio. The goal is to meaningfully reduce that gap: helping aspiring developers vibe-code and ship faster while moving their visual quality closer to that of professional production standards.

For my personal usage, l am going to use it to indulge in my own power fantasy imaginations, and self-determine the spell effects that I can unleash as per my whims and fancy, and not limited by heavy dependencies on what others create. I find myself often complaining and feeling dissatisfied when playing existing games: Why is the laser beam not thick and powerful enough? Why is the magic circle not intricate enough?.... But now I can simply just manifest what comes to mind.

What it does

The end product of this project is a reusable pre-built Unreal Engine VFX knowledge graph that will serve as the "second brain" to direct Codex in its artistic and technical decisions, saving it the trouble of iterating through trial and errors.

A developer can begin with a short creative request such as:

"Create a cosmic fireball that is encapsulated in an arcane magic circle."

The developer should not need to know which Niagara modules to use, how many emitters are required, how translucency should be balanced, which layer should own the brightest value, or how the effect should accelerate and decay.

The system translates the creative prompt and optional reference images into a technically grounded visual direction. It then retrieves relevant knowledge from a knowledge graph that is populated with recipes and documentations of pre-existing validated VFX builds, creates or modifies the effect inside Unreal Engine, and reviews the result against both artistic and technical quality criteria.

The workflow for Codex is:

  1. Interpret the user's creative intent and reference images.
  2. Retrieve relevant techniques, artistic principles, parameter guidance, and known failure modes from the knowledge graph.
  3. Convert that knowledge into a new VFX implementation plan.
  4. Build the effect through Unreal Engine Niagara capabilities, Unreal MCP, and specialized editor automation scripts.
  5. Texture of the effects are generated by Codex's ImageGen
  6. Capture the effect at multiple moments and audit its technical state against the intended goal.
  7. Revise it until it passes the quality gate.
  8. Save the successful lessons back into the shared knowledge brain.

The final output for the developer are 2 things:

  • VFX effect assets that have close parity with the developer's creative imagination
  • A growing knowledge graph with accumulating artistic taste and judgment that makes every future effect creation faster and more reliable to create.

Why this is different

Natural-language access to Unreal Engine via the official built-in MCP solves an important problem: it allows an agent to operate the editor.

However, engine access alone does not give the agent artistic taste.

It does not automatically know:

  • what makes an effect feel powerful rather than generic;
  • how brightness and darkness should be distributed;
  • how anticipation should hand off into impact and residue;
  • how slow, medium, and fast motion should coexist;
  • which implementation approaches are safe inside the editor;
  • or how to determine whether an effect is genuinely finished.

Hence, in the latest Unreal Engine 5.8 MCP capability, anyone can do prompts to assets, but the lack of artistic nuances in a simple prompt request still leads to low quality VFX assets, and iteratively improving the quality requires huge amount of time and token cost.

This project captures that missing layer.

The knowledge graph records not only what worked, but why it worked, what failed, how the failure appeared visually, which parameter ranges were proven, and how the result was validated. The knowledge is treated as reusable reasoning rather than a library of motifs. A new effect should be able to combine artistic and technical principles from multiple earlier effects instead of simply copying one of them. This is so that Codex can skip all that trial and error and just head towards what works.

Hence, a user who conveniently uses a simple prompt can still get a stunning VFX.

How I built it

I built this project using GPT-5.6 in Codex for both the VFX implementation and Knowledge Graph building.

The first step was connecting Codex to Unreal Engine through MCP. This gave the agent access to the assets.

As mentioned, MCP could let an agent operate Unreal, but it did not teach the agent what makes an effect feel powerful and cinematic.

To provide that missing artistic layer, I created specialized Codex skills for VFX art direction, implementation, and quality review. These skills encode principles such as visual hierarchy, authored darkness, localized highlights, causal choreography, anticipation, release, propagation, residue, multi-scale motion, and temporal validation.

My taste-oriented skill methodology was adapted from "leonxlnx/taste-skill" (https://github.com/leonxlnx/taste-skill), which provided an important foundation for thinking about how subjective aesthetic judgment could be translated into reusable agent instructions. The original skill is mainly used for website UI design. I extended that approach specifically for real-time Unreal Engine Niagara effects, including motion, timing, materials, camera context, technical safety, and multi-frame evaluation.

Unreal MCP could handle many general editor operations, but some operations were either unavailable or unsafe to perform through generic tools. So for these, I made Codex use "NiagaraStackAutomation", an Unreal Engine in-built editor-only C++ plugin exposing Python functions for operations.

I then used this workflow to create the Arcane Cosmic Fireball from an original visual reference generated with ChatGPT, and then GPT 5.6 in Codex iterated on it. Sol Ultra had to be used as this was a really complex task, to achieve the level of aesthetics required.

Within Codex, ImageGen was used to create texture assets, and sprite-sheet material that were responsible for the texture of the effects.

After 2 days of iteration, the Arcane Cosmic Fireball became the first knowledge source for the knowledge graph. Instead of documenting only final parameter values, I made Codex GPT 5.6 record the complete trial-and-error journey: what worked, what failed, why particular choices worked, unsafe editor operations, recovery procedures, artistic decisions, timing relationships, parameter ranges, and validation evidence.

Those reflections were converted by GPT 5.6 in Codex into atomic Obsidian-compatible Markdown pages covering case studies, recipes, techniques, taste principles, design decisions, experiments, pitfalls, and parameter sets. Codex GPT 5.6 created a schema knowledge contract with controlled vocabulary, evidence status, provenance, typed relationships, applicability conditions, transfer boundaries, and explicit rejection rules.

The authored wiki was indexed into an authoritative directed knowledge graph and made queryable through Graphify (https://github.com/Graphify-Labs/graphify). The current graph contains 18 curated source pages, 74 nodes, 866 directed relationships, and four deterministic communities. Before replacing a known-good graph, the repository automatically lints the wiki, verifies its sources and relationships, builds the authoritative extraction, and runs transfer evaluations.

GPT 5.6 in Codex also created a fixed-camera temporal validation and a formal VFX quality-review scorecard. An effect could no longer be considered finished merely because the build compiled successfully. It had to be reviewed and pass technical readbacks and satisfy a human taste review. This distinction between “compile-clean” and “effect-valid” became one of the most important lessons in the graph.

Next, I created HyperBeam as a deliberately different second knowledge source. Unlike the spherical, orbiting structure of the Fireball, HyperBeam required directional source-to-target energy, charging, axial flow, layered textured ribbons, intermittent electrical filaments, impact response, and directional residue. Its successes and failures were reflected into the same wiki, linked to the existing Fireball knowledge, and rebuilt into the graph.

Finally, I used the accumulated Fireball and HyperBeam knowledge to design and build Celestial Judgment. This was the real generalization test. GPT 5.6 in Codex had to query the graph, identify relevant techniques and taste principles, reject effect-specific motifs that did not belong, and combine transferable ideas rather than copying either parent effect.

The main product of the finished prototype is the knowledge graph that can be reused by game developers. The 3 VFX assets created are just the by-products.

Hence, the knowledge graph works in a complete learning loop: Creative intent → knowledge retrieval → art direction → Unreal execution → temporal and technical validation → human taste review → reflection → reusable graph knowledge

Each produced effect makes the VFX brain more capable, allowing future effects to begin from accumulated experience instead of starting from zero.

Challenges I ran into and Lessons Learnt

Knowledge Grinding from Scratch

I know I know, the correct approach should have been to mine the step by step knowledge straight from Unreal Engine tutorials and just index them into my knowledge graph. The really great tutorials that actually teach you how to make cinematic AAA standard VFX are paid online courses.

You might be wondering why didn't I do that and instead went through all that tribulations of trial and error to establish the starter base VFX recipes for the initializing of the knowledge graph foundation.

This was because I wanted to avoid any legal complications associated with intellectual property infringement as well as to avoid venturing into gray areas of this competition's rules of originality.

The safest was to get Codex to create the knowledge from nothing, but it also led to the biggest challenge I faced (which I still prefer over the risk of getting a legal notification)

The first base VFX, which was the Arcane Cosmic Fireball (see Image 1), took me a full 2 days to build from nothing. The only input was a reference image, which was also created with ChatGPT's Image 2.0.

It was 2 full days of continuous iteration, continuous trial and error, and token burning (only Sol Ultra was truly fit for the job in getting all the artistic nuances right. Sol Extra High could still give me something kind of good, but just not quite there, not the fall off your chair kind of good - remember, aesthetics was the whole point of this project, so I couldn't settle for a VFX that is just functioning)

There are many Unreal Engine VFX skills on Github. However, so far none exists which specifically give directions in the artistic thinking to adopt, the specific sequence of Unreal Engine MCP tools to call, the specific scripts to write, for a specific style and quality of VFX. Hence it was on me and my Codex to stumble and fall along the way to establish the recipe through this uncharted terrain.

2 full days. And this was only creating one fireball, for one spell, not even creating the game itself. Imagine how much more time it would take to create a full game with more than 10 spells? And so this is exactly why the knowledge graph containing the established methodologies is needed, so that AI game devs can go straight to getting the VFX they want to create without going through the same trial and error.

Lessons Learnt

During this initial "knowledge grinding phase", there were many times I just left Codex GPT 5.6 to its own device and only checked its work once its done, and there were many times I wasn't satisfied with the final output, even when using Sol Ultra with my custom tailored VFX Taste skills.

Then, I learnt that I needed to watch Codex while it builds, and when I see that it is not heading down the right direction (e.g. the direction of the flame particles relative to the central orb was kind of awkward at first), I had to quickly flag that up and steer it. I shouldn't leave it till the end before critiquing it, as doing so leads to token wastage, and wasted time from going down a checkpoint that should have been deemed unacceptable at the start.

Hence, this highlights the importance of human taste in the loop, which should happen during the AI build process, not after. If it's a common wisdom that static images output or Web UI layout design need to be validated with human taste, then it's even more so for complex multi layered outputs like VFX, which needs to be judged not only based on the direct visuals like color tone or texture, but on aspects that entail abstract nuances, like dynamic motion and the tempo of the kinetics. My experience says it all, that even using my custom made VFX Taste skills that was supposed to let the LLM inherit my personal taste judgement, wasn't a fool proof way of automating validation of what good looks like.

Not so ideal output from Knowledge Graph

My first output of the "Celestial Judgment" from the Knowledge Graph wasn't that great. It just didn't inherit that WOW level of quality from its parent VFXs, the "Arcane Cosmic Fireball" and the "Hyperbeam". This is a serious failure because the whole purpose of the knowledge graph was to enable AI game devs to achieve stunning effects with minimal effort.

Hence, I knew it was definitely to do with the way I was creating the wiki prior to indexing.

Lessons Learnt

1) Leave the Wiki Schema creation to the LLM, don't micro-manage: At first, I micro-managed how each knowledge grinding session should be chopped up into the different wiki subfolders. Then, after my initial failure, Codex reviewed the initial schema, and further enhanced the schema to according what it knows best of what it needs. It concluded that the performance was not doing well because the wiki was only populated with the what was done instructions (what parameter to use, what values to set etc.), but it did not include the whys (reasoning and decision making factors), which brings us to the next point.

2) Reasoning, not just parameter descriptions: I learnt that LLM Wikis would really be more powerful for taste-based tasks when the contents are encoded with reasoning, visual explanations, causality and failure conditions rather than just descriptions of the steps. This is so that when met with unprecedented scenarios, the LLM, equipped with the reasonings of why certain things are done, would be more versatile in extrapolating the logic to tweak existing methodologies to fit new challenges. Perhaps its something like why a student learns better when understanding the rationale of certain concepts, rather than just through memorization and rote learning.

Context Compaction

Throughout my sessions, I was constantly disrupted by this error notification: "stream disconnected before completion: websocket closed by server before response.completed" whenever Context Automatically Compacted. And the session becomes a dead end, because prompting Codex to continue or restarting it as a duplicate chat still results in the same error. It doesn't even give me a chance to request for compaction manually, as doing so would lead to the error as well.

I know that it is bad practice to allow compaction to happen by the time it reaches 100%, and the best practice would be to manually do compaction when it reaches about 60%.

However, most of my tasks are long running tasks, that easily reach 100% before Codex has completed a turn. Especially if I were to just leave Codex to itself, I wouldn't be there to always monitor the context percentage and stop it manually just to do context compaction or handoff to a fresh new session.

Lesson Learnt It is okay to just run into this error and not be alarmed by it. I can just treat it as an automated message prompting me to do handoff to a fresh new session. The session is not really a dead end. I can simply just look for its jsonl file under the "sessions" subfolder in ".codex" folder and then ask a new session to read that dead session's jsonl file and just continue from there.

Though, the only disadvantage is, there will still be wasted time with Codex staying idle for a portion the time that I'm away, once it runs into the error, and I will not be able to leverage on the expectations that things will be completed by the time I'm back.

Accomplishments that I'm proud of

Building as a complete beginner

I began this project with absolutely zero background in game development or Unreal Engine.

During Build Week, I completed three proof cases: the "Arcane Cosmic Fireball", "HyperBeam", and then both of which birthed the "Celestial Judgment" from the knowledge graph. Creating them required procedural VFX decomposition, layered cosmic and plasma materials, ribbons, runes, sigils, texture ingredients, editor automation, technical debugging, and repeated visual critique.

Without AI assistance, I believe I would have needed months just to learn the foundations behind these fundamentals and then years of practice to acquire genuine mastery. With GPT-5.6 Sol in Codex, I was able to build working foundations and produce these effects within the span of only a week.

Once again, just a disclaimer, I am not claiming that a beginner can instantly match an experienced professional VFX artist or game studio. What this project demonstrates is that the distance between a complete newcomer and professional production practices can be meaningfully reduced.

AI-generated games are often dismissed as “AI slop” because iteration often stop once the game becomes functional, without investing the same attention in visual quality, artistic judgment, and polish. Knowing that my work may contribute, even in a small way, to reducing (but not entirely eliminating) that stigma towards the AI game dev community, is deeply rewarding to me.

Unlikely Connection Became Real

Who would have thought of a connection between Andrej Karpathy’s LLM Wiki and visual-effects design for games? Karpathy described LLM Wiki as a way for agents to build an accumulating, interlinked body of knowledge instead of rediscovering the same information from scratch with every task.

When I first connected that idea to Unreal Engine VFX—capturing not only technical recipes, but also artistic reasoning, visual taste, parameter choices, failure modes, and validation criteria, it felt entirely theoretical and far-fetched, something that's interesting as a hypothesis, but maybe too ambitious to work in reality.

Seeing it actually come alive was therefore one of the most rewarding moments of this project. Codex could retrieve lessons from previous effects, connect related ideas, and apply and extrapolate that accumulated knowledge when designing something new.

That gave me a satisfaction that the core idea had moved beyond imagination and become a working proof. I would not claim that it has already been validated at full production scale, but Build Week gave me concrete evidence (and forced me out of my procrastination) that the concept is feasible, and that the unlikely connection I imagined is worth pursuing, with the prototype that I have now to serve as a starting point for expansion beyond Build Week.

Built With

  • c++
  • codex
  • gpt-5.6
  • graphify
  • niagara
  • obsidian
  • powershell
  • python
  • unreal-engine-5.8
  • unreal-mcp
Share this project:

Updates