-
-
Natural-language VRM editing with Codex-generated Blender Python, iterative refinement, and VRM 1.0 / Perfect Sync support.
-
A real Japanese request becomes a Blender-side result through an actual prompt, Python helper, execution, and visible verification.
-
The workbench already supports natural-language requests, model inspection, Python generation, Blender execution, validation, and evidence.
-
Older VRM assets still require inspection, careful edits, validation, and reruns—even for a small change such as eye color.
-
Current capabilities are separated from future goals, including broader VRM 1.0 support, Perfect Sync automation, and safer editing.
-
Real project evidence connects a Japanese request, generated Python, Blender execution, and a reviewed result—not just a mockup.
-
A clear before-and-after proof: the same VRM model changes from red eyes to blue after the AI-assisted Blender workflow.
Inspiration
This project began with a practical problem: I had an older VRM avatar model, very little Blender or 3D modeling knowledge, and a long list of changes I wanted to make.
English is not my working language, so I used ChatGPT and Codex extensively to understand the hackathon requirements, translate technical details, and prepare the English documentation. I translated the resulting text back into Japanese, checked it against the actual implementation, and revised anything that was inaccurate or overstated.
Advanced avatar editing in Blender often requires specialized knowledge of both Blender and 3D modeling. Although AI can now generate code, many creative workflows still require users to understand these complex tools before AI becomes truly useful.
I wanted to build an AI-assisted workbench that guides the editing workflow itself, allowing users without specialized Blender or 3D modeling knowledge to perform advanced avatar editing through natural language instead of relying solely on generated code.
What it does
The workbench establishes an iterative workflow between the user, Codex, and Blender. The user describes the desired edit in natural language, Codex generates Blender Python scripts to perform the modification, and the user reviews the resulting 3D model before deciding whether to accept the changes or refine them further.
Instead of replacing the user, the AI acts as a technical collaborator. Creative decisions always remain in the hands of the user, while repetitive scripting and technical implementation are delegated to AI.
This review-and-refine cycle can be repeated as many times as necessary until the user is satisfied with the result.
How we built it
The workbench was built around an iterative workflow combining Blender, Python, and AI.
Blender serves as the editing platform, while Python scripts perform model modifications. Instead of writing every script manually, the user collaborates with Codex to generate, refine, and reuse Blender Python code.
Each generated script is executed in Blender, and the resulting 3D model is reviewed by the user. Based on the visual results, the workflow repeats until the desired outcome is achieved.
As the project evolved, reusable prompts, workflows, and editing procedures were extracted, refined, and organized into a reusable workbench rather than remaining as one-off scripts.
Challenges we ran into
One of the biggest challenges was the complexity of Blender itself. Before users can even begin editing VRM models, they often need to learn Blender's interface, terminology, and workflow, which can be overwhelming for people without 3D modeling experience.
This meant that simply generating Python scripts with AI was not enough. The workflow itself needed to guide users so they could understand what they were doing without first becoming Blender experts.
Another challenge was identifying which editing procedures could be generalized into reusable workflows and prompt templates instead of remaining project-specific solutions.
Accomplishments that we're proud of
We are especially proud that users without specialized Blender or 3D modeling knowledge can iteratively refine their avatars through natural language. Instead of learning complex editing procedures, they can simply describe what they want, review the results, and request further revisions as many times as needed.
This allows users to focus on their creative ideas rather than Blender operations.
What we learned
This project taught us that AI is most effective as a collaborative partner rather than a replacement for the user.
We learned that the best results came from an iterative workflow where users describe their intentions in natural language, review the generated edits, and continue refining their models through conversation with AI.
We also learned that building reusable workflows and prompt templates is often more valuable than creating a single successful solution, because those assets can support future creative projects beyond Blender.
Perhaps the biggest lesson was that AI should lower the barrier to expertise, not eliminate the need for human creativity.
What's next for AI-Assisted Blender Model Editing Workbench
The next step is to expand this reusable workflow beyond Blender-based VRM editing.
We are currently extracting reusable prompts, workflows, and AI collaboration patterns from this project to build an AI-assisted workbench for Live2D creation. Our long-term goal is to establish a shared AI Support System that can support multiple creative pipelines rather than a single application.
We also plan to improve compatibility with a wider range of VRM models and continue refining the collaborative workflow based on user feedback.
Ultimately, we hope to make AI-assisted creative workflows accessible across different content creation tools, allowing users to focus on their ideas rather than software-specific expertise.
We hope this approach can make advanced avatar creation more accessible to people of all skill levels.
Log in or sign up for Devpost to join the conversation.