-
-
pot
-
bridge
Inspiration
Blender is a powerful tool, but traditional 3D modeling often requires users to understand a large number of operations, shortcuts, and Python APIs. For beginners, even when they already have a clear idea in mind, turning that idea into an editable 3D model can still be difficult and time-consuming.
To solve this problem, I developed Blender Codex AI Assistant, an AI-powered modeling assistant that runs directly inside Blender. Users only need to describe what they want to create or modify in natural language. The AI then generates the corresponding Blender Python script, which users can review and execute directly within Blender.
What it does
- Generates Blender modeling scripts from natural-language instructions
- Analyzes uploaded reference images and generates 3D modeling code
- Supports multi-turn conversations for gradually refining a model
- Supports OpenAI, DeepSeek, and other compatible API providers
- Provides optional web search to supply additional context to the AI
- Allows users to ask questions, preview code, and execute scripts directly from the Blender sidebar
- Uses only Python’s standard library to minimize installation and dependency issues
For example, users can enter instructions such as:
- “Create a spiral staircase with 20 steps”
- “Add a metallic material to the selected object”
- “Generate a basic 3D model based on this reference image”
The assistant converts these design intentions into executable Blender operations.
How we built it
The project was developed as a Blender add-on. It uses the Blender Python API to collect the user’s prompt, reference image, scene information, and conversation history, then sends the relevant context to the selected AI model.
After receiving the response, the add-on extracts and displays the generated Blender Python code. The user can review the script before executing it in the current Blender scene.
To support different AI providers, I implemented provider-specific API configuration, automatic API endpoint switching, and separate API key storage for each provider. The optional web-search feature can also inject relevant search results into the conversation context, helping the AI handle unfamiliar Blender functions or technical requirements.
The overall workflow is:
Natural-language or image input → AI analysis → Blender Python code generation → User review → Script execution in Blender → Further conversational refinement
Challenges we ran into
The biggest challenge was not simply making the AI generate code, but ensuring that the generated code was compatible with modern Blender versions and could run reliably.
During development, the AI sometimes produced outdated Blender APIs or referenced modifiers, node types, and data structures that did not actually exist. For example, it generated the nonexistent SMOOTH_BY_ANGLE modifier, incorrect shader node types, and properties from older Blender 2.7x APIs.
To address these issues, I continuously improved the system prompt, added explicit Blender API constraints, and introduced targeted rules based on real execution errors.
Another major challenge involved network requests inside Blender. Complex prompts can take a long time to process, but changing the global socket timeout may interfere with other Blender network operations. To solve this, I implemented multiple timeout protection layers and ensured that the original socket settings are restored after each request.
Accomplishments that we're proud of
One of our biggest accomplishments is turning a complex Blender workflow into a simple conversational experience. Users can describe a modeling task in natural language, receive executable Blender Python code, review it, and run it directly without leaving Blender.
We are also proud that the project supports both text and image-based workflows. Users can provide a reference image, ask the AI to analyze it, and generate a starting point for a 3D model. Multi-turn conversations then allow the model to be refined step by step instead of forcing users to restart from scratch.
Another important achievement was improving the reliability of AI-generated Blender code. We identified and fixed issues involving outdated Blender APIs, nonexistent modifiers and shader nodes, incorrect data types, network timeouts, and failed responses. These improvements made the assistant significantly more stable and better suited to Blender 4.0 and later versions.
We also implemented support for multiple AI providers, automatic API endpoint switching, provider-specific API key storage, optional web search, conversation history, code preview, and controlled script execution—all within a lightweight Blender add-on that requires no third-party Python dependencies.
Most importantly, we built a working bridge between natural-language creativity and practical 3D production, making Blender more accessible to beginners while also helping experienced users automate repetitive modeling tasks.
What we learned
This project taught me that integrating a large language model into professional software involves much more than making a single API request.
A reliable AI-assisted workflow requires carefully designed context, version constraints, error handling, timeout recovery, and a verification process before execution.
I also learned how to build Blender add-on interfaces, work with the Blender Python API, manage network requests, maintain multi-turn conversation history, and preserve user control when dealing with uncertain AI-generated output.
Instead of executing generated code automatically, the add-on allows users to inspect the script first, reducing the risk of unwanted changes to the Blender scene.
What's next for blender_codex
In the future, I plan to add automatic code validation before execution, improve the success rate of complex modeling tasks, and introduce more advanced scene-awareness and local editing capabilities.
I also hope to allow users to select a specific object or region of a model and modify it directly using natural-language instructions.
Log in or sign up for Devpost to join the conversation.