Inspiration
Turning an engineering idea into a product often requires more than a sketch. You need to think about the enclosure, materials, controls, ports, dimensions, and how the final product should look. I wanted to build a tool that bridges the gap between a rough drawing and a visual product concept while keeping engineering constraints part of the process.
That idea became VisionForge AI: a local AI-assisted design studio that helps makers, students, and hardware enthusiasts explore product concepts without depending on paid image-generation APIs.
What it does
VisionForge transforms an engineering sketch and a structured design brief into a workflow for developing and reviewing hardware concepts.
- Sketch processing: Upload a drawing, clean up its linework, preview the processed reference, and inspect an edge map and basic image diagnostics.
- Local AI reasoning: Use Microsoft's Phi-4 Mini through Ollama to refine design prompts and organize product requirements locally.
- Product concept generation: Use Stable Diffusion 1.5 through Diffusers to generate concept images locally, including image-to-image generation initialized from a sketch.
- Engineering checks: Evaluate component fit against user-provided enclosure dimensions, identify size violations, and review required features using explicit, inspectable calculations.
- Experimentation and exports: Compare design outputs, record parameters and timing, and save concept images, metadata, Markdown reports, and ZIP packages.
VisionForge is designed to keep the workflow useful without requiring a paid hosted inference API. Its generated images are conceptual visualizations—not validated CAD, circuit schematics, or manufacturing instructions.
How we built it
VisionForge is built with Python and Gradio, using a modular architecture for image processing, generation, engineering checks, and experiment records.
Pillow handles sketch preparation, orientation, resizing, cleanup, and edge previews. Ollama runs Phi-4 Mini locally for text-based design reasoning. Stable Diffusion 1.5 runs through Hugging Face Diffusers in a separate Python environment, with DirectML used as an attempted acceleration path on Windows hardware and CPU execution available as a fallback.
The application organizes the workflow into separate tabs for sketches, design briefs, generation settings, results, project history, and engineering tools. Local metadata and report exports help preserve the settings and decisions behind each concept.
Challenges we ran into
One of the biggest challenges was running image generation on a Windows laptop with integrated graphics. DirectML's GPU device became suspended during inference, forcing us to explore a CPU fallback. This made the memory and performance trade-offs of local AI much more concrete: eliminating API costs does not eliminate hardware limitations.
Integrating separate Python environments, connecting Gradio callbacks, and keeping the user interface consistent with the generation pipeline also required debugging. Another important challenge was separating genuine engineering calculations from AI-generated suggestions. A convincing image does not prove that a component fits, and an AI-generated critique is not a measurement.
Accomplishments we're proud of
We established a local generation pipeline that successfully produced and saved a product concept image, alongside a workflow for processing sketches and documenting design decisions.
We also built explicit component-fit checks, exportable engineering reports, and a set of offline tests for image processing and engineering utilities. Most importantly, we shaped the project around a useful principle: every generated concept should be accompanied by enough context for a person to review its limitations and make the next design decision.
What we learned
We learned how to connect a local language model, an image-generation pipeline, preprocessing utilities, and an interactive interface into one application. We also learned that model choice must account for hardware, that image-to-image initialization is not equivalent to ControlNet geometry conditioning, and that evaluation metrics need clear limitations.
A design tool becomes more trustworthy when it distinguishes what it has measured, what a model has suggested, and what still requires human verification.
What's next for VisionForge
The next steps are to stabilize and test the complete interface, improve performance on integrated graphics, and expand the design-evaluation workflow. Future work could include a local vision-language model for component recognition, ControlNet for stronger structural conditioning, and a repeatable design tournament that compares alternative concepts using the same requirements and transparent metrics.
The long-term goal is to make VisionForge a practical bridge between a sketch and an engineering review—not just another image generator.
AI development disclosure
AI tools, including ChatGPT, were used to assist with portions of implementation and debugging. The project integrates and tests these components locally, and generated design outputs are explicitly treated as concepts rather than verified engineering results.
Log in or sign up for Devpost to join the conversation.