ENGINEER-1

The general AI engineer that lets you create any physical product just by thinking about it! Vibe invent anything by generating 3D CAD designs from prompts and then 3D print it.

Comment

Inspiration

The gap between creative imagination and physical creation has always required specialized CAD skills and 3D modeling expertise. I wanted to bridge this gap by creating "the general AI engineer that lets you create any physical product just by thinking about it!".

The name is inspired by the AI 2027 report, which envisions what would happen when humans create exponentially smarter AI models (AGENT-1, AGENT-2, AGENT-3, ...) which ultimately automate scientific research. ENGINEER-1 is a small step towards building Engineering General Intelligence (EGI).

What it does

ENGINEER-1 is an AI-powered platform that enables users to design and create 3D models through natural conversation. Users simply describe what they want to build, and the AI agent guides them through the entire design process.

The system searches through hundreds of existing CAD designs using the ChromaDB-powered RAG system to find inspiration and reference examples. It creates mockup images using Gemini NanoBanana to help visualize the concept before creating the 3D model (Design Mode).

The project uses OpenSCAD to create precise, parametric 3D models that can be iteratively refined through conversation. Finally, users can publish their creations to our hub marketplace where others can discover, customize, and build upon their designs.

How we built it

  1. MCP Server Architecture: We implemented a custom MCP (Model Context Protocol) server for OpenSCAD using FastMCP in Python, providing tools for script creation, rendering, STL export, and hub publishing. This server maintains persistent state and handles all 3D modeling operations through the OpenSCAD CLI.

  2. RAG System with ChromaDB: We built a comprehensive vector search system by embedding a dataset of thousands of 3D designs from Thingiverse (through BrightData) using Jina Embeddings v3. The embeddings are stored in ChromaDB, allowing semantic search across design descriptions, names, and metadata. This gives the AI agent access to a vast library of reference designs and inspiration.

  3. Thinking Mode & Image Generation: We integrated Gemini NanoBanana to generate reference images when users describe abstract concepts. This "thinking mode" helps bridge the gap between verbal descriptions and visual representations before the 3D modeling begins.

  4. Web Hub/Marketplace: We created a Next.js 15 application with Turbopack, React 19, and Tailwind CSS for the frontend. The hub features advanced filtering, search capabilities, reference image galleries, and a clean interface for browsing published designs.

The entire system is orchestrated by a Claude Code as the MCP host.

Challenges we ran into

  1. OpenSCAD Integration: Getting OpenSCAD to work reliably in a server environment required careful handling of the CLI interface, especially for rendering operations with different camera angles and export formats.

  2. Embedding Quality: Finding the right embedding model and configuration for 3D design descriptions was challenging. We experimented with different models before settling on Jina Embeddings v3 with the text-matching task for optimal semantic search results.

Accomplishments that we're proud of

  1. Complete MCP Server Implementation: We successfully built a production-ready MCP server for OpenSCAD that provides comprehensive 3D modeling capabilities through a conversational interface.

  2. Intelligent Design Search: Our RAG system with hundreds of embedded examples enables the AI to find relevant inspiration and learn from existing designs, dramatically improving the quality of generated models.

  3. End-to-End Product Pipeline: We created a complete workflow from idea to shareable product - including design, rendering, export, and publishing to a marketplace.

  4. Clean Architecture: The modular design with separate MCP servers for different capabilities (OpenSCAD, RAG, thinking mode, design mode) makes the system extensible and maintainable.

What we learned

  1. Iterative Design Importance: Users rarely get their design right on the first try - building in versioning and iterative refinement capabilities from the start was crucial.

  2. Visual References Matter: Having reference images (either provided or generated) dramatically improves the quality of 3D model generation and helps align expectations between users and the AI.

  3. Integration Complexity: Orchestrating multiple AI services (Claude, Gemini), databases (ChromaDB), and specialized tools (OpenSCAD) requires careful attention to error handling and state management.

  4. CAD is VERY HARD!

What's next for ENGINEER-1

  1. Advanced Materials & Manufacturing: Integrate material selection, cost estimation, and direct integration with 3D printing services and manufacturers.

  2. Collaborative Design: Enable multiple users to work on designs together, with version control and design branching similar to Git for 3D models.

  3. Physics Simulation: Add FEA (Finite Element Analysis) capabilities to validate structural integrity and optimize designs for real-world use.

  4. Creator Monetization: Implement a marketplace economy where designers can sell their parametric designs, with automatic royalty distribution and licensing management.

  5. Extended CAD Support: Expand beyond OpenSCAD to support other CAD kernels like Fusion 360, Autodesk Inventor, or commercial tools.

  6. Mobile & AR Integration: Create mobile apps with AR visualization to let users see their designs in real-world contexts before manufacturing.

  7. AI Design Optimization: Implement generative design capabilities that can automatically optimize designs for specific constraints (weight, strength, material usage, etc.).

  8. ENGINEER-0: training a reinforcement learning based version that can go beyond human created examples and unlock completely novel ways of designing products.

Long Term:

  1. Manufacturing Network: Build partnerships with local makerspaces, 3D printing farms, and traditional manufacturers to offer instant quotes and one-click ordering.

Built With

Share this project:

Updates