GeckCo AI: Codex For Hardware

Landing Page
Lay out your parts, and decribe your idea
AI will annotate your step by step build guide
AI will annotate your step by step build guide
AI will annotate your step by step build guide
AI will upload the code directly into your microcontroller, no more touching code anymore!
AI will monitor the sensor logs and self diagnose issues, like a closed feedback loop system, and reupload the working code until it works
Documentation made easy, just project directly to github

Inspiration

Building hardware should feel magical, but for beginners it often feels like being trapped in wire spaghetti. One wrong pin, one missing ground, one scary Arduino error, and the whole project stops. We wanted to make hardware building feel more like LEGO: visual, guided, calm, and actually fun. 🧩⚡

What it does

GeckCo AI: Codex For Hardware helps users turn a rough IoT idea into a working ESP32 prototype.

Users upload a photo of their real parts, describe what they want to build, and GeckCo AI:

Identifies the useful components in the photo 📸
Annotates the parts visually
Creates a step-by-step picture-based build guide
Generates ESP32 firmware
Flashes code directly to the board, no Arduino IDE needed
Checks serial logs and webcam evidence to verify the real hardware behavior
Creates GitHub-ready documentation for the finished project

How we built it

We built GeckCo AI as a lightweight web app using HTML, CSS, and JavaScript, with a small Node.js backend for API proxying and local hardware support.

The core stack includes:

OpenAI Responses API for multimodal reasoning, hardware planning, firmware generation, and verification
Deepgram for voice-to-text idea input
Arduino CLI for compiling ESP32 firmware
Web Serial + esptool-js for flashing the board from the browser
Webcam APIs for physical behavior verification
GitHub API for one-click documentation export
Netlify Functions for hosted guide mode

Challenges we ran into

Hardware is beautifully annoying. 😭

Some of the biggest challenges were:

Making image annotations accurate and useful instead of cluttered
Getting AI responses structured enough to turn into real UI and firmware
Handling long-running AI calls without timeout issues
Flashing ESP32 firmware directly from the browser
Fixing firmware image size and flash offset issues
Designing a beginner-friendly flow that hides complexity without hiding safety
Making the AI verify not just code, but actual physical behavior

Accomplishments that we're proud of

We are proud that GeckCo AI is not just a chatbot that gives advice.

It closes the loop:

photo → guide → wiring → firmware → flashing → logs → webcam proof → GitHub docs

That means the AI does not stop at “here is some code.” It helps the user build, upload, test, and document a real hardware prototype. We are especially proud of the direct ESP32 flashing flow and the closed-loop debugging idea, because that is where many beginner hardware projects usually fall apart. 🚀

What we learned

We learned that beginner hardware support is not just about better tutorials. It is about reducing the number of scary decisions the user has to make at once.

We also learned that AI is most useful here when it acts like a careful build companion, not an all-knowing answer machine. The best experience is one small step at a time, with uncertainty shown clearly and the human still in control of physical wiring and safety.