inteGREAT

Inspiration

When an embedded project fails, the bug rarely comes with a clear label. Is it the code, the wiring, the component, or the interaction between them?

We ran into this while building with an ESP32. Our code editor could show us the firmware, but it couldn't see the breadboard. A camera could see the LEDs, but it couldn't explain which code controlled them. We built inteGREAT to bring that context together and help engineers debug across the hardware/software boundary.

What it does

inteGREAT is a graph-first debugging workspace for ESP32 Arduino projects. It connects:

Source code
Declared wiring
Runtime logs
Camera observations

This lets engineers investigate a failure in one place. With inteGREAT, engineers can:

Inspect dependencies across code and hardware
Run supported tests
Review evidence-backed hypotheses
Choose a diagnostic action
Rerun the test after a repair to check whether the observed behavior now matches the expectation

How we built it

Frontend: React + TypeScript, with an interactive dependency graph
Backend: Python FastAPI service that imports Arduino source and YAML hardware manifests to connect software functions, GPIO pins, and physical components
Vision: OpenCV powers camera-based observations, including a two-LED visual test workflow
AI:
- StepFun's Step 5 Preview for investigation coordination and synthesis
- Gemini for specialist analysis
- Devin for engineer-requested, reviewable software repair proposals
Fallback: Deterministic rules provide a foundation that keeps working when AI services are unavailable

Our ESP32 demo combines alternating LEDs, a passive buzzer, and a button that toggles playback.

Challenges we ran into

The hardest challenge was separating what the system observed from what it could only infer. A dark LED could mean a software issue, a disconnected wire, or an obstructed camera view. We had to preserve that uncertainty instead of treating every unexpected image as a hardware failure.

We also worked through:

Camera connectivity
GPIO assignments
Asynchronous AI responses
Keeping evidence linked to the correct investigation

Connecting these pieces while keeping the interface understandable was as much a design challenge as an engineering one.

What we learned

Debugging assistance is only as good as its context. A useful explanation needs traceable evidence, clear limits, and a concrete next step.

We also learned that general infrastructure and specialized measurements work well together. The graph and evidence workflow can support many systems, but each physical behavior still needs an appropriate detector and test.

What's next for inteGREAT

Validate more workflows on real hardware
Connect the LED visual workflow to the existing AI investigation pipeline
Expand support to more components and embedded platforms
Longer term: adapters for logic analyzers, multimeters, and oscilloscopes, so inteGREAT has stronger electrical evidence alongside code and camera observations