Circuit Coach

An augmented reality circuit assembly coach that guides users step by step as they build and analyze any circuit of their choosing. Designed to make circuitry less tedious and more intuitive.

Comment

Inspiration

Assembling circuits - be it on a breadboard or a PCB - often gets finnicky. It is easy to get lost within a self-spun web of connections, converting a project that looks simple on paper into into a complex network of connections that’s difficult to navigate and easy to misconfigure. A single misplaced wire or incorrectly oriented component can lead to hours of frustrating debugging, especially for beginners. We wanted to create a way to bridge this gap between theory and hands-on building, giving users real-time guidance so they can focus on learning and experimenting instead of getting stuck on tiny mistakes.

What it does

Currently, Circuit Coach analyzes a PCB schematic (such as an ECAD layout image) and breaks down its assembly into a series of step by step instructions. Each of these steps is communicated via an AI-processed version of the original PCB image that highlights only the components relevant to that step. Inside their headset, users can flip through this sequence of guided images while assembling the circuit beside them.

How we built it

Used Gemini Live to read the Pico 4's video feed and deliver live commentary on what it sees.

Challenges we ran into

  • Setting up the Pico 4 to work with Unity and support a pass through view took us longer than expected given the number of dependencies that needed to be installed.
  • We were able to cast the Pico 4's camera feed to our laptops (on which we could run a live AI model to interact verbally with the world), but struggled to find a way to get any output from our laptops onto the Pico 4 without needing to rebuild in Unity.
  • We intended to build an AR overlay of the AI processed circuit images onto a real world circuit board. For example, if a resistor needs to be added in a given step, we wanted to project an AR overlay of the resistor component onto a physical circuit board, positioned exactly as it is needed to be assembled.

Accomplishments that we're proud of

  • We were able to accurately prompt cutting edge image AI models to generate sequences of step-by-step circuit assembly images.
  • We managed to get our live model to interact with the live video feed of the Pico 4 and communicate with it.

What we learned

  • We got to interface with a virtual reality headset that we'd never worked with previously. We got to explore the specific developer benefits it provides and hear about the next workflows the Pico 4 is being developed to support.
  • We better understood the way unity handles mixed reality development and how all the components and toolkits work together to deliver an immersive experience.
  • Given the number of times we had to go back to the drawing board and rethink our overall architecture, we improved our approach to designing step-by-step, spatially guided experiences in a mixed reality environment.

What's next for Circuit Coach

In the short term, we aim to implement a satisfactory overlay feature such that the step by step instructions are not just static images, but rather dynamically adjusting visual cues that truly assist in circuit assembly. Later on, we how to add interactivity via our live AI model (or any other LLM) to make circuit coach a faster and more pleasant experience that can catch out users' assembly mistakes or even explain the significant of each component that goes into making a final PCB.

VIDEO LINK: https://drive.google.com/file/d/1U15pQMXQCHz4dnuYZL9Ggf7mD0pUCSqh/view?usp=sharing

Built With

Share this project:

Updates