Inspiration

If you've ever soldered, you know the feeling. One hand is holding the part, the other is holding the wire, and the iron is on the far side of the bench. You need a third hand.

We kept thinking about who needs that third hand most. For someone with a tremor, limited arm movement, or only one free hand, reaching across a bench for a hot tool can be hard or even dangerous.

What it does

You say "EVERA, pass me the soldering iron," and the robot arm picks it up and hands it to you. Say "got it" and it lets go. Say "stop" and it freezes right away. Is the final vision.

You can also tap tools on an iPad app, which shows a live 3D model of the arm and the camera feeds. EVERA talks back too. Ask it how to build a PCB and it'll search the web and walk you through it.

How we built it

We recorded about 80 demos of us moving the arm by hand, then trained a robot policy (ACT) on AMD's cloud GPUs. That model now runs on an AMD mini PC sitting next to the arm. We also trained a small vision model that spots Raspberry Pis, Arduinos, and soldering irons on the bench.

The iPad app is a plain web app. It uses OpenAI to understand speech, an LLM to decide what to do, and ElevenLabs for EVERA's voice.

Challenges we ran into

Our first dataset was a mess. We dropped the object somewhere different every time, so the robot learned to do something vaguely in between. We started over with everything taped down.

Accomplishments that we're proud of

We are happy to have built a working pick up arm that works autonomously with an app to go with it.

What we learned

Good data beats more data. A few careful demos teach the robot more than lots of sloppy ones. We also learned that most of hardware is cables, cameras, and patience. We learned

What's next for Evera

More skills (holding a board steady, putting tools away), one smarter model that can follow any spoken instruction, and using vision to grab whatever you ask for. Most of all, we want to make it safe enough to test with people with limited mobility, because that's where a third hand could really change things.

Also, our thing only currently works with the soldering iron. We want to be able to do all the different things not just the soldering iron.

Built With

  • ai
  • cnn
  • lerobot
Share this project:

Updates

Submission history