SARA Sees You

S.A.R.A: Small Assistant Robotic Arm. CV/ML, Unity, Arduino JSONSerial, HTTP Server

Comment

Inspiration

Needing a robotic companion because love is scarce. This robot is designed to recognize human faces and interact with people in the frame. It is inspired by notable robot arms such as Dumme from Iron Man.

What it does

Populate a corner of your desk and give personality to the space. Interprets video into neural network to make predictions and receive commands over HTTPlocalhost. These commands are fed into Unity for animation and live movement reactions to the feed via a JSON serial controller to move servo arm.

How we built it

A lot of time, and a lot of grinding and experience. Pretty swaggylicious team too ;) We split the project into 3 parts. The first was the Unity application, which was designed to communicate through serial with S.A.R.A. The second was the serial communication protocol with the device, which allows for S.A.R.A. to be reprogrammable with differing numbers of servo motors and minimum/maximum angles for all joints.

Challenges we ran into

Time + paywalls. #Proprietarysux #praisetux . Impressing dad.

Accomplishments that we're proud of

Getting the project done within 30 mins of submission. Interpreting different classes + objects such as faces, people, and hand gestures. Designing a robust arduino protocol that can be translated to as any other robotic arm system as long as it is controlled with the Arduino and runs via servo motors.

What we learned

It is important to have the ability to reallocate time during the project and adjust for delays without undercutting our accomplishments up to that point.

What's next for Sara Sees You

Collect dust on Mark's desk, optimize training model, finish unity app, hide and seek. The aim now is to finish the Unity Application with final features such as an animation suite and a robot builder tool. The aim is to release the software and the Arduino firmware such that other people can implement their own version of S.A.R.A!

Built With

Share this project:

Updates