Shades by Jo

Have you ever met someone at Uni and forgotten their name? Make those connections last longer than freshers. Our project will rememeber their name, course and where you met them. Avoid the awkward.

Comment

Inspiration

At university you meet new people all the time: freshers events, labs, and societies. It's impossible to remember everyone. But when you see them again, what course do they do, where did you meet? Or even what's their name!? Shock horror.

Avoid the embarrassment of asking someone's name for the third time. Our wearable embedded system recognises them and retrieves the information they told you from our database and relays it back to you in an ear piece. Very spy movie I agree.

What it does

Shades by Jo is a wearable system that helps you remember people in real time.

Our vision was a pair of glasses that remembers peoples faces when they speak to you so next time you speak you can be reminded of who they are.

The webcam takes in video and audio input. We decided to create an embedded system on a raspberry pi and coded in python. We used python libraries for facial recognition, speech to text, and text interpretation. This is stored in a database. When a recognised person is seen speaking the information stored about them is retrieved.

This isn't just convenience, it has applications for accessibility helping people with: Prosopagnosia (face blindness), low vision, and memory loss.

How we built it

Sweat and tears. We investigated the assorted equipment box. We initially aimed to use an arduino, but there was no compatible equipment so we pivoted to using a raspberry pi. We divided the work between us, each focusing on a different aspect, being the facial recognition, speech to text, the database and setting up the hardware.

Challenges we ran into

The first major hurdle we faced was with the raspberry pi. Our SD card was only 8GB which caused quite a few issues. With such limited space, installing packages and a compatible version of python was too much to ask, having only 2GB free after PI os was installed. After troubleshooting for a work-around, we decided to instead use a virtual machine running Ubuntu. This came with its own complications, mainly getting the virtual machine to have access to the camera and mic, however we managed to pass USB permissions to the virtual machine to get all the hardware working together.

Accomplishments that we're proud of

We are proud of how we adapted to the challenges we faced, brainstorming together to overcome sudden hurdles and adjusting the plan when needed to ensure as much of our vision could be realised.

What we learned

A physics student learned what an sd card is. Never attempt to use a raspberry pi. Stick to real pie. Check things are feasible before attempting them.

What's next for Shades by Jo?

Making a specialised microchip with all the required functionality. Our prototype is clunky but a specialised design would sleekly fit in a cool pair of shades to roll out into industry.

Share this project:

Updates