Embarrassment always happens if you do not recognize a person who you might talk in somewhere and sometime, For example, a friend you haven't met for over 10 years. It will be awkward once you have an unexpected meeting but you do not know each other name. In an important business conference, people may not know each other very well. During the conversion, it is unavoidable to mention others' names. Correctly saying attendee names will show your respect to others. This inspired us to develop a smart wearable face recognition device.

What it does

Our wearable face recognition device can help you to eliminate this embarrassment when you are in an important business meeting, an old friend you haven't met for decades, or your kiddos wants to make more friends in the kinergarden. The face recognition system can provide the basic information of the person you are looking at. Furthermore, the designed system equipped with an emotion analysis function which can suggest the right way to start your conversation.

How I built it

The system included a portable unit, a client unit, a smartphone interface, and a cloud server. It was designed with several advanced technologies such as Computer Vision, Deep Learning, Big data, and Secure ethernet communication. Face images were gathered from a portable device (Raspberry with camera) and transferred image data to the client side via UDP. The client side did the image pre-processing for face detection using the HOG algorithm and random forest classifier. To reduce the WLAN transferring expense and server processing redundancy, instead of delivering the raw video frames, we only transferred the image pre-processing result that only contained faces information. To ensure the WLAN communication security, TCP/IP with SSL protocol was used for the clients and server communication. The server-side performed further face analysis including face verification, face recognition and face analysis. The ResNet model was applied to match the facial features in our database. The face analysis result was then sent back to the client and updated on the mobile device simultaneously.

Python, VC++, Swift are the coding language we used for this project.

Challenges I ran into

  1. Initially, the ethernet communication protocol between clients and the server were used UDP. But the simple unsafe UDP communicating bothered us. After research, we finially decided to use TCP/IP with SSL to ensure reliability and safety of the ethernet communicating.
  2. To correctly transfer the data on the ethernet bothered us a lot. Initially, we spent lots of time on receiving the correct image between the client and the server. Lots of problems were met, such as data structure alignment, little endian vs big endian among different devices, the decision on receiving callback strategies.
  3. We compared several face detection and recognition models. Some models have higher detection accuracy but lower time efficiency. Some models have a higher detection rate but lower accuracy. To balance the detection accuracy and real-time detection, we finally decided to introduce the multi-thread strategy.

Accomplishments that I'm proud of

Finally, we accomplished all the setup and functions within 24 hrs. We did establish the entire system including portable video unit, client-side, and cloud side. The mobile interface for IOS design was a bonus for us which made our system much easier extendable.

What we learned

We learned different ethernet communication protocols, multi-thread programming, and IOS app development.

What's next for

  1. Cuda (GPU) programming will be used on cloud servers to perform real-time processing on more complex and better performance algorithms.
  2. The UI can be more attractive and user-friendly.
  3. The wearable imaging unit needs to be redesigned in the future for the purpose of the lightweight, compact size, and aesthetic look.
  4. Server needs to be tested for multi-client communications.

Built With

Share this project: