At UCSD, student identification process is involving multiple platforms and also very hard for student to use: taking MTS 202 requires to pull out smartphone to show the ticket in the app, checking out at Pines dining hall requires to swipe student ID, signing attendance sheet in the classroom requires to use that $40 iClicker to click in. Could those processes be simplified so student can enjoy and focus on learning at university?
What it does
UCVision replaced traditional student ID with student’s facial data. Student can simplify smile to the camera on the IOT device and UCVision can recognize student identification. UCVision will push all the identification data to its backend NoSQL database and updated to its front-end mobile app.
How we built it
UCVision contains three parts: IOT device, backend server, mobile client. (1) IOT device is a set of raspberry pi which responsible for facial recognition and incremental learning. (2) Backend server is a Google Cloud kubernetes cluster which manages the database and providing HTTP2 server push to the mobile client. (3) Mobile client is a native iOS application written in Swift 4 and it is providing a dashboard view for UCVision user such as MTS bus driver and classroom instructor to verify student identity.
Challenges we ran into
Performing heavy loading facial recognition on IOT device is very challenging to us. So we decided to split the machine learning pipeline into two step. First is a generic model which is pretrained on Google Cloud TPU machines. Second is a retraining model which is trained real-time on the IOT device which provided incremental learning.
Accomplishments that we're proud of
We successfully deployed this large scale facial recognition project on thin and low-cost IOT system.
What we learned
How to optimize Tensorflow on low performance IOT devices. How to perform model retrain on a base model using Tensorflow. How to use HTTP2 to send a server side push instead of using websocket. How to receive HTTP2 pushed on native iOS app.
What's next for UCVision
We are thinking to commercialize UCVision to retail industry. During the hackathon, we had a great demo and conversation with Macy’s tech, where we can use UCVision’s technology to help their retail store to track their customer and providing useful data for their email marketing. UCVision can also provide safety features such as shoplifter detection where retail company can flag subjects and calling security personnel when shoplifter enters the store.