Aurelia

Aurelia Logo
Homepage
Good Lighting Demonstration
Bad Lighting Demonstration
Problem
About Us Page
Our Team
Privacy Page

Inspiration

As black students, we are constantly exposed to emerging technologies. Sometimes, though, we ask ourselves, "Why can't technology see me?" The most frequent and frustrating example of this is failing to activate touchless faucets in bathrooms. It is especially frustrating watching others with lighter skin use these systems with ease. We wanted to develop a solution that paves the way for future tech that sees all skin tones.

What it does

Aurelia uses transfer learning and machine learning in JavaScript. We trained a neural network with datasets of over 650 images to precisely recognize diverse facial features in different lighting. When users activate their camera, Aurelia will indicate if their face was recognized and whether the lighting was suitable, allowing them to change lighting if needed. It will display a number between 0 and 1, with 0 being low confidence and 1 being high confidence. As users scroll through the page, we added information about colorism in facial recognition technologies so users are aware of its dangers. We also added applications and privacy tabs to increase user awareness.

How we built it

Our dataset training was based on a convolutional neural network (CNN) using TensorFlow. We created a custom dataset using our own images. Then, we implemented it onto our website using HTML for backend and JavaScript and CSS for frontend.

Challenges we ran into

CSS and JavaScript: We had trouble with integrating the CNN model we used into our application using JavaScript, as most of our team's experience was with ReactJS, which uses functions such as useState for event listeners with different syntax as well.

Defining our data: Choosing our dataset also proved to be challenging as we had to define what good lighting and bad lighting was. We decided front lighting was better than back lighting. We took various photos at different times throughout the day to have the best available dataset to tailor our model to our precise project.

Model: We ran into some issues with our model detecting the discrepancies in lighting conditions. To solve this, we considered more variables that could change the results the model generated such as different backgrounds, multiple people in frame, and having the subject only partially in frame. We then simulated those situations when implementing our dataset. This resulted in more accurate readings for a broader range of facial features and lighting conditions.

Accomplishments that we're proud of

First Hackathon: This hackathon was our first hackathon and we're super proud that we were able to finish from our problem formulation to solution implementation in 24 hours! We're also very proud that we were able to gear our hack towards an issue that will make a meaningful impact when implemented because colorism is an issue we deal with daily!

What we learned

CNN's: Each of us gained familiarity with CNNs, how they work, and real world applications.

Github Collaboration: We gained a cohesive understanding of Github with commands such as push and pull to work on the project.

Lighting: Different shots in photography and what is not susceptible to picking up on dark skinned features.

Keyboard Commands: We learned new commands like CTRL + SHIFT + ALT + WIN + L (shoutout LinkedIN), ALT + SHIFT + Up/Down (to duplicate on VSCode), and CTRL + D (to select multiple instances)!

What's next for Aurelia

Commercial Applications in Professional Photography and Videography Services: Our tool can be used in professional environments to ensure accurate lighting conditions for ALL skin tones and inclusive media representation. Additionally, our technology could be implemented in online meeting platforms to help users present themselves better under different lighting situations during virtual calls.

Applications in the Classroom: Aurelia ensures that every student is on a level playing field through allowing the chosen device to pick up each student's features accurately. Some common applications for this in the classroom include increasing connection in virtual learning environments via higher face detection and proctored test applications such as LockDown Browser.