PassAI

Inspiration

American Airlines was the first airliner to use facial recognition technology in airports. We wanted to extend this concept to within the airplane cabin.

What it does

The model consists of a camera, Convolutional Neural Network (CNN), and a processing unit. The camera monitors passengers' expressions in each plane section, and sends the frames to be processed. The CNN classifies facial expressions as (sad, angry, happy, neutral, etc). The CNN output is then passed to a movies' suggestion module, which is displayed onto the passengers' screen.

How We built it

We used Python as the main scripting language, and used OpenCV\ keras\ Pylsl packages. Three laptops (with their webcams) were used as the testbed, where they sent facial expression data using Pylsl to a central node (also a laptop). The central node displays passengers expressions' results to the flight attendants, as well as movie suggestions for each passengers to improve their mood.

OpenCV: handles camera communication and frame capturing. Keras: builds and runs the CNN that classifies input frames for passengers' expressions. Pylsl: handles networking functions between nodes (computers with cameras) using TCP-based connections.

Challenges we ran into

Since an Airplane does not have reliable fast internet connection, we could not train nor deploy the CNN model online. This limited our options into what services we can use. For example, we could not use Google Colab nor Microsoft Azure for neural network deployment.

Another challenge we faced was properly identifying the expressions made by passengers. Once these expressions were obtained, we also had to establish a local network for our computers to communicate amongst themselves (using TCP).

Accomplishments that we are proud of

1) Managing to build a reliable local network to pass fast-changing data to a central node. 2) succeeding in building an interface for quickly displaying relevant passenger mood info. 3) running a locally trained CNN, which are often difficult to run on laptops.

What we learned

improved our programming abilities, as well as identifying relevant problems to companies.

What's next for PassAI

Future work include building an interactive flight attendant interface that visualizes passengers' mood in each seat. This can give them a better idea of which area needs more attention during the trip.

Built With

• keras
• opencv
• python
• tensorflow