Hajj brings millions of people from different cultures and backgrounds all in one place; with that comes the challenge of surveying these guests about their satisfaction on the facilities and services provided during Hajj. Having access to this information in realtime will allow measuring the overall sentiment to intervene when needed, and will also help improving the services in the future.

We had to come up with an innovative and easy way to gather this feedback from as many people as possible despite the barriers of language and culture. Thanks to advancements in machine learning and cloud services, we were able to develop a solution that analyzes the sentiment of the guests as they perform their Hajj through a touch-less photo taking experience.

What it does

Our solution will help gather feedback from the guests as they perform their Hajj through different locations by capturing photos on the go. The photos are then analyzed and classified based on facial expressions. A dashboard will show the sentiment based on location.

How we built it

We used xcode to develop an iOS app that runs in smartphones and tablets to capture the photos. The photos are captured on the go as soon as a face is detected. Once the photo is captured, it is then stored and passed on to Google Cloud Vision to be analyze for sentiment. Firebase and Javascript handle the outputs and present them on the dashboard in the form of analytics.

Challenges we ran into

*Auto capturing a photo accurately by using facial expressions. *Poor support for Swift from GCP.

What's next?

*Train the model to capture demographics such as age, gender and ethnicity. It currently only captures emotions. *Enable photo sharing on (we already registered the domain name) and on social media (#Hajjak). *Improve the dashboard to provide more insights by linking the dataset to additional datapoints.

Built With

Share this project: