Sapphire

AI-Powered Reliable Hiring Platform

Comment

Elevator Pitch

Sapphire is an AI-powered Recruitment Tool. With Sapphire, applicants like yourself can finally hunt for jobs with a peace of mind! It can even help your recruiters make better hiring decisions as well.

Try Sapphire Out!

  1. Web App

  2. GitHub repo

  3. YouTube Video Demo

Inspiration

Being university students comes with its own set of difficulties, one of which are the countless interviews that one must prepare for. Throughout our experience going through the whole affair, it left us daunted. In this age of technology, there must exist many ways to streamline the process; a way to bring both humans and technology together, evaluating candidates with human empathy and machine precision.

Problem

Face-to-face interviews are still subject to a wide array of human bias; it is usually up to a single HR employee to review your application and decide whether you would fit well into the organizational culture. The idea that a single individual could play God over the whole interview process leaves job seekers like ourselves worried. On the other hand, employers fear the numerous applicants who possess a "fake-it-till-you-make-it mindset". When undeserving and incompetent individuals sneak their way into a company, significant problems are caused when they are unable to deliver.

In summary, two major problems affect hiring decisions – first, non-verbal behaviours of candidates and second, bias and perceptions of interviewers.

Overview of Sapphire

Before we go into the nitty-gritty, let’s give you an overview of how Sapphire is like.:

  1. Sapphire works for both virtual interviews and face-to-face interviews.

  2. It also works for both interviewers and interviewees.

  3. Lastly, it allows us to hire reliably without bias using Artificial Intelligence.

What Sapphire Does

Thus, Sapphire was born. We conceptualized Sapphire as an AI-powered hiring platform with strong machine learning capabilities. Sapphire can pick out distinct events throughout the whole video interview process. By leveraging on up-to-date oculesics, facial expressions and speech research, Sapphire can advise interviewers when there is a difference between what is spoken by a candidate and what can be inferred from their body language.

In addition, Sapphire is integrated with a micro-expression detection module. Studies have shown that individuals who are taught to read and lookout for micro-expressions can detect lies 25% better. Hence, by empowering interviewers with the ability to detect them, we are confident that Sapphire will be able to play a huge role in the hiring ecosystem.

Lastly, we explored the design of a machine learning model from scratch, and we plan Sapphire to be able to tailor each model to each and every individual. This will allow people from all backgrounds to have an equal chance to land the job. In contrast, some tech firms such as Amazon have created a biased machine learning model which actively discriminates against some groups. This results in the phenomenon of “garbage in, garbage out”, a common phrase used to criticize bad ML models, and in this case, the biased ML-hiring applications which Amazon previously used.

Features of Sapphire

Evaluators’ Login

First is our login page where evaluators can sign into our app. Evaluators include HR personnel and managers in charge of recruiting new hires.

Conducting Interviews

Upon logging in, evaluators will be brought to the main menu through which they can conduct on-site interviews and review offline interviews.

Main Menu

Next would be what a hiring manager would do in the typical interview process. In the main menu, interviewers will first select the position they want to carry out an interview for, then select the candidate whom they want to interview, and finally select the interview room.

Connect to Room

Sapphire connects to the room’s camera automatically and aligns it to the interviewee’s face.

Benchmarking

The first major feature we have is Sapphire Detection. To start the analytical interview process, we would calibrate the interviewee’s emotional state, facial features and voice intonation to create a base benchmark for future analytical comparisons.

Extracted Indicators

The next major feature would be extracted indicators. Sapphire is able to extract key events from the whole interview process by identifying emotional changes, which is how attached an interviewee is to a particular question or just how indifferent he or she is.

What follows is the speech to text feature which would convert what the applicant is saying into text. Lastly, we also examine the voice inflexions to figure out the key phrases that an interviewee is stressing.

This is followed by detecting micro-facial expressions as well as word phrasing and choice so that we can capture all these data to help our Machine Learning model predict a candidate’s emotional state.

Sapphire Insights

The second last major feature would be Sapphire Insights. By using the extracted data, we can then be able to accurately detect lies, build a character profile of each candidate, and lastly, create follow-up questions automatically. This is so that the interviewer can ask for further information esp when a lie is detected.

Interview Summary

The last feature would be our interview summary page powered by Sapphire Insights where interviewers can review the interview, raise the follow-up questions as mentioned and create follow-up actions for the candidates as well.

How We Built It

Our app prototype is built using Ionic Framework that allows for rapid development and deployment. You may preview Sapphire by visiting our Web App or our GitHub Repo.

On the technical visual side of the project, we have experimented with Python and C++ along with OpenCV (an open source Computer Vision Library) to detect facial landmarks. Building on this foundation, a machine learning model can be further trained to predict any significant events with high accuracy. On the auditory side of the house, we are studying automatic intonation analysis to first learn how to benchmark normal speech and then study how we can classify abnormal speech characteristics. Looking forward, we would be examining how we can combine both visual and the auditory analysis into a perfect amalgamation by capitalizing the benefits provided by both methodology.

Challenges We Ran Into

We wanted to build a prototype which would simulate every single functionality of Sapphire and thus, sought to create one that was as close as possible. This meant that we spent a considerable amount of time in trying out different algorithms and learning from it. The largest challenge we had was learning how to benchmark an emotional state of an applicant. We studied several prominent research papers on the subject matter and we are glad that we were able to figure out a good methodology as proposed by Howard & Ferris (1996) and Purkiss et al. (2006) which we subsequently adopted.

Another challenge that we faced was untangling the web of the whole interview process into a streamlined flow. We had to interview several HR managers and job seekers alike to get their valued opinion on the AS-IS workflow model before we would be able to design one that supports their required functionalities.

Requirements Met

Now we’ve got a good look at how Sapphire works, let’s see how it meets our requirements:

  1. An automated and accurate AI hiring bot allows us to hire swiftly yet reliably.

  2. The cultural fit of candidates can be ascertained using emotional evaluation and micro-expressions.

  3. Lie detection and voice inflexion analysis will be used to determine the reliability of candidates.

  4. And finally, Sapphire can even evaluate interviewers as well during the interviewing process to highlight any possible biases they may exhibit.

Benefits

First, we can enable a faster applicant process.

Second, we can employ bias-free evaluation.

Third, we can enhance the accuracy of the interview process.

Perhaps most importantly, is that Sapphire can drastically reduce the amount of time and cost to conduct an interview. A report from Forbes showed that the current cost to do one is $4k per candidate, and Sapphire targets to help firms achieve a cost saving of $2k. That’s 50% in savings!

Competitors & Our Unique Selling Point

So how does Sapphire fare against its competitors?

Current players in the market work well in accelerating the recruitment process, but none of them help to eliminate biases and assess candidates holistically like how Sapphire does.

Market Opportunity

Based on a report by CNBC, the value of recruitment automation in 2017 alone is $200b, and we believe Sapphire has much to offer and can capitalize on the trend of recruitment automation.

Business Model

We have an initial set-up fee of $250 per integration, and subsequently, a variable fee of $50 per candidate.

Potential Revenue

Using the base target of achieving 5% market share in Singapore, we expect Sapphire to potentially review 2.4k candidates a month and our revenue thus works out to be $1.4m per year in SG alone.

Feedback from Public

Because we wanted to make the creation of Sapphire as inclusive as possible, we sought to seek feedback form the public when we created Sapphire. In fact, several features of Sapphire were actually conceived from the ideas of friends and family we surveyed about how this tool can be useful for everyone.

For example, we have spoken to Mr Lau who is a hearing-impaired job seeker. He was heartened to hear that Sapphire will be able to help disabled people. In fact, the idea of having confidence-boosting online lessons came from him because he felt that many disabled job seekers have low self-esteem and thus having free practice would help them boost their confidence.

Both job seekers and hiring managers whom we interviewed agreed that bias-free recruitment automation is indeed a huge problem and they look forward to what Sapphire could potentially do to help streamline this cumbersome and unreliable process.

Here are some feedback we have garnered from our interviews:

“As a hiring manager, such analytical tools can help us to interview candidates much faster and eliminate bias.”

Ms, June Eng, Senior Manager (HR) Boys' Town Singapore

“Sapphire can empower job seekers, especially the disabled, to have more confidence and assistance during interviews through its AI technology."

Mr. Lau, Hearing-Impaired Job Seeker

“Employing AI to verify the reliability of candidates reassures applicants like myself that everyone is judged based on merit and cultural fit. Sapphire will be able to help both interviewers and interviewees attain their goals more effectively.”

Mr. Kiefer Yoon Wei Sheng, NTU Student & Internship-Seeker

Future Extensions

Empowering the disabled is something we have always bore in mind when we created Sapphire, and this is closely in line with the great work that VirtualAhan is doing.

In the future, we will target to first have a free online training AI bot which allows disabled persons to practice their interviewing skills and to help them boost their confidence in the process.

Second would be accessibility options for disabled interviewees, for instance, integrating with Amazon Echo, Google Assistant and Siri to help disabled persons tackle the interview more easily.

Third would be an AI dynamic interview bot which helps organizations conduct multi-lingual interviews autonomously depending on the responses of interviewees.

Future Benefits

We are also heartened that the future benefits will also align closely to the Sustainable Development Goals of Asean Youth Community as well.

The first goal we will work towards is Goal 4 – quality education. Through Sapphire, we aim to increase the number of people with skills and promote equal education as well.

The second goal is Goal 10 – reducing inequalities. Sapphire will be able to empower disabled groups and promote social and economic inclusion.

Last but not least is attaining Goal 17 – global partnerships. AYC and Virtualahan are the best examples of how inter-organizational and international cooperation can help to enhance support for less developed countries and disadvantaged groups, and we hope that Sapphire will be able to play a part in that noble cause as well.

Team

Sapphire is a diverse inter-school team comprising Ms. Reis Chang Kai Lin, a Digital Media Creator studying Communication Studies & Business Analytics at NTU, Mr. Chester Ong, a ML developer studying Information Systems at SMU, and Mr. Jarrett Yeo, an AI developer studying Computer Science and Business Analytics at NTU.

What's Next for Sapphire

In the future, we plan to get more feedback before expanding Sapphire. We envision a “marketplace” where individuals could post their resume and a standardized video interview for employers to reach out to them. Furthermore, we seek to create a video and audio package that could easily be deployed on the company premises to facilitate real-time interview.

We look forward to you supporting us in helping society!

References

  1. Purkiss, S. L. S., Perrewé, P. L., Gillespie, T. L., Mayes, B. T., & Ferris, G. R. (2006). Implicit sources of bias in employment interview judgments and decisions. Organizational Behavior and Human Decision Processes, 101(2), 152-167.

  2. Howard, J. L., & Ferris, G. R. (1996). The employment interview context: Social and situational influences on interviewer decisions 1. Journal of Applied Social Psychology, 26(2), 112-136.

Built With

Share this project:

Updates