HireHuman

An application that will take interview speeches in realtime and tell if the student is using LLM for his interviews or his original brain using some advanced algorithms with the power of Azure OpenAI

Comment

Inspiration

Hiring has slowed down not only by a reduced number of jobs but also by an increase in frauds by job applicants usings LLMs, AI generated deepfakes for Malpractices. This is creating a very tough time for the recruiters in finding the best and real Talent. Remote hiring has its own challenges as managers have to be alert to identify tailormade profiles of fake candidates.

What it does

This application captures interviews in real-time, takes the input just by listening, and uses some advanced algorithms like the WaterMark method to run NLP algorithms on the text and uses Azure OpenAI to compute the results at voice detection, Separation and identification in finding if the candidate response is from an LLM or Human or Mix of both. We have trained the model with thousands of datasets and got industry-level accuracy with the results.

How we built it

We have built it with the following tools and frameworks

  1. React for Frontend and Audio Capture and Audio Encoding
  2. Flask for the Backend, which integrates OpenAI Cosmos DB and outputs the results generated from the database and Models
  3. AssemblyAI for voice diarization

Challenges we ran into

  1. voice diarization was almost impossible as we deal with real-time detection here.
  2. Studying multiple algorithms and researching about them before building models and testing thousands of datasets against all the algorithms and comparing them with each other, then getting a conclusion out of them
  3. Training the bot was also challenging, as responses and formats varied
  4. Rate Limitings from Azure OpenAI, where we had to ensure we did not lose any data, but we had to cut down some tokens. The result should be the same, with no change in accuracy

Accomplishments that we're proud of

  1. We have successfully come across all the issues mentioned above with thorough research and making this app the most reliable in terms of DevOps and System design
  2. Rate limiting is achieved with no change in accuracy and no loss in data
  3. The overall most significant achievement is that this is our first application where we have integrated LLM and got real-life exposure to how companies are incorporating and using the power of LLM outside

What we learned

  1. Technically, we have learned an entirely new set of skills, starting from Handling Blobs most efficiently to integrating the power of OpenAI into the application
  2. We have learned how to build an LLM bot. Not only building it, we have learned prompt engineering and ensuring that the prompts are on point and the LLM returns the information as requested.

What's next for HireHuman

  1. We have spoken to some recruiters for the past weeks before finalizing this application and took their inputs. They are interested in the product and concept.
  2. If Microsoft permits and collaborates, I would like to initiate a project under the guidance of Microsoft as my primary investor and launch this application from a low scale to a high scale and implement more features into the application that utilizes powers of AI and become one of the most reliable solutions for interviews.

Built With

Share this project:

Updates