Cortex

Tired of overthinking reCAPTCHA boxes? Simply listen and describe to verify your human-ness using Cortex. Help underserved industries with labeled datasets for ML & join us in the future of CAPTCHAs.

Comment

Inspiration

Our team originally wanted to build AI-generated sounds for music production. However, to our disappointment, the labeled data we needed for that idea wasn’t sufficient. As a result, we pivoted to trying to make better, more accessible dataset labeling and creation for these smaller, underserved industries.

For this project, we settled on audio CAPTCHAs as an example method of online authentication. Why?

  • We saw there was a pressing need for accessible and comprehensive datasets in many underserved industries seeking to leverage machine learning.
  • To empower/support these niche markets/opportunities by supporting/enhancing their data labeling processes.
  • In the future we can bring the ability to create similar simple tasks for a variety of unharnessed datasets.

What it does

An audio authentication system that Labels ML Datasets through the reCaptcha authentication system. Users will label descriptive themes and keywords that the audio files evoke as their verification process, which can help benefit other programs overall. As users complete more CAPTCHAs, a dataset of keywords that describe the audio clips is built. For every new audio clip, users are prompted to note one word that describes the clip. Once a cache of repeated tags/themes exists and some of these keywords’ counts cover a certain threshold, users who have been given that audio clip as a CAPTCHA from then on will select/rank key words from the word bank. This allows the backend dataset of word descriptors to be fully robust and automatically crowdsourced.

How we built it

  • Our backend runs on Flask and Python, hosted on an Oracle Cloud Infrastructure Compute Instance.
  • Our frontend is powered by Retool.

Challenges we ran into

  • How to continually label data using crowdsourcing without any ground truth
  • How to get good initial datasets of unlabeled data
  • How to make the reCAPTCHA as least frustrating as possible for the users
  • Distinguishing from existing services

Accomplishments that we're proud of

Creating a frontend, simple in nature, that allows for the creation of a sophisticated dataset.

What we learned

  • How to use Retool’s prebuilt UI components and utilize their API to connect to our backend server.
  • How to design and build automated data labeling through simple user input.
  • Running Flask on the Oracle Instance to create our own custom RESTful API

What's next for

  • Validation of ease of use with customer basis (and potentially also visually impaired as a better audio CAPTCHA than current ones)
  • Potential integration to replace mobile gaming ads
  • Music instead of sound effects
  • Stronger Security Measures (More robustness toward bot attacks, too)

Built With

Share this project:

Updates