reQUACKCHA

An audio - based captcha that makes the classic "bot verification" a little more exciting - with a twist for Hack Hack Goose 2026!! Try it out at requack.biz!

Comment

reQUACKCHA

Inspiration

Our inspiration came from registering on Devpost for this hackathon, during which we encountered the all-too-familiar CAPTCHA. Performing “Are you a human?” checking to verify that bots cannot sign into websites takes a cumulative 500 years every single day worldwide, a process that is often frustrating and monotonous for users.

We sought to make this a more engaging and enriching experience, from which QuackCHA was born. reQUACKCHA is an audio-based CAPTCHA system that still validates human login but, instead, plays audio of a specific goose species and asks users to “quack” back to the CAPTCHA. After validation, a fun fact about the goose is displayed. This way, the task becomes a little more engaging and teaches users something new about a goose species they may never have encountered before.

What it Does

  • Shows screen of user logging into a particular website after which CAPTCHA pops up
  • Displays picture of goose species, along with playable reference audio of goose call
  • Picture is pulled from research-grade iNaturalist posting
  • Audio is pulled from xeno-canto wildlife sounds database
  • User can record themselves imitating the call, after which it is validated for whether it is human via RMS bursts and volume modulation within 5 second window
  • After audio is validated[doesn’t need to be exact match of bird], a fun fact of the bird is shown, and user can continue onto the desired website

How we built it

  • Used Cursor as main IDE
  • Created monorepo with Turbo and Tailwind CSS for HTML and styling ###Next.js:
  • Pulls user audio recording via MediaRecorder/WebAudio API’s
  • Stores bird recording in backend data

  • Stores iNaturalist pictures, fun fact, bird name, species ID in metadata

  • Frontend - Tailwind CSS, shadcn/Radix UI for components

  • Backend - FastAPI with REST endpoints for taking in species data and recordings

    • Processes user recording in 5 second normalized window, evaluates RMS bursts and volume modulation, and runs through Silero deep learning model to differentiate speech from rhythmic taps/noise

  • QA - Playwright for end-end checks and pytest to check FastAPI

  • Ran Next.js and Uvicorn simultaneously in terminal(npm run dev: all) and tested via local host

  • Multi-agent task split - one for frontend, one for backend, one to handle separation/handoff

  • Used Vercel to deploy web app and Render for FastAPI integration via backend web service saved as environment variable NEXT_PUBLIC_API_BASE_URL

  • We split up tasks between three agents, with two being for frontend and backend, and another used to ensure that the tasks were segregated and that handoff between them was smooth.

Challenges we ran into

Throughout our project, we ran into some major/minor issues:

  • The biggest issue was probably ensuring that our audio pipeline worked consistently, especially because the voice recording was often approved by the captcha system, even when we simply drummed on the table, and we had to experiment with using a deep-learning model, Silero, in the backend, to differentiate speech from noise after initial RMS burst/volume modulation processing.
  • We also tried additional approaches to improve accuracy, such as ZCR, to further discriminate between the voicedness of stimuli, but found that this yielded diminishing returns, and instead opted for a simple approach.
  • We also found it initially difficult to communicate with the agent about the UI changes we wanted to make, for which we had to make our prompts detailed, and write “make no mistakes” at the end of each prompt to ensure correct and thorough output.
  • Finally, we had to make sure that initially, the agent did not make its own audio files, and ensured that the .mp3 audio files were pulled from xeno-canto and saved as static files in FastAPI to be computed alongside the user’s audio, pulled from MediaRecorder/WebAudio API’s in the browser, accessed by NextJS. Then, to extract readily accessible images, we searched iNaturalist for research-grade sightings that mapped to the geese species of a particular audio recording, and also integrated presentation of fun facts/conservation stats after a captcha was complete.

Accomplishments that we are proud of

  • One accomplishment that we are particularly proud of is the implementation of human voice in the rather stale and tasteless system of traditional captchas in use today. By using audio, it allows the users to find captchas to be more fun and interesting rather than a source of rage and annoyance. Additionally, the use of different species of geese as the main theme of our captcha system allows for the awareness of wildlife preservation and recognition. With many geese populations (especially those classified as midcontinent light geese) showing a sharp decline over the years, our geese-themed project is able to focus on the public appreciation of the said species, contributing to an increased awareness by the general population.
  • We were also proud of the fact that our application was built to pull real species recordings and research-grade images from open-source databases, providing the validity of our data and ability for our app to not only provide a fun alternative to captcha, but also educate users about species in a reliable manner.

What we learned

We learned about how a web application is developed, from choosing a tool to orchestrating between the frontend and backend, testing output on a local host server, creating a thorough AGENTS.md file to specify project requirements to the agent, how to think through every aspect of user design - from potential inconveniences a user may face, to simplifying the user experience as much as possible, and other skills that we hope to build upon in the future.

What’s next for reQUACKCHA

  • In the future, we hope to implement audio classification pipelines involving machine learning algorithms/neural networks capable of discerning human-produced speech from deepfakes, such as Siri or other chatbots.
  • Our system currently discerns voice from background noise/rhythmic tapping at a similar frequency to the stimulus, but occasionally makes errors, and could benefit from a more robust model that runs in the browser.
  • Additionally, just as captcha was originally used to digitize large volumes of news content, we hope that when users complete reQUACKCHA, the data from human voice recordings can be used to train human voice recognition systems, as well as voice generation models like Eleven Labs to produce emotional inflection, volume modulation, and bursts, which are often undertrained in speech-text data. Generating more humanistic content for mediums such as YouTube Music’s AI DJ, digitally transcribed audiobooks, and even educational materials through platforms like NotebookLM is gaining prominence in today’s day and age.
  • The data from our project can also help between human and AI-generated voice, which will become increasingly relevant as improvement in generative AI models brings computer-generated text, audio, and video closer to human creation in a time where creativity is of utmost importance to maintaining human originality and authenticity, using AI as a tool for enhancing productivity and efficiency beyond what was originally possible.

Built With

  • fastapi
  • figma
  • next.js
  • shadcn
  • tailwind
  • turbo
Share this project:

Updates