Ben

• 

  mic amplifier

• 

  display, raspberry and MAX98357A

• 

  Esp32s3 keyboard

• 

  body

Inspiration

My little brother has Level 2 autism, and he currently struggles to study at the same pace as his peers. I wanted to create something that could make learning more accessible and less frustrating for him. This became the inspiration for Ben, an interactive educational assistant that turns learning problems into game-like activities and provides help when the user gets stuck.

What it does

Ben provides three types of educational activities: math, logic, and language. When a user struggles with a problem, the system can listen to their response and provide an explanation or hint through voice. Rather than simply giving the answer, the goal is to make the learning process interactive and allow the user to work through the problem with assistance.

How we built it

The main computer is a Raspberry Pi 4 Model B, running the Qwen2.5-3B language model locally. I wanted the system to work locally so that the core interaction could happen without relying entirely on an internet connection. For physical input, I built an arcade-style controller using a joystick and push buttons. An ESP32-S3 converts these inputs into keyboard commands, allowing the user to interact with the game naturally. For voice input, I used an electret microphone capsule. Since I was not familiar with audio amplifier circuits at the beginning, I designed and built my own microphone amplifier using a BJT transistor. The output consists of a 1.8-inch ST7735 display and a MAX98357A I2S Class-D mono amplifier connected to a speaker. The software architecture connects the game UI, server, and voice agent:

UI → Flask server → Voice agent → AI/TTS

When a user enters one of the three problem sets, the UI communicates with the Flask server. The voice agent retrieves the current problem through the server and uses Whisper to process the user's speech. Prewritten responses are used for common interactions so that the system can respond quickly, while the local language model helps determine what the assistant should say in more specific situations. Piper TTS then converts the response into speech.

Challenges we ran into

One of my biggest challenges was connecting the UI, server, and voice agent reliably. At first, communication through the API introduced enough delay to make the interaction feel slow. I used Flask and Flask-CORS to create the communication layer between the game interface and the voice assistant. I eventually improved the responsiveness by using prewritten responses for common situations instead of generating every response with the language model. Another major challenge was building the electret microphone amplifier. I had very little experience with amplifier theory when I started. Understanding transistor biasing, signal amplification, and how the circuit affected the microphone signal required me to spend significant time learning the underlying electronics rather than simply copying a circuit.

Accomplishments that we're proud of

One accomplishment I am particularly proud of is achieving fast interaction despite running the system locally on a Raspberry Pi. Using prewritten responses for predictable situations significantly reduced the delay between the user's action and the assistant's response. Getting the system to accept human voice input through a microphone and turn it into an interactive learning experience was another major milestone. It required me to combine electronics, embedded systems, web development, speech recognition, text-to-speech, and local AI into one working system.

What we learned

This project taught me much more about electrical circuits and audio electronics than I initially expected. BJT transistor amplification and operational amplifier concepts were unfamiliar to me, so building the microphone amplifier forced me to understand the theory behind the circuit rather than treating it as a black box. I also learned about the limitations of small language models running locally. A model such as Qwen2.5-3B can respond quickly and run on relatively inexpensive hardware, but its reasoning and language capabilities are more limited than larger cloud-based models. This taught me that a useful AI system does not necessarily need to rely entirely on a language model. Combining deterministic responses with AI can sometimes produce a faster and more reliable experience.

What's next for Ben

The next major challenge is Mongolian language support. I am from Mongolia, and I want Ben to be genuinely useful for Mongolian-speaking children. My goal is to develop a reliable translation pipeline that can translate Mongolian speech into English for the language model and then translate the response back into Mongolian before it is spoken to the user.

Built With

• claude
• electret-mic-capsule
• esp32
• flask
• javascript
• joystick
• max98357a
• piper
• push-buttons
• python
• qwen2.5:3b
• raspberry
• st7735
• whisper.ai