Inspiration
Learners frequently ask for meanings, request repetition, pause, or avoid an unfamiliar expression during conversation. Those signals usually disappear after the conversation. Generic study plans then fail to target the exact failings that interrupted communication.
What it does
Personalized Listening-Speaking Interactive Feedback Assistant (AKA: RecallSpeak) presents one guided learning loop while keeping its three processing experiences separate and connected only by authenticated D1 data:
- English Check runs an adaptive English voice conversation, explains the result in simple English and the learner's native language, and saves a four-category weakness profile.
- Today's Lesson teaches one active item with a simple-English meaning, a native-language explanation, an English example, and a native-language explanation of that example before listening and speaking practice commences.
- Real Conversation uses consent-gated, session-only speaker diarization to find explicit repair moments such as:
- "Sorry?" (places where repetition is requested)
- Grammatical issues
- Moments where understanding is limited or misunderstandings occur
To address privacy concerns, it saves only necessary learning candidates (lesson topics) for the learner's next lesson. Recordings are held only for the current request and discarded after analysis - never stored, which further strengthens steps made to assuage privacy concerns.
How we built it
This project was built by vibecoding with Codex.
Challenges we ran into
Although I am an engineer, this was my first time attempting to develop an application from scratch. Learning the structures and logic necessary for designing the application was my biggest challenge and I am pleased to say that I have succeeded in creating an application which can contribute greatly to English education across the world.
Accomplishments that we're proud of
I am proud of the fact that I have created an application, which, if released will allow users worldwide to easily learn necessary skills and acquire knowledge for listening and speaking English. The project's focus is on allowing users to bridge the gap between their weak points and cementing newly learned concepts while maintaining accessibility by having native-language support in the form of AI-created explanatory text which will act as a support structure for the learner as they continue to use the application.
What we learned
At first, I intended to use API keys in the application, so naturally, API keys were needed in the testing of the application - leading to increased cost. However, it was ultimately possible to create a local testing environment, which negated the need to use API keys during testing.
What's next for Personal Listening-Speaking Interactive Feedback Assistant
The next iteration of the project will be released - not as a web application - but as a mobile application on iOS and Android. The project is not about making a profit, but rather enriching and contributing to society and English education worldwide, creating a platform which makes learning English more accessible for people across all walks of life.
In order to realize this, further fine-tuning of the local LLM and financial considerations regarding the cost of OpenAI API key usage will be necessary, but I am confident that both of these factors will continue to see progress and exciting developments in the near future.
Log in or sign up for Devpost to join the conversation.