Inspiration
Once an avid user of a renowned language-learning app, I thought I had my language skills well-honed. Armed with practiced phrases and vocabulary, I embarked on an exciting year-long adventure in France. The prospect of immersing myself in the language and culture thrilled me.
One day, I entered a French bakery, eager to try out my language skills in a real-world setting. However, as I tried to communicate my order, my carefully practiced phrases seemed insufficient. The French baker, speaking in rapid French, couldn't understand my attempts. With a polite smile, I exited the bakery, feeling defeated and disheartened.
The app I had relied on primarily focused on vocabulary and scripted dialogues, lacking the essence and unpredictability of real-life conversations.
What it does
Through engaging storylines, role-play AI assistants and progressive challenges, the platform allows one to test their conversational skills in GenAI-curated dialogues.
How we built it
Frontend (iOS App in Swift): Language: Swift (for iOS development) UI/UX Framework: UIKit for IOS17 Features: API access to backend, two way audio streaming, text translation and transcription, interactive ux with animations to capture attention
Backend (Google Cloud Compute Engine with Flask): Cloud Platform: Google Cloud Platform (GCP) Compute Engine: For hosting your backend server. Web Framework: Flask (Python) for creating API endpoints. API Endpoint Processing: Flask's route decorators for handling /connect, /start, and /storyline endpoints. Data Storage: Cloud Firestore to save user session data Authentication: Using session id and apple sign on
OpenAI Integration (for Storyline Generation): LLM Integration / Generative AI: OpenAI GPT-3.5 for generating storylines based on user input and json mode - prompt engineering Audio Processing: Using whisper-1 model to transcribe audio streamed from user to text and pass it to storyline engine for prompting Text-to-Speech: TTS-1 model to convert AI’s storyline response to user from text to speech - streamed to frontend from backend
Challenges we ran into
Integrating Frontend, Backend and OpenAI APIs
Accomplishments that we're proud of
The human-centric product we have laid the foundation for.
What we learned
New technologies as mentioned before and ways to integrate them; perform trial and error to see which tech works and which didn't
What's next for LangQuest
Customer Discovery and testing the market for the product
Log in or sign up for Devpost to join the conversation.