SgStudyPal

Past Year Exam Papers
Syllabus Quizes
Curiousity Companion Live Video.
Homework Help

Inspiration

Kids hate homework. Period. Staring at static worksheets kills their natural curiosity, and traditional Q&A bots just feel like more work. I wanted to give my kids an educational experience that felt like exploration, not a chore.

What truly inspired this project was my own recent journey. It is crazy to think I only started building things with AI in February 2026. As a non-developer, the sheer velocity of progress you can make when you actually enjoy the process of building is mind-blowing. If I could orchestrate a complex, real-time web application just by staying curious and having fun with my AI build partner, what could my kids achieve if their learning felt the exact same way?

What it does

SgStudyPal is a complete educational ecosystem that actually makes learning fun again. It handles the basics—getting tough homework answered, gamified syllabus tracking, and easy access to past exam papers—but the absolute core of the app is the Curiosity Companion.

The Companion is a real-time, voice and vision-enabled AI tutor named Gwen, powered by the Gemini Multimodal Live API over WebSockets. Instead of forcing a rigid lesson, we use a "State Gate" Pre-flight UI. Kids just drop in their name and 3 things they love (like Space, Cello, or Dinosaurs). Gwen reads this instantly, completely solving the AI "cold start" problem. A student can hold up a drawing or play an instrument, and Gwen "sees" and "hears" it in real-time, letting the kid's actual interests drive the conversation naturally.

How we built it

I orchestrated a strict Turborepo architecture with a Next.js (React) frontend, integrating the @ai-sdk/google to manage communication with the Gemini models. To keep the app perfectly stable, I used a "Vibe Coding" approach with robust state management, ensuring heavy media pipelines stay completely dormant until the user explicitly hits "Connect."

To manage the heavy payload of live video frames without crashing the browser, I built a custom Canvas Optimizer. Instead of blasting raw, continuous video over the WebSocket, the optimizer selectively debounces the frames—only capturing and sending snapshots at specific intervals. This drastically cuts down the bandwidth requirements while keeping the visual context sharp for the Gemini API to read.

Finally, the entire application is deployed using an automated infrastructure-as-code (IaC) setup. I wrote a custom shell script that packages the Next.js standalone build into a minimal Docker container and ships it straight to Google Cloud Run (asia-southeast1) for scalable, low-latency execution that bypasses standard serverless timeouts.

Challenges we ran into

The biggest headache was managing that continuous multimodal data without degrading the UI. Pushing audio chunks and video frames simultaneously over WebSockets is incredibly resource-intensive. Debugging complex state bugs, buffer parsing for image payloads, and deployment failures was painful, but it forced me to maintain a tight, continuous feedback loop with my AI build partner to push through the friction and stabilize the application.

Accomplishments that we're proud of

Going from zero coding experience to deploying a stateful, multimodal real-time AI architecture in just a few weeks. Achieving my strict "Definition of Done" (0 TS errors and a stable local build) and successfully automating the Dockerized deployment to Google Cloud Run so the app is live and accessible to the world.

What we learned

Ideas are the new code: With AI, as long as you have the vision and the persistence, almost anything on the web can be built now.

The "Last Mile" is the hardest: Launching a basic prototype is easy, but building a product you are genuinely proud of—one that is rock-solid on the backend and highly intuitive on the frontend—requires relentless iteration.

Partnership is key: Debugging is a pain, but getting constant feedback and leveraging AI for agentic orchestration is the ultimate superpower.