EDGEucator

Inspiration As a tutor, educator, and mentor, I (Mahdi) have seen the same pattern from both sides of the classroom: even the best teaching means nothing if the student doesn’t care. Engagement is the real bottleneck to learning. Kids flourish when they feel excited, curious, and emotionally connected to what they’re exploring—not when they’re pushed through worksheets or generic content. EDGEucator was born from that realization. We wanted to create an AI companion that helps children discover and grow genuine interests—STEM, art, sports, nature, reading, anything that sparks them. And just as importantly, we wanted to give parents and educators the context and tools to nurture those passions in the real world. And because this is meant for kids, privacy had to be foundational. That’s why EDGEucator runs entirely on an edge device, keeping all data local, secure, and centered around growth—not surveillance.

What It Does EDGEucator is an AI-powered voice companion that helps children discover and develop their passions through natural, engaging conversations—all processed locally on a Snapdragon-powered device.

For Children Chat naturally with an AI mentor (Harry Potter–themed character). Explore topics they’re curious about: science, art, sports, reading, nature, and more. Experience responses through a fully animated 3D avatar with realistic lip-sync. Enjoy personalized conversations that adapt to their interests and emotional state.

For Parents & Educators A dedicated dashboard provides rich, privacy-preserving insights: interest discovery to understand what sparks genuine curiosity, emotional wellbeing tracking based on tone analysis, engagement patterns revealing when and how children learn best, personalized afterschool program and activity recommendations aligned to the child’s interests, multiple child profiles with individual data tracking, and a full conversation history with long-term growth metrics.

Privacy-First Design 100% on-device processing, Snapdragon NPU acceleration, no cloud or external data transmission, and full parental control.

How We Built It EDGEucator is a fully integrated edge-AI system powered by the Qualcomm Snapdragon X Elite NPU.

Core Voice Pipeline (Python) Wake word detection using Picovoice (“Harry Potter”). Speech-to-text handled by Whisper (encoder + decoder) running on the Snapdragon NPU. LLM reasoning powered by Llama 3.2 (1B Instruct) for character-driven responses. Text-to-speech provided by Piper TTS. Emotion detection performed by a Wav2Vec2 model on the NPU to analyze vocal tone.

Interactive 3D Avatar (Three.js / TypeScript) Ready Player Me avatar rendering. Rhubarb Lip Sync for phoneme-accurate visemes. Custom morph target interpolation for smooth, natural mouth movements. WebSocket-based synchronization enabling seamless idle → listening → talking transitions.

Parent Dashboard (Next.js / React) Real-time visualizations of interests, emotions, conversations, and recommendations. Multiple dashboard views including Overview, Interests, Emotions, Conversations, and Recommendations. Recharts for data visualization, FastAPI backend for processing, and local JSON storage for complete offline functionality.

Conversation Analysis (Background Python Worker) Automatic extraction of topics, emotions, and key phrases from every conversation. Insight generation using the local Llama model. Longitudinal tracking of interest trends, engagement levels, and emotional wellbeing. A recommendation engine that maps interests to real-world activities and programs.

Tech Stack Edge AI: Qualcomm QNN Runtime 2.31, ONNX Runtime, qai_hub_models Backend: Python, FastAPI, llama.cpp Frontend: Next.js 14, React, TypeScript, Tailwind, Three.js Audio: sounddevice, soundfile, librosa, Web Audio API Models: Whisper Base, Llama 3.2-1B, Wav2Vec2

Challenges We Ran Into NPU model deployment complexity: Qualcomm’s QNN Runtime requires strict ONNX model structures, and debugging NPU execution provider issues took significant time, solved through careful study of QAIRT examples and corrected loading pipelines. Real-time lip-sync accuracy: Volume-based lip movements were robotic, so we integrated Rhubarb and built interpolation logic to achieve film-quality visemes. Balancing response speed with quality: Early LLM responses were too slow and verbose; prompt engineering and personality tuning fixed this. Privacy-preserving analytics: We needed rich insights without cloud usage, so we built a fully local Llama-powered analysis engine. Cross-platform audio handling: Connecting Python audio output to JavaScript avatar playback required building a robust, synchronized WebSocket communication layer. Windows ARM compatibility: Many ML libraries lacked ARM64 support, but Qualcomm’s QAIRT SDK and ONNX QNN EP resolved compatibility and performance issues.

Accomplishments We’re Proud Of Complete on-device AI pipeline running entirely on the Snapdragon NPU. Engaging 3D avatar with professional-grade lip-sync. Actionable parent insights that reveal real interest and emotion patterns. Real NPU acceleration for Whisper, Llama, and Wav2Vec2. Privacy by design with zero cloud dependency. Smart recommendations connecting interests to real-world opportunities. Smooth orchestration across Python, FastAPI, Next.js, and Three.js.

What We Learned Technical insights: optimizing ONNX models for QNN Runtime, advanced prompt engineering for character-driven LLMs, WebGL rendering pipelines and morph target animation, and event-driven architectures with low-latency WebSockets. Edge AI insights: Snapdragon NPUs can run full multi-model pipelines locally; ONNX is powerful but sensitive to model structure; edge-first design can deliver cloud-level performance while preserving privacy. Product design insights: children require fast responses to stay engaged, visual feedback enhances immersion, parents need context rather than raw data, and privacy must be built in from the start. Human insights: interest discovery is about nurturing natural excitement, emotion tracking can reveal meaningful early signals, and AI should augment human connection, not replace it.

What’s Next for EDGEucator Short-term enhancements: multi-character mentor personalities, persistent conversation memory, Android and tablet support, and visual emotion detection using the device camera. Medium-term goals: learning goals system for parents and children, local community resources integration, multi-language support, and a teacher-focused dashboard. Long-term vision: federated learning with privacy preservation, curriculum integration, parental collaboration tools, and potential hardware partnerships with Qualcomm. Research directions: studying long-term effects of AI mentorship on learning outcomes, developing ethical child-AI interaction frameworks, and tailoring conversation patterns for different age groups and learning styles.

Built With

• arm
• cpu
• edgeai
• javascript
• json
• llama
• node.js
• npu
• numpy
• picovoice
• python
• pytorch
• qaihub
• qualcomm
• readyplayerme
• rhubarblipsync
• three.js
• typescript
• whisper