Sokomu

Inspiration

Most Nigerian language resources treat Yoruba, Igbo, and Hausa as single uniform languages. In reality, someone who only learns textbook Standard Yoruba struggles to follow relatives speaking Ijebu or Ekiti, and Central Igbo sounds very different from Onitsha or Owerri dialects. On top of that, mobile data in Nigeria is expensive and connectivity drops regularly. A cloud-only AI tutor burns data on every single practice message and stalls the moment your network fluctuates.

When Firebase AI Logic introduced hybrid inference with Gemini Nano on-device, it gave us a practical way to solve both problems at once: run interactive text drills locally on the user's device at zero data cost, and only use the cloud to generate and cache multi-speaker regional audio when Wi-Fi is available.

What it does

Sokomu (blending Sọ in Yoruba, Koya in Hausa, and Mụ̀ta in Igbo) is an offline-first dialect and tonal tutor for Yoruba, Igbo, Hausa, and Nigerian Pidgin.

• Regional Dialect & Scenario Selection: Learners pick a target language, a specific regional dialect, and a practical scenario such as bargaining at a market, greeting elders, or navigating transit.
• Cached Multi-Speaker Audio (IN_CLOUD): While online, Sokomu generates a two-speaker dialogue comparing standard phrasing with the chosen regional dialect and synthesizes audio using gemini-3.1-flash-tts-preview. Both the audio blob and the scenario's dialect glossary are immediately saved to browser IndexedDB.
• Offline Interactive Drills (PREFER_ON_DEVICE): Once cached, learners can switch off Wi-Fi or mobile data completely. Gemini Nano runs locally in Chrome to grade typed translations, fix missing tonal diacritics, and explain regional idioms with zero network latency.
• Live Hybrid Telemetry: Every response reads result.response.inferenceSource and renders a live ON_DEVICE or IN_CLOUD badge in the UI so learners can verify when inference is happening locally.
• Native Trainer Flywheel: Vetted native speakers sign in to submit regional idioms and tonal corrections to Cloud Firestore, which sync down to learners' local IndexedDB glossaries on their next connection.

How we built it

• Frontend & Offline Storage: Built with Next.js (App Router), TypeScript, and Tailwind CSS using Bun. Synthesized .mp3 audio blobs and dialect glossaries persist in IndexedDB so lessons survive page reloads and airplane mode.
• Hybrid AI Orchestration: Integrated the Firebase AI Logic Web SDK (firebase/ai) with GoogleAIBackend. Interactive grading uses InferenceMode.PREFER_ON_DEVICE (running Gemini Nano locally in Chrome, with gemini-3.5-flash-lite as the cloud fallback), while multi-speaker dialogue audio routes via ONLY_IN_CLOUD to gemini-3.1-flash-tts-preview.
• Cost & Bandwidth Compression: By decoupling one-time audio synthesis (c_{\text{tts}}) from (N) interactive practice turns, total cloud inference cost (C_{\text{cloud}}) stays flat regardless of how long a student practices offline:

$$C_{\text{cloud}} = c_{\text{tts}} + \sum_{i=1}^{N} c_{\text{drill}}(i) = c_{\text{tts}} + 0$$

• Security & Dynamic Prompts: Enforced Firebase App Check (reCAPTCHA Enterprise for production and debug tokens for local testing) to eliminate exposed client-side API keys. Used Firebase Remote Config to update scenario prompt templates dynamically without redeploying, alongside Firebase Authentication and Cloud Firestore for user progress and verified dialect rules.

Challenges we ran into

• Next.js SSR vs. Browser-Only AI APIs: Chrome's LanguageModel runtime and Firebase App Check only exist in the browser. Early builds crashed during server-side rendering until we isolated SDK initialization behind client-side typeof window !== "undefined" guards and custom React hooks.
• Local Model Hardware Thresholds: Getting Gemini Nano to download on a standard development laptop required enabling chrome://flags/#optimization-guide-on-device-model (Enabled BypassPerfRequirement) and maintaining at least 22 GB of free disk space so Chrome would not evict the local weights.
• Tonal Diacritics on a Quantized Model: Smaller on-device models can drop Yoruba sub-dots (ẹ, ọ, ṣ) or Igbo/Hausa tonal markers when translating regional dialects. Injecting compact, Firestore-synced dialect glossaries from IndexedDB into the local prompt context kept translations grounded in authentic regional spelling.

Accomplishments that we're proud of

• Building a complete airplane-mode study loop where cached two-speaker dialect audio plays smoothly offline while Gemini Nano grades typed responses locally with zero network packets sent.
• Surfacing inferenceSource telemetry directly in the interface so hybrid routing is transparent and verifiable rather than hidden in developer logs.
• Using Firestore glossaries and Remote Config to improve low-resource dialect accuracy without expensive model fine-tuning or app redeploys.

What we learned

• Hybrid AI requires designing for asynchronous model downloads, storage quotas, and offline state transitions from the very first commit—not as an afterthought.
• On-device models handle low-resource African languages far more reliably when paired with concise, locally cached glossary grounding instead of open-ended zero-shot prompts.
• Combining Firebase App Check with Remote Config makes client-side generative AI both safer against API abuse and faster to iterate on.

What's next for Sokomu

• Native Android Build: Extending Sokomu to Android via the Firebase AI Logic Kotlin SDK and AICore so learners can practice regional dialects on mobile devices.
• Offline Speech-to-Text Grading: Adding local audio input evaluation so learners can speak dialect phrases aloud offline and get pronunciation feedback.
• Broader Language & Dialect Coverage: Onboarding native dialect trainers for additional Nigerian languages—including Ibibio, Tiv, Edo, and Kanuri—and piloting structured dialect modules with diaspora families and Nigerian schools.

Built With

• bun
• chrome-built-in-ai
• firebase
• firebase-ai-logic
• firebase-app-check
• firebase-auth
• firebase-remote-config
• firestore
• gemini
• gemini-nano
• google-ai
• indexeddb
• nextjs
• react
• tailwindcss
• typescript