TerpforWeb

An AI-native web script and browser extension that dynamically translates web page text and media into real-time Sign Language delivered via a hyper-realistic 3D avatar overlay.

Comment

Inspiration

Over 1.5 billion Deaf and hard of hearing individuals globally - including 40 million across Africa - face systemic exclusion from essential digital services because the web is built almost entirely for text and audio. Spoken/written languages are often secondary languages for native signers, creating immense friction when navigating healthcare portals, legal services, e-government platforms, or emergency travel updates. We were inspired to build TerpforWeb to solve this grand challenge at its root. By leveraging generative AI, hyper-realistic 3D avatars, and lived-experience feedback from Deaf users, we set out to build an AI-native accessibility companion that integrates into any website with a single script, breaking communication barriers without requiring site operators to undergo costly website redesigns.

What it does

TerpforWeb is a plug-and-play web script, browser extension, and Content Management System (CMS) plugin that automatically translates website text and multimedia into real-time, context-aware Sign Language delivered via a hyper-realistic 3-Dimensional (3D) avatar overlay. Acting as an on-demand accessibility bridge across the Professional Services Access category, TerpforWeb dynamically parses Document Object Model (DOM) elements, converts complex English , Swahili , Spanish , Portuguese and all language variations syntax into Sign Language gloss structures, and renders 3-Dimensional (3D) avatar gestures in Web Graphics Library (WebGL) with under 250-millisecond latency - empowering Deaf users to independently execute banking transactions, review legal contracts, and access public healthcare platforms.

How we built it

We engineered TerpforWeb using a high-performance, privacy-first technical stack: Core Web & Extension Stack: Built with TypeScript, JavaScript, HTML5/CSS3, and WebGL (Three.js) for lightweight, client-side 3D avatar rendering directly in the browser. Backend & Containerization: Powered by Java, Shell scripts, and Docker containers deployed on Google Cloud Run for auto-scaling serverless execution. AI & Language Engine: Integrated the Gemini API via Vertex AI to handle real-time semantic processing, text-to-gloss grammar conversion, and synthetic training data generation across specialized legal and medical domains. Dataset Foundation: Trained on proprietary, Deaf-validated motion capture arrays backed by 8,000+ hours of user testing and 7,000+ hours of dataset curation.

Challenges we ran into

Cross-Document Object Model Rendering & Zero-Conflict Execution: Injecting an interactive Web Graphics Library (WebGL) avatar widget across diverse web architectures and WordPress themes without breaking client Cascading Style Sheets (CSS) or JavaScript execution was a major engineering hurdle. We built a sandboxed Shadow Document Object Model (Shadow DOM) architecture to isolate styling and state management. Grammar & Cultural Fidelity: Sign Language is not a word-for-word translation of spoken text; it possesses distinct grammar, spatial orientation, and non-manual markers (facial expressions). We utilized Gemini via Vertex Artificial Intelligence (Vertex AI) to map complex sentence structures into accurate Sign Language gloss syntax rather than literal translation. Low-Latency Client-Side Performance: To prevent slowing down host websites, we offloaded 3-Dimensional (3D) skeleton trajectory calculations to Web Graphics Library (WebGL) shaders, ensuring zero noticeable impact on page load speeds or host site Search Engine Optimization (SEO) performance.

Accomplishments that we're proud of

90-Day Build Execution & Realized Revenue: Took TerpforWeb from an early prototype to a production-ready, revenue-generating Software as a Service (SaaS) solution within the 90-day competition window - generating $21,320.00 USD in realized, arms-length Business-to-Business (B2B) revenue across 5 enterprise client accounts with zero related-party funding. Deaf-Validated Accuracy & Lived-Experience Co-Creation: Built on 8,000+ hours of direct user testing with the Deaf community and over 7,000 hours of technical dataset curation, achieving high translation fidelity and cultural accuracy validated directly by native signers and certified Sign Language interpreters. Seamless AI-Powered Integration: Successfully engineered a lightweight, client-side script and plugin driven by the Gemini Application Programming Interface (Gemini API) via Vertex Artificial Intelligence (Vertex AI), enabling everyday website administrators to deploy 3-Dimensional (3D) avatar accessibility across public-facing portals in under two minutes with zero code friction.

What we learned

AI-native Optimization: We learned how to combine a Large Language Model (LLM) driven semantic understanding (via Gemini) with spatial 3D keypoint retargeting to achieve real-time, context-rich translation. Scalable Web Accessibility: Gained deep insights into Web Content Accessibility Guidelines (WCAG) and how client-side WebGL rendering can dramatically lower server infrastructure costs (>75% gross margins) while maintaining strict data privacy standards. The Power of Lived Experience: Real-world feedback from Deaf community members taught us that facial expressions and body posture carry up to 50% of the linguistic meaning in Sign Language, shaping our gesture animation pipeline.

What's next for TerpforWeb

Self-Serve Enterprise & CMS Plugin Launch: Distributing our production-ready plugin and browser extension via official app marketplaces and proprietary web distribution to accelerate self-serve adoption for corporates, governments , individuals and Small and Medium Enterprises (SME). Generative Motion Engine Upgrades: Fine-tuning custom generative gesture models using Gemini Flash models on Vertex Artificial Intelligence (Vertex AI) to expand localized regional sign dialects across the globe. Enterprise Expansion: Expanding active enterprise pilots across our pipeline (including Safaricom, Kenya Airways, Aeropuertos Argentina, and E-Citizen government portals) to capture market share within the global $10 Billion digital global accessibility market on our path toward $3.5 Million Annual Recurring Revenue ($3.5M ARR).

Share this project:

Updates