Team
Landing Page
Login
Home
Vocal

Smarteg: Voice AI for Zero-Waste Wartegs

Inspiration

Our journey began with a stark reality: Indonesia loses 48 million tons of food annually, costing $20 billion USD, burdening local economies and the environment. This crisis hits hardest at Wartegs, Indonesia’s beloved small food stalls, where owners rely on outdated manual systems, leading to significant food waste and financial losses. Inspired by the resilience of these UMKM (Usaha Mikro, Kecil, dan Menengah), we envisioned an accessible, voice-powered Progressive Web App (PWA) to empower Warteg owners to reduce waste, boost profits, and contribute to a sustainable future.

What it does

Smarteg is a voice-activated PWA designed to transform Warteg operations through intuitive voice commands, leveraging AI for efficiency and sustainability:

Voice-Driven Stock & Sales Tracking: Record initial stock (e.g., "Ayam Goreng 20 potong") and customer orders (e.g., "Nasi, ayam, tempe") using Web Speech API, with real-time updates synced via WebSocket.
Frontend Demand Prediction: JavaScript-based algorithms (e.g., weighted moving averages) forecast daily cooking needs, minimizing over-preparation.
Smart Waste Monitoring: Calculate daily waste by recording leftover stock, generating PDF reports for green financing using jsPDF.
Social Media Suggestions: Create AI generated poster for promotions (e.g., "Tempe limited edition!") triggered by voice input.
Real-Time AI Insights: WebSocket connects to Google Gemini AI for background discrepancy detection (e.g., employee theft) and stock validation.
Google OAuth Login: Secure user authentication via Google accounts.

Smarteg reduces food waste, enhances profitability, and promotes sustainable practices for Wartegs, all while being user-friendly for non-tech-savvy owners.

How we built it

Smarteg was crafted as a frontend-focused PWA for the hackathon MVP, leveraging modern web technologies and AI integration:

Frontend: Built with Next.js, React, and TypeScript for a dynamic, responsive UI, styled with Tailwind CSS.
Voice Interface: Integrated Web Speech API for Speech-to-Text (STT) and Text-to-Speech (TTS) to enable voice commands in Indonesian and English.
Data Management: Used IndexedDB for temporary data caching (stock, sales, reports), with real-time syncing to a temporary WebSocket server (wss://temporary-server.com) for AI processing.
AI Processing: Connected via WebSocket to Google Gemini AI for real-time discrepancy detection and stock validation, ensuring live insights.
Predictive Analytics: Implemented JavaScript-based weighted moving averages in the frontend for demand forecasting, fetching historical data via WebSocket.
Authentication: Integrated conventional Google OAuth using Google Identity Services SDK, redirecting to smarteg.app/oauth-callback?token= and validating tokens at api.smarteg.app/auth.
Report Generation: Used jsPDF for local PDF waste reports, compatible with future uploads to api.smarteg.app/reports.
Deployment: Hosted on Vercel for seamless performance and PWA installability.

Challenges we ran into

Building Smarteg presented several hurdles:

Speech Recognition in Noisy Environments: Ensuring Web Speech API accuracy in bustling Warteg settings with diverse Indonesian accents.
WebSocket Stability: Managing reliable real-time connections to the temporary Gemini AI server under variable network conditions.
Frontend-Only Predictive Analytics: Developing accurate demand forecasts using limited IndexedDB data and JavaScript algorithms.
Google OAuth Integration: Implementing a secure, conventional OAuth flow within the hackathon timeline, ensuring compatibility with api.smarteg.app/auth.
User Adoption: Designing an intuitive voice-first interface for non-tech-savvy Warteg owners to overcome resistance to new technology.
Scope Management: Balancing ambitious AI and voice features with the hackathon’s tight deadline.

Accomplishments that we're proud of

Functional MVP in 2 Days: Delivered a working PWA with voice, AI, and authentication features within hackathon constraints.
Seamless Voice & AI Integration: Combined Web Speech API and WebSocket-based Gemini AI for a user-friendly, real-time experience.
Secure Authentication: Implemented conventional Google OAuth with smarteg.app/oauth-callback?token= and api.smarteg.app/auth for secure login.
Frontend Predictions: Built accurate demand forecasting using JavaScript, demonstrating value without backend dependency.
Sustainability Impact: Showcased Smarteg’s potential to reduce Warteg food waste by 20-30%, supporting Indonesia’s Net Zero Emissions by 2060.
User-Centric Design: Crafted an accessible interface tailored for UMKM owners, with TTS-guided onboarding.

What we learned

Voice Technology’s Power: Voice interfaces significantly enhance usability for non-technical users in fast-paced environments.
AI for Small Businesses: Real-time AI insights (via WebSocket) and frontend analytics can drive practical, impactful solutions for UMKM.
PWA Accessibility: PWAs offer a low-friction way to reach diverse users, especially in mobile-first markets like Indonesia.
Data-Driven Sustainability: Effective data management (even in IndexedDB) is crucial for waste reduction and financial insights.
OAuth Simplicity: Conventional Google OAuth streamlines authentication while ensuring security and scalability.
Iterative Development: Rapid prototyping under hackathon constraints taught us to prioritize core features and modular design.

What's next for Smarteg: Voice AI for Zero-Waste Eateries

Our vision is to make Smarteg the leading platform for sustainable F&B UMKM in Indonesia:

Refine & Scale: Improve STT accuracy, optimize frontend predictions, and expand to more cities.
Partnerships: Collaborate with waste management firms, food banks, and green financiers to amplify impact.
Feature Expansion: Add multi-user access, support for broader F&B segments, and advanced analytics.
Sustainability Leadership: Drive zero-waste practices across Indonesia’s F&B UMKM, contributing to global sustainability goals.