Recyclens

RecycLens is an AI-powered recycling assistant that uses Gemini AI to identify waste through voice and vision, gamifying sustainability with a leaderboard to make recycling easy, accurate, and fun.

Comment

Inspiration

We've all stood in front of a recycling bin, holding an item, unsure if it belongs in the blue bin, the green bin, or the trash. "Wish-cycling"—tossing things in the recycling hoping they're recyclable—actually contaminates the waste stream, doing more harm than good. We wanted to solve this confusion not just by building a scanner, but by creating an interactive companion that makes sustainability engaging and competitive.

What it does

RecycLens is a web application that turns your smartphone into a smart recycling assistant.

  • Real-Time AI Assistant: Users can open the "Ask Gemini" feature to have a live, conversational video chat with the AI. You simply show an item to the camera and ask, "Is this recyclable?" The AI sees what you see and responds instantly with voice, explaining exactly how to dispose of it (e.g., "That's a coffee cup; the lid is plastic recycling, but the cup is trash because of the wax lining").
  • Gamification: To encourage good habits, users can "Submit Proof" of their recycling. Snapping a photo earns them points.
  • Leaderboard: Users compete on a global leaderboard, turning eco-friendly actions into a friendly competition.

How we built it

Frontend: Built with React, TypeScript, and Vite for a fast, responsive mobile-first experience. AI Integration: We utilized the Google Gemini Multimodal Live API (Gemini 3 Pro Preview) via WebSockets. This allows for low-latency, two-way communication where the model processes both the live video stream and audio input simultaneously. Audio Processing: We implemented a custom AudioWorklet to handle PCM audio streaming efficiently, ensuring the voice interaction feels natural and lag-free. Backend/Storage: We used Supabase to manage user profiles and the real-time leaderboard, ensuring data persists across sessions. Styling: Tailwind CSS was used to create a sleek, dark-mode aesthetic that feels modern and eco-conscious.

Challenges we ran into

Audio Latency: Getting the AI to respond instantly without an awkward delay was tough. We had to move away from deprecated script processors to modern AudioWorklets to handle the audio stream smoothly. Real-time Video: Streaming video frames to the Gemini API while maintaining performance on mobile devices required careful optimization of frame rates and resolution. State Management: Handling the complex state of a live WebSocket connection alongside the user's navigation and camera permissions was a balancing act.

Accomplishments that we're proud of

Successfully integrating the Gemini Multimodal Live API for a truly "Iron Man Jarvis-like" experience where the AI sees and hears in real-time. Creating a seamless UI that works beautifully on mobile devices, feeling like a native app. Building a working gamification system from scratch that actually updates in real-time.

What's next for Recyclens

Object Detection Validation: Currently, points are awarded for submission. We plan to use Gemini to verify the photo actually contains recyclable items before awarding points. Geolocation: Adding local recycling rules based on the user's GPS location (since recycling rules vary by city). Social Features: Allowing users to challenge friends or create "Neighborhood Teams" to compete on the leaderboard.

Built With

Share this project:

Updates