LiveLifeLoop

Inspiration We asked ourselves: What if every object in the world had its own soundtrack? A rusty bicycle, a ceramic mug, a glass window — each material vibrates at its own frequency, each surface tells a story. LLL was born from the idea that music isn't just something you listen to — it's something the world is already playing. We just needed to build the translator.

What it does LLL (Life Live Loop) transforms your surroundings into a personal music studio. Point your camera at any scene, and our AI identifies objects and their materials — wood, metal, glass, fabric. Choose an emotion, and our 5-dimensional matching algorithm ($S = 0.3T + 0.25M + 0.2A + 0.15J + 0.1E$) scores 25 tracks across tempo, mood, material affinity, tag similarity, and energy to find your perfect soundtrack. Every match becomes a collectible music card you can revisit — browsed through a gesture-controlled 3D carousel using just your hands, no touch required.

How we built it Backend: FastAPI + Google Gemini API for real-time object recognition and scene description generation Frontend: React 19 + Vite + Tailwind CSS 4 with Framer Motion animations Gesture System: MediaPipe HandLandmarker (WASM + GPU) running at 30fps, with custom algorithms for fist/palm/pinch detection, zone-based scrolling, and a charge-to-play mechanic 3D Carousel: CSS perspective + translateZ/rotateY transforms driven by a physics engine with friction, velocity interpolation, and boundary bounce Music Library: 25 royalty-free tracks spanning lofi, jazz, classical, hip-hop, DnB, R&B, and world music Challenges we ran into Bridging MediaPipe's imperative WASM API with React's declarative rendering model was the hardest part. We solved it using useRef for all mutable state in the animation loop, keeping React completely out of the 30fps detection cycle. Gesture debouncing (3-frame confirmation + 500ms cooldown) was essential to prevent false triggers.

Accomplishments that we're proud of The gesture system feels magical — pinch your fingers to charge and play music, open your palm to stop, move your hand to scroll through floating 3D cards with particle effects and glowing progress rings. It's an entirely touchless music browsing experience.

What we learned Performance-critical UI requires escaping React's render cycle. Direct DOM manipulation via refs, requestAnimationFrame throttling, and imperative particle systems taught us where the framework should end and raw browser APIs should begin.

What's next for LiveLifeLoop Firebase multiplayer — collaborative sessions where friends contribute objects to build shared playlists AI-generated music — replace the static library with real-time music generation based on detected materials AR overlay — visualize sound waves emanating from identified objects in camera view Format in Markdown, with LaTeX support for math. Inspiration What if every object around you was secretly a musical instrument? A ceramic mug hums differently from a metal railing. A wooden desk resonates unlike a glass window. We built LLL because we believe the world is already making music — it just needs someone to listen. Inspired by synesthesia and the physicality of sound, we wanted to blur the line between seeing and hearing.

What it does LLL (Life Live Loop) turns any photograph into a personalized soundtrack. Snap or import a photo, and Google Gemini identifies objects and their physical materials (wood, metal, glass, fabric, ceramic, plastic, organic). Select an emotion, and our 5-dimensional weighted scoring algorithm finds the optimal track:

$$S = 0.3,T + 0.25,M + 0.2,A + 0.15,J + 0.1,E$$

where $T$ = tempo fit, $M$ = mood match, $A$ = material affinity, $J$ = Jaccard tag similarity, and $E$ = energy distance. The result is saved as a collectible music card, browsable in a gesture-controlled 3D carousel — pinch to play, open palm to stop, wave to scroll — entirely touchless via MediaPipe hand tracking.

How we built it The backend runs FastAPI with Gemini API for object recognition and description generation, feeding into a custom music matcher that scores across a 25-track library spanning lofi, jazz, classical, hip-hop, drum-and-bass, R&B, and world music. The frontend is React 19 + Vite + Tailwind CSS 4 with Framer Motion for page transitions. The gesture system uses MediaPipe HandLandmarker (WASM + GPU delegate) at 30fps, with custom detection functions:

$$\text{isFist} = \frac{1}{4}\sum_{i \in {8,12,16,20}} d(\text{tip}_i, \text{wrist}) < 0.23$$

A physics engine drives the 3D card carousel with CSS perspective transforms ($\text{translateZ} = -|d|^{1.3} \times 50$), friction-based velocity decay ($v_{t+1} = v_t \times 0.92$), and boundary bounce.

Challenges we ran into MediaPipe's imperative WASM API clashes fundamentally with React's declarative model. Running hand detection at 30fps inside requestAnimationFrame while avoiding React re-renders required all mutable state to live in useRef — cursor position, detection timestamps, velocity — with React state only for UI-visible changes. Gesture debouncing (3-frame confirmation + 500ms cooldown) was critical to prevent phantom triggers from brief hand transitions.

Accomplishments that we're proud of The touchless music browsing experience feels genuinely magical. Floating your hand over a 3D carousel of glowing music cards, pinching to charge a progress ring with rising particles, and hearing your scene's soundtrack begin — it bridges the physical and digital in a way that feels like the future of interaction.

What we learned Performance-critical UI demands knowing where React should end and raw browser APIs should begin. We learned to treat the framework as a layout tool while using imperative DOM manipulation for anything running at 30+ fps. We also learned that a well-designed scoring function with just 5 dimensions can produce surprisingly musical results.

What's next for LiveLifeLoop Real-time AI music generation — replace the static library with models that compose music from detected material timbres Firebase multiplayer sessions — friends contribute objects from different locations to co-create playlists AR sound visualization — overlay animated sound waves on detected objects in the camera view Haptic feedback — vibration patterns matching the music's rhythm for a fully embodied experience