Lullaby You

Transform loved ones voices into soothing lullabies. Advanced AI technology creates natural-sounding lullabies with your voice. Create lasting memories with personalised lullabies for your loved ones.

Comment

Inspiration

In our hyper-connected yet increasingly disconnected world, I wanted to create something that bridges the gap between technology and genuine human intimacy. The inspiration came from realizing that while we have apps for everything, we're losing those precious, irreplaceable moments of human connection—like a parent's voice singing their child to sleep. LullaBy You transforms the most personal thing we have, our voice, into a timeless gift of comfort and love. It's about using AI not to replace human connection, but to preserve and amplify it, ensuring that even when we can't be physically present, our love and care can still wrap around those we cherish most.

What it does

LullaBy You transforms any recorded voice into personalized lullabies using advanced voice cloning technology. Users can record their voice or upload audio files, select from emotional templates (Sweet Dreams, Miss You, Brave Dreams, Calm Down), and our system clones their voice, generates melodic content, and mixes it with backing tracks to create custom lullabies. The final product is a downloadable audio file that preserves the emotional connection between loved ones, perfect for parents, grandparents, or anyone wanting to comfort children with their own voice even when they can't be physically present.

How we built it

We built LullaBy You using Next.js 15 with TypeScript for a robust, type-safe foundation. The frontend leverages Shadcn UI components with a beautiful purple-pink gradient design system. Bolt.new's prompt-based website generation capabilities were instrumental in rapidly prototyping our landing page components and smaller frontend tasks, allowing us to focus on the core audio processing functionality.

The voice cloning and audio generation heavily relies on ElevenLabs API, which handles voice model creation, speech synthesis, and melody generation. We integrated Tone.js for in-browser audio processing, mixing, and effects. RecordRTC manages voice recording with real-time waveform visualization, while our custom audio pipeline handles the complete flow from voice input to final lullaby output. We're eagerly awaiting ElevenLabs API V3 release to explore enhanced voice modulation capabilities that could make our lullabies even more soothing and natural.

Challenges we ran into

  • Audio Processing Complexity: Implementing real-time audio processing in the browser while maintaining quality and performance
    • Voice Cloning Quality: Achieving natural-sounding lullabies from short voice samples while working within ElevenLabs' current API limitations
    • Cross-browser Compatibility: Ensuring consistent audio recording and playback across different browsers and devices
    • File Size Management: Balancing audio quality with reasonable file sizes for web delivery
    • Emotional Authenticity: Making AI-generated melodies feel genuinely soothing and lullaby-like rather than robotic

Accomplishments that we're proud of

  • Complete End-to-End Pipeline: Successfully built a working system from voice recording to final lullaby generation
    • Intuitive User Experience: Created a seamless 4-step wizard that makes advanced AI technology accessible to anyone
    • Real-time Audio Visualization: Implemented beautiful waveform displays and processing animations that enhance user engagement
    • Robust Error Handling: Built comprehensive fallbacks and user feedback systems for various failure scenarios
    • Mobile-Responsive Design: Delivered a polished experience across all device types in just 5 days

What we learned

  • Browser Audio APIs: Mastered Web Audio API, RecordRTC, and Tone.js for complex audio processing
    • Rapid Prototyping: Learned how tools like Bolt.new can accelerate development of UI components and landing pages
    • User-Centric Design: Discovered the importance of emotional design when building products that handle deeply personal content like family voices
    • Performance Optimization: Understood the challenges of processing large audio files in real-time web applications

What's next for Lullaby You

Enhanced Voice Modulation: Our primary focus is improving the voice modulation system to make generated lullabies sound more naturally soothing and less AI-generated. We're exploring advanced audio processing techniques and eagerly awaiting ElevenLabs V3 API access for better voice control.

AI-Generated Content Improvements: We plan to enhance our AI-generated melodies and lyrics to be more contextually appropriate for lullabies, incorporating child psychology research and traditional lullaby patterns.

Advanced Templates: Expanding beyond our current 4 templates to include culturally diverse lullaby styles, seasonal themes, and age-appropriate variations.

Smart Personalization: Implementing machine learning to analyze voice characteristics and automatically optimize template selection and audio processing parameters for each user.

Extended Audio Features: Adding features like multiple voice layers, environmental sounds, and adaptive volume that responds to ambient noise levels.

Built With

  • bolt
  • elevenlabs
  • nextjs
  • shadcn
  • supabase
  • vercel
Share this project:

Updates