Sleep n' Sound

Saving busy parents up to 3 hours of sleep a night - one tale at a time

Comment

Sleep n' Sound: AI-Powered Bedtime Stories 🌙✨

📂 Pitch Deck

Check out our Pitch Deck to learn more about Sleep n' Sound and how we’re revolutionizing bedtime stories for busy parents:
👉 View Pitch Deck on Canva

🌟 Inspiration

As developers passionate about solving real-world problems, we were struck by the challenges parents face in balancing work, family, and sleep. According to research, 70% of mothers with a college or post-grad degree struggle to balance family life with work, and 7/10 parents lose an average of 3 hours of sleep a night in their child’s first year. This inspired us to create Sleep n' Sound, an AI-powered bedtime story app designed to save busy parents up to 3 hours of sleep a night—one tale at a time.

🛠️ What it does

Sleep n' Sound is a magical bedtime story app that:

  1. Generates Custom Stories: Using OpenAI’s GPT-4, it creates unique, engaging stories tailored to your child’s preferences.
  2. Brings Stories to Life: Each story is divided into 4 segments, with:
    • AI-Generated Images: Created using Fal.AI’s fal-ai/flux-pro/v1.1-ultra model.
    • Voice Narration: Powered by ElevenLabs’ text-to-speech model, allowing users to select a voice for storytelling.
  3. Interactive Playback: Stories are played back segment by segment, with new images and narration every 15 seconds.
  4. Save and Share: Users can save their favorite stories to their library or share them with others.
  5. Story Library:
    • Your Stories: A personal library of stories created by the user.
    • Public Stories: A community-driven library where users can browse and enjoy stories shared by others, fostering creativity and connection.

🏗️ How we built it

  • Planning: Before coding, we created workflow diagrams and wireframes using Excalidraw (check out our workflow here).
  • Frontend: Built using Lovable, with inspiration from 21st.dev for building polished UIs faster.
  • Backend: Powered by Supabase for database management and edge functions.
  • AI Integrations:
    • Story Generation: OpenAI’s GPT-4 for creating structured, JSON-formatted stories.
    • Image Generation: Fal.AI’s flux-pro/v1.1-ultra model for generating vivid, story-specific images.
    • Voice Narration: ElevenLabs’ text-to-speech API for lifelike voiceovers.
  • Version Control: Managed via GitHub for seamless collaboration and deployment.

🚧 Challenges we ran into

  1. Integrating Multiple AI Tools:
    • Combining Fal.AI, ElevenLabs, and OpenAI required careful handling of API responses, especially when merging base64-encoded audio and image URLs.
    • Lovable initially struggled to debug issues with Fal.AI integration, so we manually resolved them by diving into the documentation and refining our API calls.
  2. Supabase Limitations:
    • Lovable couldn’t directly inject SQL scripts into Supabase, so we had to manually handle database updates and edge functions since we could not get much help from discord to resolve this.
  3. Time Constraints:
    • We wanted to integrate ElevenLabs voice cloning, a conversational AI agent for character interactions, and unique ambient sound effects for each segment but ran out of time.
  4. Performance Concerns:
    • We noticed the end to end completion response times for GPT o4 turbo were initially quite long (45s+), which was a blocker for us given the target market and their typically shortened attention spans. By switching to GPT-4 we sped up edge function return times (15s) while maintaining high-quality story generation and avoiding the need to switch to a smaller model like GPT o1-mini.

🎉 Accomplishments that we're proud of

  • Successfully Integrated 3 AI Tools: We combined Fal.AI, ElevenLabs, and OpenAI to create a seamless, end-to-end storytelling experience.
  • Built an MVP in a Weekend: Despite the challenges, we delivered a functional, visually appealing app that solves a real problem for parents.
  • Overcame API Challenges: We manually debugged and resolved issues with Fal.AI and ElevenLabs integrations, ensuring smooth data flow between services.
  • Community-Driven Story Library: We implemented the Public Stories feature, allowing users to share and explore stories created by others.

📚 What we learned

  • Building with Multiple AI Tools: We learned how to connect and orchestrate multiple AI APIs to create a cohesive product just in a weekend (something we thought would take a long time)!
  • Debugging Complex Integrations: Working through API response formatting and edge cases taught us the importance of thorough testing and documentation.

🚀 What's next for Sleep n' Sound

  1. Voice Cloning: Integrate ElevenLabs’ voice cloning feature to allow parents to narrate stories in their own voice.
  2. Conversational AI Agent: Add a chatbot feature so kids can interact with characters from the story.
  3. Ambient Sound Effects: Enhance each story segment with unique background sounds (e.g., rain, forest, ocean) for a more immersive experience.

👥 Team Information

  • Sijan Poudel:

    • Came up with the idea.
    • Built the workflow diagram and wireframes for the app.
    • Built the frontend with Lovable and backend edge functions/database with Supabase.
    • Handled AI integrations.

  • Eddy:

    • Product flow schematics and workflows
    • Contributed to backend functions for story creation using Open AI
    • Automated story library updates
    • Utility assessment, market analysis and deck formation

✅ Compliance

We confirm that our project adheres to all hackathon rules and deadlines. All work was completed within the designated timeframe, and all third-party tools and APIs were used in compliance with their respective terms of service.

💡 Final Thoughts

Sleep n' Sound started as a simple idea to help parents save time and sleep better. Through hard work, creativity, we built a platform that not only tells stories but also brings them to life. With features like Public Stories, we’re fostering a community of storytellers and making bedtime a little easier—and a lot more magical—for families everywhere.

Built With

Share this project:

Updates