Drum machine DAW (Beatonic)

Ever wanted to make a drum beat? Edit samples to get that perfect tone with Gemini AI, arrange synths and tracks to make your dream drum beat in less than a minute.

Comment

Drum Machine DAW - Hackathon Submission

Inspiration

We wanted to democratize music production by creating an intuitive, AI-powered drum machine and synthesizer that anyone can use, regardless of their musical background. Traditional DAWs have steep learning curves, but we envisioned a tool where you could simply describe the sound you want in plain English and have AI generate it for you. Our goal was to combine the accessibility of modern AI with the creative power of professional audio tools.

What it does

Drum Machine DAW is a full-featured digital audio workstation with three main components:

  1. Drum Sequencer: A 16-step pattern-based drum machine where users can:

    • Create and layer multiple tracks with custom samples
    • Edit track names and assign Desmos-style colors for visual organization
    • Upload custom audio samples or create new ones
    • Control volume, tempo, and playback
    • Add or remove tracks on the fly

  2. Wave Editor: An AI-powered sample generator and editor that allows users to:

    • Generate custom audio samples using natural language (e.g., "deep bass kick for techno")
    • Fine-tune waveform parameters (sine, square, sawtooth, triangle, noise)
    • Adjust frequency, amplitude, duration, and ADSR envelope
    • Apply effects like normalize, reverse, fade in/out
    • Test samples with a virtual keyboard
    • Export samples directly to the drum sequencer

  3. AI Integration: Powered by Google Gemini 2.5 Flash, the system:

    • Interprets natural language descriptions of sounds
    • Generates creative and musically appropriate synthesis parameters
    • Provides contextual suggestions based on sound design principles
    • Maintains conversation history for iterative refinement

How we built it

Frontend:

  • React + TypeScript for type-safe UI development
  • Vite for fast development and building
  • Radix UI for accessible, composable components
  • Tailwind CSS for responsive, modern styling
  • Web Audio API for real-time audio synthesis and playback
  • Custom AudioEngine class for buffer management and sound generation

Backend:

  • Python Flask server for API endpoints
  • Google Gemini 2.5 Flash for AI-powered parameter generation
  • Google Cloud Secret Manager for secure API key storage
  • Docker containerization for consistent deployment
  • Cloud Run for scalable, serverless hosting

Infrastructure:

  • Docker Compose for local development
  • Google Cloud Build for CI/CD
  • Cloud Run for production deployment
  • Environment-based configuration for dev/prod parity

Audio Processing:

  • Custom ADSR envelope implementation
  • Real-time waveform visualization with canvas rendering
  • Oscilloscope-style animation during playback
  • Sample editing tools (normalize, reverse, fades)
  • Multi-track mixing and playback synchronization

Challenges we ran into

  1. Web Audio API Complexity: Managing AudioContext lifecycle, buffer creation, and real-time playback required deep understanding of browser audio constraints and timing issues.

  2. AI Response Parsing: Google Gemini returns plain text, not structured JSON. We had to implement robust parsing with cleanup logic to handle markdown code blocks and ensure valid JSON extraction.

  3. State Management: Synchronizing audio buffers, playback state, and UI updates across multiple components required careful state architecture and React hooks management.

  4. Secret Manager Integration: Setting up Google Cloud Secret Manager with proper IAM permissions and environment variables for both local development and Cloud Run deployment was tricky.

  5. Waveform Animation: Creating smooth, real-time waveform visualization that works with both generated synthesis and uploaded samples required multiple iterations and performance optimization.

  6. Cross-Component Communication: Implementing the "Edit Waveform" feature that passes audio buffers between the drum machine and wave editor while maintaining proper state required careful prop drilling and callback management.

Accomplishments that we're proud of

  1. AI Sound Design: Successfully integrated Google Gemini to generate musically appropriate synthesis parameters from natural language descriptions, making professional sound design accessible to everyone.

  2. Seamless Workflow: Created an intuitive workflow where users can generate samples in the Wave Editor and instantly add them to tracks in the Drum Machine with a single click.

  3. Real-time Audio Visualization: Built a custom oscilloscope-style waveform display that animates in sync with audio playback, providing visual feedback during sound creation.

  4. Production-Ready Deployment: Successfully deployed to Google Cloud Run with proper secret management, making the app accessible online with enterprise-grade security.

  5. Polished UI/UX: Designed a modern, responsive interface with Desmos-inspired color coding, editable track names, and intuitive controls that feel professional yet approachable.

  6. Flexible Architecture: Built a modular system where the drum sequencer, wave editor, and synthesizer can work independently or in perfect harmony.

What we learned

  1. Web Audio API Mastery: Gained deep knowledge of browser audio capabilities, AudioContext management, buffer manipulation, and real-time synthesis.

  2. AI Prompt Engineering: Learned how to craft effective prompts for generative AI that produce consistent, musically appropriate results while handling edge cases and parsing challenges.

  3. Cloud Architecture: Mastered Google Cloud services including Cloud Run, Secret Manager, Cloud Build, and proper IAM configuration for secure, scalable deployment.

  4. Audio Processing Theory: Developed understanding of ADSR envelopes, waveform synthesis, frequency manipulation, and audio effects implementation.

  5. React Patterns: Refined skills in complex state management, prop drilling solutions, custom hooks, and performance optimization for real-time applications.

  6. Full-Stack Integration: Learned to seamlessly connect a Flask backend with a React frontend, handling CORS, environment variables, and deployment configurations.

What's next for Drum Machine DAW

  1. Pattern Sequencing: Implement song mode where users can chain multiple patterns together to create full compositions with arrangement tools.

  2. More AI Features:

    • AI-powered drum pattern generation based on genre and mood
    • Intelligent mixing suggestions
    • Automatic mastering with AI

  3. Export and Sharing:

    • Export patterns as MIDI files
    • Render full tracks to WAV/MP3
    • Cloud storage for saving and sharing projects
    • Collaboration features for multi-user sessions

  4. Advanced Audio Effects:

    • Add compression, EQ, distortion, and modulation effects
    • VST plugin support
    • Real-time effect automation

  5. Sample Library:

    • Built-in library of professionally designed samples
    • Community-contributed sample marketplace
    • AI-generated sample packs based on genre

  6. Mobile Support:

    • Responsive design improvements for tablet/phone
    • Touch-optimized controls
    • Native mobile apps with offline capabilities

  7. Advanced Synthesis:

    • FM synthesis
    • Granular synthesis
    • Wavetable synthesis
    • Multi-oscillator modulation routing

  8. Monetization:

    • Freemium model with advanced features
    • Professional tier with unlimited tracks and exports
    • Marketplace for samples and presets

Built With

Share this project:

Updates