Session — Real-Time AI Music Generation

Session is a mobile app that lets you generate and shape music in real time using Gemini 3 and Gemini Lyria. Think of it as a DJ controller for AI: you describe a vibe, Gemini 3 turns it into prompts, and Lyria plays a continuous, evolving stream that reacts to your choices as you go. The app is built with React Native (Expo) and uses two Gemini models together. Gemini 3 (gemini-3-pro-preview) acts as the creative director: in “Create” mode you type a vibe (e.g. “rainy night in Tokyo”) and it returns 15 music prompts tuned for Lyria. On any style—Hip Hop, EDM, Rock, Drum Machine—you can tap the AI button to get a new set of prompts. Gemini Lyria (lyria-realtime-exp) is the performer: it takes those prompts and your mix settings and streams AI-generated music in real time with no pre-recorded tracks. You shape the sound with 16 prompt knobs (weights 0–2), plus live controls for tempo (70–160 BPM), key, reverb, echo, and a lowpass filter. The stream updates as you change anything. A waveform and ambient halos give visual feedback. Everything runs in real time: describe a vibe, adjust the mix, press play, and Lyria keeps generating. Session is a single place to explore AI music with Gemini 3 and Lyria in one flow.

Powered by Gemini 3 + Gemini Lyria

Session uses two Gemini models working together via the official @google/genai SDK:

Gemini 3 (gemini-3-pro-preview) — The creative director. Understands musical genres and natural language vibes, and generates optimized prompts for Lyria.
Gemini Lyria (lyria-realtime-exp) — The performer. Takes weighted prompts and generates real-time streaming music.

How Session uses Gemini 3

Feature	Gemini Model	How It's Used	Code Location
Vibe-to-Prompts	`gemini-3-pro-preview`	User describes a vibe (e.g., "rainy night in Tokyo") → Gemini 3 generates 15 musically-rich prompts optimized for Lyria	`src/services/GeminiPromptGenerator.ts`
AI Style Refresh	`gemini-3-pro-preview`	Tap the AI button on any genre → Gemini 3 generates fresh, creative sub-genre prompts for that style	`src/services/GeminiPromptGenerator.ts`
Prompt Generation	`gemini-3-pro-preview`	`ai.models.generateContent({ model: "gemini-3-pro-preview" })` interprets musical context and returns structured JSON prompt arrays	`src/services/GeminiPromptGenerator.ts`

How Session uses Gemini Lyria

Feature	Gemini Model	How It's Used	Code Location
Real-time music streaming	`lyria-realtime-exp`	`ai.live.music.connect()` opens a persistent streaming session that generates audio in real time	`src/EngineBridge.tsx`
Weighted prompt mixing	`lyria-realtime-exp`	`session.setWeightedPrompts()` sends the Gemini 3-generated prompts with user-adjusted weights to shape the music	`src/EngineBridge.tsx`
Live tempo & config	`lyria-realtime-exp`	`session.setMusicGenerationConfig()` adjusts BPM and temperature on the fly	`src/EngineBridge.tsx`
Audio streaming	`lyria-realtime-exp`	Receives base64-encoded PCM16 audio chunks (48kHz stereo) for gapless playback	`src/EngineBridge.tsx`

Gemini Pipeline Flow

User describes a vibe: "dark underground club"
                    ↓
    ┌─────────────────────────────────┐
    │  Gemini 3 (gemini-3-pro-preview)│  ← ai.models.generateContent()
    │  Interprets vibe, generates 15  │
    │  musical prompts as JSON        │
    └───────────────┬─────────────────┘
                    ↓
    User adjusts prompt weights via knobs (0–2)
                    ↓
    ┌─────────────────────────────────┐
    │  Lyria (lyria-realtime-exp)     │  ← ai.live.music.connect()
    │  Streams real-time AI music     │
    │  based on weighted prompts      │
    └───────────────┬─────────────────┘
                    ↓
    Web Audio API → effects (reverb, echo, filter) → speakers

Gemini 3 writes the music prompts. Lyria performs them. Every note the user hears is AI-generated — there is no pre-recorded audio.

Features

Gemini 3 Vibe-to-Music — Describe any vibe in natural language, Gemini 3 generates the prompts, Lyria plays them
AI Prompt Refresh — Tap the AI button to have Gemini 3 generate fresh prompts for any genre
Real-time AI music generation — Streams audio from Gemini Lyria with gapless playback
4 genre styles + Create — Hip Hop, EDM, Rock, Drum Machine, plus a freeform "Create" mode powered by Gemini 3
Weighted prompt mixing — Dial prompts up or down (0–2) to blend styles
Live controls — Adjust tempo (70–160 BPM), musical key, reverb, echo, and lowpass filter in real time
Session recording — Record your AI-generated sessions as WAV files
Visual feedback — Ambient color halos that pulse with the beat, waveform visualization, tempo ticker
Background playback — Music keeps playing when you minimize the app

Architecture

┌──────────────────────────────────────────────────────┐
│                  React Native UI                      │
│  (Expo Router, Reanimated, SVG, Skia)                │
│                                                       │
│  ┌─────────────────────────────────────────────────┐ │
│  │  GeminiPromptGenerator (gemini-3-pro-preview)   │ │
│  │  Generates prompts from vibes & styles          │ │
│  └──────────────────────┬──────────────────────────┘ │
│                         ↓                             │
│  ┌─────────┐  ┌────────┐  ┌──────────────┐          │
│  │PromptGrid│  │Mixer   │  │PlayPauseBtn  │          │
│  │(16 knobs)│  │Controls│  │RecordBtn     │          │
│  └────┬─────┘  └───┬────┘  └──────┬───────┘          │
│       │            │               │                  │
│       ▼            ▼               ▼                  │
│  ┌─────────────────────────────────────────────────┐ │
│  │          EngineBridge (WebView)                  │ │
│  │  ┌───────────────────────────────────────────┐  │ │
│  │  │  @google/genai SDK (ESM import)           │  │ │
│  │  │  ↕ Lyria RealTime (lyria-realtime-exp)    │  │ │
│  │  │  ↕ Web Audio API (effects chain)          │  │ │
│  │  └───────────────────────────────────────────┘  │ │
│  └─────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────┘

Tech Stack

Layer	Technology
AI — Prompts	Gemini 3 (`gemini-3-pro-preview`) via `@google/genai` SDK
AI — Music	Gemini Lyria (`lyria-realtime-exp`) via `@google/genai` SDK (`v1alpha`)
Framework	Expo SDK 54, React Native 0.81
Audio	Web Audio API (in WebView), expo-av
Graphics	react-native-svg, @shopify/react-native-skia
Animations	react-native-reanimated
Routing	expo-router
Recording	ScriptProcessorNode → WAV encoding

Getting Started

Prerequisites

Node.js 18+
Expo CLI (npx expo)
iOS Simulator or physical device (Xcode required for iOS)
A Google Gemini API key with access to the Lyria model

Setup

Clone the repository:

git clone <repo-url>
cd session/session

Install dependencies:

npm install

Create a .env file in the session/ directory:

EXPO_PUBLIC_API_KEY=your_gemini_api_key_here

Or start the dev server:

npx expo start

 npm start

How It Works

Select a style or describe a vibe — Choose from Hip Hop, EDM, Rock, Drum Machine, or tap "Create" and describe any vibe in natural language
Gemini 3 generates prompts — For "Create" mode, Gemini 3 interprets your vibe and generates 15 musical prompts. For any style, tap the AI button to have Gemini 3 refresh prompts
Shape the sound — Each style has 16 prompt knobs. Turn knobs up to add that flavor to the mix
Press play — The app connects to Gemini Lyria and starts streaming AI-generated music based on your weighted prompts
Mix in real time — Adjust prompts, tempo, key, and effects while the music plays. Lyria adapts the stream on the fly
Record — Hit the record button to capture your session as a WAV file

Built With

Updates

D P started this project — Feb 08, 2026 01:39 PM EST

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.