Timbre

Timbre is an AI agent that analyzes the emotion in your video to automatically create custom soundtracks that capture its feeling.

Comment

Inspiration

Current video soundtracking methods are often generic or complex. We aimed to use AI to automatically create custom music based on a video's emotional content.

What it does

Timbre is an AI agent that analyzes the emotion and content of uploaded videos to automatically generate fitting custom soundtracks.

How we built it

We used a serverless AWS architecture:

  • Frontend: Next.js with AWS Amplify
  • Authentication: Amazon Cognito
  • API Layer: Amazon API Gateway
  • Backend Logic: AWS Lambda functions
  • AI Agent Core: AWS Strands & Amazon Bedrock (ReAct pattern)
  • Analysis Tools: Lambda wrappers for Amazon Rekognition & Transcribe
  • Orchestration: AWS Step Functions
  • Storage: Amazon S3 & Amazon DynamoDB
  • Deployment: AWS SAM (Serverless Application Model)

Challenges we ran into

  • Configuring AWS service integrations and permissions (IAM, API Gateway, Cognito).
  • Debugging CORS issues.
  • Managing the asynchronous serverless workflow.
  • Refining the agent's prompt engineering.

Accomplishments that we're proud of

  • Built a secure, end-to-end serverless pipeline.
  • Successfully implemented an AI agent using Strands/Bedrock for intelligent analysis.
  • Integrated multiple core AWS services via infrastructure-as-code (AWS SAM).

What we learned

  • Practical experience with AWS serverless services and SAM.
  • AI agent design (Strands/ReAct) and LLM integration (Bedrock).
  • Secure web application patterns (Cognito, S3 presigned URLs).
  • Asynchronous processing and cloud debugging techniques.

What's next for Timbre

  • Implement real analysis and music generation tools.
  • Replace status polling with WebSockets for real-time updates.
  • Refine the user feedback loop for music regeneration.
  • Enhance the UI/UX for analysis visualization and user control.

Built With

Share this project:

Updates