MusicBucket

Photos

What it does

Invite your friends, family, or strangers to a karaoke room, upload a song that will use a machine learning model to extract voice from background music that you can choose to sing to. Practice to see if you're in tune with only the artists voice or do a solo with only background music of the song.
Access a event's camera streams to see different angles of the performance from your phone.

It's a completely Serverless App

AWS Machine Learning Models Utilized:

Quantiphi - Source Separation

Quantiphi - Barcode/QR-code Scanner

Sample input:

Audio Output: A .out file that needs to be unzipped

Scan Sample Input: A image/jpeg

Ticket Scan Output: JSON

"{"09161b30-bdcb-4f03-afac-4e35ac9aba6e": "QRCODE"}"

How I called the Sagemaker Models

AWS Cognito credentials used to generate presignedurls.
Audio file is uploaded to s3 bucket.
Once the Audio File is uploaded it triggers an s3 bucket function.
Trigger function invokes Sagemaker endpoint, stores s3 bucket urls/metadata into RDS. Function then grabs websocket connection IDs from DynamoDB and returns JSON through API Gateway websocket connection back to user.

Music Room

Built with Twilio WebRTC integrated with Lambda function for generating tokens. User first invites friends by making a request through API Gateway(RestAPI) to a lambda function to generate tokens, and those tokens are sent to the invited users in json by going through their respective API Gateway(Websockets) connections which generates a notification which they can click on, and it joins the music room.
The Chat is built with API Gateway Websockets and connections managed with DynamoDB.
Lyrics created by placing the vocal track returned by source seperation model into AWS Transcribe.

MusicBucket Architecture