Shark Video-on-Demand

You know how you've always had to use sub-par platforms to watch shark videos, well I have built a platform dedicated entirely to stream shark videos with the added security benefit as well.

Comment

Inspiration

Well my biggest inspiration was the fact that good SaaS based video streaming services were hard to maintain and deploy which led me to work on a Video-on-Demand platform that could extensively scale and support proper security like OAuth2.0 and Cloudfront Key Pairs

What it does

  • The admins can upload MP4 videos into the S3 bucket which get converted into HLS with multiple supported resolutions and thumbnails with a MediaConvert job
  • The admins can invite users through the Cognito dashboard from where they receive an email about the invitation to the platform and temporary passwords which have to be changed on the first login
  • The users can log in to the platform and get a list of all videos and resolutions available on the platform
  • They can generate CloudFront key pairs and start streaming the videos through an HLS player

How we built it

  • Initially I started off by trying to automate the MediaConvert job so that it gets triggered on every upload to the S3 bucket. Currently, I use a lambda function which job config JSON that gets triggered on mp4 uploads to S3
  • Next we came to the challenge of storing the video metadata into DynamoDB after the MediaConvert job was finished, since MediaConvert runs asynchronously, I used EventBridge to trigger an event firing up the Lambda function as soon as the MediaConvert job was over. The event had the metadata for converted video location and thumbnails which we stored into DynamoDB.
  • Now I started with the user interaction part, firstly I create a Cognito user pool and linked it with SES, after that I created the API Gateway.
  • On the API Gateway, we created 2 endpoints, one for Reading (GET) and one for Updating (PATCH). Both of them triggering routes-read and routes-ud lambda functions respectively, both the functions helped in performing their respective operations with DynamoDB.
  • All API GW endpoints used the Cognito authorizer with the videos/read OAuth scope.
  • I also created an additional gateway endpoint called /keys which triggered the cookie-gen lambda function, generating Pre-Signed Cloudfront Cookies for accessing the videos and thumbnails
  • After storing those cookies you could access the videos with the plain CloudFront URL

Challenges we ran into

  • Initially we used a different approach for storing video metadata after the MediaConvert job, which was to publish a Cloudwatch Event which triggered an SNS notification which in turn triggered the lambda function, but that seemed inefficient, after some research, I found that EventBridge cuts the path for us and saves down on costs as well.
  • CORS was one of the biggest issues with CloudFront while trying to keep security intact due to a major lack of documentation
  • Initially I was under the impression that pre-signed CloudFront URLs would work fine but soon came to realize that they won't because HLS breaks down videos into segments with multiple files instead of a single file like MP4, while looking into solutions I came across pre-signed Cookies and decided to implement them, although I knew nothing about cookies so it took up a lot of my time
  • This is my first time using a lot of AWS services like API Gateway, event-bridge, Cognito, MediaConvert which led me to spend a lot of time on unclear documentation and trial and error all day.

Accomplishments that we're proud of

  • The scalability, I performed a load test on the API using locust and it performed extremely well till the point where my own network bottlenecked
  • An absolutely serverless solution, this application backend uses an entirely end-to-end serverless approach as you can see in the architecture diagram
  • Ideally small microservices keeping the codebase absolutely minimal.

What we learned

  • The blend of AWS services which I thought I would never get to work with like MediaConvert.
  • Keeping everything minimal from code to delivery.

What's next for Shark Video-on-Demand

  • I plan to turn this into a fully running SaaS video-on-demand platform and expand the territory for more than Shark and blahaj videos :)

Built With

Share this project:

Updates