How to build a YouTube-like website with AWS?

How does video processing work?

First things first, you have to get your videos ready in a particular video format because the web does not support the wide variety of formats available in the wild.

Let’s say you shoot your video at 4K 60 FPS. So the video takes over 4 Gigabytes per 10 minutes. Let’s say your videos are made in .mp4 format. Well, the first step is done.

Uploading to cloud

Amazon Web Services offers a lot of services, but right now, we are interested in S3. S3 bucket provides an immense amount of data storage capability. So S3 can handle a lot of data transactions with the process going haywire. So we have found out a server to store our data.

Services offered by AWS (thinkwik)

There is no restriction to stick to AWS S3. There are many more cloud providers that offer this feature.
So once we have our signed S3 URL, you can upload your video data to the S3 bucket.

Working of SQS

Amazon SQS stands for Simple Queue Service‌. SQS is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and server less applications. SQS is preferred because it is highly reliable and scalable for receiving tons of data every second.

Working of Cloud Watch

Amazon Cloud Watch helps us set certain alarms. Let’s say we keep an alarm when our SQS query is greater than 0 queries.

So now the alarm is responsible for launching the AWS EC2 machine with the help of Amazon AGS (Auto Scaling Groups), and the EC2 launched using a snapshot/template. Therefore when a new query is registered in the queue, the Cloud Watch Alarm triggers and fires up an EC2 machine.

Now when the EC2 machine has launched, it has two jobs.

First job of Cloud Watch Alarm

When queue > 0 in the Amazon SQS

The first job is to read the SQS queue’s message and start processing the first message in the queue. The first message contains the location file for the video file from the S3. Now the Job of the EC2 Machine is to process the video file. The video is processed using FFMPEG convert into three resolutions, say 1090,720,480 and upload them back to the Production S3 buckets and update the database.

Now, we use Amazon Cloud Front as the CDN to securely access the video files from the S3 bucket and deliver them.

Second job of Cloud Watch Alarm

When queue = 0 in the Amazon SQS

So now the queue is zero. The Amazon ASG tries to downscale the EC2’s as there are no videos to process. Hey, but what if ASG tries to downscale when a video is processing? That’s when locking and unlocking the EC2 machine can prevent descaling when a video instance is running.

Cost to make video delivery infrastructure

You may be thinking that this architecture may cost heavily, but it’s not true. S3 does not charge for ingress & first download of the data & the first upload from the EC2 instance, so technically, there is no limit for the data you upload to your S3 bucket. Now comes the paying part. Cloud Front charges $0.085 per GB, and S3 charges 0.023 for the data transfer after that.

Data streaming and storage cost for 10 GB = $1.08

Implementing this level architecture will be a little complex compared to the Amazon media package, but it will be 20 times cheaper than its price.

Hope this tutorial gave you an abstract idea about how to create a scalable video architecture.

Watch the full Video on Codedamn YouTube Channel.