URL Shortener? Use S3!

URL Shortener? Use S3! 

Create a URL Shortener Service is the first question most people attempt, when studying system design! and the answer usually involves load balancers, servers, NoSQL databases and more, but I created a URL shortener service without creating a single server, load balancer or database, just an S3 bucket and some clever tricks! 


Before starting with the design, lets first of all define what we are trying to create, clarify the functional and non-functional requirements!  

Functional Requirements: 

  • Allow users to generate custom short links(Aliases) for longer URLs.
  • When users access a short link, redirect to the original link.
  • Links should expire automatically after a period of non-use!

 Non-functional Requirements: 

  • The system should be highly available. This is required because, if our service is down, all the URL redirections will start failing.
  • URL redirection should happen in real-time with minimal latency.

The Architecture: 

URL shortening Service Architecture Diagram, with layers of servers DB and backgroud services

This solution is pretty amazing! it meets all of the functional and non-functional requirements, but configuring and managing all of these components and resources is a nightmare, and even small errors could lead to downtime or poor service! Not to mention the additional cost of having servers, cache and databases up and running at all times! 

The S3 Implementation: 

We can achieve the same level of performance, or better by offloading a lot of this architecture to AWS! AWS offers some amazing tools and services all of which are designed to make the lives of system designers and architects easier! So why not leverage it for this task? 

Idea: 

The core of the idea is to use S3 buckets and object metadata to serve redirects quickly and efficiently! s3 objects can have metadata fields such as `x-amz-website-redirect-location`! This specific metadata field when served, results in a 301 permanently moved response and the requester is redirected to the stored URL! This is perfect for our task!  

The Services we are going to use:

Method: 

  1. First of all, create an S3 bucket!
  2. the objects within the bucket need to be publicly accessible, so disable block all public access! Also create a bucket policy that allows getObject on all objects in the bucket!
  3. Next we need to host a static website using the bucket, you can use an empty index.html file to host a static website for now!
  4. Use AWS cloudshell to add an object to the bucket, this will be out test-object! use a command like: 
  5. Now test if visiting '<https://s3_static_website_url/xyz>' redirects you to youtube
  6. Now you can create a CNAME record in your DNS Zones (Route53 or elsewhere)! Something like this: 
    `<domain_name>  CNAME  <s3_bucket_static_website_url>`
  7. Now whenever you visit `http://<domain_name>/key` you will get redirected to the corresponding URL immediately!
  8. Create a simple web-application for the creation of URLs using something like Next.js! This application only needs to create objects in the S3 bucket!
  9. Deploy the application on vercel to get a fully serverless URL shortener!

Optional Improvements: 

  • Create a cloud front distribution on the S3 bucket and modify the CNAME record to point to the cloudFront URL! this will allow the redirects to happen with extremely low latency (10-20ms)! 
  • Use ACM with cloudfront to allow for HTTPS redirects! 

The benefits: 

  • S3 buckets provide tremendous throughput and have extremely high object durability and with replication provide very high uptime! 
  • S3 buckets come with 5 GB of free storage, since each object is empty (0 bytes) and only metadata is set, each redirect should come in at less than 100 bytes! which means you can store tens of millions of redirects for practically free! 
  • Use of cloud front minimises reads to s3 bucket(which are charged)! It also allows for very low latency! 
  • You can utilise s3 lifecycle policies to expire records after a period of non-use!
  • No need to have constantly running servers, load balancers, cache or databases! Everything is handled by AWS! 

Try it out: 

So I have created a simple URL shortener using this approach! You can try it out at:  

You can also checkout my portfolio: here

Comments

Post a Comment