Hey again, Emmanuel here. Another week of surviving the unstable field of software development. According to advice I see online from many senior devs, one of the best skills you can build right now is understanding system design. Learning how the components of an app fit together, analyzing trade-offs, and justifying architectural decisions are skills I didn’t possess. However, like with any skill, learning and practice are all you need to get “good.”

That’s why I started learning the basics of system design. I’m not talking about designing software at the level of platforms like Netflix or Spotify. No, I’m starting with something far more manageable for beginners like me: a URL Shortener. So, what exactly is system design?

So, What Exactly Is System Design?

System design is the process of planning the architecture of a software system. It’s essentially a kind of blueprint that defines the various components of a system, their relationships, and how data flows between them. To keep it fairly brief, I’ll use an analogy of a building. In this scenario, coding is like laying individual bricks or placing the wiring. System design, however, is like planning how many floors the building has, where the plumbing goes, and how it can support people without collapsing.

When I say “components,” I mean servers, databases, and stuff like that. Before you even start designing anything, you need to understand the requirements of the system. These usually fall into two categories: functional and non-functional requirements. Functional requirements are the essentials of the system, as they describe what it must do to function. On the other hand, the non-functional requirements are how well the system performs. For non-functional requirements, we consider factors like speed, availability, reliability, and security.

System design is typically split into two main layers: High-Level Design (HLD) and Low-Level Design (LLD). As the name hints, the HLD focuses on the high-level architecture. It maps out the connections of components like web servers, CDNs, databases, and client-facing applications. On the other hand, the LLD focuses on the smaller details. Stuff like the logic of individual modules, classes, and data structures within particular services. So, you usually start with the HLD, then move on to the LLD when designing a system.

Now, I initially thought system design was something I had to worry about when creating large production-grade systems. That’s not the case, though, as even small apps require architectural decisions (though we often overlook them). And what better way to understand this than trying to design a system ourselves?

Our First Problem: A URL Shortener

As I mentioned in the intro, I plan to design a URL shortener. Why? Because it’s a fairly common example with a straightforward design, and it also stops me from going overboard. If you don’t know what a URL shortener is, it’s simply a system that takes a long URL and produces a shorter link that redirects visitors to the original destination. For instance, I might submit a link like “example.com/articles/learning-system-design” and receive a shortened link like “short.ly/aB92x.”

Firstly, we need to understand the core functional requirements of our URL shortener. It needs to:

  • Accept a URL
  • Generate a unique short code
  • Map the short link to the original long URL in the database
  • Redirect visitors when they access the shortened link

If we’re talking about the functional requirements, then these are enough for our basic URL shortener. You see that it clearly describes what the system must do. For non-functional requirements, we might consider the speed of redirects, handling database failures, and making the application handle increasing traffic.

Note that I won’t be considering extra stuff like user accounts, analytics, or link expiration to keep this design simple. Don’t judge me :(

Now that we have a good idea of the requirements of our URL shortener, we can finally move on to actually designing the system.

How Would We Design It?

Our system requirements are now well-defined, so we can actually start the design process. We need to identify the major components, which act as the building blocks of our entire system. For such a simple URL shortener, I decided to split it into three components:

  • Client: A client-facing component where you can submit URLs and access shortened links
  • Backend/Server: Handles requests, generates short codes, and redirects
  • Database: Stores the mapping of the short codes and original long URLs

With the HLD components mapped out, we need to understand the URL creation flow. I’ve summed it up in a few steps:

  • A user submits a URL in the client interface
  • The long URL is sent to the backend through an HTTP request
  • Next, the backend checks the submitted value to ensure it's a valid URL
  • The backend server generates a short code to represent the original URL (e.g., "aB92x")
  • The short code and original URL are stored in the database
  • Finally, the backend returns the shortened URL (e.g., "short.ly/aB92x")

Breaking the URL creation process down like this makes it quite easy to understand and implement. Besides that, the redirection process is the next major part of our URL shortener:

  • The visitor opens the shortened URL (let’s say “short.ly/aB92x”)
  • The backend extracts the code “aB92x” from the request
  • The database is queried for the associated long URL
  • If a matching record is found, the original URL is retrieved
  • If a record isn’t available, an HTTP 404 Not Found response is sent
  • The visitor is finally redirected to the original URL, with a 302 status code

And just like that, you get a decent overview of the whole system without needing to write even a single line of code. Now, having this design gives you a clearer starting point when implementation begins.

What Could Go Wrong?

One thing to keep in mind about the design of our URL shortener is that it’s frankly as simple as it gets. However, system design isn’t just about finding a solution that works. It also requires you to understand where things break and how to fix them as requirements evolve. If our URL shortener suddenly has to serve millions of users, several things could go wrong.

Let’s say you process two different original URLs, and somehow, our underlying code generates the same short code for both. Obviously, that is a major problem, as it gets two different links to redirect visitors to. To fix this, we would need to handle uniqueness in our short code generation. We need sufficient collision handling to check for duplicated codes and regenerate until a unique one is obtained. You can also add a UNIQUE constraint to the database itself to avoid storing duplicate values under the short code column/attribute.

Here’s another problem to consider. We get thousands of visitors accessing our shortened links, which leads to many requests to our backend server and database. Due to all this high traffic, our system starts struggling to process requests. For this, we may decide to scale out our system by adding more server instances to handle requests. Scaling out might mean adding a load balancer to distribute requests among our multiple servers. To avoid hitting our database on every request, we can add a cache to our system to store frequently accessed URL mappings. Only when the cache doesn’t contain the requested mapping do we query our database.

Lastly, we need to consider database failures. What happens when our database fails and can’t return our mappings? That’s why you need to have database backups to help restore data after data loss, corruption, or certain failures. We can also create database replicas to maintain copies of data, and they can potentially take over when the primary database becomes unavailable.

One thing I can’t stress enough is that these changes are for when your URL shortener needs to handle more users. Don’t try to add all these for a shortener that only serves a few people. It’s best to identify performance issues as they occur and fix them accordingly.

What I've Learned So Far

So, what did I learn after all of this? Well, firstly, system design isn’t as clear-cut as I initially thought. It’s less about finding the perfect approach and more about understanding trade-offs and requirements. You get to consider where things break and improvements that can be made while still avoiding unnecessary complexity.

Personally, it also helps in keeping my plans grounded. I usually start coding without much thought and then run into different unplanned features. Eventually, I abandon the project because I get overwhelmed by the numerous features I could build. Now, I try my best to design the system before anything else.

Should you learn system design? Yes, I wholeheartedly recommend doing that. Even if AI makes building things quite easy, understanding how things work together, making decisions, and recognizing trade-offs are still important.