High Level Design

These are used to send data over IP networks.

• Impractical in real distributed systems because you really need partition tolerance, as network partitions are unavoidable.
• Good for single node dbs on a single machine

• System prefers correctness over uptime
• The nodes may refuse requests untill consistency is restored.
• Examples: HBase, MongoDB

• System prefers availability.
• Preferred when you don't always need to show the latest data. For example, you don't really need to see the most accurate Instagram follower count. It will eventually be updated so it's fine.
• The system is always available.
• I really like this one.
• Eg: Cassandra, DynamoDB, CouchDB, DNS

• A load balancer distributes requests evenly across different servers.
• So if there are a lot of requests, a load balancer will send different requests to different servers evenly, so that no server is overloaded.
• It may be a networking device or a software application.
• If a server fails, the load balancer routes requests to servers in good condition instead.
• They enable horizontal scaling.

• A technique for temporarily storing frequently requested data for quick access.
• Sending requests to databases can be slower, so caching some data helps speed up things.
• Specialized caching software is used like Redis, which stores this temporary data in RAM which is fast.

• A CDN is a specialized network of servers, which hosts media files like images, videos, audios, webpages, or other stuff.
• Whenever a client requests this type of data, the request is handled by a nearby CDN server which has cached copy of this content, instead of the main server.

• An API gateway provides an interface for clients to access different services of a backend system.
• It has routes which the client can send requests to for different purposes and receive a response.

• They store values in key-value pairs just like hashmaps do.
• If it is a persistent store, the data is saved on disk.

• They restrict the rate at which a system or application responds to requests.
• These can help prevent DDoS attacks.

• These systems are used to collect and analyze various metrics like performance data.

• They allow communication between two components of a system using a queue (FIFO).
• One service sends a message to the queue and then goes back to doing its work.
• The service which needs to act on this message receives it and does the required action.
• Both are independent of each other and don't need to interact directly.

• They create unique IDs to help identify different components of a distributed system.

• Responsible for scheduling and executing tasks in a distributed system.

• Throughput is defined as the amount of information processed in a certain period of time.
• You can think of it like the number of actions performed per unit of time.
• Higher throughput is considered better.
• Unit is bits per second.

• Amount of time required for a single packet of information to be delivered.
• It is measured in milliseconds.
• Generally, we aim for high throughput and decent latency.
• It mainly depends on -
  • Network delays
  • Delays due to mathematical calculations

• The amount of time a system is available which is measured as - \(availability = \frac{uptime}{uptime + downtime}\)

An application is said to be scalable if it is able to support growth or manage increasing workload without compromising performance. It helps in -

• Ensuring availability
• Maintaining cost effectiveness
• Increasing performance
• Managing growth

• A load balancer could use a hashing function to determine which node to store the data on.
• For instance, if you had three databases, then a simple hash function k % 3 would determine the target db's index.
• For example, for user 4, the data would be stored on server 4 % 3 which is 1.
• Now if let's say you want to add another database node, you would have to shift ALL the data because the hash function would provide you a different value.
• This is problematic and may be very expensive if there is a LOT of data.

• We imagine that our servers are spread across a clock (1-12).
• There are 3 servers at positions 1, 3 and 9.
• Let's say our hash function returned 8. There's no server at position 8, so moving clockwise, it would reach server 9.
• So we can do this for all requests.
• Let's say another node is added on position 7.
• Previously, for hash values 4, 5, 6, 7 and 8, all data was stored in server 9.
• But now the values 4, 5, 6, and 7 should correspond to the server at position 7, so we just have to some data present in server 9 to server 7.
• We can leave other DBs unchanged, which saved us cost and time.

• Let's say we have a simple system
• If the server fails, the entire system goes down, this is single point of failure.
• To avoid this, we can add multiple server nodes
• But how do the clients know now which server to connect to?
• For that, we will add a load balancer.
• The server thing is sorted. Yay! But what if the DB goes down? Just add more… This is called a master slave configuration. There are other configurations as well (discussed later)
• Similarly we have to add multiple load balancers as well so that if one goes down, others can work. We also had to add a DNS service which allows clients to connect to a load balancer.
• You may be wondering what if the DNS service goes down? For that we use multiple DNS service providers. Sorted.
• What if the entire application is hosted in Japan and suddenly there's an earthquake? For that, we host our application on different regions.

• They are horizontally scalable (SQL dbs are vertically scalable).
• They are more available.
• They have a flexible schema, which you can change easily.

• ACID properties are not guaranteed.
• Less consistency.
• These are not read optimized (SQL DBs are better at reads).
• No information about relations between different entities.
• Joins are hard.

• When you need strong ACID properties. For example, banking systems (transactions), inventory management, payments etc..
• When your data has a fixed, well-defined schema.
• When you require complex queries for analytics (SQL is a powerful language!).
• When data integrity matters.
• When you expect moderate-high read/write consistency.

• When you have flexible, evolving schemas or unstructured data. For example, storing analytics, json documents, etc.
• When you need horizontal scaling and partitioning (a lot of horizontally distributed SQL DBs do exist though).
• When you need simple queryies.
• When you need low latency at massive scale.

• There is a master DB which handles all write requests.
• There are multiple slave DBs which handle all read requests.
• When data is written to master, it is asynchronously or synchronously copied to the slaves.
• Advantage is that it provides easy READ scaling.
• Disadvantage is that the master becomes a single point of failure :(

• Multiple nodes can accept writes.
• Changes are replicated to all other nodes to maintain consistency.
• There is no single point of failure -> high availability.
• Conflicts may arise as two nodes may write different values to the same record simultaneously.
  • Timestamps may be used to fix this issue. The latest timestamp wins.

• There is an active component and a passive component.
• When the active component goes down, the passive one takes over.
• Simpler but passive component is idle most of the time.

• Multiple nodes are active at once.
• If one fails, others continue running without disruption.
• For example, multiple servers behind a load balancer.
• More efficient use of resources, but harder to maintain consistency.

• A heartbeat is a signal which indicates if a server node is alive or not.
• Each server keeps on sending a signal like "I'm alive".
• If there is no heartbeat, the node is considered down. So load balancers remove them from rotation.

• A node periodically pings different servers.
• For example, there may be a /health endpoint which may send response 200 OK if the server is fine.
• In both cases, failure mechanisms are used to take action.

• They protect your system from cascading failures (cascading mean something like a domino effect where one crash causes the next node to crash then the next and so on).
• For example, If a payment service is down, your API will stop sending requests to that service until it is healthy again.

• When receiving a failed response, clients may retry a few more times.
• Exponential backoff may be used. For example, try at 1sec, then 2sec, then 4sec, then 16sec, then 32sec and so on to avoid flooding a struggling service.
• It is combined with circuit breakers to prevent infinite retries.

• A message queue is a middleware that lets different services communicate asynchronously by passing messages.
• So service A does not call service B directly. It passes its message to the message queue. Whenever service B is available, it picks it up.
• Service A and B can keep doing their work independently without blocking other clients.
• Some examples are Kafka, RabbitMQ, AWS SQS, etc..

• One publisher broadcasts a message which multiple subscribers can consume.
• For example, Service A broadcasts "Order created". Service B (billing) and Service C (inventory), which subscribed to it, will now act on it.
• Multiple consumers can react differently on the same event at the same time.

• This is a server architecture where there is just a single, unified codebase.
• All services resides in just one codebase.
• There is only one thing to deploy here.
• It is simple to build but very hard to scale.
• You would have to scale the entire server instead of an individual service.
• Tight coupling.

• System is decomposed into multiple small services.
• Each service can be scaled individually depending on the requirements.
• One service goes down, the rest are unbothered.
• Services are loosely coupled.
• For inter-service communication, gSRP is preferred over REST APIs.

Recommended video: https://www.youtube.com/watch?v=qSJAvd5Mgio