When a website starts receiving more visitors, increasing server capacity is not the only option. There comes a point when a single server is too risky: if it becomes overwhelmed or experiences a failure, the entire service is affected. This is where a load balancer becomes relevant.
In simple terms, a load balancer is a component that receives requests from users and forwards them to one or more servers behind it. From the user's perspective, the accessed address remains the same. From the infrastructure side, requests can be distributed across multiple machines that work as a single service.
However, a load balancer is not a magic button to make a website faster. It helps manage traffic and improve availability, but it still depends on the health of the application, database, network, and server configurations behind it.
How does a load balancer work?
Imagine a restaurant with one entrance and several cashiers. Visitors do not need to choose a cashier themselves. The staff at the entrance directs them to the available cashier to balance the queues. A load balancer operates on a similar principle.
In services like AWS Elastic Load Balancing, the load balancer acts as a contact point for clients and forwards requests to registered targets. These targets can be instances, IP addresses, or other services, depending on the type of load balancer used. The system can also monitor the health of the targets and avoid unhealthy servers. AWS documentation explains this pattern officially.
Traffic distribution can use several approaches, such as:
- Round robin: requests are distributed in turn to each server.
- Least connections: requests are directed to the server with the fewest active connections.
- Weighted routing: stronger servers are given a higher weight.
- Location-based or rule-based routing: users or specific types of requests are directed to certain groups of servers.
The available algorithms vary by provider and type of load balancer. Therefore, do not choose based solely on feature names. Consider how the algorithm fits with your traffic patterns and application characteristics.
Health checks are more important than just splitting traffic
A feature often considered trivial is actually a crucial part: health checks. Load balancers periodically test whether a server is still fit to receive requests. The checks can involve connecting to a specific port or making an HTTP request to a specific endpoint.
For example, you create an endpoint /health that only returns the application status. If that endpoint fails to respond according to the rules, the server can be temporarily removed from the target list. Google Cloud also uses health checks to determine which backend is ready to accept connections. Google Cloud's health check concepts explain that healthy or unhealthy status is determined based on thresholds and configured check results.
A common mistake is making health checks too simple or too heavy. An endpoint that only checks if the web process is alive may not be aware that the database connection is failing. Conversely, an endpoint that runs complex queries every few seconds can burden the system.
A more sensible approach is to separate the checks:
- Liveness check: is the application process still running?
- Readiness check: is the application ready to receive traffic, including critical dependencies like databases or internal services?
Use checks that align with your goals. Not all database disruptions should remove the entire server from rotation, especially if the application can still serve certain pages or functions.
When does a website start needing a load balancer?
A load balancer makes sense when there are clear operational reasons, not just because the infrastructure wants to look more “enterprise.” Some signs to watch for include:
- You are running more than one application server. If multiple servers need to receive traffic from a single public address, a load balancer simplifies access management.
- Downtime of one server should not take down the service. With health checks and multiple backends, traffic can be redirected when one server has issues.
- Deployment needs to be done gradually. Some traffic can be directed to the new version before all users are migrated.
- Traffic load is variable. A load balancer can serve as an entry point for autoscaling systems, although autoscaling itself still needs to be configured separately.
- The application has multiple service regions. In a multi-region architecture, configurations can direct users to the most appropriate pool or backend.
For personal blogs or internal applications with a single server and low traffic, a simple reverse proxy or direct configuration on the web server is often sufficient. Adding a managed load balancer too early can create fixed costs, new network rules, and more points to monitor.
Commonly overlooked aspects when setting up a load balancer
1. User sessions
If the application stores user sessions only in the server's local memory, users may experience logouts or loss of context when the next request hits a different server. Solutions can include centralized session storage like Redis or a database, or using sticky sessions if appropriate.
Sticky sessions bind users to a specific server. This is practical but can reduce traffic balancing and overload certain servers. Therefore, centralized session storage is usually more flexible for applications that want to scale.
2. TLS termination
A load balancer can handle HTTPS connections from users and forward traffic to the backend. This pattern is called TLS termination. The benefit is that certificate management becomes more centralized. However, the path from the load balancer to the server still needs to be considered. For sensitive data, internal encryption may still be necessary.
3. Timeout and request size
Timeout settings that are too short can terminate processes that are still running, while timeouts that are too long can cause connections to pile up. Also, pay attention to upload size limits, WebSocket connections, streaming, and long-running requests. Default values may not suit your application's behavior.
4. Logs must retain context
If all requests pass through the load balancer, application logs should retain information such as forwarded client addresses, request IDs, response times, and which backend handled the request. Without this, investigating disruptions can feel like searching for a needle in a haystack.
What you can do now
Before purchasing or activating a load balancer service, create a simple map:
- How many application servers are truly needed?
- Does the application store sessions or files locally?
- What endpoint will be used for health checks?
- What is the normal request duration, and which requests are allowed to run long?
- How will traffic be redirected if all backends are considered unhealthy?
- Have costs and monitoring been calculated, not just server costs?
Also, intentionally test failure scenarios. Turn off one backend, see if traffic shifts, then restore that server and ensure it does not immediately receive the full load before it is truly ready.
In summary: a load balancer is most useful when a website requires availability, load distribution, or a pathway to a larger architecture. But the foundation remains the same: applications should be as stateless as possible, health checks should make sense, logs should be complete, and every routing rule needs to be tested before a real incident occurs.
Sources & further reading
- How Elastic Load Balancing works - AWS Documentation
- Target groups for your Application Load Balancers - AWS Documentation
- Health checks overview - Google Cloud Documentation
- Backend services overview - Google Cloud Documentation
– Rio Yotto @rioyotto