Skip to content
Prompt Words
History

Load balancer

Also called: LB, application load balancer, ALB.

A server in front of several copies of your app that passes each request to one of them, using a rule such as round robin or least connections. It health-checks each copy, for example GET /ready, and stops sending traffic to a copy that fails.

The balancer hands requests to the 3 servers in turn: 1, 2, 3, then 1 again (round robin). Take server 2 down to see health checks find it and take it out.

Server 1: 0 · Server 2: 0 · Server 3: 0 · Failed: 0 · Balancer sends to: 1, 2, 3

No requests yet. All 3 servers are healthy.

    Served 0 0 0 failed 0 healthy 1 2 3

    Say it in a prompt

    Put an application load balancer in front of 3 app instances with round-robin routing. Health-check GET /ready (200 only when the instance can serve requests) every 10 seconds; take an instance out after 2 failures in a row and put it back after 3 passes. Keep the app stateless: store sessions in Redis, not in server memory.

    Vague vs precise prompt

    Vague prompt

    we have more users now, make the server handle it

    Typical resultRaises the worker or thread count on the one server. When that server goes down, the whole site goes down with it.

    Precise prompt

    Run 3 app instances behind an application load balancer with round-robin routing. Health-check GET /ready every 10 s; remove after 2 failures, re-add after 3 passes. Move sessions to Redis so any instance can serve any user.

    Typical resultTraffic splits across 3 instances, a crashed instance stops getting requests within about 20 seconds, and users stay logged in whichever instance answers.

    Seen on

    • NGINX: Its upstream module spreads requests round-robin by default, with least-connected and ip-hash as options, and marks a server as failed after max_fails failed attempts.
    • AWS Application Load Balancer: Sends health-check requests to every target and routes only to healthy ones; by default a target is taken out after 2 failed checks in a row.

    You might describe it as

    • spread the traffic over several servers
    • one address in front of many app copies
    • stop sending users to the server that crashed

    Not to be confused with

    • CDN

      A load balancer forwards every request to one of your servers; a CDN answers from its own cached copy when it can and only asks your server on a miss.

    • Horizontal scaling

      A load balancer spreads requests across the copies you have; horizontal scaling is adding those copies.

    • Health check

      A load balancer spreads requests over the copies of your app; a health check decides which copies may get them.