Horizontal scaling means adding more machines to a system to increase its capacity — three servers instead of one, then thirty. It’s usually called scaling out, in contrast to vertical scaling, which makes a single machine bigger.
The appeal is that there’s no ceiling. A single server eventually runs out of CPU sockets and memory slots; a fleet doesn’t. Horizontal scaling also gives you redundancy for free: if one of thirty servers dies, you’ve lost 3% of your capacity rather than your entire service.
What it requires
Scaling out only works if the work can be split up, which puts real constraints on how the system is built:
- A load balancer in front, to distribute requests across the fleet. Without one, extra servers just sit idle.
- Stateless application servers. If a server keeps a user’s session in local memory, that user has to keep landing on the same machine. Push session state into a shared store and any server can handle any request.
- A shared data layer. Every instance has to see the same data, which is why the database is usually the last thing to scale out and the hardest.
- Coordination for scheduled work. Ten instances each running the same nightly job at midnight is ten times the work and, often, ten copies of the same email.
Scaling out a database
Application servers are the easy part. Databases resist horizontal scaling because they hold state, and the strategies for splitting that state each come with a cost:
- Read replicas spread read traffic across copies of the data. Straightforward, and the usual first move for analytics — but writes still all go to one primary.
- Sharding partitions rows across independent databases by some key, such as customer ID. Genuinely unlimited scale, at the price of cross-shard queries becoming hard and rebalancing becoming a project.
- Distributed query engines, the model most cloud data warehouses use, spread a single query’s work across many nodes. This is why an analytical warehouse can scan a billion rows in seconds where a single server would grind.
The trade-offs
Scaling out costs more in complexity than it saves in hardware, at least at first. You now have deployments to coordinate, logs spread across machines, and failure modes — partial outages, inconsistent versions mid-deploy, a replica lagging behind — that a single server simply doesn’t have. Capacity also comes in chunks: you add a whole instance, not the 15% more headroom you actually needed.
The usual practical advice holds. Scale up until it gets expensive or you need redundancy, then scale out — and make sure you’re watching resource utilization closely enough to know which one you’re short of.
Key article
Related terms
Put it to work
- Cloud cost analytics — Overview
- Kubernetes cost dashboard — Dashboard
- Resource utilization — Metric
