Scalability

Scalability is the ability of a system to handle growth (more traffic, more data, more users) by adding resources, while keeping latency and cost roughly proportional. A system is not scalable if handling twice the load requires four times the money or doubles your response time. In an interview, scalability is the thread that connects almost every other decision.

Vertical vs Horizontal Scaling

Vertical scaling (scale up) means using a bigger machine: more CPU, RAM, and disk. It is simple and needs no code changes, but it has a hard ceiling, gets expensive fast, and leaves you with a single point of failure. Horizontal scaling (scale out) means adding more machines and spreading load across them. It scales nearly without limit and adds redundancy, but it requires the system to be designed for it. In interviews, favor horizontal scaling for the parts that must grow.

The Golden Rule: Keep the App Tier Stateless

A stateless server keeps no client-specific data in local memory between requests, so any server can handle any request. That is what lets a load balancer freely add and remove servers and lets autoscaling react to traffic. Push state out: sessions into Redis or a token, files into object storage, and durable data into the database. Statelessness is the single most important enabler of horizontal scale.

The Scaling Toolkit

• Load balancing: distribute requests across a stateless, autoscaling fleet of app servers.

• Caching at every layer: browser, CDN, application (Redis), and database. The cheapest request is one you never make.

• Database read replicas: for read-heavy workloads, replicate the primary and send reads to replicas.

• Sharding/partitioning: for write-heavy workloads, split data across shards by a key (user id, tenant). This scales writes but adds complexity and the risk of a hot shard if one key is too popular. Consistent hashing keeps re-sharding cheap when nodes are added.

• Asynchronous processing: push slow work (sending email, encoding video, updating feeds) onto a queue like Kafka or SQS and process it in the background. Queues also absorb spikes so a traffic burst becomes a longer queue instead of dropped requests.

• Precomputation and denormalization: for read-heavy paths, compute results ahead of time (for example, fan-out on write to build feeds) so reads are cheap.

Estimate Before You Scale

Do quick back-of-the-envelope math to justify choices. Example: 100 million daily active users doing 10 requests each is about 1 billion requests per day, which is roughly 11,000 requests per second on average, and peak is typically 3 to 5 times the average, so plan for around 50,000 requests per second. These numbers tell you whether one database is enough or whether you must shard.

Find the Real Bottleneck

Measure end to end first (p95 and p99 latency), then decompose by tier. The usual suspects are the database, a single-threaded or single-instance component, a hot key or hot shard, lock contention, and N+1 query patterns. Scale the tier that is actually the bottleneck; adding app servers does nothing if the database is the limit.

Common Mistakes

• Keeping state on app servers, which blocks horizontal scaling.

• Sharding prematurely, before a single database with replicas has been outgrown.

• Scaling the app tier when the database is the actual bottleneck.

• Proposing a design with no capacity estimate, so you cannot tell if it holds.

Interview Soundbite

"I will keep the app servers stateless behind a load balancer with autoscaling, cache reads in Redis and static content in a CDN, and move slow work to a queue. For the database I will start with a primary plus read replicas, and shard by user id only once write volume, which I estimate at about 50k requests per second at peak, outgrows a single primary."