Everything on One Server
Your app just launched. 50 users, fast responses, zero drama. This chapter is about what that looks like under the hood — and exactly what happens when a thousand people show up at once.
You ship. It works.
Imagine you just launched a food delivery app. Users open it, browse restaurants, place orders. It all works beautifully. Your entire backend is one server — a $50/month machine sitting in a data center — running one program. That program does everything: handles web requests, runs your business logic, reads and writes to the database.
This is called a monolith. Not because it's old-fashioned or wrong. It's called that because it's one solid piece. And for a new product, it's the smartest thing you can build. Amazon, Twitter, Shopify — all started exactly this way.
The monolith is not the enemy. Starting with one is the right call. This chapter is about understanding it deeply so you know exactly when — and why — it breaks.
What's actually inside that one server
When a user taps "Place Order", here's what happens inside your server — all within the same program, on the same machine:
- It receives the request: Your server is listening on a port (like port 3000). The user's phone sends an HTTP request — basically a formatted text message that says "POST /api/orders, here's the order details." Your server reads that message. This takes about 0.01ms — basically instant.
- It figures out what to do: Your code matches the request to the right function. "POST /api/orders? Run the placeOrder handler." Still basically instant.
- It runs your business logic: Is the user logged in? Is the restaurant open? Is the item in stock? This is your actual application code running. Could be 1ms for a simple check, could be 50ms if it's complex.
- It talks to the database: Your server asks the database to save the order and fetch some data. The database is also on the same machine, so this is like talking to a roommate rather than calling someone on the phone — very fast. About 1–10ms.
- It sends back a response: The server packages up the result as JSON and sends it back to the user's phone. Done. Total time: roughly 5–60ms. Fast enough that users never notice.
The key thing to notice: everything talks to everything else directly, without going over the internet. It's all in one place. That's why it's so fast.
Now 1,000 people show up at once
Your app gets featured on a blog. Traffic spikes. Suddenly a thousand people are placing orders at the same time — each one triggering that same five-step process. Watch what happens to your server:
Why does it break? The three killers.
Your server has three finite resources. When any one of them runs out, users start getting errors. Here's each one explained from the ground up.
Think of your server's CPU like a chef in a kitchen. A $50/month server has 2 CPUs — two chefs. Each one can only work on one thing at a time.
Most of the time, your chefs are actually waiting — waiting for the database to respond, waiting for a file to load. While they wait, they can start on another order. That's why 2 CPUs can handle hundreds of requests simultaneously when those requests spend most of their time waiting.
But some tasks keep the CPU busy the whole time — resizing a profile photo, generating a PDF, encrypting a file. When many of those pile up, your 2 chefs are fully occupied and can't start new work. New requests pile up at the door. Response times go from 50ms to 30 seconds. Users think your app is broken.
This is called CPU saturation, and it typically hits around 200 concurrent heavy requests on a basic server.
RAM is your server's short-term memory — the space it uses to hold everything it's currently working on. Your $50 server has 4GB of it.
Here's the problem: your server and your database are sharing that 4GB. The database alone wants about 1GB to keep frequently-accessed data in memory so it doesn't have to read from disk every time. Your application then uses roughly 1MB per active user connection — session data, request buffers, response buffers.
Do the math: 3,000 simultaneous users × 1MB = 3GB, plus 1GB for the database = 4GB total. You've just used every last byte of RAM. The operating system panics and starts killing processes to free up memory. Usually it kills your app, or the database, or both. Everything goes down.
This is called memory exhaustion, and it's often the thing that kills servers before CPU does.
Every time your app talks to the database, it needs a connection — a dedicated communication channel. Think of it like a phone line: if all lines are busy, new callers get a busy signal.
By default, most databases only allow 100 simultaneous connections. Each one takes a small amount of memory and a small amount of CPU to maintain. That limit of 100 exists for good reason — more connections than that and the database itself starts struggling.
So your 101st user gets this:
FATAL: remaining connection slots are reserved
for non-replication superuser connectionsTranslation: "We're full. Go away." And this often happens at surprisingly low traffic — not because your server is slow, but just because nobody changed that default setting. This is the most common first production incident engineers experience.
The obvious fix: get a bigger server
Your first move when you hit these limits is simple: rent a bigger machine. Upgrade from 2 CPUs and 4GB RAM to 8 CPUs and 32GB RAM. Now you can handle roughly 4x the traffic, and you didn't have to change a single line of code.
4 GB RAM
256 GB RAM 🔥
This is called vertical scaling — making your one server more powerful. It works great, and you should always do it before anything more complex.
The biggest server you can rent on AWS has 448 CPUs and 24TB of RAM. It costs around $220 per hour. That's both impressive and irrelevant to most of us — because long before you need that, you'll hit a different wall: a single server is a single point of failure.
If that one big machine crashes — power failure, hardware fault, a bad software deploy — your entire app is down. No backup, no redundancy. Every user sees an error page. Vertical scaling makes your server bigger, but it never makes it safer.
The real question isn't "how big can I make this server?" It's "when do I stop fighting the ceiling and split things up instead?" The answer is usually around 50,000 daily active users — when the database and the app start visibly competing for the same resources. That's what Chapter 2 is about.
When an interviewer asks you to design a system, don't immediately reach for microservices and Kubernetes. Start with: "At launch, a single server handles everything. Here's when I'd know it's time to split things up: when the database and app are fighting for RAM, when one slow endpoint is making everything else slow, or when I need redundancy." Interviewers at top companies specifically watch for candidates who don't over-engineer. The monolith answer, given confidently with clear reasoning, is often the impressive one.