One of the biggest myths in software engineering is that growth inevitably makes software slower.
We almost expect it.
More users mean more database queries. More requests mean more servers. More features mean more bugs. Eventually someone says, “We’ll optimize it later,” and everyone silently agrees that the application will only get slower from here.
But when you study companies like Google, Netflix, Discord, Amazon, Stripe, and Cloudflare, something interesting appears.
Many parts of their systems actually become faster as they grow.
That sounds impossible.
How can adding millions of users make software faster?
The answer is simple.
They’re no longer optimizing for individual requests.
They’re optimizing the entire system.
This shift in thinking completely changes how software is designed.
Small Applications Think Locally
Imagine you’re building a simple e-commerce website.
A customer requests a product page.
The backend performs the following steps:
Query the product
Query inventory
Query reviews
Query recommendations
Query seller information
Return the response
Everything seems fine.
The database has only 10,000 products.
Each query takes around 20 milliseconds.
Life is good.
Then your business grows.
Now you have:
50 million products
Millions of daily users
Thousands of sellers
Real-time inventory updates
Suddenly every page performs six expensive queries.
The database becomes overloaded.
Latency increases.
CPU usage spikes.
Developers add more servers.
Nothing improves.
Why?
Because they’re solving the wrong problem.
Bigger Systems Stop Doing Work
This is probably the most important lesson I’ve learned from studying distributed systems.
The fastest operation is the one you never perform.
As software grows, mature systems become obsessed with removing unnecessary work.
Instead of asking,
“How can we make this query faster?”
they ask,
“Why are we running this query at all?”
Those are completely different engineering mindsets.
Cache Everything That Doesn’t Need Fresh Data
Beginners often think caching is an optimization.
Experienced engineers know caching is architecture.
Imagine an article on a news website.
Without caching:
User Request
↓
Database
↓
Generate HTML
↓
Return Response
Every single visitor repeats the same expensive process.
Now introduce a cache.
First Request
↓
Database
↓
Cache
↓
Future Requests
↓
Cache Only
Suddenly thousands of users are served without touching the database.
Ironically, more users increase the cache hit rate.
As popularity grows, performance improves.
Growth actually makes the system faster.
Hot Data Becomes Cheap
One fascinating property of large systems is that popularity creates efficiency.
Suppose your application has one million products.
Only 2% of them are viewed constantly.
Those popular products remain in memory.
They stay inside Redis.
They remain inside CPU caches.
They’re already loaded.
Fetching them becomes incredibly cheap.
Meanwhile the unpopular products can remain on disk.
Large companies intentionally exploit this behavior.
Not all data deserves equal treatment.
Stop Calculating the Same Thing
Imagine your homepage displays:
Trending products
Best sellers
Most liked articles
Top creators
A beginner calculates these lists every time someone refreshes the page.
A better system calculates them once every few minutes.
Millions of users then read the same precomputed result.
The workload shifts from:
Millions of expensive calculations
to
One scheduled calculation
Millions of cheap reads
Notice the pattern.
Growth didn’t require faster hardware.
It required smarter work.
Move Work Away From Users
Users hate waiting.
Servers don’t.
Instead of calculating everything during a request, modern systems calculate before anyone asks.
Examples include:
Generating recommendations overnight.
Rendering static pages ahead of time.
Building search indexes every hour.
Preprocessing analytics continuously.
Compressing images immediately after upload.
When users eventually request the information, it’s already waiting.
Latency disappears.
Event-Driven Systems Age Better
Traditional systems often look like this:
User Uploads Image
↓
Resize Image
↓
Generate Thumbnail
↓
Scan Malware
↓
Extract Metadata
↓
Store Database
↓
Notify Followers
↓
Return Success
The user waits for everything.
As traffic increases, uploads become slower.
Instead, event-driven systems separate the work.
Upload
↓
Store Image
↓
Return Success
↓
Publish Event
↓
Background Workers
• Resize
• Notify
• Scan
• Compress
• Generate AI Tags
Now new features barely affect upload speed.
You simply subscribe another worker.
The request remains fast.
Growth no longer increases latency.
Read More Than You Write
Most applications read far more often than they write.
Think about YouTube.
A video is uploaded once.
It may be watched 50 million times.
Optimizing uploads makes little sense.
Optimizing playback changes everything.
Large companies optimize for the common case.
Every engineering decision asks:
“What happens most often?”
Then they make that path incredibly efficient.
Partition Everything
Eventually one database becomes too large.
Many developers panic.
Instead of buying a bigger server, successful companies divide the problem.
Imagine customer IDs.
Users
1–1,000,000
↓
Database A
1,000,001–2,000,000
↓
Database B
2,000,001+
↓
Database C
No single database carries the entire workload.
Each one becomes smaller.
Queries become faster.
Maintenance becomes easier.
Growth actually reduces pressure per machine.
The Network Is Faster Than You Think
Developers often fear adding services.
They imagine every network request is expensive.
But sometimes splitting systems improves performance.
Why?
Specialization.
A dedicated search service becomes incredibly efficient at searching.
A recommendation service becomes excellent at recommendations.
A media server focuses entirely on video.
Instead of one overloaded application doing everything poorly, several focused services become experts.
Complexity increases.
Latency often decreases.
Batch Everything
Small systems love individual operations.
Large systems love batches.
Imagine inserting one million analytics events.
Bad approach:
Insert
Insert
Insert
Insert
Better approach:
Insert 10,000 Rows
Insert 10,000 Rows
Insert 10,000 Rows
The database performs fewer transactions.
Disk writes become sequential.
Network overhead disappears.
Batching alone can improve throughput by orders of magnitude.
Make Popularity Work for You
Social media platforms discovered something interesting.
Popular content keeps becoming cheaper to serve.
Why?
CDNs.
If millions of users watch the same video, copies are stored around the world.
Requests never reach the original server.
Instead they reach nearby edge servers.
More viewers create more cached copies.
More cached copies reduce latency.
Popularity becomes a performance optimization.
Predict the Future
No, not with AI.
With probability.
If someone opens a product page, they’re likely to click the next image.
They’re likely to view related products.
They’re likely to read reviews.
Modern systems quietly fetch these resources before users request them.
When the click finally happens, the data already exists locally.
The interface feels instant.
The user believes the application is incredibly fast.
In reality, the application simply guessed correctly.
Design Around Queues
Queues are one of the most underrated performance tools.
Instead of overwhelming downstream services, queues absorb spikes.
Imagine Black Friday.
Without queues:
100,000 Orders
↓
Payment API
↓
Everything Crashes
With queues:
100,000 Orders
↓
Queue
↓
Workers
↓
Payment Processing
The system remains stable.
Customers receive confirmations immediately.
Workers process requests at a sustainable pace.
Large scale becomes manageable.
The Architecture Evolves
Small software often looks like this:
Frontend
↓
Backend
↓
Database
Elegant.
Simple.
Easy to understand.
As the business grows, architecture evolves.
Users
↓
Load Balancer
↓
API Gateway
↓
Microservices
↓
Message Queue
↓
Redis
↓
Search Engine
↓
Databases
↓
Analytics Pipeline
↓
Background Workers
↓
CDN
It looks more complicated.
Because it is.
But every additional component exists for one reason:
To remove work from the critical request path.
That’s why mature systems remain responsive despite serving millions of users.
The Biggest Lesson
Most engineers spend their careers trying to make code execute faster.
The best engineers spend their careers making code unnecessary.
They eliminate duplicate work.
They cache aggressively.
They batch operations.
They move expensive computation into the background.
They partition data intelligently.
They precompute results.
They embrace asynchronous processing.
Most importantly, they stop thinking about optimizing functions and start thinking about optimizing systems.
That’s the difference between software that slows down as it grows and software that becomes more efficient with every new user.
Growth doesn’t automatically create performance problems.
Poor architecture does.
If you design your software so that each additional user reinforces your caches, warms your data, distributes your workload, and improves your infrastructure utilization, something remarkable happens.
Your application doesn’t merely survive success.
It gets better because of it.
And that’s one of the most satisfying moments in software engineering—not when your system handles its first million users, but when you realize that serving the second million is actually easier than serving the first.
답글 남기기