Estimation, Latency, Availability and SLOs

PDF page 11. Why do we estimate before designing? p. 1

Knowing that an app has "millions of users" is not enough. Estimates help answer practical questions: how much traffic must the system handle, how quickly data grows, and which parts need more capacity. The goal is a useful order of magnitude, not perfect arithmetic.

2. Useful numbers and shortcuts p. 1

Conversion formulas p. 1

Average QPS = requests per day / 86,400. Daily storage = uploads per day x average size. Data per second = requests per second x data per request; multiply bytes per second by 8 to get bits per second.

PDF page 23. Worked example: a photo-sharing app p. 2

Assume 100 million daily active users; 100 photo views per user per day; 20% of users upload one photo daily; each processed photo is 2 MB; and each served image averages 200 KB. These are illustrative assumptions, not production measurements.

Step 1: calculate reads and writes p. 2

Views/day: 100 million x 100 = 10 billion. Average reads = 10 billion / 86,400 = approximately 115,700 requests/s. Uploads/day: 100 million x 20% = 20 million. Average writes = 20 million / 86,400 = approximately 231 uploads/s. If peak traffic is assumed to be 3 times average:

There are roughly 500 reads for each write, making this a read-heavy workload. The peak multiplier is a stated assumption; real systems should use measured traffic where possible.

Step 2: calculate storage p. 2

20 million uploads x 2 MB = 40 TB per day. Over a year, 40 TB x 365 = 14.6 PB of raw uploads. If all uploads are retained and three copies are stored, the annual upload cohort would require about 43.8 PB for those copies, before other overhead.

PDF page 3Worked example (continued) Step 3: estimate bandwidth p. 3

Average read traffic is about 115,700 requests/s. If each view transfers 200 KB:

This estimate excludes protocol overhead, cache hit rates, compression, retries and traffic variation. A peak multiplier of 3 would imply about 555 Gbps of peak image egress under the same assumptions.

What does this tell us? p. 3

The read-heavy workload creates very high outbound bandwidth demand. A CDN and caching strategy are strong candidates because popular images can be served closer to users instead of repeatedly traveling to the origin. Storage must also accommodate rapid growth, retention rules and redundancy.

4. Latency and throughput p. 3

Latency is how long one operation takes. Throughput is how much work the system completes per unit of time. A system can have high throughput and still have high latency - for example, batching can increase total work processed while making individual requests wait longer.

PDF page 45. Why latency percentiles matter p. 4

Averages hide slow requests. Imagine most users get a response quickly, while a smaller group experiences a long delay. Percentiles show where those slower requests sit.

Tail latency: why fan-out hurts p. 4

Suppose one request contacts 100 independent backend services and waits for every result. Assume each service has a 1% chance of exceeding its p99 latency threshold.

Probability at least one backend is slow:

This is an illustrative probability model. Real delays may be correlated, and not every design waits for all downstream services. Reduce unnecessary dependencies, use timeouts, cache results, or consider hedged requests where appropriate.

PDF page 56. Two useful laws for system design Little's Law: how many requests are in progress? p. 5

Little's Law connects the arrival rate with the time work spends in the system.

Example: 1,000 requests arrive each second and the average time in the system is 200 ms (0.2 seconds).

This helps reason about concurrency, connection pools and capacity. It does not mean every server needs exactly 200 threads.

Amdahl's Law: why adding machines has limits p. 5

Some work can run in parallel; some must remain sequential. The sequential part limits the benefit from adding more workers.

If 5% of a job is sequential, even infinitely many parallel workers cannot achieve more than 1 / 0.05 = 20x theoretical speedup.

Why utilization matters p. 5

When a service runs close to 100% utilization, small traffic increases can create growing queues and large latency spikes. Keeping headroom is useful for latency-sensitive workloads. A 60-70% utilization target can be a starting assumption for some systems, but the right level depends on workload variability and recovery needs.

PDF page 67. Availability: how often does a system work? p. 6

Availability is the fraction of time a service can provide its intended functionality. The "nines" give a quick way to compare reliability targets.

These figures assume continuous service and a fixed time window. Contractual availability calculations can use different definitions and measurement rules. Higher targets usually require more redundancy, monitoring and recovery investment.

Components in series: all must work p. 6

Components in parallel: either one can work p. 6

These improvements assume independent failures. Shared databases, network paths, deployments or configuration can cause replicas to fail together.

PDF page 78. SLI, SLO, SLA and error budgets p. 7

These terms help teams measure reliability and agree on an acceptable level of service.

How an error budget supports decisions p. 7

For a 30-day month, a 99.9% time-based availability target allows 0.1% downtime, or approximately 43.2 minutes. An error budget helps product and engineering teams decide when to continue delivery and when to prioritize reliability. It is not a reason to ignore reliability while budget remains.

Disaster-recovery targets p. 7

Chapter recap p. 7

• Estimate requests, storage and bandwidth before choosing architecture.
• Track latency percentiles as well as averages.
• Use Little's Law for work in progress and Amdahl's Law for scaling limits.
• Leave capacity headroom when traffic variability and latency demand it.
• Required dependencies reduce availability; independent redundant replicas can improve it.
• SLIs measure, SLOs set targets, SLAs define commitments, and error budgets guide decisions.

PDF page 89. Check your understanding p. 8

Choose an answer for each question, then review the answer key and explanations.

1. Two services, each 99.9% available, must both work for a request to succeed. What is the combined availability, assuming independent failures?

• A. 99.9%
• B. About 99.8%
• C. 99.99%

2. A service processes 1,000 requests/s. Average time in the system is 200 ms. Approximately how many requests are in progress?

• A. 5
• B. 200
• C. 5,000

3. If 5% of a job must run sequentially, what is its theoretical maximum speedup?

• A. 5x
• B. 20x
• C. Unlimited

4. Why does the photo-sharing example suggest a CDN?

• A. Uploads are only about 700/s at peak
• B. Image reads create very high outbound bandwidth demand
• C. A CDN guarantees database availability

Answer key and explanations

• B. For two required services in series, multiply availability: 0.999 x 0.999 ≈ 99.8%.
• B. Little's Law gives 1,000 x 0.2 = 200 requests in progress on average.
• B. The sequential fraction caps speedup at 1 / 0.05 = 20x.
• B. The estimated average image traffic is about 185 Gbps, making distributed caching a strong design choice.