CHAPTER 37
Cheat-Sheets, Glossary and Study Plan
Quick-reference tables, a glossary, practice questions and a 12-week plan
Source: PDF pp. 102–107
Core PDF content
PDF page 102Cheat-sheet 1: technology selection p. 102
| Need | Typical choices | Pick when |
|---|---|---|
| Relational OLTP | PostgreSQL, MySQL, Aurora, Cloud SQL | Default; transactions, joins, flexible queries |
| Massive KV / wide-column | DynamoDB, Cassandra, ScyllaDB, Bigtable | Known access patterns, huge scale, multi-region writes |
| Document store | MongoDB, Firestore, Couchbase | Flexible schema, nested documents |
| Distributed SQL | Spanner, CockroachDB, TiDB, YugabyteDB | Horizontal scale with strong consistency |
| Cache / counters / locks | Redis, Memcached, Dragonfly | Sub-ms reads, rate limits, leaderboards, sessions |
| Search | Elasticsearch / OpenSearch, Solr, Typesense | Full-text, facets, log search |
| Vector search | pgvector, Qdrant, Weaviate, Milvus, Pinecone, FAISS | Embeddings for RAG / recommendations |
| Object storage / lake | S3, GCS, Azure Blob; Iceberg, Delta, Hudi | Media, backups, datasets, lakehouse |
| Message queue | SQS, RabbitMQ, Pub/Sub | Job queues, decoupling |
| Event log / streaming | Kafka, Pulsar, Redpanda, Kinesis | High-throughput events, replay, stream processing |
| Stream processing | Flink, Kafka Streams, Spark Streaming | Windows, joins, real-time features |
| Batch / orchestration | Spark, dbt, Airflow, Dagster, Temporal | ETL, scheduled pipelines, durable workflows |
| OLAP | BigQuery, Snowflake, Redshift, ClickHouse, Druid, Pinot | Analytics, dashboards, real-time aggregates |
| Load balancing / edge | NGINX, Envoy, HAProxy, cloud LBs, CDN (CloudFront, Fastly, Cloudflare) | L4/L7 routing, TLS, caching, DDoS |
| Coordination | etcd, ZooKeeper, Consul | Leader election, config, locks |
| Observability | Prometheus, Grafana, OpenTelemetry, Loki / ELK, Jaeger / Tempo | Metrics, logs, traces, alerts |
| ML platform | MLflow, Kubeflow, Feast, KServe, Triton, Ray | Tracking, features, serving, distributed compute |
| LLM serving | vLLM, TensorRT-LLM, SGLang, TGI; managed model APIs | Self-hosting vs hosted inference |
| LLM apps | LangGraph, LlamaIndex, DSPy, Langfuse, MCP servers | Orchestration, retrieval, tracing, tool integration |
PDF page 103Cheat-sheet 2: symptom to remedy p. 103
| Symptom | First things to try |
|---|---|
| DB CPU high, reads dominate | Add indexes, cache hot reads, read replicas, CDN for static data |
| DB writes saturated | Batch writes, queue + async, partition/shard, LSM-based store, reduce indexes |
| Hot key / celebrity user | Local cache, key splitting, replicate hot items, request coalescing |
| Cache stampede after expiry | Single-flight, jittered TTL, stale-while-revalidate, early refresh |
| High tail latency (p99) | Timeouts, hedged requests, fewer fan-outs, isolate noisy neighbours, load shedding |
| Cascading failures | Circuit breakers, bulkheads, backpressure, retry budgets with jitter |
| Duplicate processing | Idempotency keys, dedupe table, outbox + idempotent consumers |
| Consumers lagging | More partitions/consumers, batch, fix slow handler, backpressure, DLQ poison messages |
| Stale reads after write | Read-your-writes routing, read from leader, version tokens |
| Cross-service consistency | Saga, outbox, idempotent steps, reconciliation jobs |
| (AI) LLM too slow | Stream, shorter prompts, prefix caching, smaller model, speculative decoding, parallel tool calls |
| (AI) LLM too expensive | Route to smaller models, prompt caching, shorter outputs, batch APIs, distil / fine-tune a small model |
| (AI) Hallucinations | RAG with citations, rerank, constrain format, abstention, verification step, better evals |
| (AI) Model quality decaying | Check data drift and label delay, retrain on fresh data, fix features, slice analysis |
Cheat-sheet 3: classic trade-offs p. 103
| Choice | Option A | Option B | Decide by |
|---|---|---|---|
| Consistency vs availability | Strong (CP) | Eventual (AP) | Cost of stale vs unavailable data |
| Latency vs throughput | Small batches, immediate | Large batches, amortised | User-facing vs offline work |
| SQL vs NoSQL | Joins, transactions | Scale, flexible model | Access patterns, size, team skills |
| Push vs pull (fan-out) | Precompute on write | Compute on read | Read/write ratio, follower skew |
| Sync vs async | Simple, immediate result | Decoupled, resilient | Need for an answer now |
| Normalise vs denormalise | Single source of truth | Fast reads, duplication | Read-heavy and tolerance for inconsistency |
| Monolith vs microservices | Simplicity, one deploy | Independent scale and teams | Org size and domain boundaries |
| Batch vs stream | Accuracy, simplicity | Freshness | Latency requirement of the decision |
| Cache write-through vs | Consistent, slower writes | Fast writes, loss risk | Durability need |
write-back
(AI) Prompt vs RAG vs Cheap, flexible Fresh/private facts; behaviour change Gap shown by evals
fine-tune
| (AI) API vs self-host | Frontier quality, no ops | Control, privacy, unit cost at scale | Volume, privacy, latency, team |
|---|---|---|---|
| (AI) Large vs small model | Quality, reasoning | Latency and cost | Routing by task difficulty |
PDF page 104Glossary p. 104
| Term | Meaning | Term | Meaning |
|---|---|---|---|
| ACID | Atomicity, consistency, isolation, durability of transactions | Load shedding | Dropping excess work to protect the system |
| Agent | LLM in a loop that plans, calls tools and observes results | LoRA | Parameter-efficient fine-tuning with low-rank adapters |
| Anycast | Many servers share one IP; routing sends users to the nearest | LSM tree | Write-optimised log-structured storage engine |
| API gateway | Edge service for auth, rate limits, routing | MCP | Open protocol connecting AI apps to tools and data |
| Backpressure | Signal that makes producers slow down when consumers lag | Merkle tree | Hash tree for efficient difference detection |
| Bloom filter | Probabilistic set; no false negatives, some false positives | MLOps | Practices to build, deploy and monitor ML reliably |
| BM25 | Keyword relevance scoring function (improved TF-IDF) | MVCC | Concurrency via multiple row versions |
| Bulkhead | Resource isolation so one failure cannot exhaust everything | Outbox | Write event with the data in one transaction, then publish |
| CAP | Under partition choose consistency or availability | p99 | 99th-percentile latency |
| CDC | Change data capture: stream of committed DB changes | PACELC | CAP extended with latency/consistency trade-off when no partition |
| CDN | Geo-distributed edge caches | Partitioning | Splitting data across nodes (sharding) |
| Circuit breaker | Stops calls to a failing dependency, probes for recovery | Point-in-time join | Join features as they were at each label timestamp |
| Concept drift | Relationship between inputs and target changes over time | Prefill / decode | LLM phases: process prompt / generate tokens |
| Consistent hashing | Key placement on a ring with minimal remapping | Prompt caching | Reusing computation for an identical prompt prefix |
| Context window | Maximum tokens an LLM can attend to | Prompt injection | Malicious instructions hidden in model inputs |
| CQRS | Separate write and read models | PSI | Population stability index, a drift metric |
| CRDT | Data type that merges concurrent updates without coordination | Quantisation | Lower-precision weights for smaller, faster models |
| Data drift | Input feature distribution shifts from training | Quorum | Majority needed for a decision; W + R > N |
| DLQ | Dead-letter queue for messages that repeatedly fail | RAG | Retrieve evidence, then generate grounded answers |
| DPO / RLHF | Methods to align models with human preferences | Raft / Paxos | Consensus algorithms |
| Embedding | Dense vector capturing semantic meaning | Read repair | Fix stale replicas when a read reveals divergence |
| Eventual consistency | Replicas converge if updates stop | Reranker | Second-stage model that reorders retrieved candidates |
| Exactly-once | At-least-once delivery plus idempotent processing | RPO / RTO | Max tolerable data loss / downtime |
| Fan-out | One event delivered to many recipients | Saga | Series of local transactions with compensations |
| Feature store | Shared offline/online feature definitions and storage | Service mesh | Sidecar proxies for service-to-service traffic |
| Fencing token | Increasing number to reject stale lock holders | SLI / SLO / SLA | Indicator, internal objective, customer contract |
| Fine-tuning | Further training of a model on task data | Speculative decoding | Draft-and-verify to speed up generation |
| Gossip | Peer-to-peer state dissemination | Split brain | Two nodes both think they are leader |
| Guardrails | Checks and constraints around model inputs/outputs/actions | Tail latency | High-percentile response times |
| Hallucination | Fluent but unsupported or false model output | Temperature | Sampling randomness control |
| Hedged request | Duplicate request after a delay to cut tail latency | TTFT | Time to first token |
| HNSW | Graph-based ANN index | Two-phase commit | Atomic commit protocol across participants |
| Idempotency | Repeating an operation has the same effect as once | Training-serving skew | Features differ between training and inference |
| Inverted index | Term to document postings for search | Vector clock | Per-node counters detecting concurrent updates |
| Term | Meaning | Term | Meaning |
|---|---|---|---|
| KV cache | Stored attention keys/values for generated tokens | Vector database | Store optimised for similarity search over embeddings |
| Lamport clock | Logical clock giving causal ordering | WAL | Write-ahead log for durability and replication |
| Leader election | Choosing one coordinator among nodes | Zero trust | Authenticate and authorise every request |
| Linearizability | Strongest consistency: operations appear instantaneous |
Practice questions p. 105
Classic system design
- Design a URL shortener, pastebin, or distributed ID generator.
- Design a news feed, Twitter, or Instagram.
- Design WhatsApp / Slack / a group chat system.
- Design YouTube / Netflix; design a live-streaming platform.
- Design Uber / Lyft / a "nearby friends" or proximity service.
- Design Google Drive / Dropbox; design a collaborative editor (Google Docs).
- Design a web crawler; a search engine; typeahead.
- Design a rate limiter; an API gateway; a distributed cache; a distributed lock service.
- Design a key-value store; a message queue; a metrics and monitoring system; a job scheduler.
- Design a ticketing / flash-sale system; a payment system; a digital wallet; a hotel booking system.
- Design a notification system; an email service; a leaderboard; a top-K / trending service.
- Design an ad-click aggregation pipeline; a log aggregation system.
AI system design p. 105
- Design a recommendation system for a video / e-commerce / music app (retrieval + ranking + cold start).
- Design a fraud, spam or abuse detection system.
- Design a search ranking system with hybrid retrieval.
- Design an ML platform (feature store, training, registry, serving, monitoring) for a company.
- Design a ChatGPT-like assistant and its GPU serving fleet.
- Design enterprise document Q&A with permissions (RAG).
- Design an AI coding assistant; a customer-support agent; a voice assistant; a text-to-image service.
- Design a content-moderation pipeline using ML and LLMs plus human review.
- Design an LLM evaluation and monitoring platform; a prompt-management system; an LLM gateway.
- Design a semantic caching layer; a document-ingestion pipeline; a multi-agent research assistant with tools and budgets.
PDF page 106A 12-week plan p. 106
| Week | Focus | Output to produce |
|---|---|---|
| 1 | Ch. 1-3: framework, networking, estimation | Memorise latency numbers; do 10 estimation drills |
| 2 | Ch. 4-5: load balancing, caching, CDN | Draw LB + cache + CDN architecture from memory |
| 3 | Ch. 6: databases, indexing, replication, sharding | Choose stores for 5 scenarios and justify |
| 4 | Ch. 7-8: distributed theory, messaging | Explain CAP, quorum, Raft, saga, Kafka in 2 minutes each |
| 5 | Ch. 9-10: APIs, security, resilience, deployment | Design a rate limiter; write a failure-mode table |
| 6 | Ch. 11-12 + cases 13-14 | URL shortener and key-value store, timed, no notes |
| 7 | Cases 15-18: feed, chat, notifications, video | Two timed mocks; get feedback |
| 8 | Cases 19-24: ride-hailing, typeahead, crawler, storage, ticketing, analytics | Two more mocks; review weak areas |
| 9 | Ch. 25-28: ML foundations, features, serving, recommenders | Design a recommender and a fraud system on paper |
| 10 | Ch. 29-30: LLM inference, embeddings, RAG | Size a GPU fleet; design a RAG system with an eval plan |
| 11 | Ch. 31-33: agents, LLM apps, tuning, evals, safety | Design an agent with guardrails; write an eval suite outline |
| 12 | Ch. 34-37: AI case studies, playbook; full mocks | Three full mock interviews (2 classic, 1 AI); fix gaps |
Going deeper (primary sources) p. 106
This guide aims to be self-contained, but everything in it is grounded in a small set of foundational works. If you want the original arguments and details:
- Books: Designing Data-Intensive Applications (Kleppmann); Site Reliability Engineering (Google); System Design Interview vols. 1-2 (Xu); Designing Machine Learning Systems and AI Engineering (Huyen).
- Distributed systems papers: Google File System, Bigtable, Spanner; Amazon Dynamo; Kafka; Raft ("In Search of an Understandable Consensus Algorithm"); Chord / consistent hashing.
- ML and LLM papers and posts: "Hidden Technical Debt in Machine Learning Systems"; "Attention Is All You Need"; PagedAttention (vLLM); FlashAttention; LoRA; DPO; "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"; ReAct; Anthropic's "Building effective agents"; the MCP specification.
- Practice beats reading: build a small version of two or three designs (a cache-backed API, a Kafka consumer pipeline, a RAG app with an eval set). Operating something, even a toy, turns vocabulary into judgement.
Review questions for this chapter have not been written and reviewed yet. The chapter text above is the full source content.
References
System Design & AI System Design: The Complete Guide — 2026 Edition, pp. 102–107.
Tip: use ← and → keys to move between chapters.