CHAPTER 16

Design a Chat System

Persistent connections, ordering, delivery guarantees, presence and sync

Source: PDF p. 53

Core PDF content
Functional requirementsNon-functional requirements and scale
- 1:1 and group chat (up to 500 members) - Delivery and read receipts, online presence - Multi-device sync, message history, media sharing - Push notification when offline- Delivery latency < 200 ms when both online - No message loss; per-conversation ordering - 50 M DAU, ~10 M concurrent connections, ~40 messages/user/day - Optional end-to-end encryption
QuantityEstimateDesign implication
Messages50 M x 40 = 2 B/day = ~23,000/s (peak ~70,000)Write-heavy; needs a write-optimised store
Storage2 B x ~100 B = 200 GB/day = ~73 TB/year (before replication)Wide-column store, TTL / archive old data
Connections10 M concurrent WebSockets / ~100 k per gateway = ~100 gateways (+headroom)Stateful gateways; need a session registry
MediaImages/videos are the bulk of bytesObject storage + CDN, thumbnails, signed URLs
Figure 37. Chat architecture: WebSocket gateways hold connections; the chat service persists first, then routes via the session registry or push. · PDF p. 53

Message flow and guarantees p. 53

1. A sends a message over its WebSocket with a client-generated message ID (idempotency). The gateway forwards it to the chat service. 2. The chat service assigns a per-conversation sequence number (or time-sortable ID), persists it, then acknowledges A ("sent"). 3. It looks up B's gateway(s) in the session registry and forwards the message; B's device acks ("delivered"), and later "read". 4. If B is offline, the message stays in the store and a push notification is sent; when B reconnects it syncs: "give me messages in conversation C after seq 4,812".

Deep dives p. 53

  • Ordering: only conversation-level order matters. Route all messages of a conversation through the same shard/partition and use a monotonic sequence; clients sort by it. Global ordering is unnecessary and expensive.

Review questions for this chapter have not been written and reviewed yet. The chapter text above is the full source content.

Try it

Open “Chat system” in Labs

References

System Design & AI System Design: The Complete Guide — 2026 Edition, p. 53; figures 37.

Ch. 15Ch. 17: Design a Notification System

Tip: use ← and → keys to move between chapters.