CHAPTER 28
Recommendation, Search and Ranking Systems
Multi-stage funnels, embeddings, learning-to-rank and feedback loops
Source: PDF pp. 74–76
Core PDF content
PDF page 74The multi-stage funnel p. 74
You cannot run a heavy model over 100 million items per request. Production recommenders and search engines narrow the set in stages, each using a more expensive model on fewer items:
Candidate generation p. 74
| Approach | Idea | Notes |
|---|---|---|
| Collaborative filtering / matrix factorisation | Learn user and item vectors from the interaction matrix (ALS, SVD) | Strong signal, suffers cold start |
| Item-to-item similarity | People who engaged with X also engaged with Y (co-occurrence, embeddings) | Simple, explainable, used everywhere |
| Two-tower retrieval | Neural user tower and item tower; score = dot product; items indexed in an ANN index | Industry standard for large catalogs |
| Content-based | Match item features (text, image, category) to user profile | Handles new items |
| Graph-based | Random walks / GNNs on user-item graph (PinSage) | Captures multi-hop relations |
| Popularity / trending / editorial | Global or segment-level hot items | Needed for cold start and fallback |
Ranking p. 75
- Features: user (demographics, long- and short-term interests, history), item (content, quality, age, popularity), context (device, time, location), and crosses (user-item affinity, historical CTR of this user on this category).
- Models: GBDT for strong tabular baselines; deep ranking models (Wide & Deep, DeepFM, DCN, DIN, transformers over behaviour sequences) for high-cardinality sparse features with learned embeddings.
- Multi-task learning: predict several outcomes (click, like, share, dwell time, purchase) and combine them with tuned weights, since optimising clicks alone promotes clickbait.
- Loss and calibration: pointwise (log-loss), pairwise or listwise (LambdaRank/LambdaMART) objectives. Calibrate scores when they are combined or used in auctions.
- Position bias: items shown higher get more clicks regardless of quality. Log positions, train with position as a feature that is neutralised at serving, or use inverse propensity weighting.
- Exploration: without it the system only learns about what it already shows. Use epsilon-greedy, Thompson sampling or UCB bandits, a small exploration slice, and boosts for new items.
- Cold start: new users (onboarding questions, popularity by segment, context), new items (content embeddings, exploration quota, creator signals).
- Re-ranking: enforce diversity (MMR, determinantal point processes), freshness, creator fairness, safety, frequency caps and ad placement after scoring.
Search ranking and ads p. 75
| System | How it differs from recommendations |
|---|---|
| Web / product search | Starts from an explicit query: query understanding (spelling, intent, entities), hybrid retrieval (BM25 + dense embeddings), learning-to-rank with query-document features and click models, cross-encoder reranking (Ch. 30) |
| Ads (CTR / CVR prediction) | Extremely sparse features, calibrated probabilities feed an auction (second-price / GSP), budget pacing, and strict latency (< 50 ms); explore-exploit and delayed conversions complicate labels |
| Feeds (Ch. 15) | Ranking over a mixed candidate pool from graph + recommenders, with integrity filters and diversity |
Evaluating recommenders p. 75
- Offline: Recall@K for retrieval; NDCG@K / MAP for ranking; evaluate on a time-split of future interactions; slice by user cohort and item popularity. Offline gains often fail online because logged data is biased by the previous model.
- Online: A/B tests on engagement and long-term value (retention, satisfaction, diversity), plus guardrails (latency, complaints, creator health). Interleaving for fast ranker comparison.
- Feedback loops: popularity bias and filter bubbles reinforce themselves; track coverage, novelty, Gini of exposure, and keep exploration traffic.
Review questions for this chapter have not been written and reviewed yet. The chapter text above is the full source content.
Try it
References
System Design & AI System Design: The Complete Guide — 2026 Edition, pp. 74–76; figures 51, 52.
Tip: use ← and → keys to move between chapters.