Evaluation / monitoring
Logs traces, scores outputs and tracks quality, cost and drift.
Problem it solves: AI quality regressions are invisible without evals.
Where it sits: Async side-path from LLM/model services.
When to use
- All production AI.
Failure modes
- Sampling too little misses regressions.
Metrics to watch
- Eval scores
- Cost per request
- Drift
Configuration in Labs (illustrative defaults)
- Instances (count) = 1 — Identical copies behind this logical component. Capacity scales linearly in this model.
- Capacity per instance (req/s) = 2000 — Maximum requests one instance completes per second before it saturates.
- Base service time (ms) = 5 — Processing time per request at zero load. Queueing adds to this as utilisation rises.