AI Foundry Agent Service
Managed agents with tools and memory.
Problem it solves: Multi-step tasks need control flow around the model.
Where it sits: Between the API and LLM/tool servers.
When to use
- Tasks needing tools and multiple steps.
When not to use
- Single-shot Q&A.
Failure modes
- Unbounded loops multiply cost and latency.
- Tool errors cascade.
Metrics to watch
- Steps per task
- Task success rate
- Cost per task
Alternatives
- Fixed workflow
Configuration in Labs (illustrative defaults)
- Instances (count) = 2 — Identical copies behind this logical component. Capacity scales linearly in this model.
- Capacity per instance (req/s) = 300 — Maximum requests one instance completes per second before it saturates.
- Base service time (ms) = 10 — Processing time per request at zero load. Queueing adds to this as utilisation rises.
Vendor examples (pattern only): Azure managed service. Simulated as a generic “Agent / orchestrator”.