Router Configuration
The inference router selects which model backend handles each agent request.
Basic setup
[router]
default_route = "local"
[[router.routes]]
name = "local"
provider = "ollama"
model = "llama3.2"
base_url = "http://127.0.0.1:11434"
[[router.routes]]
name = "cloud"
provider = "anthropic"
model = "claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"
Route selection
The router picks a route based on:
- Agent hint — The agent can request a specific model via
infer_with_model. - Route profile — Weighted selection based on configured thresholds.
- Default route — Falls back to
default_routeif no match.
Route profiles
Fine-tune routing behavior per agent:
[[agents]]
name = "researcher"
[agents.route_profile]
prefer = "cloud"
fallback = "local"
max_latency_ms = 5000
cost_weight = 0.3
quality_weight = 0.7
| Field | Description |
|---|---|
prefer | Preferred route name |
fallback | Route to use if preferred is unavailable |
max_latency_ms | Switch to fallback if latency exceeds this |
cost_weight | Weight for cost optimization (0.0-1.0) |
quality_weight | Weight for quality optimization (0.0-1.0) |
Provider reference
| Provider | provider value | Required fields |
|---|---|---|
| Ollama | ollama | base_url, model |
| Anthropic | anthropic | api_key, model |
| OpenAI-compatible | openai | base_url, api_key, model |
Testing routes
Verify routing with:
akshi config-check --test-routes
This sends a test prompt to each configured route and reports latency and availability.