📄️ Router - Load Balancing
LiteLLM manages:
📄️ [BETA] Request Prioritization
Beta feature. Use for testing only.
📄️ Auto Routing
LiteLLM can auto select the best model for a request based on rules you define.
📄️ Proxy - Load Balancing
Load balance multiple instances of the same model
📄️ UI - Router Settings for Keys and Teams
Configure router settings at the key and team level to achieve granular control over routing behavior, fallbacks, retries, and other router configurations. This enables you to customize routing behavior for specific keys or teams without affecting global settings.
📄️ Budget Routing
LiteLLM Supports setting the following budgets:
📄️ Fallbacks
If a call fails after num_retries, fallback to another model group.
📄️ [New] Fallback Management Endpoints
Dedicated endpoints for managing model fallbacks separately from the general configuration.
📄️ Tag Based Routing
Route requests based on tags.
📄️ Tag Regex Routing
Route requests to specific deployments based on regex patterns matched against request headers — without requiring per-user tag configuration.
📄️ Timeouts
The timeout set in router is for the entire length of the call, and is passed down to the completion() call level as well.
📄️ Provider specific Wildcard routing
Proxy all models from a provider