Provider routing

Each model is served by one or more upstream providers. By default the gateway picks for you and falls back to another provider if one fails. Add a provider object to a request to steer that choice.

The gateway removes provider before forwarding, so it is never sent upstream. The shape is compatible with OpenRouter's.

json

{
  "model": "deepseek/deepseek-v4-pro",
  "messages": [{ "role": "user", "content": "Hello" }],
  "provider": {
    "order": ["deepinfra", "together"],
    "allow_fallbacks": false
  }
}

Options#

Field	Type	What it does
`only`	string[]	Allow only these providers.
`ignore`	string[]	Never use these providers.
`order`	string[]	Preferred order. Earlier entries are tried first; the rest remain as fallbacks.
`sort`	string	`"price"`, `"latency"`, or `"throughput"`. Picks deterministically along that axis.
`allow_fallbacks`	boolean	`false` tries the first provider once, with no retry on another. Default `true`.
`quantizations`	string[]	Pin quantization variants, for example `["fp8"]`.
`zdr`	boolean	`true` uses only providers committed to zero data retention. See Data policy.
`data_collection`	string	`"deny"` uses only providers committed not to train on your data. See Data policy.

Naming providers#

A provider name matches every region that provider serves. To target one region, use name/region:

json

{ "provider": { "only": ["deepinfra", "together/eu"] } }

Names are case-insensitive. The provider list is on Providers, and each model page shows which providers serve it.

When nothing qualifies#

only, ignore, quantizations, zdr, and data_collection are filters: a provider that fails one is not used. If no provider for the model passes, the request fails with a 503. For the data-policy filters the error is no_provider_matches_data_policy. The gateway never falls back to a provider that does not meet your filters.

Invalid values#

A malformed provider field is a 400, never a silent no-op. A routing preference that is ignored would still return 200, and you would have no way to tell.

Limiting providers for a whole account#

A provider object applies to one request. For a rule that must always hold, such as a data policy, set it once on the account instead of in every request. See Access controls.