Models and routing

Choose which models each AI system and key may use, share traffic across credentials, and fall back when a provider fails.

This is in development. Details may change before it is available.

Model access

The models each provider credential lists are the models your organization can reach. Within those, owners and admins choose for each key, under Providers and models, which providers and models it may use. Every provider and model lets the key use whatever your credentials allow; turn it off to choose providers one by one:

• Allowed models: only these models can be called. Allow every model of a provider with *, or name them. A name can end in * to match a family of models.
• Blocked models: these models are refused, even when the provider is allowed.

A request for a model the key may not use is refused with an error that names the model. GET /v1/models lists only the models your organization has a credential for and the key may use.

Share traffic across providers

When a key may use more than one provider, give each a Weight. Altrum spreads requests between them in proportion to their weights, and tries the heavier ones first when one fails.

Fallbacks

A request can name other models to try, in order, when it fails because the provider is down, overloaded or times out. Altrum tries each in turn, using your organization’s own credential for each provider. Guardrails, budgets and rate limits apply to every attempt. Without a chain on the request, the other providers a key may use are tried in order of weight.