Interface ModelRouter

Router is the model routing configuration for the request.

interface ModelRouter {
    family_model?: string[];
    max_cost?: number;
    models?: string[];
    optimization?: string;
    quality_tradeoff?: number;
    region?: string;
}

Properties

family_model? max_cost? models? optimization? quality_tradeoff? region?

Properties

Optional family_model

family_model?: string[]

Model families to include into the assessment in the case of smart routing.

Optional max_cost

max_cost?: number

Maximum cost of the model to include into the assessment in the case of smart routing.

Optional models

models?: string[]

Model names to include into the assessment in the case of smart routing.

Optional optimization

optimization?: string

Model selection optimization parameter. Options are "efficacy" (default) and "cost".

Optional quality_tradeoff

quality_tradeoff?: number

How much of quality possible to sacrifice during most optimal model selection, value between 0 and 1. Where, e.g. 0.8 means the quality can be reduced by 20%.

Optional region

region?: string

The region of the model to include into the assessment in the case of smart routing.

Settings

Member Visibility

Theme

On This Page

family_modelmax_costmodelsoptimizationquality_tradeoffregion