Model Gateway (BETA)

Note

Model Gateway is in currently in beta stage and available only on IBM watsonx.ai for IBM Cloud. Breaking changes in API may be introduced in the future.

Model Gateway provides proxy for inference requests to many model providers. The feature contain easy model usage with load balancing.

Gateway

class ibm_watsonx_ai.gateway.Gateway(*, credentials=None, verify=None, api_client=None)[source]

Model Gateway class.

clear_secrets_manager()[source]

Clear Model Gateway configuration.

Returns:

status (“SUCCESS” if succeeded)

Return type:

str

set_secrets_manager(secrets_manager, name='Watsonx AI Model Gateway configuration')[source]

Configure Model Gateway by, among others, setting Secrets Manager url.

Parameters:

  • secrets_manager (str) – Secrets Manager url

  • name (str, optional) – Model Gateway configuration name

Providers

class ibm_watsonx_ai.gateway.providers.Providers(api_client)[source]

Model Gateway providers class.

create(provider, name=None, data=None)[source]

Create provider in Model Gateway.

Parameters:

  • provider (str) – provider name

  • name (str, optional) – name of provider for display

  • data (dict, optional) – data required to connect to provider api

Returns:

provider details

Return type:

dict

delete(provider_id)[source]

Delete provider.

Parameters:

provider_id (str) – unique provider ID

Returns:

status (“SUCCESS” if succeeded)

Return type:

str

get_available_models_details(provider_id)[source]

Get available models details for given provider.

Parameters:

provider_id (str) – unique provider ID

Returns:

details of available models for provider

Return type:

dict

get_details(provider_id=None)[source]
Get provider/providers details:

  • provider_id is set - details for given provider are returned

  • provider_id is None - details for all providers are returned

Parameters:

provider_id (str, optional) – unique provider ID

Returns:

provider/providers details

Return type:

dict

static get_id(provider_details)[source]

Get provider ID from provider details.

Parameters:

provider_details (dict) – details of the provider in Model Gateway

Returns:

unique provider ID

Return type:

str

list()[source]

List providers.

Returns:

dataframe with providers details

Return type:

pandas.DataFrame

list_available_models(provider_id)[source]

List available models for provider.

Parameters:

provider_id (str) – unique provider ID

Returns:

dataframe with available models details

Return type:

pandas.DataFrame

Models

class ibm_watsonx_ai.gateway.models.Models(api_client)[source]

Model Gateway models class.

create(provider_id, model, alias=None, metadata=None)[source]

Register model in Model Gateway.

Parameters:

  • provider_id (str) – unique provider ID obtained from provider details

  • model (str) – model name as supported by provider

  • alias (str, optional) – alias for registered model, can be used later as model name during embeddings or text/chat completions calls

  • metadata (dict, optional) – additional metadata which can be added for the model

Returns:

model details

Return type:

dict

delete(model_id)[source]

Unregister model from Model Gateway.

Parameters:

model_id (str) – unique model ID obtained from model details

Returns:

status (“SUCCESS” if succeeded)

Return type:

str

get_details(*, model_id=None, provider_id=None)[source]
Get details of model or models:

  • model_id is set - details for single model are returned, provider_id if set is ignored

  • provider_id is set, model_id is None - details for all models for given provider are returned

  • both model_id and provider_id are None - all models details are returned

Parameters:

  • model_id (str, optional) – unique model ID

  • provider_id (str, optional) – unique provider ID, ignored if model_id is set

Returns:

details of model/models

Return type:

dict

static get_id(model_details)[source]

Get model ID from model details.

Parameters:

model_details (dict) – details of the model registered in Model Gateway

Returns:

unique model ID

Return type:

str

list(provider_id=None)[source]

List models registered in Model Gateway. List can be filtered by provider_id.

Parameters:

provider_id (str, optional) – ID of provider added into Model Gateway

Returns:

dataframe containing list results

Return type:

pandas.DataFrame

Policies

class ibm_watsonx_ai.gateway.policies.Policies(api_client)[source]

Model Gateway policies class.

create(action, resource, subject, effect=None)[source]

Create policy.

Parameters:

  • action (str) – action for policy

  • resource (str) – resource for policy

  • subject (str) – subject for policy

  • effect (str, optional) – effect for policy

delete(action, resource, subject, effect=None)[source]

Delete policy.

Parameters:

  • action (str) – action for policy

  • resource (str) – resource for policy

  • subject (str) – subject for policy

  • effect (str, optional) – effect for policy

Returns:

status (“SUCCESS” if succeeded)

Return type:

str

get_details()[source]

Get policies details.

Returns:

policies details

Return type:

dict

list()[source]

List policies.

Returns:

dataframe with policies details

Return type:

pandas.DataFrame