Interface DeploymentsTextChatStreamParams

If specified, context will be inserted into messages. Depending on the model, context may be inserted into the content with system role; or into the content of the last message of user role.

In the example, context "Today is Wednesday" is inserted as such content of user becomes "Today is Wednesday. Who are you and which day is tomorrow?".

The id_or_name can be either the deployment_id that identifies the deployment or a serving_name that allows a predefined URL to be used to post a prediction. The deployment must reference a prompt template with input_mode chat.

The WML instance that is associated with the deployment will be used for limits and billing (if a paid plan).

The messages for this chat session. You cannot specify system role in the messages. Depending on the model, the content of system role may be from system_prompt of the prompt template, and will be automatically inserted into messages.

As an example, depending on the model, if system_prompt of a prompt template is "You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", a message with system role having content the same as system_prompt is inserted.