Skip to main content

0.22.0 → 0.30.0

This release introduces a typed request and response hierarchy to support the new ModelGatewayService alongside the existing ChatService and DeploymentService. Most changes are in the chat API surface.

ChatResponse is now a base class​

ChatResponse has been refactored from a final class into an open base class. The fields that are specific to IBM watsonx.ai native models (modelId, modelVersion, createdAt, moderations, detections) have been moved to the new TextChatResponse subclass.

Return types are now specific to each service:

Servicechat()chatStreaming()
ChatServiceTextChatResponseCompletableFuture<ChatResponse>
DeploymentServiceTextChatResponseCompletableFuture<ChatResponse>
ModelGatewayServiceModelGatewayChatResponseCompletableFuture<ChatResponse>

ModelGatewayChatResponse is a subclass of TextChatResponse. In most cases no change is needed if you use var or assign to the concrete type.

Before:

ChatResponse response = chatService.chat("Hello!");
String modelId = response.modelId();
Map<String, List<ChatResponse.ModerationResult>> moderations = response.moderations();

After:

TextChatResponse response = chatService.chat("Hello!");
String modelId = response.modelId();
Map<String, List<TextChatResponse.ModerationResult>> moderations = response.moderations();

The inner record types ModerationResult, DetectionEntry, and DetectionResult are now nested inside TextChatResponse instead of ChatResponse.

PartialChatResponse has three new components​

PartialChatResponse gained serviceTier, systemFingerprint, and cached at the end of its component list, to carry the metadata returned by the OpenAI-compatible endpoints used by the Model Gateway. Reading the record is unaffected, but any code calling the canonical constructor (typically test fixtures or custom REST clients) must pass the three additional arguments.

Before:

new PartialChatResponse(id, object, modelId, model, choices, created,
    modelVersion, createdAt, usage, moderations, detections);

After:

new PartialChatResponse(id, object, modelId, model, choices, created,
    modelVersion, createdAt, usage, moderations, detections,
    serviceTier, systemFingerprint, cached);

ChatRequest.deploymentId() removed​

The deploymentId field has been removed from ChatRequest. It was previously accepted but ignored by ChatService. When using DeploymentService, use the new dedicated DeploymentChatRequest type instead.

Before:

ChatRequest request = ChatRequest.builder()
    .deploymentId("my-deployment-id")
    .messages(UserMessage.text("Hello!"))
    .build();

deploymentService.chat(request);

After:

DeploymentChatRequest request = DeploymentChatRequest.builder()
    .deploymentId("my-deployment-id")
    .messages(UserMessage.text("Hello!"))
    .build();

deploymentService.chat(request);

DeploymentService now requires DeploymentChatRequest​

The chat() and chatStreaming() methods of DeploymentService now accept DeploymentChatRequest instead of ChatRequest. DeploymentChatRequest has the same builder API as ChatRequest plus the deploymentId() field.

Before:

ChatRequest request = ChatRequest.builder()
    .deploymentId("my-deployment-id")
    .messages(UserMessage.text("Hello!"))
    .parameters(ChatParameters.builder().temperature(0.7).build())
    .build();

ChatResponse response = deploymentService.chat(request);

After:

DeploymentChatRequest request = DeploymentChatRequest.builder()
    .deploymentId("my-deployment-id")
    .messages(UserMessage.text("Hello!"))
    .parameters(ChatParameters.builder().temperature(0.7).build())
    .build();

TextChatResponse response = deploymentService.chat(request);

ChatParameters.ToolChoiceOption moved to BaseChatParameters​

The ToolChoiceOption enum has been extracted from ChatParameters to the new BaseChatParameters class, which is the common parent for both ChatParameters and ModelGatewayParameters.

Because ChatParameters extends BaseChatParameters, the old ChatParameters.ToolChoiceOption reference still compiles through inheritance, so updating the import is a recommended cleanup rather than a required change.

Before:

import com.ibm.watsonx.ai.chat.model.ChatParameters.ToolChoiceOption;

After:

import com.ibm.watsonx.ai.chat.model.BaseChatParameters.ToolChoiceOption;

Interceptors are now parameterized by request type​

MessageInterceptor, ToolInterceptor, and InterceptorContext take a type parameter R extends BaseChatRequest, which each service pins to the request type it accepts:

ServiceInterceptor type
ChatServiceMessageInterceptor<ChatRequest>, ToolInterceptor<ChatRequest>
DeploymentServiceMessageInterceptor<DeploymentChatRequest>, ToolInterceptor<DeploymentChatRequest>
ModelGatewayServiceMessageInterceptor<ModelGatewayChatRequest>, ToolInterceptor<ModelGatewayChatRequest>

If you register interceptors as inline lambdas on the service builder, no change is required - the type argument is inferred from the setter.

The type argument only needs to be written out when you declare the interceptor separately or implement the interface in a named class. In exchange, context.request() returns the concrete request type, so no cast is needed:

Before:

MessageInterceptor interceptor = (context, message) -> {
    ChatRequest request = context.request();
    // ...
};

After:

MessageInterceptor<ChatRequest> interceptor = (context, message) -> {
    ChatRequest request = context.request();
    // ...
};

InterceptorContext.invoke() accepts the same concrete type, so passing a request meant for a different service is now a compile error instead of a runtime failure.

ChatProvider is now generic​

ChatProvider has become ChatProvider<R extends BaseChatRequest, C extends ChatResponse>. An implementation declares both the request type it accepts and the response type it returns, so chat() receives R and returns C. Previously the interface was hard-wired to ChatRequest and ChatResponse, so a provider serving deployments or the Model Gateway had no way to express the types it actually worked with.

Before:

public class MyChatProvider implements ChatProvider {

    @Override
    public ChatResponse chat(ChatRequest chatRequest) {
        // ...
    }

    @Override
    public CompletableFuture<ChatResponse> chatStreaming(ChatRequest chatRequest, ChatHandler handler) {
        // ...
    }
}

After:

public class MyChatProvider implements ChatProvider<ChatRequest, TextChatResponse> {

    @Override
    public TextChatResponse chat(ChatRequest chatRequest) {
        // ...
    }

    @Override
    public CompletableFuture<ChatResponse> chatStreaming(ChatRequest chatRequest, ChatHandler handler) {
        // ...
    }
}

The implements clause gains the two type arguments and chat() must return the response type declared as C, while chatStreaming() keeps returning CompletableFuture<ChatResponse>. The combinations used by the SDK services are:

ServiceDeclaration
ChatServiceChatProvider<ChatRequest, TextChatResponse>
DeploymentServiceChatProvider<DeploymentChatRequest, TextChatResponse>
ModelGatewayServiceChatProvider<ModelGatewayChatRequest, ModelGatewayChatResponse>

Passing a request meant for another service is now rejected at compile time.

ChatClientContext is now generic​

This affects you only if you provide a custom REST client through the REST Client SPI. The chatStreaming() method of ChatRestClient, DeploymentRestClient, and ModelGatewayRestClient now receives a ChatClientContext parameterized with the service's request type, and an override must match it:

Before:

@Override
public CompletableFuture<ChatResponse> chatStreaming(String transactionId, TextChatRequest textChatRequest,
    ChatClientContext context, ChatHandler handler) {
    // ...
}

After:

@Override
public CompletableFuture<ChatResponse> chatStreaming(String transactionId, TextChatRequest textChatRequest,
    ChatClientContext<ChatRequest> context, ChatHandler handler) {
    // ...
}

Use ChatClientContext<DeploymentChatRequest> in a DeploymentRestClient and ChatClientContext<ModelGatewayChatRequest> in a ModelGatewayRestClient.

New ModelGatewayService​

This release adds ModelGatewayService for sending chat requests to third-party models through the IBM watsonx.ai Model Gateway endpoint. It is a new, additive API with no migration required. See the Model Gateway documentation for details.

toAssistantMessage() and toAssistantMessages() now throw EmptyChatResponseException​

ChatResponse.toAssistantMessage() and toAssistantMessages() previously threw IllegalStateException when the response contained no usable output. They now throw EmptyChatResponseException, a dedicated runtime exception that carries the FinishReason of the empty choice, its zero-based index, and the original ChatResponse.

Callers that caught IllegalStateException to handle empty responses must be updated:

Before:

try {
    AssistantMessage message = response.toAssistantMessage();
} catch (IllegalStateException e) {
    // handle empty response
}

After:

try {
    AssistantMessage message = response.toAssistantMessage();
} catch (EmptyChatResponseException e) {
    FinishReason reason = e.finishReason(); // e.g. LENGTH, CONTENT_FILTER
    int index = e.index();                  // -1 when no choices at all
    ChatResponse original = e.response();
}

Callers that did not catch IllegalStateException are unaffected at runtime, but should update import statements.

AssistantMessage record gained a thinking field​

The AssistantMessage record gained a thinking component that carries the model's reasoning content. The canonical constructor now takes an additional String thinking argument between content and name:

Before:

new AssistantMessage("assistant", content, name, refusal, toolCalls);

After:

new AssistantMessage("assistant", content, thinking, name, refusal, toolCalls);

Code that uses the static factory methods (AssistantMessage.text(), AssistantMessage.tools()) or constructs the record via toAssistantMessage() is unaffected. Only code calling the canonical 5- or 6-argument constructor directly needs to be updated.

New ModelGatewayCatalogService​

This release adds ModelGatewayCatalogService for listing and fetching models configured in the IBM watsonx.ai Model Gateway. It is a new, additive API with no migration required. See the Model Gateway Catalog documentation for details.

New ClusterSchemaService​

This release adds ClusterSchemaService for grouping a set of document schemas into semantically similar clusters. It is a new, additive API with no migration required.