Skip to main content

0.30.1 → 0.40.0

This release renames the Model Gateway chat types, moves EmptyChatResponseException into a dedicated package, introduces ModerationException for responses blocked by the moderation system, makes every collection the SDK exposes unmodifiable, adds DeveloperMessage, PartialResponseInterceptor, and streaming support for MessageInterceptor, and ships GraalVM native image reachability metadata for both modules.

What's new​

GraalVM native image support​

Both watsonx-ai-core and watsonx-ai ship GraalVM reachability metadata under META-INF/native-image/. No extra configuration is required: add the SDK to a native-image project and the reflection, proxy, and resource entries are picked up automatically.

DeveloperMessage​

DeveloperMessage is a new ChatMessage subtype that carries the OpenAI developer role. Newer models use it in place of SystemMessage to set the assistant's behavior. It is accepted only by the Model Gateway chat APIs (ModelGatewayChatService and ModelGatewayTextChatRequest).

import com.ibm.watsonx.ai.chat.model.DeveloperMessage;

messages.add(DeveloperMessage.of("You are a helpful assistant"));

Passing a DeveloperMessage to ChatService or DeploymentService throws an IllegalArgumentException. Use SystemMessage there instead.

PartialResponseInterceptor and MessageInterceptor in streaming​

MessageInterceptor now fires in streaming mode as well. Previously it was applied only to non-streaming responses. In 0.40.0 it is invoked once per assistant message with the complete aggregated content, both when calling chat(...) and when calling chatStreaming(...).

A new PartialResponseInterceptor companion intercepts each individual content token before it is delivered to ChatHandler.onPartialResponse. Register both on ChatService, DeploymentService, and ModelGatewayChatService through their builders:

ChatService.builder()
    // called once with the full message (streaming and non-streaming)
    .messageInterceptor((ctx, content) -> content == null ? "" : content.strip())
    // called once per token in streaming mode
    .partialResponseInterceptor((ctx, token) -> token.replace("foo", "bar"))
    ...

PartialResponseInterceptor receives the token before it reaches onPartialResponse, so what the handler sees is already the transformed text. ctx.response() is always empty inside a PartialResponseInterceptor because no complete response exists at that point. See Chat - Interceptors for the full reference.

Breaking changes​

Model Gateway chat types are qualified with Chat​

Chat completions were the only Model Gateway operation in 0.30.1, so the chat types owned the unqualified ModelGateway* name. Now that the gateway also exposes ModelGatewayCatalogService, ModelGatewayEmbeddingService, and ModelGatewayImageService, the chat types are named after the API they target too.

BeforeAfter
ModelGatewayServiceModelGatewayChatService
ModelGatewayParametersModelGatewayChatParameters
ModelGatewayRestClientModelGatewayChatRestClient
ModelGatewayUtilityModelGatewayChatUtility

Packages and method signatures are unchanged, and so are ModelGatewayChatRequest, ModelGatewayChatResponse, and ModelGatewayTextChatRequest, which were already qualified. The nested types of the parameters class (ReasoningEffort, ServiceTier, Prediction, StreamOptions, Cache, and Router) keep their names as well, only the enclosing class changes.

Custom REST clients​

If you provide your own Model Gateway chat client through the Service Provider Interface, extend ModelGatewayChatRestClient and implement its nested ModelGatewayChatRestClientBuilderFactory, then rename the ServiceLoader registration file accordingly.

Before:

META-INF/services/com.ibm.watsonx.ai.gateway.chat.ModelGatewayRestClient$ModelGatewayRestClientBuilderFactory

After:

META-INF/services/com.ibm.watsonx.ai.gateway.chat.ModelGatewayChatRestClient$ModelGatewayChatRestClientBuilderFactory

EmptyChatResponseException moved to a dedicated package​

Chat exceptions now live in their own package, so EmptyChatResponseException moved out of com.ibm.watsonx.ai.chat. The class, its constructor, its accessors (finishReason(), index(), response()), and the NO_CHOICE constant are unchanged, only the import has to be updated.

BeforeAfter
com.ibm.watsonx.ai.chat.EmptyChatResponseExceptioncom.ibm.watsonx.ai.chat.exception.EmptyChatResponseException

Before:

import com.ibm.watsonx.ai.chat.EmptyChatResponseException;

After:

import com.ibm.watsonx.ai.chat.exception.EmptyChatResponseException;

Moderation-blocked responses throw ModerationException​

When the moderation system blocks a chat response entirely, the API returns the moderation results without any usable choice. Building an assistant message out of that response previously surfaced as an EmptyChatResponseException, which said nothing about the reason, and the streaming path failed with a NullPointerException on the moderation sentinel chunk.

Both paths now report com.ibm.watsonx.ai.chat.exception.ModerationException, which carries the detector results that triggered the block:

try {
    AssistantMessage message = response.toAssistantMessage();
} catch (ModerationException e) {
    // Map from detector name ("pii", "hap", "granite_guardian") to the flagged spans
    e.moderations().forEach((detector, results) ->
        logger.warn("{} blocked the response: {} match(es)", detector, results.size()));
}

In streaming mode the returned CompletableFuture completes exceptionally with the same exception, and ChatHandler.onError receives it.

If you were catching EmptyChatResponseException to handle blocked responses, catch ModerationException instead. EmptyChatResponseException keeps its original meaning of a response with no usable output for reasons unrelated to moderation, such as truncation by maxCompletionTokens. To branch without exceptions, TextChatResponse.isBlockedByModeration() answers the same question on the response object. See Error Handling.

Collections exposed by the SDK are unmodifiable​

Every response record, request, and parameters class that carries a collection now takes a defensive copy at construction time. Two consequences:

  • Accessors return unmodifiable collections. Mutating a list or map obtained from the SDK, for example tokenizationResponse.result().tokens().add("x") or embeddingResponse.results(), now throws UnsupportedOperationException. Copy it into a collection of your own first.
  • Collections passed to the SDK are snapshotted. Changing your own list after handing it over no longer affects the object you built. ForecastData.from(map) copies the map instead of wrapping it, and RetryInterceptor.retryOn(...), ChatService.Builder.tools(...), and InputSchema.Builder.idColumns(...) behave the same way.

A null collection is still accepted and preserved as null, but a collection that contains null elements is now rejected with a NullPointerException, because the copy is taken with List.copyOf and Map.copyOf.

Behavior changes​

These do not break compilation, but they change what your code observes at runtime.

Streams can be cancelled​

chatStreaming and generateStreaming return a CompletableFuture whose cancel(...) now aborts the response body subscription and closes the connection, so the model stops streaming. Previously cancelling the future left the stream running in the background. After cancellation no further callback reaches the handler, not even onError, and the future ends in the cancelled state so a later get() or join() throws a CancellationException. See Chat.

watsonx.ai API version​

The API version sent with every request moved from 2026-07-08 to 2026-08-19. No request or response shape changed for the operations covered by the SDK.

502 Bad Gateway is retried​

502 joined 429, 503, 504, and 520 in the set of status codes retried automatically with exponential backoff. A gateway error that used to fail immediately is now retried, so a failing call can take longer before it throws. Tune it with the WATSONX_RETRY_STATUS_CODES_* variables described in Environment Variables.