Service Provider Interface

The IBM watsonx.ai Java SDK exposes several Service Provider Interfaces that allow framework integrators to replace or customize its core infrastructure (HTTP transport, thread management, and JSON serialization) without changing any application code. All SPIs are resolved at runtime via the Java ServiceLoader mechanism.

REST Client SPI

Every service delegates HTTP communication to an abstract WatsonxRestClient, with one subclass per service (e.g. ChatRestClient). The concrete implementation is discovered at startup via ServiceLoader. If none is registered, the SDK falls back to its built-in DefaultRestClient (based on the Java HttpClient).

Service to REST client mapping

Service	REST client
`ChatService`	`ChatRestClient`
`TextGenerationService`	`TextGenerationRestClient`
`EmbeddingService`	`EmbeddingRestClient`
`RerankService`	`RerankRestClient`
`TokenizationService`	`TokenizationRestClient`
`DetectionService`	`DetectionRestClient`
`TextClassificationService`	`TextClassificationRestClient`
`TextExtractionService`	`TextExtractionRestClient`
`CreateSchemaService`	`CreateSchemaRestClient`
`ImproveSchemaService`	`ImproveSchemaRestClient`
`MergeSchemaService`	`MergeSchemaRestClient`
`ClusterSchemaService`	`ClusterSchemaRestClient`
`TimeSeriesService`	`TimeSeriesRestClient`
`FoundationModelService`	`FoundationModelRestClient`
`ToolService`	`ToolRestClient`
`DeploymentService`	`DeploymentRestClient`
`ModelGatewayChatService`	`ModelGatewayChatRestClient`
`ModelGatewayCatalogService`	`ModelGatewayCatalogRestClient`
`ModelGatewayEmbeddingService`	`ModelGatewayEmbeddingRestClient`
`ModelGatewayImageService`	`ModelGatewayImageRestClient`
`FileService`	`FileRestClient`
`BatchService`	`BatchRestClient`

Each RestClient performs the lookup this way:

static ChatRestClient.Builder builder() {
    return ServiceLoader.load(ChatRestClientBuilderFactory.class).findFirst()
        .map(Supplier::get)
        .orElse(DefaultRestClient.builder());
}

Providing a custom REST client

The recommended pattern is to keep the factory and builder as static nested classes inside the custom RestClient. The example below shows a full implementation for ChatRestClient using Quarkus and the RESTEasy Reactive client.

Step 1 - Define the JAX-RS interface

Declare the API endpoints using JAX-RS annotations. The SDK's request/response types can be used directly:

@Path("/ml/v1")
public interface ChatRestApi {

    @POST
    @Path("text/chat")
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    TextChatResponse chat(
        @HeaderParam("X-Request-Id") String requestId,
        @HeaderParam("X-Global-Transaction-Id") String transactionId,
        @QueryParam("version") String version,
        TextChatRequest request);

    @POST
    @Path("text/chat_stream")
    @Consumes(MediaType.APPLICATION_JSON)
    @RestStreamElementType(MediaType.APPLICATION_JSON)
    Multi<String> chatStreaming(
        @HeaderParam("X-Request-Id") String requestId,
        @HeaderParam("X-Global-Transaction-Id") String transactionId,
        @QueryParam("version") String version,
        TextChatRequest request);
}

Step 2 - Implement RestClient

Extend ChatRestClient, build the framework-native client in the constructor using the inherited fields, and implement the abstract methods:

public final class QuarkusChatRestClient extends ChatRestClient {

    private final ChatRestApi client;

    QuarkusChatRestClient(Builder builder) {
        super(builder);
        try {
            client = QuarkusRestClientBuilder.newBuilder()
                .baseUrl(URI.create(baseUrl).toURL())
                .clientHeadersFactory(new BearerTokenHeaderFactory(authenticator))
                .connectTimeout(timeout.toSeconds(), TimeUnit.SECONDS)
                .readTimeout(timeout.toSeconds(), TimeUnit.SECONDS)
                .build(ChatRestApi.class);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public TextChatResponse chat(String transactionId, TextChatRequest textChatRequest) {
        // Quarkus implementation
        // ...
    }

    @Override
    public CompletableFuture<ChatResponse> chatStreaming(
            String transactionId,
            TextChatRequest textChatRequest,
            ChatClientContext<ChatRequest> context,
            ChatHandler handler) {
        // Quarkus implementation
        // ...
    }

    // Factory and Builder as static nested classes
    public static final class QuarkusChatRestClientBuilderFactory
            implements ChatRestClientBuilderFactory {
        @Override
        public Builder get() {
            return new QuarkusChatRestClient.Builder();
        }
    }

    static final class Builder extends ChatRestClient.Builder {
        @Override
        public ChatRestClient build() {
            return new QuarkusChatRestClient(this);
        }
    }
}

Note: ChatClientContext is parameterized by the request type of the service it belongs to, and the override must match. Use ChatClientContext<ChatRequest> in a ChatRestClient, ChatClientContext<DeploymentChatRequest> in a DeploymentRestClient, and ChatClientContext<ModelGatewayChatRequest> in a ModelGatewayChatRestClient.

Step 3 - Register via `ServiceLoader`

Create the file:

META-INF/services/com.ibm.watsonx.ai.chat.ChatRestClient$ChatRestClientBuilderFactory

The $ separator is the JVM convention for nested class names in ServiceLoader registration files. The file content is the fully qualified name of the factory:

io.quarkiverse.langchain4j.watsonx.runtime.client.impl.QuarkusChatRestClient$QuarkusChatRestClientBuilderFactory

Once registered, any ChatService built in that runtime will automatically use your implementation.

Real-world example: Quarkus integration

The quarkus-langchain4j-watsonx integration uses this SPI to replace the default Java HttpClient with Quarkus's reactive RESTEasy client. This allows the SDK to participate in Quarkus's managed thread model, reactive pipelines (Mutiny), and GraalVM native compilation. A *BuilderFactory is registered for each service via CDI, so the consuming application uses the same ChatService, EmbeddingService, etc. API without any modification.

See quarkus-langchain4j-watsonx for the complete reference implementation.

Executor SPI

The SDK uses three distinct executors internally, each replaceable independently via ServiceLoader.

SPI interface	Default behavior	Used for
`CpuExecutorProvider`	`ForkJoinPool.commonPool()`	CPU-bound tasks: JSON parsing, data transformation
`IOExecutorProvider`	Virtual threads (Java 21+), cached thread pool (Java 17–20). A fixed pool can be forced via `WATSONX_IO_EXECUTOR_THREADS`	HTTP response processing, SSE stream parsing
`CallbackExecutorProvider`	Virtual threads (Java 21+), cached thread pool (Java 17–20)	User callbacks in `ChatHandler` and `TextGenerationHandler`

The three executors are intentionally separate to prevent user callback code from blocking the SSE parsing thread, and to keep CPU-bound work off the I/O thread.

The same pattern applies to CpuExecutorProvider and IOExecutorProvider. The executor is loaded once at startup and cached for the lifetime of the JVM.

Providing a custom executor

Each executor provider is a functional interface with a single executor() method. The SDK loads the provider via ServiceLoader at startup:

private static CallbackExecutorProvider loadCallbackExecutorProvider() {
    return ServiceLoader.load(CallbackExecutorProvider.class)
        .findFirst().orElse(null);
}

Here's a complete example of replacing the CallbackExecutorProvider with a custom fixed thread pool:

Step 1 - Implement the provider

package com.example.watsonx.executor;

import com.ibm.watsonx.ai.core.spi.executor.CallbackExecutorProvider;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

public class CustomCallbackExecutorProvider implements CallbackExecutorProvider {
    
    private static final Executor EXECUTOR;
    
    static {
        AtomicInteger counter = new AtomicInteger(1);
        EXECUTOR = Executors.newFixedThreadPool(10, r -> {
            Thread thread = new Thread(r);
            thread.setName("watsonx-callback-" + counter.getAndIncrement());
            thread.setDaemon(true);
            return thread;
        });
    }

    @Override
    public Executor executor() {
        return EXECUTOR;
    }
}

Step 2 - Register via `ServiceLoader`

Create the file:

META-INF/services/com.ibm.watsonx.ai.core.spi.executor.CallbackExecutorProvider

With the content:

com.example.watsonx.executor.CustomCallbackExecutorProvider

Once registered, all user callbacks (ChatHandler, TextGenerationHandler) will execute on your custom thread pool instead of the default virtual threads or cached thread pool.

Example: Custom CPU executor

Similarly, you can replace the CPU executor used for JSON parsing and data transformation:

package com.example.watsonx.executor;

import com.ibm.watsonx.ai.core.spi.executor.CpuExecutorProvider;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;

public class CustomCpuExecutorProvider implements CpuExecutorProvider {
    
    private static final Executor EXECUTOR =
        Executors.newWorkStealingPool(Runtime.getRuntime().availableProcessors());

    @Override
    public Executor executor() {
        return EXECUTOR;
    }
}

META-INF/services/com.ibm.watsonx.ai.core.spi.executor.CpuExecutorProvider

With:

com.example.watsonx.executor.CustomCpuExecutorProvider

JSON SPI

By default, the SDK uses Jackson for JSON serialization and deserialization, configured with snake_case property naming and NON_NULL inclusion. The JsonProvider SPI allows replacing this with any other JSON library.

public interface JsonProvider {
    <T> T fromJson(String json, Class<T> clazz);
    <T> T fromJson(String json, TypeToken<T> typeToken);
    String toJson(Object object);
    String prettyPrint(Object object);
    boolean isValidObject(String json);
}

TypeToken<T> is a utility class provided by the SDK to capture generic type information at runtime, used for deserializing parameterized types like List<ChatMessage> or DetectionResponse<DetectionTextResponse>.

The SDK loads the provider via ServiceLoader at startup:

private static JsonProvider loadProvider() {
    return ServiceLoader.load(JsonProvider.class)
        .findFirst().orElse(new JacksonProvider());
}

Jackson configuration & mix-in annotations

The default JacksonProvider is configured with:

new ObjectMapper()
    .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
    .setDefaultPropertyInclusion(Include.NON_NULL)
    .setPropertyNamingStrategy(PropertyNamingStrategies.SNAKE_CASE)
    .findAndRegisterModules();

Additionally, the SDK uses Jackson mix-in annotations (via WatsonxJacksonModule) to handle special serialization/deserialization requirements for SDK types:

Custom builders with @JsonPOJOBuilder(withPrefix = "")
Ignored fields (e.g., @JsonIgnore on AssistantMessage.thinking())
Dynamic properties with @JsonAnyGetter/@JsonAnySetter (e.g., ToolArguments)
Suppressed auto-detection with @JsonAutoDetect(getterVisibility = NONE), so a type is written from its explicitly annotated accessors only (e.g., ModelGatewayEmbeddingResponse.Embedding)

If you replace Jackson with another JSON library, you must replicate these configurations. The mix-ins are defined in com.ibm.watsonx.ai.WatsonxJacksonModule and cover all SDK request/response types.

Providing a custom JSON provider

The example below shows the structure of a custom provider. The implementation details (naming strategy, annotations handling) depend on your chosen JSON library:

Step 1 - Implement the provider

package com.example.watsonx.json;

import com.ibm.watsonx.ai.core.exception.JsonException;
import com.ibm.watsonx.ai.core.spi.json.JsonProvider;
import com.ibm.watsonx.ai.core.spi.json.TypeToken;

public class CustomJsonProvider implements JsonProvider {
    
    @Override
    public <T> T fromJson(String json, Class<T> clazz) {
        // Deserialize JSON string to object
        // Must handle all SDK types with their specific requirements
        // On failure, throw JsonException
    }

    @Override
    public <T> T fromJson(String json, TypeToken<T> typeToken) {
        // Deserialize JSON string to generic type
        // Extract Type from typeToken.getType() and use library's type-aware deserialization
        // On failure, throw JsonException
    }

    @Override
    public String toJson(Object object) {
        // Serialize object to JSON string
        // Apply snake_case naming and NON_NULL inclusion
        // On failure, throw JsonException
    }

    @Override
    public String prettyPrint(Object object) {
        // Serialize with pretty-printing (indentation)
        // Handle both objects and JSON strings
        // Best-effort: fall back to a plain representation instead of throwing
    }

    @Override
    public boolean isValidObject(String json) {
        // Validate if string is a valid JSON object (not array or primitive)
        // Return false for null, blank, or invalid JSON
    }
}

Step 2 - Register via `ServiceLoader`

Create the file:

META-INF/services/com.ibm.watsonx.ai.core.spi.json.JsonProvider

With the content:

com.example.watsonx.json.CustomJsonProvider

Once registered, all JSON operations in the SDK will use your custom provider instead of Jackson.

Using TypeToken for generic types

The SDK provides TypeToken<T> to capture generic type information at runtime, used for deserializing parameterized types like List<ChatMessage> or Map<String, Object>. Your JsonProvider implementation must handle TypeToken by extracting the Type via typeToken.getType() and passing it to your JSON library's type-aware deserialization method.

There are three ways to obtain a TypeToken:

// 1. Anonymous subclass - captures any type, including nested generics and wildcards
TypeToken<Map<String, List<Integer>>> nested = new TypeToken<>() {};

// 2. listOf - convenience factory for List<T>
TypeToken<List<String>> list = TypeToken.listOf(String.class);

// 3. parameterizedOf - factory accepting one or more type arguments, e.g. Map<K, V>
TypeToken<Map<String, Integer>> map = TypeToken.parameterizedOf(Map.class, String.class, Integer.class);

Limitation of parameterizedOf. Each type argument must be a raw Class, so only flat parameterized types can be expressed (e.g. Map<String, Integer>). For nested generics (Map<String, List<Integer>>) or wildcards (List<? extends Number>), use the anonymous subclass form, which captures the full type.

REST Client SPI​

Service to REST client mapping​

Providing a custom REST client​

Step 1 - Define the JAX-RS interface​

Step 2 - Implement RestClient​

Step 3 - Register via ServiceLoader​

Real-world example: Quarkus integration​

Executor SPI​

Providing a custom executor​

Step 1 - Implement the provider​

Step 2 - Register via ServiceLoader​

Example: Custom CPU executor​

JSON SPI​

Jackson configuration & mix-in annotations​

Providing a custom JSON provider​

Step 1 - Implement the provider​

Step 2 - Register via ServiceLoader​

Using TypeToken for generic types​

REST Client SPI

Service to REST client mapping

Providing a custom REST client

Step 1 - Define the JAX-RS interface

Step 2 - Implement RestClient

Step 3 - Register via `ServiceLoader`

Real-world example: Quarkus integration

Executor SPI

Providing a custom executor

Step 1 - Implement the provider

Step 2 - Register via `ServiceLoader`

Example: Custom CPU executor

JSON SPI

Jackson configuration & mix-in annotations

Providing a custom JSON provider

Step 1 - Implement the provider

Step 2 - Register via `ServiceLoader`

Using TypeToken for generic types