Interface Inference

The details of an inference API.

interface Inference {
    sse?: boolean;
    url: string;
    uses_serving_name?: boolean;
}

Properties

sse?: boolean

This is true if the inference API supports SSE streaming.

url: string

The inference URL.

uses_serving_name?: boolean

This is true if the inference API uses the serving_name that was defined in this deployment.