genai.extensions.llama_index package

Extension for LLamaIndex library

class genai.extensions.llama_index.IBMGenAILlamaIndex[source]

Bases: LLM

__init__(*, client, model_id, callback_manager=None, **kwargs)[source]

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters:

  • client (Client)

  • model_id (str)

  • callback_manager (CallbackManager | None)

  • kwargs (Any)

async achat(messages, **kwargs)[source]

Async chat endpoint for LLM.

Parameters:

  • messages (Sequence[ChatMessage]) – Sequence of chat messages.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Chat response from the LLM.

Return type:

ChatResponse

Examples

```python from llama_index.core.llms import ChatMessage

response = await llm.achat([ChatMessage(role=”user”, content=”Hello”)]) print(response.content) ```

async acomplete(*args, **kwargs)[source]

Async completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

  • prompt (str) – Prompt to send to the LLM.

  • formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Completion response from the LLM.

Return type:

CompletionResponse

Examples

`python response = await llm.acomplete("your prompt") print(response.text) `

async astream_chat(messages, **kwargs)[source]

Async streaming chat endpoint for LLM.

Parameters:

  • messages (Sequence[ChatMessage]) – Sequence of chat messages.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

ChatResponse – An async generator of ChatResponse objects, each containing a new token of the response.

Return type:

AsyncGenerator[ChatResponse, None]

Examples

```python from llama_index.core.llms import ChatMessage

gen = await llm.astream_chat([ChatMessage(role=”user”, content=”Hello”)]) async for response in gen:

print(response.delta, end=””, flush=True)

```

async astream_complete(*args, **kwargs)[source]

Async streaming completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

  • prompt (str) – Prompt to send to the LLM.

  • formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

CompletionResponse – An async generator of CompletionResponse objects, each containing a new token of the response.

Return type:

AsyncGenerator[CompletionResponse, None]

Examples

```python gen = await llm.astream_complete(“your prompt”) async for response in gen:

print(response.text, end=””, flush=True)

```

chat(messages, **kwargs)[source]

Chat endpoint for LLM.

Parameters:

  • messages (Sequence[ChatMessage]) – Sequence of chat messages.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Chat response from the LLM.

Return type:

ChatResponse

Examples

```python from llama_index.core.llms import ChatMessage

response = llm.chat([ChatMessage(role=”user”, content=”Hello”)]) print(response.content) ```

classmethod class_name()[source]

Get the class name, used as a unique ID in serialization.

This provides a key that makes serialization robust against actual class name changes.

Return type:

str

client: Client
complete(*args, **kwargs)[source]

Completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

  • prompt (str) – Prompt to send to the LLM.

  • formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Completion response from the LLM.

Return type:

CompletionResponse

Examples

`python response = llm.complete("your prompt") print(response.text) `

conversation_id: str | None
data: PromptTemplateData | None
property metadata: LLMMetadata

LLM metadata.

Returns:

LLM metadata containing various information about the LLM.

Return type:

LLMMetadata

model_id: str
moderations: ModerationParameters | None
parameters: TextGenerationParameters | None
parent_id: str | None
prompt_id: str | None
prompt_template_id: str | None
stream_chat(messages, **kwargs)[source]

Streaming chat endpoint for LLM.

Parameters:

  • messages (Sequence[ChatMessage]) – Sequence of chat messages.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

ChatResponse – A generator of ChatResponse objects, each containing a new token of the response.

Return type:

Generator[ChatResponse, None, None]

Examples

```python from llama_index.core.llms import ChatMessage

gen = llm.stream_chat([ChatMessage(role=”user”, content=”Hello”)]) for response in gen:

print(response.delta, end=””, flush=True)

```

stream_complete(*args, **kwargs)[source]

Streaming completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

  • prompt (str) – Prompt to send to the LLM.

  • formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

  • kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

CompletionResponse – A generator of CompletionResponse objects, each containing a new token of the response.

Return type:

Generator[CompletionResponse, None, None]

Examples

```python gen = llm.stream_complete(“your prompt”) for response in gen:

print(response.text, end=””, flush=True)

```

trim_method: str | TrimMethod | None
use_conversation_parameters: bool | None
class genai.extensions.llama_index.IBMGenAILlamaIndexEmbedding[source]

Bases: BaseEmbedding

classmethod class_name()[source]

Get the class name, used as a unique ID in serialization.

This provides a key that makes serialization robust against actual class name changes.

Return type:

str

client: Client
embed_batch_size: int
execution_options: dict | CreateExecutionOptions | None
model_id: str
parameters: dict | TextEmbeddingParameters | None

Submodules