genai.extensions.llama_index package

Extension for LLamaIndex library

class genai.extensions.llama_index.IBMGenAILlamaIndex

Bases: LLM

__init__(*, client, model_id, callback_manager=None, **kwargs)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Parameters:

• client (Client)

• model_id (str)

• callback_manager (CallbackManager | None)

• kwargs (Any)

async achat(messages, **kwargs)

Async chat endpoint for LLM.

Parameters:

• messages (Sequence[ChatMessage]) – Sequence of chat messages.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Chat response from the LLM.

Return type:

ChatResponse

Examples

```python from llama_index.core.llms import ChatMessage

response = await llm.achat([ChatMessage(role=”user”, content=”Hello”)]) print(response.content) ```

async acomplete(*args, **kwargs)

Async completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

• prompt (str) – Prompt to send to the LLM.

• formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Completion response from the LLM.

Return type:

CompletionResponse

Examples

`python response = await llm.acomplete("your prompt") print(response.text) `

async astream_chat(messages, **kwargs)

Async streaming chat endpoint for LLM.

Parameters:

• messages (Sequence[ChatMessage]) – Sequence of chat messages.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

ChatResponse – An async generator of ChatResponse objects, each containing a new token of the response.

Return type:

AsyncGenerator[ChatResponse, None]

Examples

```python from llama_index.core.llms import ChatMessage

gen = await llm.astream_chat([ChatMessage(role=”user”, content=”Hello”)]) async for response in gen:

print(response.delta, end=””, flush=True)

```

async astream_complete(*args, **kwargs)

Async streaming completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

• prompt (str) – Prompt to send to the LLM.

• formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

CompletionResponse – An async generator of CompletionResponse objects, each containing a new token of the response.

Return type:

AsyncGenerator[CompletionResponse, None]

Examples

```python gen = await llm.astream_complete(“your prompt”) async for response in gen:

print(response.text, end=””, flush=True)

```

chat(messages, **kwargs)

Chat endpoint for LLM.

Parameters:

• messages (Sequence[ChatMessage]) – Sequence of chat messages.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Chat response from the LLM.

Return type:

ChatResponse

Examples

```python from llama_index.core.llms import ChatMessage

response = llm.chat([ChatMessage(role=”user”, content=”Hello”)]) print(response.content) ```

classmethod class_name()

Get the class name, used as a unique ID in serialization.

This provides a key that makes serialization robust against actual class name changes.

Return type:

str

client: Client

complete(*args, **kwargs)

Completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

• prompt (str) – Prompt to send to the LLM.

• formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Returns:

Completion response from the LLM.

Return type:

CompletionResponse

Examples

`python response = llm.complete("your prompt") print(response.text) `

conversation_id: str | None

data: PromptTemplateData | None

property metadata: LLMMetadata

LLM metadata.

Returns:

LLM metadata containing various information about the LLM.

Return type:

LLMMetadata

model_id: str

moderations: ModerationParameters | None

parameters: TextGenerationParameters | None

parent_id: str | None

prompt_id: str | None

prompt_template_id: str | None

stream_chat(messages, **kwargs)

Streaming chat endpoint for LLM.

Parameters:

• messages (Sequence[ChatMessage]) – Sequence of chat messages.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

ChatResponse – A generator of ChatResponse objects, each containing a new token of the response.

Return type:

Generator[ChatResponse, None, None]

Examples

```python from llama_index.core.llms import ChatMessage

gen = llm.stream_chat([ChatMessage(role=”user”, content=”Hello”)]) for response in gen:

print(response.delta, end=””, flush=True)

```

stream_complete(*args, **kwargs)

Streaming completion endpoint for LLM.

If the LLM is a chat model, the prompt is transformed into a single user message.

Parameters:

• prompt (str) – Prompt to send to the LLM.

• formatted (bool, optional) – Whether the prompt is already formatted for the LLM, by default False.

• kwargs (Any) – Additional keyword arguments to pass to the LLM.

Yields:

CompletionResponse – A generator of CompletionResponse objects, each containing a new token of the response.

Return type:

Generator[CompletionResponse, None, None]

Examples

```python gen = llm.stream_complete(“your prompt”) for response in gen:

print(response.text, end=””, flush=True)

```

trim_method: str | TrimMethod | None

use_conversation_parameters: bool | None

class genai.extensions.llama_index.IBMGenAILlamaIndexEmbedding

Bases: BaseEmbedding

classmethod class_name()

Get the class name, used as a unique ID in serialization.

This provides a key that makes serialization robust against actual class name changes.

Return type:

str

client: Client

embed_batch_size: int

execution_options: dict | CreateExecutionOptions | None

model_id: str

parameters: dict | TextEmbeddingParameters | None

Submodules

• genai.extensions.llama_index.embeddings module
  • IBMGenAILlamaIndexEmbedding
    • IBMGenAILlamaIndexEmbedding.callback_manager
    • IBMGenAILlamaIndexEmbedding.class_name()
    • IBMGenAILlamaIndexEmbedding.client
    • IBMGenAILlamaIndexEmbedding.embed_batch_size
    • IBMGenAILlamaIndexEmbedding.execution_options
    • IBMGenAILlamaIndexEmbedding.model_id
    • IBMGenAILlamaIndexEmbedding.model_name
    • IBMGenAILlamaIndexEmbedding.num_workers
    • IBMGenAILlamaIndexEmbedding.parameters
• genai.extensions.llama_index.llm module
  • IBMGenAILlamaIndex
    • IBMGenAILlamaIndex.__init__()
    • IBMGenAILlamaIndex.achat()
    • IBMGenAILlamaIndex.acomplete()
    • IBMGenAILlamaIndex.astream_chat()
    • IBMGenAILlamaIndex.astream_complete()
    • IBMGenAILlamaIndex.callback_manager
    • IBMGenAILlamaIndex.chat()
    • IBMGenAILlamaIndex.class_name()
    • IBMGenAILlamaIndex.client
    • IBMGenAILlamaIndex.complete()
    • IBMGenAILlamaIndex.completion_to_prompt
    • IBMGenAILlamaIndex.conversation_id
    • IBMGenAILlamaIndex.data
    • IBMGenAILlamaIndex.messages_to_prompt
    • IBMGenAILlamaIndex.metadata
    • IBMGenAILlamaIndex.model_id
    • IBMGenAILlamaIndex.moderations
    • IBMGenAILlamaIndex.output_parser
    • IBMGenAILlamaIndex.parameters
    • IBMGenAILlamaIndex.parent_id
    • IBMGenAILlamaIndex.prompt_id
    • IBMGenAILlamaIndex.prompt_template_id
    • IBMGenAILlamaIndex.pydantic_program_mode
    • IBMGenAILlamaIndex.query_wrapper_prompt
    • IBMGenAILlamaIndex.stream_chat()
    • IBMGenAILlamaIndex.stream_complete()
    • IBMGenAILlamaIndex.system_prompt
    • IBMGenAILlamaIndex.trim_method
    • IBMGenAILlamaIndex.use_conversation_parameters
  • to_genai_message()
  • to_genai_messages()