Installation¶

Requirements¶

• Python: 3.12 or 3.13 (strictly required)
• Operating System: macOS or Linux
• A model provider: a foundation model and an embedding model reachable over any OpenAI-compatible endpoint (a hosted API, a self-managed vLLM/TGI/Ollama server, or an OpenShift MaaS deployment), accessed through the openai SDK — or your own BaseFoundationModel / BaseEmbeddingModel implementation
• A vector store: Milvus Lite (embedded, local-file, no setup required), or a running Milvus server/PostgreSQL (pgvector) instance for server-backed retrieval — all support hybrid search

Basic Installation¶

Install ai4rag using pip:

pip install "git+https://github.com/IBM/ai4rag.git@main"

This installs the core package with all required dependencies. Using "@main" will download and install latest version of ai4rag. If you want to use specific version, please use e.g. "@v0.1.1"

Vector store clients — pymilvus (with the milvus-lite extra), pgvector, and asyncpg — are core dependencies and install automatically. There is no separate vector-store extra to install. No extra step is needed to use a remote Milvus server, embedded Milvus Lite, or PostgreSQL/pgvector as a vector store.

pip install "ai4rag[text-extraction] @ git+https://github.com/IBM/ai4rag.git@main"

Development Installation¶

This project uses uv for dependency and environment management. Install it first if you haven't already:

curl -LsSf https://astral.sh/uv/install.sh | sh

Then clone and set up the project:

# Clone the repository
git clone https://github.com/IBM/ai4rag.git
cd ai4rag

# Install all development dependencies (creates .venv automatically)
uv sync --extra dev

The dev optional dependencies include:

• Testing tools (pytest, pytest-cov, pytest-mock)
• Code quality tools (black, pylint, isort)
• Documentation tools (mkdocs, mkdocs-material)
• Development utilities (beautifulsoup4, dotenv, ipykernel)

To install only a specific subset of dependencies, use the corresponding extra name (see pyproject.toml):

uv sync --extra test        # testing tools only
uv sync --extra code_check  # linting/formatting tools only
uv sync --extra docs        # documentation tools only

Optional Extras¶

Model Provider Setup (OpenShift MaaS)¶

ai4rag reaches foundation and embedding models through the stock openai SDK (installed automatically as a core dependency), so it works with any OpenAI-compatible endpoint — a hosted API, a self-managed server (vLLM, TGI, Ollama, …), or an OpenShift AI Models-as-a-Service (MaaS) deployment. The steps below use MaaS, the provider ai4rag ships helpers for; to use a different endpoint, point the same openai client at its URL (or supply your own BaseFoundationModel / BaseEmbeddingModel implementation). The vector store is configured independently, via direct clients (remote Milvus, embedded Milvus Lite, or PGVector) — see Vector Store Setup below.

1. Get Access to a MaaS Deployment¶

Obtain access to an OpenShift AI MaaS deployment that exposes:

• At least one foundation model (e.g., qwen3-8b-fp8-dynamic)
• At least one embedding model (e.g., bge-m3)

MaaS serves all models from a single OpenAI-compatible endpoint — MAAS_BASE_URL. No extra package is required — the openai SDK ships with ai4rag as a core dependency.

2. Note Your Credentials¶

Record the MaaS base URL and API key for use in ai4rag:

• MAAS_BASE_URL — the OpenAI-compatible endpoint URL (e.g. https://<host>); a trailing /v1 is appended automatically if not already present
• MAAS_API_KEY — a single API key for the single client that serves listing, chat, and embeddings

Vector Store Setup¶

ai4rag connects to the vector store directly — no MaaS deployment is required for this part. Pick a provider and pass its config to AI4RAGExperiment as vector_store_config:

from ai4rag.rag.vector_store import MilvusConfig, MilvusLiteConfig, PGVectorConfig

# Zero-config, embedded Milvus Lite backed by a local file (great for local experimentation)
vector_store_config = MilvusLiteConfig(db_path="./ai4rag.db")

# Or build configs from environment variables
vector_store_config = MilvusConfig.from_env()       # reads MILVUS_URI (must be http(s)://), MILVUS_TOKEN, MILVUS_SERVER_CERT
vector_store_config = MilvusLiteConfig.from_env()   # reads MILVUS_LITE_DB_PATH (optional; defaults to "./ai4rag_milvus_lite.db")
vector_store_config = PGVectorConfig.from_env()     # reads PGVECTOR_HOST, PGVECTOR_PORT, PGVECTOR_DB, PGVECTOR_USER, PGVECTOR_PASSWORD

Each config class exposes the environment variables it reads via its env_vars attribute, and can be constructed explicitly instead of from the environment, e.g. MilvusConfig(uri="https://localhost:19530") for a remote server or MilvusLiteConfig(db_path="./ai4rag.db") for embedded local storage.

MaaS Environment Configuration¶

Store your MaaS credentials securely in a .env file:

# .env
MAAS_BASE_URL="<maas_endpoint_url>"
MAAS_API_KEY="<maas_api_key>"

Load environment variables in your code:

import os
from dotenv import load_dotenv, find_dotenv

load_dotenv(find_dotenv())

base_url = os.getenv("MAAS_BASE_URL")
api_key = os.getenv("MAAS_API_KEY")

Verify Installation¶

Check that ai4rag is installed correctly:

import ai4rag
print(ai4rag.__version__)

Test MaaS connectivity:

import os
from ai4rag.utils.clients.maas_client import create_maas_client

# Single client — serves listing, chat, and embeddings for every model.
client = create_maas_client(
    base_url=os.getenv("MAAS_BASE_URL"),
    api_key=os.getenv("MAAS_API_KEY"),
)

# List available models
models = client.models.list().data
print(f"Available models: {[m.id for m in models]}")

Next Steps¶

• Quick Start Guide - Run your first optimization
• User Guide - Comprehensive usage documentation