Generative AI

Generative AI refers to AI systems that can produce new content — text, code, summaries, answers — rather than just classifying or predicting from existing data. Large language models (LLMs) are the most prominent example. Unlike traditional automation that requires explicit rules, generative AI can reason over ambiguous inputs, synthesize information from multiple sources, and adapt its output to context — making it useful across a wide range of business functions.

The business impact is concrete. Organizations are using generative AI to reduce the time analysts spend writing reports, help developers work faster by generating boilerplate and explaining unfamiliar code, surface insights buried in unstructured data like emails and service notes, and give non-technical staff direct access to data they previously needed IT to retrieve. For industries that run on IBM i — manufacturing, distribution, finance, insurance — these capabilities translate directly into operational efficiency, faster decision-making, and reduced reliance on specialized expertise for routine tasks.

For IBM i practitioners specifically, generative AI opens up a new class of capabilities: natural language interfaces to business data, automated code generation, document summarization, and intelligent assistants that understand your specific business domain.

What generative AI can do for IBM i

Natural language queries over Db2 for i

LLMs can translate plain English questions into SQL, allowing non-technical users to query IBM i data without knowing the schema. Combined with tools like watsonx.ai, this enables conversational BI directly over your existing Db2 for i tables.

RPG and CL code generation

LLMs trained on or fine-tuned with IBM i code can assist developers in writing RPG, CL, and SQL. Tools like IBM Bob and Continue (via the Developer Experience journey) integrate LLM code assistance directly into the IDE.

Document and record summarization

Unstructured text stored in IBM i — notes fields, text files on IFS, scanned documents — can be summarized, classified, or extracted from using generative AI APIs.

Conversational interfaces to business applications

LLM-powered chatbots can be built on top of IBM i applications, allowing users to ask questions like “What’s the inventory status for part number X?” and receive natural language answers backed by live Db2 for i queries.

Running LLMs on IBM i

Several options exist for running generative AI close to your IBM i data:

LLaMa.cpp (on IBM i via PASE)

LLaMA.cpp can run directly on IBM i in the PASE environment, hosting LLMs locally. This keeps data entirely on-premises. See Running AI natively on IBM i for setup details.

Calling an LLM from IBM i applications

The Db2 for i AI SDK provides SQL-based APIs for interacting with LLM endpoints — including watsonx.ai, Ollama, and any OpenAI-compatible API — directly from ILE programs, without requiring Python or any additional runtime.

Running LLMs on other services

Many on-premise and cloud options exist for running LLMs outside of IBM i, including but certainly not limited to:

Key considerations to remember

Data privacy — For sensitive data, prefer on-premises LLM options (Ollama, LLaMA.cpp, watsonx.ai on Cloud Pak for Data) over cloud APIs
Model selection — IBM Granite models are optimized for enterprise use cases and available via watsonx.ai; Llama 3 and Mistral are strong open-weight alternatives
Prompt engineering — The quality of LLM output depends heavily on how questions and context are structured; invest time in prompt design
Hallucination — LLMs can produce plausible-sounding but incorrect output; for factual business queries, always ground responses with RAG or validate against source data